This chapter reviews current methodologies used to measure and evaluate the effects of international scholarship programs for higher education. Just as these programs are diverse in their design, there is no “one size fits all” approach to evaluation. The overview of methodologies uses several analytic lenses and identifies their respective challenges: understanding the theory of change; choosing the unit of analysis; the timeline for evaluation; and approaches to quantitative and qualitative data collection. The chapter concludes with the importance of transparency in relaying evaluation outcomes to key audiences to improve programs and influence research in the field.
- U.S. Agency For International Development (USAID)
- Scholarship Recipients
- International Educationinternational Education
- International Fellowship Program (IFP)
- Mutual understandingMutual Understanding
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, access via your institution.
Tax calculation will be finalised at checkout
Purchases are for personal use onlyLearn about institutional subscriptions
While the report conducted by IIE was internal, the topics that emerged were discussed in a roundtable held at the Institute in December 2012 and were outlined on IIE’s blog: http://www.iie.org/Blog/2012/December/Alumni-Tracking#.V-CODc6cHIU
Amos, L.B., Windham, A, de los Reyes, I.B., Jones, W., and Baran, V. (2009). Delivering on the promise: An impact evaluation of the Gates Millennium Scholars Program. Washington, DC: American Institutes for Research.
Banerjee, A., Cole, S., Duflo, E. and Linden, L. (2005). Remedying education: Evidence from two randomized experiments in India, No. w11904, National Bureau of Economic Research. Available at: http://www.nber.org/papers/w11904.pdf (Accessed 23 Jan 2017).
Bamberger, M. (1999). Ethical Issues in Conducting Evaluation in International Settings. New Directions for Evaluation, 82, pp. 89–97.
Bamberger, M., Rugh, J. and Mabry, L. (2011). RealWorld evaluation: Working under budget, time, data, and political constraints. Thousand Oaks, CA: SAGE Publications, Inc.
Bhandari, R. and Belyavina, R. (2011). Evaluating and measuring the impact of citizen diplomacy: Current status and future directions. New York: Institute of International Education.
Boeren, A., Bakhuisen, K., Christian-Mak, A.M., Musch, V. and Pettersen, K. (2008). Donor policies and implementation modalities with regard to international postgraduate programmes targeting scholars from developing countries. Brussels: Vlaamse Interuniversitaire Raad.
CARE. (2009). Girls’ leadership development in action: CARE’s experience from the field. Available from: http://www.care.org/sites/default/files/documents/EDU-GE-2012-Girls-Leadership-Development-in-Action.pdf (Accessed 1 August 2016).
Center for Theory of Change. (2016). Center for Theory of Change home page. Available from: www.theoryofchange.org (Accessed 1 September 2016).
Chen, P., Weiss, F. L., Nicholson, H. J. and Girls Incorporated (2010). Girls Study Girls Inc.: Engaging Girls in Evaluation through Participatory Action Research. American Journal of Community Psychology, 46(1–2), pp. 228–237.
Chesterfield, R., and Dant, W. (2013). Evaluation of LAC higher education scholarships program: Final report. Washington, DC: U.S. Agency for International Development.
Chouinard, J.A. and Cousins, J.B. (2009). A Review and Synthesis of Current Research on Cross-Cultural Evaluation. American Journal of Evaluation, 30(4), pp. 457–494.
CIDA. (2005). Evaluation of the Canadian Francophone Scholarship Program (CFSP), 1987–2005. Quebec: CIDA.
Cook, T.D., Scriven, M., Coryn, C.L. and Evergreen, S.D. (2009). Contemporary thinking about causation in evaluation: A dialogue with Tom Cook and Michael Scriven. American Journal of Evaluation, 31(1), pp. 105–117.
Cosentino, C., Rangarajan, A., Sloan, M., Fortson, J., Moorthy, A., Humpage-Liuzzi, S. and Thomas, C. (2015). Monitoring, Evaluation, and Learning Design for The MasterCard Foundation Scholars Program. Washington, DC: Mathematica Policy Research.
Creed, C., Perraton, H and Waage, J. (2012). Examining development evaluation in higher education interventions: a preliminary study. London: London International Development Centre.
Gertler, P.J., Martinez, S., Premand, P., Rawlings, L.B. and Vermeersch, C.M. (2011). Impact evaluation in practice. Washington, DC: The World Bank.
Hansel, B. and Chen, Z. (2008). AFS long term impact study, 20 to 25 years after the exchange experience, AFS alumni are compared with their peers, Report 1. New York: AFS International.
Hesse-Biber, S.N. (2010). Mixed methods research: Merging theory with practice. New York: Guilford Press.
Hofmann-Pinilla, A. and Kallick Russell, J. (2009). Evaluation of the Leadership Development for Mobilizing Reproductive Health Program, Final Report Executive Summary. New York: Research Center for Leadership Action.
Institute of International Education. (2015). Schlumberger Faculty for the Future: Program impact study. New York: Institute of International Education.
Institute of International Education. (2016). LOTUS Annual Progress Report FY2016. Cairo, Egypt: IIE Cairo.
Jadad, A.R. and Enkin, M. (2007). Randomized controlled trials: questions, answers, and musings (2nd ed). Malden, MA: Blackwell Publishing.
Kirkpatrick, D.L. (1979). Techniques for evaluating training programs. Training and Development Journal, 33 (6), pp. 78–92.
Kirkpatrick, D. L. (1994). Evaluating training programs: the four levels. San Francisco: Berrett-Koehler.
Kusek, J.Z. and Rist, R.C. (2004). Ten steps to a results-based monitoring and evaluation system: A handbook for development practitioners. Washington, DC: The World Bank.
Marsh, R., Baxter, A., Cliff, R., Di Genova, L., Jamison, A. and Madden, M. (2016). Career choices, return paths and social contributions: The African alumni project, Abridged report. Toronto: The MasterCard Foundation.
Martel, M. and Bhandari, R. (2016). Social Justice and Sustainable Change: The Impacts of Higher Education, Ford Foundation International Fellowships Program Alumni Tracking Study Report No. 1. New York: Institute of International Education.
Martinez, A., Epstein, C. and Parsad, A. (2015). Evaluation of the National Science Foundation’s Partnerships for International Research and Education (PIRE) Program, Volume 1: Final report. Cambridge, MA: Abt Associates.
Mason, L., Powers, C. and Donnelly, S. (2015). The Boren Awards: A Report of Oral Language Proficiency Gains during Academic Study Abroad. New York, NY: Institute of International Education.
Mawer, M. (2014). A study of research methodology used in evaluations of international scholarship schemes for higher education. London: Commonwealth Scholarship Commission in the United Kingdom.
Mayne, J. (2011). Addressing cause and effect in simple and complex settings through contribution analysis, in Forss, K., Mara, M., and Schwartz, R. (eds), Evaluating the complex: Attribution, contribution and beyond. New York: Transaction Publishers.
Perna, L., Orosz, K., Gopaul, B., Jumakulov, Z., Ashirbekov, A., and Kishkentayaeva, M. (2014). Promoting human capital development: A typology of international scholarship programs in higher education. Educational Researcher, 43(2), pp. 66–73.
PPMI. (2012). Interim evaluation of Erasmus Mundus II (2009–2013), Brussels: European Commission Directorate-General of Education and Culture.
Ramboll Management Consulting. (2012). Final report: Evaluation of NPT and NICHE. Berlin: Ramboll Management Consulting.
Research Solutions International. (2016). The Benjamin A. Gilman International Scholarship Program Evaluation Report. Washington, DC: U.S. Department of State, Bureau of Educational and Cultural Affairs.
Rotem, A., Zinovieff, M., and Goubarev, A. (2010). A framework for evaluating the impact of the United Nations fellowship programmes. Human Resources for Health, 8(7). Available at: https://human-resources-health.biomedcentral.com/articles/10.1186/1478-4491-8-7 (Accessed 23 Jan 2017).
Scott, J., and Carrington, P. J. (2011). The SAGE handbook of social network analysis. Thousand Oaks, CA: SAGE publications.
SRI International. (2005). Outcome assessment of the visiting Fulbright Student Program. Washington, DC: U.S. Department of State, Bureau of Educational and Cultural Affairs.
The MasterCard Foundation. (2016). Education in Support of Social Transformation: Learning from the First Five Years of The MasterCard Foundation Scholars Program. Toronto: The MasterCard Foundation.
Tvaruzkova, M. (2012). Evaluation: Advances in measuring the effectiveness of networks. Paper presented at the Global Leadership Consortium, December 11, 2012, Washington, DC.
USAID. (2004). Generations of quiet progress: the development impact of US long-term university training on in Africa from 1963–2003, Report prepared by Aguirre International. Washington, DC: USAID.
Uyeki, E. (1993). As Others See Us: A Comparison of Japanese and American Fulbrighters. New York: Institute of International Education.
Valuy, E. and Martel, M. (2016). HER initiative to lead change: The power of education. New York: Institute of International Education.
Valuy, E. (2016). Centroamerica Adelante Final Report. New York: Institute of International Education.
Editors and Affiliations
© 2018 The Author(s)
About this chapter
Cite this chapter
Martel, M. (2018). Tracing the Spark that Lights a Flame: A Review of Methodologies to Measure the Outcomes of International Scholarships. In: Dassin, J., Marsh, R., Mawer, M. (eds) International Scholarships in Higher Education. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-62734-2_14
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-319-62733-5
Online ISBN: 978-3-319-62734-2