Impact Evaluations: Persistent Limitations, Alternative Approaches, Possible Responses

  • D. Brent EdwardsJr.


This chapter addresses how the studies reviewed in this manuscript are generally problematic for two reasons that go beyond their specific findings. The first problem is the econometric nature of the studies, while the second is the political-financial-intellectual complex from which these studies were borne and back into which they went as they furthered the interests of that complex. In addition to discussing these issues, this chapter focuses on how the combination of these issues contributes to relations of dependence between international researchers with expertise in these methods and their counterparts in low- and middle-income countries. In response to this situation, alternative methodological approaches are advocated. A number of suggestions are also made for addressing the political-economic challenges that confront knowledge production in the field of global education policy.


Knowledge production Impact evaluation Alternatives World Bank Political economy Global education policy International organization 


  1. Ball, S. (2012). Global education, inc.: New policy networks and the neo-liberal imaginary. New York: Routledge.Google Scholar
  2. Bennett, A., & Elman, C. (2006). Qualitative research: Recent developments in case study methods. Annual Review of Political Science, 9, 455–476.CrossRefGoogle Scholar
  3. Broad, R. (2006). Research, knowledge, and the art of “paradigm maintenance”: The World Bank’s development economics Vice-Presidency (DEC). Review of International Political Economy, 13(3), 387–419.CrossRefGoogle Scholar
  4. Chan, J. (2007). Between efficiency, capability, and recognition: Competing epistemes in global governance reforms. Comparative Education, 43(3), 359–376.CrossRefGoogle Scholar
  5. Dumas, M., & Anderson, G. (2014). Qualitative research as policy knowledge: Framing policy problems and transforming education from the ground up. Education Policy Analysis Archives, 22(11), 1–24. Scholar
  6. Edwards, D. B., Jr. (forthcoming). Accountability through community-based management? Implications from the local level implementation in El Salvador of a globally-popular model. In K. A. Heidemann & R. A. Clothey (Eds.), Another way: Decentralization, democratization, and the global politics of community-based schooling. Rotterdam: Sense.Google Scholar
  7. Edwards, D. B., Jr. (2018). The trajectory of global education policy: Community-based management in El Salvador and the global reform agenda. New York: Palgrave Macmillan.CrossRefGoogle Scholar
  8. Edwards, D. B., Jr., & Ávalos, E. (2015). Santa Marta y el programa EDUCO: Una experiencia de resistencia, adaptación e inversión de la lógica neoliberal. [Santa Marta and the EDUCO Program: An experience of resistance, adaptation and inversion of neoliberal reform logic.] San Salvador, El Salvador: Universidad Centroamericana.Google Scholar
  9. Edwards, D., Jr., & Klees, S. (2015). Unpacking “participation” in development and education governance: A framework of perspectives and practices. Prospects, 45(4), 483–499.CrossRefGoogle Scholar
  10. Erickson, F., & Gutierrez, K. (2002). Culture, rigor, and science in educational research. Educational Researcher, 31(8), 21–24. Scholar
  11. Gillies, J. (Ed.) (2010). Education system reform and aid effectiveness: The power of persistence. Washington, DC: USAID/Equip 2. Retrieved from
  12. Goldie, D., Linick, M., Jabbar, H., & Lubienski, C. (2014). Using bibliometric and social media analyses to explore the “echo chamber” hypothesis. Educational Policy, 28(2), 281–305.CrossRefGoogle Scholar
  13. Klees, S., & Edwards, D. B., Jr. (2014). Knowledge production and technologies of governance. In T. Fenwick, E. Mangez, & J. Ozga (Eds.), World yearbook of education 2014: Governing knowledge: Comparison, knowledge-based technologies and expertise in the regulation of education (pp. 31–43). New York: Routledge.Google Scholar
  14. Kvernbekk, T. (2013). Evidence-based practice: On the function of evidence in practical reasoning. Studier i Pedagogisk Filosofi, 2(2), 19–33.CrossRefGoogle Scholar
  15. La Londe, P., Brewer, T., & Lubienski, C. (2015). Teach for America and teach for all: Creating and intermediary organization network for global education reform. Education Policy Analysis Archives, 23(7), 1–25. Scholar
  16. Leamer, E. (1983). Let’s take the con out of econometrics. American Economic Review, 73(1), 31–41.Google Scholar
  17. Lieblich, A., Tuval-Mashiach, R., & Zilber, T. (1998). Narrative research: Reading, analysis and interpretation. Thousand Oaks, CA: Sage Publications.CrossRefGoogle Scholar
  18. Lubienski, C., Brewer, T., & La Londer, P. (2016). Orchestrating policy ideas: Philanthropies and think tanks in the US education policy advocacy networks. Australian Educational Researcher, 43(1), 55–73.CrossRefGoogle Scholar
  19. Moen, T. (2006). Reflections on the narrative research approach. International Journal of Qualitative Methodology, 5(4), Article 5. Retrieved from CrossRefGoogle Scholar
  20. Pawson, R. (2006). Evidence-based policy: A realist perspective. London: Sage.CrossRefGoogle Scholar
  21. Pawson, R., & Tilley, N. (1997). Realistic evaluation. London: Sage.Google Scholar
  22. Quint, R., & Lucas, S. (2014). The power and potential of southern think tanks for the post-2015 agenda. : What comes after the MDGs? Retrieved from
  23. Samoff, J. (1991). The façade of precision in education data and statistics: A troubling example from Tanzania. The Journal of Modern African Studies, 29(4), 669–689.CrossRefGoogle Scholar
  24. Samoff, J. (1993). The reconstruction of schooling in Africa. Comparative Education Review, 37(2), 181–222.CrossRefGoogle Scholar
  25. Samoff, J. (1996). Chaos and certainty in development. World Development, 24(4), 611–633.CrossRefGoogle Scholar
  26. Schroeder, L., Sjoquist, D., & Stephan, P. (1986). Understanding regression analysis. London: SAGE.CrossRefGoogle Scholar
  27. Spring, J. (2015). Economization of education: Human capital, global corporations, skills-based schooling. New York: Routledge.Google Scholar
  28. Upshur, C. (1995). What is participatory evaluation (PE)? What are its roots?, The Evaluation Exchange, 1(3/4). Retrieved from
  29. Vavrus, F., & Bartlett, L. (2006). Comparatively knowing: Making a case for the vertical case study. Current Issues in Comparative Education, 8(2), 95–103. Retrieved from
  30. Verger, A. (2012). Framing and selling global education policy: The promotion of PPPs in education in low-income countries. Journal of Education Policy, 27(1), 109–130.CrossRefGoogle Scholar
  31. Verschuren, P. (2001). Holism versus reductionism in modern social science research. Quality & Quantity, 35, 389–405. Scholar
  32. Weiss, C. (1995). The four “I’s” of school reform: How interests, ideology, information, and institution affect teachers and principals. Harvard Education Review, 65(4), 571–592.CrossRefGoogle Scholar
  33. Whitehead, T. (2002). Introduction to ethnographic assessment and evaluation systems. Ethnographic Assessment & Evaluation Systems Working Paper #2. Departments of Anthropology, University of Maryland. Retrieved from

Copyright information

© The Author(s) 2018

Authors and Affiliations

  • D. Brent EdwardsJr.
    • 1
  1. 1.University of Hawaii at ManoaHonoluluUSA

Personalised recommendations