Skip to main content

The Institutionalisation of Evaluation: Theoretical Background, Analytical Framework and Methodology

  • Chapter
  • First Online:
The Institutionalisation of Evaluation in Asia-Pacific

Abstract

This book contributes to the research on the institutionalisation of evaluation in a global comparative perspective by investigating the role of evaluation in the national political system, civil society and systems of professions, using a comparative research methodology that includes inter- and intra-regional comparisons. Specifically, it examines the role of evaluation in three social sub-systems: the national political system as the main authority for political decision-making, civil society for controlling and supporting political governance and systems of professions for implementing and improving knowledge and skills production. Drawing on modernisation theories and a social science perspective of institutions and organisations, the book analyses evaluation practices, regulations, laws and policies, and their embeddedness in decision-making structures. It also explores the organisational anchoring of evaluation in parliaments, audit offices and other authorities, as well as its use in civil society for improving social services and state control. In addition, it investigates the implementation of evaluation in academic study programmes, research infrastructure and voluntary organisations for professional evaluation in systems of professions. The introductory chapter provides an outline of the theoretical background and analytical framework, including its central theoretically derived categories, theoretical specifications regarding Asia-Pacific, and methodological considerations, such as the applied comparative research design. Overall, this book offers valuable insights into the institutionalisation of evaluation and its role in promoting better governance and social outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    In 2022 Diwakar and colleagues updated their report with a new focus and country selection (Diwakar et al., 2022). This time they included Afghanistan, Australia, Azerbaijan, Bangladesh, Bhutan, Cambodia, Fiji, India, Indonesia, Japan, Nepal, Pakistan, Philippines, Sri Lanka, and Tajikistan (n = 15) and the research focused on four dimensions, namely the ‘enabling environment’, ‘institutional capacities’, ‘standardising capacities’ and ‘VOPE capacities’.

  2. 2.

    The term societal system is used here when speaking of various functionally differentiated subsystems such as law, politics, religion, health, etc. It is essential to distinguish this from one of the specific societal subsystems under investigation, which is called “social system” in the study (see Sect. 3.2).

  3. 3.

    The same can be said when looking at the world’s leading countries in institutionalising evaluation, but with a different time frame.

  4. 4.

    The general added value of such comparative studies has been pointed out not least by the Comparative Area Studies (CAS) research programme (Ahram, 2011; Basedau & Köllner, 2007; Berg-Schlosser, 2012; Mainwaring & Pérez-Liñán, 2007).

  5. 5.

    For a general discussion of how changing economic and political circumstances—especially in the light of the globally evolving ‘knowledge-based economies’—affect the way science and the universities are organised and knowledge is produced (see Etzkowitz & Leydesdorff, 1998, 2000; Weingart 1997.

  6. 6.

    For an overview of this design please refer to the following literature: Elder (1976), Andreß et al. (2019), Livingstone (2003, 2013), Harkness (2012), Goerres et al. (2019), Ragin (2006, 2007, 2014), Hantrais (2008), and Wong (2014).

  7. 7.

    Since the number of possible cases in so-called “cross-national studies” is limited, Lijphart suggests changing the unit of analysis under certain circumstances. Then, for example, “multinational but cross-individual” research can be conducted (cf. for this conceptual distinction Hopkins & Wallerstein, 1967). With regard to the study of evaluation systems, such a design can be used to study organisations or individuals (evaluation experts, parliamentarians, government officials, representatives of civil society, etc.). This allows for more meaningful statistical analyses and hypothesis testing, since it is possible to move to probabilistic large-N samples, in which many minimum requirements of inductive statistics are first met. The nation-state context would then be included as an additional explanatory variable.

  8. 8.

    This and other strengths of small-N cross-national comparisons are discussed by Ebbinghaus (2005).

  9. 9.

    Overall, the Evaluation Globe research project combines intra- and inter-regional comparative research strategies. The selection of cases has to be balanced between the number of cases and the heterogeneity of the case conditions. While intra-regional comparisons tend to be homogeneous in order to allow for a most similar systems comparison, inter-regional designs tend to be heterogeneous in order to allow for a most different systems comparison and exploratory research. The sample in the Evaluation Globe allows for both: the case selection consists of relatively homogeneous cases with individual contrasting case studies; with respect to the comparison between the volumes, i.e., the world regions, the diversity of the systems studied and their basic conditions then increases by leaps and bounds, i.e., approaches a most different case design (cf. Basedau & Köllner, 2007).

References

  • Abbott, A. (1988). The system of professions: An essay on the division of expert labour. Chicago University Press.

    Google Scholar 

  • Ahmed, V., & Bamberger, M. (1989). Monitoring and evaluating development projects: The South Asian experience, seminar report series. World Bank, Economic Development Institute.

    Google Scholar 

  • Ahram, A. I. (2011). The theory and method of comparative area studies. Qualitative Research, 11(1), 69–90.

    Article  Google Scholar 

  • Akpan, W., Minkley, G., & Thakrar, J. (2012). In search of a developmental university: Community engagement in theory and practice. South African Review of Sociology, 43(2), 1–4.

    Article  Google Scholar 

  • Alexander, J. C. (2001). Soziale Differenzierung und kultureller Wandel: Essays zur neofunktionalistischen Gesellschaftstheorie. Campus-Verlag.

    Google Scholar 

  • Andreß, H. J., Fetchenhauer, D., & Meulemann, H. (2019). Cross-national comparative research—Analytical strategies, results, and explanations. KZfSS Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 71(1), 1–28.

    Article  Google Scholar 

  • Ansell, C., & Torfing, J. (Eds.). (2016). Handbook on theories of governance. Edward Elgar Publishing.

    Google Scholar 

  • Ansell, C., & Torfing, J. (2021a). Co-creation: The new kid on the block in public governance. Policy & Politics, 49(2), 211–230.

    Article  Google Scholar 

  • Ansell, C., & Torfing, J. (2021b). Public governance as co-creation: A strategy for revitalizing the public sector and rejuvenating democracy. Cambridge University Press.

    Google Scholar 

  • Armingeon, K. (2016). Political institutions. In H. Keman, & J. J. Woldendorp (Eds.), Handbook of research methods and applications in political science (pp. 234–247). Edward Elgar.

    Google Scholar 

  • Aulich, C. (2009). From citizen participation to participatory governance in Australian local government. Commonwealth Journal of Local Governance, 2, 44–60.

    Article  Google Scholar 

  • Bamberger, M. (1989). The monitoring and evaluation of public sector programs in Asia: Why are development programs not evaluated? Evaluation Review, 13(1), 223–242.

    Article  Google Scholar 

  • Bamberger, M. (1991). The politics of evaluation in developing countries. Evaluation and Program Planning, 14(4), 325–339.

    Article  Google Scholar 

  • Bank, V., & Lames, M. (2000). Über Evaluation. bajOsch-Hein.

    Google Scholar 

  • Barker, R. M. (2010). Corporate governance, competition, and political parties: Explaining corporate governance change in Europe. Oxford University Press.

    Book  Google Scholar 

  • Basedau, M., & Köllner, P. (2007). Area studies, comparative area studies, and the study of politics: Context, substance, and methodological challenges. Zeitschrift Für Vergleichende Politikwissenschaft, 1(1), 105–124.

    Article  Google Scholar 

  • Bazeley, P. (2017). Integrating analyses in mixed methods research. Sage.

    Google Scholar 

  • Beckert, J. (2010a). How do fields change? The interrelations of institutions, networks, and cognition in the dynamics of markets. Organization Studies, 31(5), 605–627.

    Article  Google Scholar 

  • Beckert, J. (2010b). Institutional isomorphism revisited: Convergence and divergence in institutional change. Sociological Theory, 28(2), 150–166.

    Article  Google Scholar 

  • Beeson, M. (2008). Institutions of the Asia-Pacific: ASEAN, APEC and beyond. Routledge.

    Book  Google Scholar 

  • Berg-Schlosser, D. (2012). Comparative area studies—Goldener Mittelweg zwischen Regionalstudien und universalistischen Ansätzen? Zeitschrift Für Vergleichende Politikwissenschaft, 1(6), 1–16.

    Article  Google Scholar 

  • Bertelli, A. M. (2012). The political economy of public sector governance. Cambridge University Press.

    Book  Google Scholar 

  • Bevir, M. (2012). Governance: A very short introduction. Oxford University Press.

    Book  Google Scholar 

  • Blomeyer, R., & Eussner, A. (2020). European Union. In R. Stockmann, W. Meyer, & L. Taube (Eds.), The institutionalisation of evaluation in Europe (pp. 463–479). Palgrave Macmillan.

    Chapter  Google Scholar 

  • Borrmann, A., & Stockmann, R. (2009). Evaluation in der deutschen Entwicklungszusammenarbeit. Band 1: Systemanalyse. Band 2: Fallstudien. Sozialwissenschaftliche Evaluationsforschung, Band 8. Waxmann.

    Google Scholar 

  • Borthwick, M. (2018 [1992]). Pacific century: The emergence of modern Pacific Asia. Routledge.

    Google Scholar 

  • Borthwick, M., & Latz, G. (2022). The Pacific century study guide. Routledge.

    Google Scholar 

  • Bose, S., & Jalal, A. (2023). Modern South Asia: History, culture, political economy. Routledge.

    Google Scholar 

  • Brante, T. (1988). Sociological approaches to the professions. Acta Sociologica, 31(2), 119–142.

    Article  Google Scholar 

  • Brook, C., & McGrew, A. (2013). Asia-Pacific in the new world order. Routledge.

    Book  Google Scholar 

  • Bundi, P. (2018). Parliamentarians’ strategies for policy evaluations. Evaluation and Program Planning, 69, 130–138.

    Article  Google Scholar 

  • Burns, T. R. (2006). The sociology of complex systems: An overview of actor-system-dynamics theory. World Futures, 62(6), 411–440.

    Article  Google Scholar 

  • Bussmann, W., Klöti, U., & Knoepfel, P. (1997). Einführung in die Politikevaluation. Helbig & Lichtenhahn.

    Google Scholar 

  • Carman, J. C. (2009). Nonprofits, funders, and evaluation: Accountability in action. The American Review of Public Administration, 39(4), 374–390.

    Google Scholar 

  • Chan, A. S., & Fisher, D. (Eds.). (2008). The exchange university: Corporatisation of academic culture. UBS-Press.

    Google Scholar 

  • Cheung, A. B. (2005). The politics of administrative reforms in Asia: Paradigms and legacies, paths and diversities. Governance, 18(2), 257–282.

    Article  Google Scholar 

  • Cheung, A. B. (2016). NPM in Asian countries. In T. Christensen, & P. Lægreid (Eds.), The Ashgate research companion to new public management (pp. 147–160). Routledge.

    Google Scholar 

  • Cheung, A. B., & Scott, I. (2003). Governance and public sector reforms in Asia: Paradigms, paradoxes and dilemmas. In A. Cheung, & I. Scott (Eds.), Governance and public sector reform in Asia (pp. 12–35). Routledge.

    Chapter  Google Scholar 

  • Christensen, T., & Lægreid, P. (Eds.). (2007). Transcending new public management: The transformation of public sector reforms. Ashgate.

    Google Scholar 

  • Christensen, T., & Lægreid, P. (2022). Taking stock: New public management (NPM) and post-NPM reforms-trends and challenges. In A. Ladner, & F. Sager (Eds.), Handbook on the politics of public administration (pp. 38–49). Edgar Elgar.

    Chapter  Google Scholar 

  • Church, P. (2017). A short history of South-East Asia. Wiley.

    Google Scholar 

  • Clark, B. (2001). The entrepreneurial university: New foundations for collegiality, autonomy, and achievement. Higher Education Management, 13(2), 9–24.

    Google Scholar 

  • Coase, R. H. (1937). The nature of the firm. Economica, 4(16), 386–405.

    Article  Google Scholar 

  • Coase, R. H. (1991). The institutional structure of production. Journal des économistes et des études humaines, 2(4), 431–440.

    Article  Google Scholar 

  • Cohen, J. L., & Arato, A. (1994). Civil society and political theory. MIT Press.

    Google Scholar 

  • Colbeck, C. L. (2008). Professional identity development theory and doctoral education. New Directions for Teaching and Learning, 113, 9–16.

    Article  Google Scholar 

  • Coleman, J. S. (1986). The idea of the developmental university. Minerva, 24(4), 476–494.

    Article  Google Scholar 

  • Coleman, J. S. (1990). Foundations of social theory. Harvard University Press.

    Google Scholar 

  • Cram, F. (2015). Lessons on decolonizing evaluation from Kaupapa Māori evaluation. Canadian Journal of Program Evaluation, 30(3), 296–312.

    Article  Google Scholar 

  • Cutt, J., & Murray, V. (2000). Accountability and effectiveness evaluation in non-profit organisations. Routledge.

    Book  Google Scholar 

  • Della Porta, D. (2008). 11 Comparative analysis: Case-oriented versus variable-oriented research. In D. Della Porta, & M. Keating (Eds.), Approaches and methodologies in the social sciences (pp. 198–222). Cambridge University Press.

    Google Scholar 

  • Dhakal, T. R. (2014). Institutionalization and use of evaluations in the public sector in Nepal. Journal of MultiDisciplinary Evaluation, 10(23), 51–58.

    Article  Google Scholar 

  • Diwakar, Y., De Mel, R. L., & Samarsinghe, C. H. (2021). A study on the status of national evaluation policies and systems in Asia Pacific region. Asia Pacific Evaluation Association.

    Google Scholar 

  • Diwakar, Y., De Mel, R., Samarsinghe, H., Liyanagamage, M., Mahmodi, S., & Kalugampitiya, A. (2022). A study on the status of national evaluation policies and systems in the Asia Pacific Region—2022. APEA. https://asiapacificeval.org/programs/regional-evaluation-strategy/promoting-national-evaluation-updates/. Accessed on 13 March 2023.

  • Drori, G. S., Höllerer, M. A., & Walgenbach, P. (Eds.). (2014). Global themes and local variations in organization and management. Routlege.

    Google Scholar 

  • Drori, G. S., & Preminger, B. (2018). Between institutional theory and Lefebvre: Sensemaking, logics and enactment of, and in, space. In K. Dale, S. F. Kingma, & V. Wasserman (Eds.), Organizational space and beyond (pp. 104–130). Routledge.

    Google Scholar 

  • Ebbinghaus, B. (2005). When less is more: Selection problems in large-N and small-N cross-national comparisons. International Sociology, 20(2), 133–152.

    Article  Google Scholar 

  • Ebener, A. (2008). Institutional evolution and the political economy of governance. In A. Ebener, & N. Beck (Eds.), The institutions of the market: Organisations, social systems, and governance (pp. 287–308). Oxford University Press.

    Google Scholar 

  • Edwards, M. (2020). Civil society (4th ed). Polity Press.

    Google Scholar 

  • Elder, J. W. (1976). Comparative cross-national methodology. Annual Review of Sociology, 2(1), 209–230.

    Article  Google Scholar 

  • Esser, H. (2002). Soziologie: Institutionen (Vol. 5). Campus Verlag.

    Google Scholar 

  • Etzkowitz, H. (2003). Research groups as ‘quasi-firms’: The invention of the entrepreneurial university. Research Policy, 32(1), 109–121.

    Article  Google Scholar 

  • Etzkowitz, H. (2004). The evolution of the entrepreneurial university. International Journal of Technology and Globalisation, 1(1), 64–77.

    Article  Google Scholar 

  • Etzkowitz, H. (2013). Anatomy of the entrepreneurial university. Social Science Information, 52(3), 486–511.

    Article  Google Scholar 

  • Etzkowitz, H., & Leydesdorff, L. (1998). The endless transition: A “Triple Helix” of university-industry-government relations: Introduction. Minerva, 36(3), 203–208.

    Article  Google Scholar 

  • Etzkowitz, H., & Leydesdorff, L. (2000). The dynamics of innovation: From national systems and “Mode 2” to a Triple Helix of university–industry–government relations. Research Policy, 29(2), 109–123.

    Article  Google Scholar 

  • Fakis, A., Hilliam, R., Stoneley, H., & Townend, M. (2014). Quantitative analysis of qualitative information from interviews: A systematic literature review. Journal of Mixed Methods Research, 8(2), 139–161.

    Article  Google Scholar 

  • Fetterman, D. M. (2023). Empowerment evaluation and social justice. Routledge.

    Google Scholar 

  • Forbrig, J. (2002). The nexus between civil society and democracy suggesting a critical approach. In W. Reichel (Ed.), Political priorities between East and West: Europe’s rediscovered wealth—What the accession-candidates in Eastern and Central Europe have to offer (pp. 79–103). Institute for the Danube Region and Central Europe.

    Google Scholar 

  • Freidson, E. (2001). Professionalism: The third logic. Polity Press.

    Google Scholar 

  • Friedman, A., & Phillips, M. (2004). Balancing strategy and accountability: A model for the governance of professional associations. Nonprofit Management and Leadership, 15(2), 187–204.

    Article  Google Scholar 

  • Furubo, J. E., Rist, R. C., & Sandahl, R. (2002). International atlas of evaluation. Transaction Publishers.

    Google Scholar 

  • Giddens, A. (1990). Consequences of modernity. Polity Press.

    Google Scholar 

  • Goerres, A., Siewert, M. B., & Wagemann, C. (2019). Internationally comparative research designs in the social sciences: Fundamental issues, case selection logics, and research limitations. KZfSS Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 71(1), 75–97.

    Article  Google Scholar 

  • Goodnow, F. J. (2003). Politics and administration: A study in government. Routledge.

    Google Scholar 

  • Greenwood, R., Suddaby, R., & Hinings, C. R. (2017). Theorizing change: The role of professional associations in the transformation of institutionalised fields. Academy of Management Journal, 45(1), 58–80.

    Article  Google Scholar 

  • Guerrero, M., & Urbano, D. (2012). The development of an entrepreneurial university. The Journal of Technology Transfer, 37, 43–74.

    Article  Google Scholar 

  • Hall, P. A., & Taylor, R. C. R. (1996). Political science and the three new institutionalisms. Political Studies, 44(5), 936–957.

    Google Scholar 

  • Han, Y. (2019). The impact of accountability deficit on agency performance: Performance-accountability regime. Public Management Review, 22(6), 927–948.

    Article  Google Scholar 

  • Hanberger, A. (2001). Policy and program evaluation, civil society, and democracy. American Journal of Evaluation, 22(2), 211–228.

    Article  Google Scholar 

  • Hantrais, L. (2008). International comparative research: Theory, methods and practice. Palgrave Macmillan.

    Google Scholar 

  • Harkness, J. A. (2012). Comparative survey research: Goals and challenges. In E. D. de Leeuw, J. Hox, & D. Dillman (Eds.), International handbook of survey methodology (pp. 56–77). Routledge.

    Google Scholar 

  • Hay, K. (2010). Evaluation field building in South Asia: Reflections, anecdotes, and questions. American Journal of Evaluation, 31(2), 222–231.

    Article  Google Scholar 

  • Hay, K. E. (2014). Building the field of evaluation in South Asia: A framework and ideas. In K. E. Hay, & S. Kumar-Range (Eds.), Making evaluation matter: Writings from South Asia (p. 39). Sage.

    Google Scholar 

  • He, L. (2010). Social movement tradition and the role of civil society in Japan and South Korea. East Asia, 27(3), 267–287.

    Article  Google Scholar 

  • Holcombe, C. (2017). A history of East Asia. Cambridge University Press.

    Book  Google Scholar 

  • Holzer, B. (2006). Spielräume der Weltgesellschaft: Formale Strukturen und Zonen der Informalität. In T. Schwinn (Ed.), Die Vielfalt und Einheit der Moderne: Kultur- und Strukturvergleichende Analysen (pp. 259–279). Springer Link.

    Chapter  Google Scholar 

  • Holzinger, M. (2014). Fehlschlüsse über die‘Weltgesellschaft’. Kölner Zeitschrift Für Soziologie & Sozialpsychologie, 66(2), 267–289.

    Article  Google Scholar 

  • Hopkins, T. K., & Wallerstein, I. (1967). The comparative study of national societies. Social Science Information, 6(5), 25–58.

    Google Scholar 

  • Howell, J., & Pearce, J. (2001). Civil society and development: A critical exploration. Lynne Rienner.

    Google Scholar 

  • Inglehart, R. (1998). Modernisierung und Postmodernisierung: Kultureller, wirtschaftlicher und politischer Wandel in 43 Gesellschaften. Campus-Verlag.

    Google Scholar 

  • Ingvarson, L. (1998). Professional development as the pursuit of professional standards: The standards-based professional development system. Teaching and Teacher Education, 14(1), 127–140.

    Article  Google Scholar 

  • Jacob, S., Speer, S., & Furubo, J. E. (2015). The institutionalisation of evaluation matters: Updating the international atlas of evaluation 10 years later. Evaluation, 21(1), 6–31.

    Article  Google Scholar 

  • Jeong, M. G., & Oh, S. G. (2010). Searching for participatory governance in Korea. Journal of Contemporary Asia, 40(2), 275–290.

    Article  Google Scholar 

  • Kalm, S., & Uhlin, A. (2015). Civil society and the governance of development: Opposing Global Institutions. Palgrave Macmillan.

    Book  Google Scholar 

  • Kalugampititya, A. (2021). Critical factors for institutionalizing evaluation at national level: Study on four countries in Asia- Sri Lanka, Nepal, Philippines and Bangladesh (PhD Thesis). Department of Sociology, Saarland University, Saarbücken, Germany.

    Google Scholar 

  • Kapitanova, J. (2013). Regeln in Sozialen Systemen. Springer VS.

    Book  Google Scholar 

  • Kassomeh, N. (2021). Revisiting the democratic transition in South Korea and Taiwan. Asian Political Science Review, 5(1), 1–12.

    Google Scholar 

  • Kettl, D. F. (2000). The transformation of governance: Globalization, devolution, and the role of government. Public Administration Review, 60(6), 488–497.

    Article  Google Scholar 

  • Kettl, D. F. (2015). The transformation of governance: Public administration for the twenty-first century. JHU Press.

    Book  Google Scholar 

  • Kettl, D. F. (2018). Politics of the administrative process. Sage/CQ Press.

    Google Scholar 

  • Kettl, D. F. (2019). From policy to practice: From ideas to results, from results to trust. Public Administration Review, 79(5), 763–767.

    Article  Google Scholar 

  • Khan, M. A. (1989). A South Asian regional study on current thoughts and practices in monitoring and evaluation. Economic Development Institute of the World Bank.

    Google Scholar 

  • Khan, M. A. (1990). Monitoring and evaluation of development projects in South East Asia, the Experience of Indonesia, Malaysia, the Philippines and Thailand. Economic Development Institute of the World Bank.

    Google Scholar 

  • Khan, M. A. (1998). Evaluation capacity building: An overview of current status, issues and options. Evaluation, 4(3), 310–328.

    Google Scholar 

  • Khan, M. A., de Guzman, G. G., Trinidad, E. A., & Selda, G. G. (1993). Study of government monitoring and evaluation systems: The case of the Philippines. Central Evaluation Office, United Nations Development Program.

    Google Scholar 

  • Kieser, A., & Ebers, M. (Eds.). (2019). Organisationstheorien. Kohlhammer Verlag.

    Google Scholar 

  • Kim, P. S. (2011). Civic engagement, politics and policy in South Korea: Significant developments but a considerable way to go. Public Administration and Development, 31(2), 83–90.

    Article  Google Scholar 

  • Kirschsieper, D., & Brose, H. G. (2014). Un-/Gleichzeitigkeit und Synchronisation. Zum Verhältnis von Diachronie und Synchronie in der Theorie Sozialer Systeme. Zeitschrift für Theoretische Soziologie, 3(2), 172–219.

    Google Scholar 

  • Kiser, L., & Ostrom, E. (1982). The three worlds of action: A metatheoretical synthesis of institutional approaches in strategies of political inquiry. In E. Ostrom (Ed.), Strategies of political inquiry (pp. 172–222). Sage.

    Google Scholar 

  • Kollmorgen, R. (2014). Modernisierungstheoretische Ansätze. In R. Kollmorgen, W. Merkel, & H. J. Wagener (Eds.), Handbuch Transformationsforschung. Springer-Verlag.

    Google Scholar 

  • Krohn, W. (2000). Wissenschaftssoziologie: zwischen Modernisierungstheorie und Sozialkonstruktivismus auf schwankendem epistemischem Boden. Soziologische Revue, 23(Supplement), 314–325.

    Article  Google Scholar 

  • Kroneberg, C. (2014). Frames, scripts, and variable rationality: An integrative theory of action. In G. Manzo (Ed.), Analytical sociology. Actions and Networks (pp. 95–123). Wiley.

    Google Scholar 

  • Larivière, V., Haustein, S., & Mongeon, P. (2015). The oligopoly of academic publishers in the digital era. PLoS ONE, 10(6), 1–15.

    Article  Google Scholar 

  • Lauth, H. J., Pickel, G., & Pickel, S. (2014). Vergleich politischer Systeme. Eine Einführung. utb.

    Google Scholar 

  • Laville, J. L., Young, D. R., & Eynaud, P. (Eds.). (2015). Civil society, the third sector and social enterprise: Governance and democracy. Routledge.

    Google Scholar 

  • Lee, Y., & Park, T. (2009). Civil participation in the making of a new regulatory state in Korea: 1998–2008. Korea Observer, 40(3), 461.

    Google Scholar 

  • Lerner, D. (1968). Modernisation: Social aspects. In L. David (Ed.), International encyclopedia of the social sciences (pp. 386–402). Macmillan.

    Google Scholar 

  • Lijphart, A. (1971). Comparative politics and the comparative method. American Political Science Review, 65(3), 682–693.

    Article  Google Scholar 

  • Lijphart, A. (1975). II. The comparable-cases strategy in comparative research. Comparative Political Studies, 8(2), 158–177.

    Google Scholar 

  • Livingstone, S. (2003). On the challenges of cross-national comparative media research. European Journal of Communication, 18(4), 477–500.

    Article  Google Scholar 

  • Livingstone, S. (2013). Challenges to comparative research in a globalizing media landscape. In F. Esser, & T. Hanitzsch (Eds.), The handbook of comparative communication research (pp. 437–451). Routledge.

    Google Scholar 

  • Lovan, W. R., Murray, M., & Shaffer, R. (Eds.). (2003). Participatory governance: Planning, conflict mediation and public decision making in civil society. Routledge.

    Google Scholar 

  • Lowe, D., & Potter, C. (2018). Understanding legislation: A practical guide to statutory interpretation. Bloomsbury Publishing.

    Book  Google Scholar 

  • Luhmann, N. (1981). The improbability of communication. International Social Science Journal, 33(1), 122–132.

    Google Scholar 

  • Luhmann, N. (1983). Legitimation durch Verfahren. Suhrkamp.

    Google Scholar 

  • Luhmann, N. (1988). Soziale Systeme: Grundriß einer allgemeinen Theorie. Suhrkamp.

    Google Scholar 

  • Luhmann, N. (1991). Operational closure and structural coupling: The differentiation of the legal system. Cardozo Law Review, 13, 1419.

    Google Scholar 

  • Luhmann, N. (1992). What is communication? Communication Theory, 2(3), 251–259.

    Article  Google Scholar 

  • Luhmann, N. (1995). Social systems. Stanford University Press.

    Google Scholar 

  • MacDonald, K. M. (1985). Social closure and occupational registration. Sociology, 19(4), 541–556.

    Article  Google Scholar 

  • MacDonald, K. M. (1995). The sociology of the professions. Sage.

    Google Scholar 

  • Mackay, K. (1999). Evaluation capacity development: A diagnostic guide and action framework. World Bank, Operations Evaluation Department.

    Google Scholar 

  • Mainwaring, S., & Pérez-Liñán, A. (2007). Why regions of the world are important: Regional specificities and region-wide diffusion of democracy. In G. Munk (Ed.), Regimes and democracy in Latin America: Theories and methods (pp. 199–229). Oxford University Press.

    Google Scholar 

  • Malik, K., & Roth, C. (1999). Evaluation capacity development in Asia. UNDP, NCSTE and The World Bank.

    Google Scholar 

  • March, J. G., & Olsen, J. P. (1983). The new institutionalism: Organizational factors in political life. American Political Science Review, 78(3), 734–749.

    Article  Google Scholar 

  • Marche, J. C., & Olsen, J. P. (1989). Rediscovering institutions. The Organizational Basis of Politics. The Free Press.

    Google Scholar 

  • Maxwell, J. A. (2010). Using numbers in qualitative research. Qualitative Inquiry, 16(6), 475–482.

    Article  Google Scholar 

  • Mayntz, R. (2017). From government to governance: Political steering in modern societies. In D. Scheer, & R. F. Rubik (Eds.), Governance of integrated product policy (pp. 18–25). Routledge.

    Google Scholar 

  • McCowan, T. (2019). The developmental university. In T. McCowan (Ed.), Higher Education for and beyond the sustainable development goals (pp. 91–114). SpringerLink.

    Google Scholar 

  • McDougall, D. (2016). Asia Pacific in world politics (2nd ed.). Lynne Rienner Publishers.

    Book  Google Scholar 

  • Mehrotra, S. (2013). The government monitoring and evaluation system in India: A work in progress (ECD Working Paper Series, 28). Independent Evaluation Group, The World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/reports/ecd_wp28_india_me_0.pdf. Accessed on 23 June 2021.

  • Mertens, D. M. (1998). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches. Sage.

    Google Scholar 

  • Merton, R. K. (1942). The normative structure of science. In R. K. Merton (Ed.), The Sociology of Science: Theoretical and Empirical Investigations (pp. 267–278). University of Chicago Press.

    Google Scholar 

  • Meyer, J. W. (2017). Reflections on institutional theories of organizations. In R. Greenwood, C. Oliver, T. B. Lawrence, & R. E. Meyer (Eds.), The Sage handbook of organizational institutionalism (pp. 831–852). Sage.

    Chapter  Google Scholar 

  • Meyer, W., Stockmann, R., & Szentmarjay, L. (2022). The institutionalisation of evaluation: Theoretical background, analytical concept and methods. In R. Stockman, W. Meyer, & L. Szentmarjay (Eds.), The institutionalisation of evaluation in the Americas (pp. 3–37). Springer International Publishing.

    Chapter  Google Scholar 

  • Meyer, W., Stockmann, R., & Taube, L. (2020). The institutionalisation of evaluation theoretical background, analytical concept and methods. In R. Stockmann, W. Meyer, & L. Taube (Eds.), The Institutionalisation of evaluation in Europe (pp. 3–34). Palgrave Macmillan.

    Chapter  Google Scholar 

  • Mill, J. S. (1843). A system of logic (2 Vols). John W. Parker

    Google Scholar 

  • Mok, K. H., & Forrest, R. (Eds.). (2008). Changing governance and public policy in East Asia. Routledge.

    Google Scholar 

  • North, D. C. (1986). The new institutional economics. Journal of Institutional and Theoretical Economics (jite)/Zeitschrift für die gesamte Staatswissenschaft, 142(1), 230–237.

    Google Scholar 

  • North, D. C. (1987). Institutions, transaction costs and economic growth. Economic Inquiry, 25(3), 419–428.

    Article  Google Scholar 

  • North, D. C. (1991). Instiutions. Journal of Economic Perspectives, 5(1), 97–112.

    Article  Google Scholar 

  • Ojo, A., & Mellouli, S. (2018). Deploying governance networks for societal challenges. Government Information Quarterly, 35(4), 106–112.

    Article  Google Scholar 

  • Oliver-Lumerman, A., & Drori, G. S. (2021). From Ivory tower to academic commitment and leadership: The changing public mission of universities. Edward Elgar Publishing.

    Google Scholar 

  • Ostrom, E. (2009). The institutional analysis and development framework and the commons. Cornell Law Review, 95, 807.

    Google Scholar 

  • Ostrom, E. (2011). Background on the institutional analysis and development framework. Policy Studies Journal, 39(1), 7–27.

    Article  Google Scholar 

  • Parsons, T. (1939). The professions and social structure. Social Force, 17(4), 457–467.

    Article  Google Scholar 

  • Parsons, T. (1954). The present position and prospects of systematic theory in sociology. In T. Parsons (Ed.), Essays in sociological theory (pp. 212–237). Free Press.

    Google Scholar 

  • Parsons, T. (1971). Evolutionäre Universalien in der Gesellschaft. In W. Zapf (Eds.), Theorien des sozialen Wandels (pp. 55–74). Kiepenheuer & Witsch.

    Google Scholar 

  • Pattyn, V. R., & Brans, M. (2015). Organisational analytical capacity: Policy evaluation in Belgium. Policy and Society, 34(3–4), 183–196.

    Article  Google Scholar 

  • Payne, A., & Phillips, N. (Eds.). (2014). Handbook of the international political economy of governance. Edward Elgar Publishing.

    Google Scholar 

  • Peel, M., & Twomey, C. (2017). A history of Australia. Bloomsbury Publishing.

    Google Scholar 

  • Peters, B. G. (2019). Institutional theory in political science: The new institutionalism. Bloomsbury Publishing.

    Google Scholar 

  • Pierre, J., & Peters, B. G. (2021). Advanced introduction to governance. Edward Elgar Publishing.

    Google Scholar 

  • Pollack, D. (2016). Modernisierungstheorie – Revised: Entwurf einer Theorie moderner Gesellschaften. Zeitschrift Für Soziologie, 45(4), 219–240.

    Article  Google Scholar 

  • Popper, K. R. (1972). Objective knowledge. Clarendon press.

    Google Scholar 

  • Popper, K. R. (2011 [1945]). The Open Society and its Enemies (First published in two volumes in 1945, 5th Edition 1966). Published in 2011 in Routledge Classics. Routledge.

    Google Scholar 

  • Ragin, C. C. (1991). Introduction: The problem of balancing discourse on cases and variables in comparative social science. In C. C. Ragin (Ed.), Issues and alternatives in comparative social research (pp. 1–8). Brill.

    Google Scholar 

  • Ragin, C. C. (2006). How to lure analytic social science out of the doldrums: Some lessons from comparative research. International Sociology, 21(5), 633–646.

    Article  Google Scholar 

  • Ragin, C. C. (2007). Comparative methods. In W. Outhwaite, & S. P. Turner (Eds.), The Sage handbook of social science methodology (pp. 67–80). Sage.

    Chapter  Google Scholar 

  • Ragin, C. C. (2014). The comparative method: Moving beyond qualitative and quantitative strategies. University of California Press.

    Book  Google Scholar 

  • Ragin, C. C., & Sonnett, J. (2005). Between complexity and parsimony: Limited diversity, counterfactual cases, and comparative analysis. In S. Kropp, & M. Minkenberg (Eds.), Vergleichen in der Politikwissenschaft (pp. 180–197). VS Verlag für Sozialwissenschaften.

    Chapter  Google Scholar 

  • Reid, A. (2015). A history of Southeast Asia: Critical crossroads. Wiley.

    Google Scholar 

  • Richardson, A. J. (1997). Social closure in dynamic markets: The incomplete professional project in accountancy. Critical Perspectives on Accounting, 8(6), 635–653.

    Article  Google Scholar 

  • Robins, J. A. (1987). Organizational economics: Notes on the use of transaction-cost theory in the study of organizations. Administrative Science Quarterly, 32(1), 68–86.

    Article  Google Scholar 

  • Rosenbaum, A. (2006). Cooperative service delivery: The dynamics of public sector-private sector-civil society collaboration. International Review of Administrative Sciences, 72(1), 43–56.

    Article  Google Scholar 

  • Rosenstein, B. (2013). Mapping the status of national evaluation policies . Parliamentarians Forum on Development Evaluation in South Asia and EvalPartners. http://www.pfde.net/index.php/publications-resources/global-mapping-report-2015. Accessed on 14 October 2020.

  • Rosenstein, B. (2015). Status of national evaluation policies: Global mapping report (2nd ed.). Parliamentarians Forum on Development Evaluation in South Asia jointly with EvalPartners. https://globalparliamentarianforum.files.wordpress.com/2016/02/the-status-of-evaluation-policies.pdf. Accessed on 7 January 2020.

  • Rosenstein, B., & Kalugampitiya, A. (2021). Global mapping of the status of national evaluation policies (3rd ed.). Global Evaluation Initiative.

    Google Scholar 

  • Roy, R. F., & Wijayasuriya, P. M. W. (1992). A study of government monitoring and evaluation systems: The case of Sri Lanka. Central Evaluation Office, United Nations Development Program.

    Google Scholar 

  • Rüschemeyer, D. (1983). Professional autonomy and the social control of expertise. In R. Dingwall, & P. Lewis (Eds.), The sociology of the professions: Lawyers, doctors and others (pp. 38–58). Macmillan St. Martin’s Press.

    Google Scholar 

  • Samaratunge, R., Alam, Q., & Teicher, J. (2008). The new public management reforms in Asia: A comparison of South and Southeast Asian countries. International Review of Administrative Sciences, 74(1), 25–46.

    Article  Google Scholar 

  • Sandelowski, M., Voils, C. I., & Knafl, G. (2009). On quantitizing. Journal of Mixed Methods Research, 3(3), 208–222.

    Article  Google Scholar 

  • Schiele, B., Claessens, M., & Shi, S. (Eds.). (2012). Science communication in the world: Practices, theories and trends. Springer Science & Business Media.

    Google Scholar 

  • Schimank, U. (1988). Gesellschaftliche Teilsysteme als Akteurfiktionen. Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 40, 619–639.

    Google Scholar 

  • Schimank, U. (1996). Theorien gesellschaftlicher Differenzierung. Springer-Verlag.

    Book  Google Scholar 

  • Schimank, U. (2001). Teilsysteminterdependenzen und Inklusionsverhältnisse. Ein differenzierungstheoretisches Forschungsprogramm zur System- und Sozialintegration moderner Gesellschaft. In E. Barlösius, H. Müller, & S. Sigmund (Eds.), Gesellschaftsbilder im Umbruch. Soziologische Perspektiven in Deutschland (pp. 109–130). Springer-Verlag.

    Google Scholar 

  • Schimank, U. (2006). Teilsystemische Autonomie und politische Gesellschaftssteuerung – Beiträge zur akteurzentrierten Differenzierungstheorie. 2. Band. VS Verlag.

    Google Scholar 

  • Schimank, U. (2015). Modernity as a functionally differentiated capitalist society: A general theoretical model. European Journal of Social Theory, 18(4), 413–430.

    Article  Google Scholar 

  • Schimank, U. (2018). Capitalist society: A view from the theory of functional differentiation: Capitalism and labor: Towards critical perspectives. In K. Dörre et al. (Eds.), Capitalism and labor: Towards critical perspectives. International Labour Studies (Vol. 16, pp. 171–184). Campus Verlag.

    Google Scholar 

  • Schimank, U., & Volkmann, U. (2012). Economizing and marketization in a functionally differentiated capitalist society—A theoretical conceptualization. In U. Schimank & U. Volkmann (Eds.), The marketization of society: Economizing the non-economic. Welfare Societies Conference Paper (pp. 37–63). Research Cluster ‘Welfare Societies’.

    Google Scholar 

  • Schoenefeld, J. J., & Jordan, A. J. (2020). Environmental policy evaluation in the EU: Between learning, accountability, and political opportunities. In A. Zito, C. Burns, & A. Lenschow (Eds.), The future of European Union environmental politics and policy (pp. 179–198). Routledge.

    Chapter  Google Scholar 

  • Schofield, N., & Caballero, G. (Eds.). (2015). The political economy of governance: Institutions, political performance and elections. Springer.

    Google Scholar 

  • Schoonenboom, J., & Johnson, R. B. (2017). How to construct a mixed methods research design. Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 69(Suppl 2), 107–103.

    Article  Google Scholar 

  • Schubert, K., & Klein, M. (2016). Das Politiklexikon. Begriffe, Fakten, Zusammenhänge. Bundeszentrale für politische Bildung.

    Google Scholar 

  • Schwarz, H. P. (2003). Die neueste Zeitgeschichte. Vierteljahresheft für Zeitgeschichte, 51(1), 5–28.

    Google Scholar 

  • Shah, A. (Ed.). (2007). Performance accountability and combating corruption. World Bank Publications.

    Google Scholar 

  • Speer, S., Pattyn, V., & De Peuter, B. (2015). The growing role of evaluation in parliaments: Holding governments accountable? International Review of Administrative Sciences, 81(1), 37–57.

    Article  Google Scholar 

  • Stachura, M., Bienfait, A., Albert, G., & Sigmund, S. (Eds.). (2009). Der Sinn der Institutionen: Mehr-Ebenen-und Mehr-Seiten-Analyse. Springer-Verlag.

    Google Scholar 

  • Stern, E. (2009). Evaluation policy in the European Union and its institutions. New Directions for Evaluation, 123, 67–85.

    Article  Google Scholar 

  • Stichweh, R. (1995). Zur Theorie der Weltgesellschaft. Soziale Systeme, 1(1), 29–45.

    Google Scholar 

  • Stichweh, R. (2006). Strukturbildung in der Weltgesellschaft—Die Eigenstrukturen der Weltgesellschaft und die Regionalkulturen der Welt. In T. Schwinn (Ed.), Die Vielfalt und Einheit der Moderne: Kultur- und strukturvergleichende Analysen (pp. 239–257). SpringerLink.

    Chapter  Google Scholar 

  • Stichweh, R. (2008). Das Konzept der Weltgesellschaft: Genese und Strukturbildung eines globalen Gesellschaftssystems. Rechtstheorie, 39(2–3), 329–355.

    Article  Google Scholar 

  • Stockmann, R. (2008). Evaluation and quality development: Principles of impact-based quality management. Peter Lang.

    Google Scholar 

  • Stockmann, R. (2013). Evaluation in der Entwicklungszusammenarbeit. In J. Wilhelm, & H. Ihne (Eds.), Einführung in die Entwicklungspolitik (pp. 541–562, 3rd ed.). LIT Verlag.

    Google Scholar 

  • Stockmann, R. (2022). Einführung in die Evaluation. In R. Stockmann (Ed.), Handbuch zur Evaluation. Eine praktische Handlungsanleitung. Sozialwissenschaftliche Evaluationsforschung, Band 16. (2nd ed.) Waxmann.

    Google Scholar 

  • Stockmann, R. (2024). An introduction to evaluation. In R. Stockmann (Ed.), A practitioner handbook on evaluation. (2nd ed.) Edward Elgar.

    Google Scholar 

  • Stockmann, R., & Meyer, W. (2014). Evaluation. Eine Einführung (2nd edition). Waxmann.

    Google Scholar 

  • Stockmann, R., & Meyer, W. (Eds.). (2016). The future of evaluation: Global trends, new challenges, shared perspectives. Palgrave Macmillan.

    Google Scholar 

  • Stockmann, R., Meyer, W., & Szentmarjay, L. (Eds.). (2022). The institutionalisation of evaluation in the Americas. Springer Nature.

    Google Scholar 

  • Stockmann, R., Meyer, W., & Taube, L. (Eds.). (2020). Institutionalisation of evaluation in Europe. Palgrave Macmillan.

    Google Scholar 

  • Tobin, I. (2003). Bureaucratic power and the NPM reforms in Korea. International Review of Public Administration, 8(1), 89–102.

    Article  Google Scholar 

  • Tocqueville, A. (2003). Democracy in America and two essays on America. Penguin.

    Google Scholar 

  • Traugott, M. (2013). Emile Durkheim on institutional analysis. Chicago University Press.

    Google Scholar 

  • Trede, F., Macklin, R., & Bridges, D. (2012). Professional identity development: A review of the higher education literature. Studies in Higher Education, 37(3), 365–384.

    Article  Google Scholar 

  • UNDP (2015). Towards a baseline study: Insights on national evaluation capacities in 43 countries. Independent Evaluation Office and the International Policy Centre for Inclusive Growth. United Nations Development Programme.

    Google Scholar 

  • Van Voorst, S., Zwaan, P., & Schoenefeld, J. J. (2022). Policy evaluation: An evolving and expanding feature of EU governance. In P. R. Graziano, & J. Tosun (Eds.), Elgar encyclopedia of European Union Public Policy (pp. 595–603). Edward Elgar Publishing.

    Chapter  Google Scholar 

  • Varone, F., Bundi, P., & Gava, R. (2020). Policy evaluation in parliament: Interest groups as catalysts. International Review of Administrative Sciences, 86(1), 98–114.

    Article  Google Scholar 

  • Vedung, E. (2010). Four waves of evaluation diffusion. Evaluation, 16(3), 263–277.

    Article  Google Scholar 

  • Vogl, S. (2017). Quantifizierung. KZfSS Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 69(Suppl 2), 287–312.

    Article  Google Scholar 

  • Weingart, P. (1997). From “Finalization” to “Mode 2”: Old wine in new bottles? Social Science Information, 36(4), 591–613.

    Article  Google Scholar 

  • Weng, B. S. (2010). A short history of Taiwan’s democracy movement. In B. Bridges, & L. Sang Ho (Eds.), Public governance in Asia and the limits of electoral democracy (pp. 115–146). Edward Elgar Publishing.

    Google Scholar 

  • Wollmann, H. (2017). Policy evaluation and evaluation research. In F. Fischer, & G. J. Miller (Eds.), Handbook of public policy analysis (pp. 419–428). Routledge.

    Google Scholar 

  • Wong, J. (2014). Comparing beyond Europe and North America. In I. Engeli, & A. C. Rothmayr (Eds.), Comparative policy studies: Conceptual and methodological challenges (pp. 163–184). Palgrave Macmillan.

    Google Scholar 

  • Zapf, W. (Ed.). (1991). Die Modernisierung moderner Gesellschaften: Verhandlungen des 25. Deutschen Soziologentages in Frankfurt am Main 1990 (Vol. 25). Campus Verlag.

    Google Scholar 

  • Zapf, W. (1996). Die Modernisierungstheorie und unterschiedliche Pfade der gesellschaftlichen Entwicklung. Leviathan, 24(1), 63–77.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Niklas Zierke .

Editor information

Editors and Affiliations

Appendices

Appendix 1: Glossary for the Evaluation Globe—Compendium on the Institutionalisation of Evaluation

  1. 1.

    Evaluation: “Evaluation is the systematic investigation of the merit or worth of an object (program) for the purpose of reducing uncertainty in decision making” (Mertens, 1998)

  2. 2.

    Monitoring: “The systematic documentation of aspects of project performance (process, output, outcomes, impacts, results) that are indicative of whether the project is functioning as intended or according to some appropriate standard” (Rossi/Freeman/ Lipsey 1999).

  3. 3.

    Performance Audit: Objective and systematic examination of effectivity (goal-achievement) and efficiency (economic use of resources), based on external criteria.

  4. 4.

    Meta-Evaluation/Analyses: Meta-Evaluation/Analyses reanalyse evaluation (reports) with regard to specific analytical questions. This could be the content of the different evaluations as well as the methodological approaches or quality aspects. Another goal of meta-evaluations/analyses could be to aggregate results.

  5. 5.

    Comment: We assume that different countries or even sectors have different ways in dealing with this issue. Due to the fact that we would not like to exclude any form of meta-analysis we would also subsume evaluation synthesis reports, meta-evaluations or other forms under this term.

  6. 6.

    Internal & External Evaluation: External Evaluation describes evaluations carried out by persons (experts) who are not part of the implementing or funding organisation. Internal Evaluation describes evaluations carried out by the same organisation that is implementing the programme. Self-evaluation is a specific form of an internal evaluation, in which case the programme implementing persons are the same as the programme evaluators (cf. Stockmann & Meyer, 2014)

  7. 7.

    Impact vs. Process Evaluation: Process Evaluations look at the stage of planning and implementing activities, basically their inputs and outputs whereas impact evaluations focus on the actual results and impact of programme measures. Impact refers to changes and reasons for change, thereby investigating expected as well as unexpected impacts.

  8. 8.

    Functions of Evaluation: According to Reinhard Stockmann and Wolfgang Meyer evaluation has three societal functions:

    1. i.

      Planning and Steering: The first one encompasses use of evaluation for management purposes. The instrument of evaluation is used to improve effectivity, efficiency and impact of measures and interventions.

    2. ii.

      Accountability and Legitimacy: In the understanding of the second function, evaluation serves for the purpose of good governance. The implementation of politically intended strategies is controlled and public accountability is provided through use of evaluations.

    3. iii.

      Enlightenment: For the purpose of enlightenment, evaluations can provide necessary facts to enable public dialogue. Relevant questions for this dialogue are, whether administration (where implementation and steering of programmes and measures takes place) and government & parliament (where political steering takes place) follow society’s values and goals, or not.

  9. 9.

    Institutionalisation: Institutionalisation indicates emergence and permanent anchoring of formal and informal rules, structures and processes that guide and stabilise behavior of individuals as well as groups, thereby regulating living together in society.

  10. 10.

    Evaluation Culture: Due to the fact that the term of culture is rather vague we kindly ask every author to work with the concept of “institutionalisation” as defined above and furthermore described within our research proposal.

  11. 11.

    Evaluation Regulations

    1. I.

      National and sectoral laws and regulations: Regulation in our case refers to legal norms that are established by state’s executive instead of legislative. They are issued for implementing or complementing existing laws and are generally binding (cf. Schubert & Klein, 2016).

    2. II.

      National and sectoral strategies or policies: The term “policy” can be defined as coherent strategy for action focusing on a specific thematic area. The extent of the structure can be varying from the description of concrete next steps to a very low level of structure. All policies (strategies) have in common that they postulate desirable goals (cf. Bussmann et al., 1997, p. 66f., 83).

    3. III.

      Administrative regulations (instructions, guidelines): This term encompasses regulations addressed at administrative authorities.

  12. 12.

    Independent evaluation institute: Institute, which is supposed to examine politics either of one specific sector or all national politics. Thereby the institute can decide completely independent on which thematic area it wishes to conduct an evaluation. A state’s Federal Audit Office can be such an institute, if it has an evaluation function.

  13. 13.

    Independent evaluation units: Ministerial departments that are provided with financial and personal resources and have got the freedom to carry out evaluations independent of the operational units.

  14. 14.

    Quality of Evaluations/Evaluators: It can be differentiated between mechanisms that are supposed to foster the quality of evaluations, such as standards, and those that foster the quality of evaluators, such as Codes of Ethics, Gentlemen’s Agreements or Guiding Principles for Evaluators.

  15. 15.

    Civil Society: The concept of civil society encompasses an area of self-organisation and self-administration by citizens, namely in form of associations organizations. For the book’s purpose we understand civil society as organized in form of civil society organisations as well as non-governmental organisations, which we do not differentiate any further.

  16. 16.

    Sector: Sector in this case refers to different policy-fields, such as the health sector, the education sector, the sector of development cooperation. From a system theoretical perspective, they can be seen as subsystems of the political or social system.

Appendix 2: Analytical Framwork: Compendium on the Institutionalisation of Evaluation

I. Political System: Institutional Structures and Processes

I.1 Evaluation regulations

I.2 Evaluation practice

I.3 Use of evaluations

• Are there national laws or regulations about evaluation or use of evaluation? If yes, which?

• Are there sectoral laws or regulations about evaluation or use of evaluation (e.g. a law about school evaluation or evaluation in the higher education system as example for laws in the educational sector)? If yes, which?

• Are there policies or strategies about evaluation or use of evaluation, either national or sectoral? If yes, which?

• Are there administrative regulations about evaluation or use of evaluation in different policy fields (instructions, guidelines, etc.)? If yes, which?

• What is the content of these laws/regulations/policies/strategies or administrative regulations regarding independence of evaluation, quality, impact orientation and available budget?

– Is use of evaluation specified? If yes, how?

– How binding are specifications regarding use of evaluation?

– What are aspired functions of evaluation (e.g. Planning and Steering, Accountability and Legitimacy, Enlightenment)?

• Is evaluation and use of evaluation findings embedded in parliamentary structures? If yes, how?

– Do parliamentarians in your country deal with evaluation findings for their own political work? If yes, to what extent (how often/how detailed do they use evaluation findings)?

– Do parliamentarians in your country demand evaluations for their own political work? If yes, to what extent? (How often? Do they commission evaluations? Do they publicly demand evaluations)?

• With regard to the whole country: How would you describe the scope of conducted evaluations? Is it possible to speak of a frequent rhythm of evaluations, for instance for every new legislation or for every national programme? Or is it rather non-specific? Does evaluation take place in all sectors/policy fields of a country (instead of only in the field of development cooperation for example)? And within one sector, is evaluation applied for measures funded in different ways or maybe only the ones that received funding by the European Union?

• With regard to the whole country: How would you describe the relation between internal and external evaluations? Which form is carried out more often and for what purposes?

• What are possible reasons for this (e.g. determination in laws, policies or regulations)?

• Is this relation differing with regard to sector or state level?

• With regard to the whole country: How would you describe the relation between process and impact/outcome evaluations? Which form is used more often and for what purposes?

– What are possible reasons for this (e.g. determination in laws, policies or regulations)?

– Is this relation differing with regard to sector or state level?

• Does an independent evaluation institute exist in your country?

– With a national responsibility?

– With a responsibility for a specific sector or policy field?

• Do independent internal departments exist, in ministries or elsewhere?

• Which sectors are “good performers” regarding use of evaluation and evaluation findings? Please describe up to 3 sectors that can be considered as leading in the field of evaluation’s use

• Which sectors are “bad performers” regarding use of evaluation and evaluation findings? Please describe up to 3 sectors that are lagging behind in the field of evaluation’s use

• On which aspect do most evaluations focus in these sectors (e.g. Planning and Steering, Accountability and Legitimacy, Enlightenment)?

– In the case of different findings in different sectors: What might be possible reasons for these differences?

• Which professional groups use evaluation and evaluation findings regularly (e.g. political decision makers, programme or project manager, administrative staff)?

– For what reasons are evaluations and/or evaluation findings used by these groups?

– How is the use of evaluation findings guaranteed (for instance: management response mechanisms, implementation of monitoring for evaluation results, others)?

– Are there differences with regard to different sectors?

• How is the quality of evaluations guaranteed (e.g. regular conduction of meta-evaluations analyses, competence requirements for evaluators, quality requirements for evaluations)?

II Social System: Dissemination and Acceptance of Evaluation in Society

II.1 Institutionalised use of evaluations by civil society

II.2 Public perception and discussion of evaluation and evaluation findings

II.3 Civil societies demand evaluations

• Is it usual practice in your country that evaluations are used to provide knowledge for referenda or political decision making on a communal basis?

– If yes, how regularly does this happen? If not, what might be possible hindering factors?

• Are evaluations and evaluation findings used by individual citizens/civil society organisations and or private enterprises or other actors?

– If yes, for what reasons (e.g. enforcement of their interests, knowledge or proof for work related issues, knowledge or proof for voluntary activities etc.) and how regularly? If not, what might be possible hindering factors?

• Is it usual practice in your country that citizens or civil society organisations (NGOs, CSOs, churches etc.) are participating in evaluations (as stakeholder)?

– If yes, how regularly does this happen? What are different forms of participation (e.g. as interview partners, as clients, as users of evaluation findings etc.)? If not, what might be possible hindering factors?

• How well-known is the instrument of evaluation in society?

• Are evaluation reports (full version) made publicly available?

• Is the general use of evaluation publicly discussed in media (benefits of evaluation, quality of evaluations, and professionalisation of evaluation)?

– If yes, to what extent? If not, what might be possible hindering factors?

• Are findings of actual evaluations publicly discussed (surprising findings, different possibilities of dealing with these findings)?

– If yes, to what extent? If not, what might be possible hindering factors?

• Do individual citizens, civil society organisations, private enterprises or other actors in your country demand evaluations, e.g. from political decision-makers?

– If yes, how often does this happen and under which circumstances/for what reasons? If not, why not? What might be possible hindering factors?

III. System of Professions: Evaluation as a Discipline

III.1 Academic study courses, further training et cetera

III.2 Profession/discipline

III. 3 Compliance to standards and quality obligations

• Do programmes of higher university education for evaluators (Diploma, Master) exist in your country? If yes, how many and where?

• In which other scientific disciplines is evaluation instructed as scientific subject? Please give as many examples as possible.

• Do other forms of academic or non-academic training exist? (e.g. e-learning, training by consultancies, else)?

• Which professional journals, newsletters or other ways/media of communication (e.g. e-Mail or discussion lists) exist?

• Which professional journals from other scientific disciplines deal with evaluation regularly?

• Does a professional organisation (VOPE—Volunteer Organisations for Professional Evaluation) exist in your country?

– How is it organised (closed associations or open network)?

– What is their actual number of members?

• Do standards, guiding principles for evaluators or s.th. similar exist in your country?

– Developed by the VOPE?

– Adopted from another VOPE?

• Would you say that the evaluation market in your country is mostly dominated by freelancers (people calling themselves evaluators), consulting firms or scientific research institutes?

• Does a certification system for evaluators exist in your country?

• Does an authority, which might be asked to conciliate in case of a dispute, arbitration board exist in your country, like an arbitration board or ombudsman?

• Does a professorship for evaluation exist in your country?

• Do professional organisations ask their members to follow standards or guiding principles? If yes, how obligatory is this?

• Do clients demand a certain evaluation quality and/or compliance to standards? How does this demand look like (is it obligatory)?

• To what extent do evaluators (and clients) follow these standards and/or quality obligations?

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zierke, N., Stockmann, R., Meyer, W. (2023). The Institutionalisation of Evaluation: Theoretical Background, Analytical Framework and Methodology. In: Stockmann, R., Meyer, W., Zierke, N. (eds) The Institutionalisation of Evaluation in Asia-Pacific. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-031-36918-6_1

Download citation

Publish with us

Policies and ethics