Keywords

There is wide consensus in public policy today that globalization urges countries and their state authorities to bring scientific truth into policymaking processes. However, many scholars and policy analysts have also addressed ambiguity and complexity in regard to how scientific knowledge is used in policymaking processes (Cairney, 2016; Smith et al., 2020; Steiner-Khamsi et al., 2020; Weingart & Guenther, 2016). Even though evidence is highly integral in terms of advanced, specialized knowledge, few scholars expect science and politics to serve the same roles and rationales. Moreover, the status and role of empirical evidence can be different between policy projects depending on the realms and knowledge these projects address, as well as their political contexts. Therefore, researchers have long expressed great interest in investigating how experts pursue their responsibilities and how they provide evidence in education policy (Heggen et al., 2010; Krejsler, 2013; Steiner-Khamsi, 2013).

In order to gain deeper insights into education policy and the way policymakers and experts deploy knowledge to legitimize school reforms, the authors in this edited volume examine citation patterns in corpuses of governmental papers written under the auspices of state authorities in the Nordic region of Europe. All empirical chapters are end results from the research project Policy Knowledge and Lesson Drawing in Nordic School Reform in an Era of International Comparisons (POLNET), funded by the Norwegian Research Council (NRC 283467). In this project, five Nordic research teams and a research team at Teacher College, Columbia University, New York, developed a shared database by systematizing references in policy documents. By collecting and analyzing bibliographic meta-data extracted from sources referenced in policy documents, the authors examine how these sources are tied together into networks of references in and across reform-making processes in five Nordic countries (Denmark, Iceland, Finland, Norway, and Sweden). Such sources include governmental documents and other types of work published by non-academic authors alongside research-based evidence produced at research institutions. In addition, we have interviewed key experts who have experience from public inquiry bodies and ministry or agency officials with a particular responsibility for assisting leaders and members of the inquiry bodies. Some interviewees had also experience with referencing the inquiries in their work of authoring white papers for the government.

In the POLNET study, our main interest is to examine how scientific knowledge and research-based evidence become affiliated with other knowledge sources in reference networks that legitimize policymaking processes. Our findings and interpretations result from a systematic analysis of white and green papers. In the Nordic contexts, white papers are documents produced by governments to legitimize political recommendations that are reviewed and debated in parliamentary processes. These papers may have a profound impact on how discussions proceed in parliamentary processes and how the bureaucracy and institutions within various sectors, such as education, eventually handle political problems. Due to the complexity of these problems as well as institutional demands, both politicians and governments are in a need of catalyzing changes and surveying people and practices to make sure that they adopt solutions that benefit society. Therefore, these policymakers also ask for public inquiry reports, or what we in the volume call green papers. These green papers are most often written before a white paper is produced to evaluate issues and clarify the pros and cons of various options and solutions.

In some countries, public inquiries are conducted by one senior officer within a ministry or by an external expert formally mandated by the ministry. This person reviews documents, collects data, and draws on various types of expertise to write a report about a particular topic. In the Nordic region of Europe, such inquiries are also made by advisory panels consisting of stakeholders, representatives from the sector, and/or experts who have the mandate to independently author a green paper to make recommendations for how to reform or renew policies in a particular area. These inquiries are organized to enlighten the work of policymakers and politicians and to inform the public, since they target political, social, cultural, and economic conditions of relevance for large parts of the population in a country. Besides these inquires, ministries may ask experts to review a particular knowledge field and write a report without a government-issued mandate. In any case, debates based on these inquiries and expert reports can have various consequences. Formally, they may lead to political decisions within the parliament, which has the formal duty to represent the interests of the citizens in the country by making laws, legitimizing reforms, and overseeing the work of the government in terms of hearings, inquiries, and evaluations. Informally, discussions upon these reports can engage stakeholders and others in dialogues that make impact on education policy.

A core aim of our analysis has been to discover differences between reference patterns in the countries’ white and green papers for the latest school reform related to compulsory education (grade 1–9/10), and to explore the role of knowledge usage in the policy processes that resulted in these papers. By drawing on Paul Cairney’s (2016, p. 3) definition, “‘evidence’ is assertion backed by information” (p. 3), we regard the knowledge sources that are referenced as the actual evidence. Nutley et al. (2019) have provided an overview of evidence-promoting organizations and their ways of identifying, labeling, and ranking “good evidence,” on which the organizations have not reached any consensus yet. Thereby, we consider the relative merits of various types of evidence as an open question for academic institutions and public branches, such as ministries and agencies, to have an ongoing debate about.

A common view is that evidence-based policies draw on knowledge from outside of public policies, including from universities and research-based institutes where dedicated researchers produce scientific knowledge of relevance for policymakers. However, evidence and expertise are not only produced at universities and research institutes but also result from various activities and innovations across a variety of branches. These activities are arranged through networks of academic and non-academic partners that consist of public ministries, state agencies, non-governmental organizations, and even grassroots initiatives (Nelson & Campbell, 2019). Various types of meetings, projects, and networks bring together partners in joint efforts to advance expertise that enlightens and legitimizes political actions and solutions. Researchers have offered good reasons to think of policy-relevant knowledge as developed across traditional division lines between science and politics (Godin & Shauz, 2016). In this context, policymakers and experts are expected to assess the evidence to determine “whether the source providing us with information is trustworthy” (Eyal, 2019, p. 34) and relevant. Although evidence is expected to be scientific in a narrow sense, they may anyhow apply a broad concept of knowledge, including information, ideas, and arguments; well-tested beliefs; and lay, professional, and academic knowledge (Radaelli, 1995). Against this backdrop, relevant knowledge in public policy, authorized as a trustworthy source, can be broadly defined to include both academic work developed by scholars at universities or research institutes and knowledge that is regarded as less scientific because it is produced in non-academic contexts. Consequently, evidence can be based on or affiliated with various knowledge sources depending on where and how policymakers and experts interact as well as how institutional reputation and legitimization serve as the modus operandi for the way policy is governed (Ball & Junemann, 2012). A key point in Steiner-Khamsi’s (2013) work is that the production of knowledge, especially the design of comparative studies, optimizes evidence for executing at least three options of different types of knowledge use. That is, references can be used either (a) as evidence that informs policy planning within particular contexts, (b) as normative guidelines for how to change educational processes concerning global problems, or (c) as projecting best practices that are evaluated against a set of international performance standards. There are good reasons for regarding the production of knowledge, especially the design of comparative research projects, as optimizing evidence for executing one or more of the three alternatives. The chapters that follow elaborate on these and related purposes by drawing on various theories as well as empirical data collected within the Nordic research project.

Topics and Perspectives

The POLNET study pays special attention to policy processes, in particular to the nexus of national, regional, and international policy brokers and their knowledge provision and usage (Steiner-Khamsi, 2013). The combination of policy learning and borrowing (or reception) and the sociology of policy transfer serves as an analytical lens to study these topics. Drawing on globalization studies (Verger et al., 2018) and research on policy borrowing and lending (see, for instance, Steiner-Khamsi, 2012; Waldow, 2012), the book’s overall purpose is to enlighten a broad discussion on what counts as evidence for policymakers and which roles various types of expertise play in policymaking processes. Following chapters address one or several of the following research questions:

  • How do policymakers and experts in the Nordic countries (Denmark, Finland, Iceland, Norway, and Sweden) draw on domestic, regional, and international knowledge in their papers on school reform policy?

  • How do they legitimize national school reform policy by referencing various types of knowledge sources?

  • How do they authorize evidence and expertise in their attempt to propose reform agendas, develop new or modified policy options, or issue new or revised school reforms in their respective countries?

Our primary concern has been to explore the significance of evidence as a lever in policy formulation processes, notably by the prominence and role of single or individual references in terms of citations to specific articles, book chapters, books, reports, and other documents. When source documents refer to clusters of references that share particular features, we can speak of spatial reference types. A central question that begs analysis is how policy processes provide reference patterns of both direct and spatial types within collections of governmental documents in areas of interest.

It is our assumption that both single references and spatial reference patterns can project systems as well as outcomes. They can, for example, locate knowledge in different geographical entities (e.g., locally, nationally, globally) or refer to particular types of knowledge (e.g., books, journal articles, governmental papers, reports) or to particular realms that are characterized by a set of thematic areas of interest. They can also project preferences for and prominence of certain types of policy analysis, such as systematic reviews, evaluations, sector analyses, and comparative studies in terms of international large-scale assessments that project best practices in terms of evidence-based standards. By conducting comparative network analysis, there are many possibilities to identify both individual and spatial reference types that reveal insights into the way policymakers and experts co-construct knowledge that legitimize education policy.

Another feature we have investigated relates to semantic patterns that can be extracted from content analysis of documents. The use of references and spatial patterns that evolve from policy processes results in thematic areas or policy realms that, in our case, relate to school reforms in the Nordic countries. These capture legislative reforms, curriculum and assessment policy, and policies addressing the development of literacy among children and adolescents. Therefore, we also look into the policy texts by describing the content of reforms (see Chaps. 4, 5, 6, 7, and 8) and by unraveling core themes, decision strategies, and conceptual schemes that can be more or less outcome-oriented (see Chap. 7 in this volume). Eventually, we examine key narratives to discern how policymakers and experts make sense of evidence provided by public inquiry bodies and other branches (see Chap. 12).

The following three bodies of research have been important for designing sub-studies in the POLNET project that are included in this edited volume: (a) research on network governance; (b) studies of “travelling reform” (i.e., diffusion vs. reception studies); and (c) research on evidence-based policy. In the next section, we will briefly present each of these bodies.

Research on Network Governance

First, this book draws on research about the role network governance plays in education policies and how changing conditions for state regulation can help with interpreting variations in reference patterns. Several researchers have pinpointed the recent transformation of network governance that extends new public management policies introduced in most European countries. Helgøy et al. (2007) have described the contemporary changes in education policy as a re-regulation of the society by ways networks of actors make use of new governance tools. In the era of comparative education analysis, Ball and Junemann (2012) have contributed to a renewed understanding of how policy networks are changing education policy by including philanthropy and business partners in public policymaking. Combining social network analysis and ethnographic research, the authors unraveled how new communities of social actors, such as think tanks and interest organizations, influence public policy by actively engaging in conversations beyond institutional procedures. By referencing political scientists (Bevir & Rhodes, 2003), Ball and Junemann (2012) argued that there is a shift from government to governance, or from a unitary state where government is “carried out through hierarchies or specifically within administrations and by bureaucratic methods” to governance, which is “accomplished through the ‘informal authority’ of diverse and flexible networks” (p. 168). Moreover, these networks move beyond the methods of new public management by being configured in terms of tightly connected policy communities or more loosely connected issue networks. Such networks arise from the exchange of resources, such as money, information, or expertise that enable policymakers to accomplish their goals.

Our book contributes new insights about the development, use, and reception of expertise in public policy. Building upon Ball and Junemann’s (2012) theorizing about network governance, we extend the idea of loosely coupled issue networks. By looking for spatial reference patterns in policy documents, we explore how network governance underpins agenda setting and policy formulation processes in reform areas such as curriculum and assessment policies. Certainly, both of these areas are emphasized in Nordic school policy today and have roots in reform trajectories, developed by professional communities from the early twentieth century and onward. Throughout the 1900s, both intellectuals and reformists developed knowledge and expertise about assessment that resulted in progressive reform movements in the Nordic countries (Ydesen et al., 2013). More recently, international comparisons have been conducted in all Nordic countries (Sivesind, 2019) as a key monitoring tool by the national governments to make policy decisions and assess the quality of teachers, schools, districts, and the education system itself (Prøitz et al., 2017; Camphuijsen et al., 2020; Skedsmo et al., 2020). These comparisons enable international organizations to serve as policy actors and knowledge providers, which have implications for how national assessment systems are designed (Tveit & Lundahl, 2018) and how curriculum policy is formulated and enacted under the auspices of national governments (Mølstad & Karseth, 2016; Nordin & Sundberg, 2016; Sivesind & Wahlström, 2016).

However, countries differ in the expansion of non-state actors in their educational policy processes. Thompson (2003) explained this difference by noting that networks result from interactions between various components, such as ideas or concepts, people, institutions, social practices, or bodies of knowledge, and thereby constitute a form of “assemblage.” Moreover, networks are also characterized by a feedback loop mechanism that transforms policies by their own logics that more often than not result in configurations that connect hierarchies with the market (p. 363). While researchers in both political and educational sciences first assumed that network governance was a reform strategy that changed the way governments could govern public services, several scholars have highlighted the fact that this strategy does not necessarily transform democracies in all that matters. As a result, network governance does not necessarily change the hierarchies of bureaucracies into a totally new shape. Based upon this discovery, researchers have begun to consider networks as being characterized by hierarchies in some form. To what degree bureaucracies are shaped by network governance is thereby an empirical question.

This notion led to new insights about the role of the state in public administration and the idea of hybrid forms of governance, which is a highly contested concept as well. Renowned scholars in the field of education and political science have questioned whether public policy has transformed into a post-bureaucratic mode of governance (Maroy, 2012). This notion motivated Philippopoulos-Mihalopoulos (2012, p. 12) to write, “There is no such thing as a hybrid (yet)” (p. 12), meaning that there can be hybrid governance modes present in public policies, but not necessarily everywhere. The same author argued that the appellation hybrid more often than not refers to pre-existing connections with “a certain otherness” (p. 12). Thus, formal bureaucracies that are hierarchically organized and governed by formal and substantial rationales in the Weberian way can still survive within a network-based society; however, they are observed differently within the context of post-bureaucratic governance (Hall & Sivesind, 2015). For example, Theisens et al. (2016, p. 467) argued from the perspective of New Public Governance that the state still plays a dominant role in the political context where public sector organizations operate as knowledge providers. This is certainly true for the Nordic countries, which in many ways moved beyond the era of new public management during the 1980s (Christensen & Lægreid, 2007).

Following the same argument, Greve et al. (2016, p. 192) argued that reform policy today seems to emphasize state-centered solutions while acknowledging the mix of governance mechanisms and institutional complexity that characterizes the public sector. They concluded that state bureaucracies are still operative institutions that govern reform processes in the Nordic countries, but they are “less driven by politicians, more planned, and less contested by the unions, and there is more public involvement” (p. 201). Consequently, public policy is still observed as shaped by hierarchical structures in terms of government; however, it is observed differently than before, due to evolving networks that are re-ordering the environment of bureaucracies. This situation has generated renewed interest in collaborative governance. In this volume, authors argue that there are many more reform actors on the international stage who pursue their own strategies and reform activities besides national governments; furthermore, this situation does actually lead to deregulation in particular policy realms “as a process of removing or reducing state regulations” (Dovemark et al., 2018, p. 123). Against this backdrop, there are good reasons for looking at the orchestration of global governance and at international organizations as intermediary bodies that influence national governments (Abbott et al., 2015).

Studies of Traveling Reform: Diffusion Versus Reception Studies

Second, in part because of new relations and connections between local and global venues for policymaking, authors in this volume draw on academic research about the increased reliance on externalization in education policy. This theory is based on a sociological theory about how systems, such as education, politics, and science, interplay within a world characterized by increased complexity. The theory pinpoints the importance of researching how education policy is legitimized (Steiner & Waldow, 2012), and therefore we are examining how policymakers reference international and regional sources and/or cross-sectorial works to justify reforms in their Nordic countries. This dimension, which focuses on the legitimization of reforms, helps us to explore how policy processes change character through policy transfer (i.e., traveling reforms) and, in particular, if and how policymakers and experts borrow solutions from abroad or draw lessons about how to reform education in specific contexts (Steiner-Khamsi, 2012).

A group of institutional researchers studying traveling policy looks at the diffusion and dissemination of international standards or global education regulations as the key mechanisms for researching the transfer of policy knowledge. Their views enable them to understand why certain policies are expanded and promoted while others are not. Obviously, their views allow us to understand the active role of international organizations in lending or disseminating specific policies and programs, which are referred to as “best practices” or “international standards” (Bromley & Meyer, 2015; Krücken & Drori, 2009). However, this “from above” view, which is often associated with the theory of world culture, provides only one of several perspectives for understanding the dissemination of global education policy. Another angle clarifies how, why, and when national or local policy actors selectively learn from global education policies based on local conditions (Baek et al., 2018). When borrowing global education policies, such as competency-based curriculum reforms, accountability reforms, or public–private partnership policies at the local or national level, this “bottom-up perspective” focuses on the process of both reception and translation (Steiner-Khamsi, 2015).

In the past few years, more studies have emerged that have advanced the “look from below” view in important ways. The Scandinavian institutionalist approach belongs to this tradition, which differs from other institutional research traditions by being concerned with non-strategic approaches and involving researchers who proclaim that ideas and practices undergo profound changes when deployed in new organizational settings (Czarniawska & Sevón, 1996; Røvik, 1996; Sahlin-Andersson, 1996). In this theory, stories, images, narratives, and master ideas serve as lenses to think of policy transfer that is shaped globally without necessarily attracting everybody’s attention or direct involvement across the globe.

There is also a long tradition in comparative education of studying the reasons for “transnational policy attraction” within local and regional contexts, that is, to study why the government borrowed, imitated, or transferred others’ policies (Phillips, 2004). Similarly, researchers have studied the role of public–private partnerships and privatization in such contexts to examine the question: Why does global education policy resonate in a specific context; in other words, why do policy participants “buy” the policy? Verger (2014) has offered an explanatory framework to understand how reform plans are “sold” to the governments of low-income countries (Verger et al., 2016). In this case, an economic theory helps to explain why projects and plans are bought by non-affluent countries. Recent studies have also asserted that global education policies resonate for different reasons in different contexts, however, not necessarily due to the logic of quasi-markets (Maroy & Pons, 2019). The idea of evaluative state and international organizations as standard-setters makes sense for understanding why international large-scale assessments (ILSAs), such as the Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS), resonate differently in different countries (Addey et al., 2017; Martens et al., 2010; Pizmony-Levy, 2017; Ydesen, 2019). Nonetheless, the reception and use of ILSAs are most often dependent on how national policies project their own goals and conceptions of best practices (Waldow & Steiner-Khamsi, 2019).

Verger (2014) emphasized that policy ideas need to be placed into a context for understanding how global policies get transformed once they have penetrated a local arena. Political institutions, administrative and regulatory viability, government ideology, domestic political contention, and periods of crisis create a context for strategic selectivity regarding the reception of policy ideas. Maroy and Pons (2019) demonstrated the importance of context in understanding how transnational models of accountability are recontextualized and translated into national policies in various ways. Furthermore, Karseth and Solbrekke (2010) analyzed the Bologna policy and the translation of the European Qualifications Framework in terms of how national qualification frameworks have been developed in different countries. All the powerful institutions within these countries examined, emphasized the importance of learning outcomes as a reference point; however, they interpreted the Bologna policy differently, as they created various windows of opportunities for reform and change within their respective contexts.

In European countries and elsewhere, policy knowledge has initiated changes in various directions. On the national level, civil society organizations, companies, and local associations have expanded their influence in public education. On the transnational level, international organizations have evaluated, compared, and ranked education systems, producing various diffusion mechanisms that have made non-state actors into standard-setters (Peters et al., 2009) and by advocating what Steiner-Khamsi (2013) characterizes as “best practices.” Moreover, scientific communication itself, mediated by international journals and their media strategies, legitimizes reforms in the education sector (Mølstad et al., 2017). Although non-state-actor organizations have no formal mandate to govern education reform, they still execute informal power through new “soft” but effective knowledge products within policy networks that are often characterized by a particular language, where measurements in terms of calculations and numbers serve as the core instrument (Grek, 2008). As a result of new technology for governing education, policy processes have come to be configured by the production and use of knowledge (Abbott et al., 2015; Brøgger, 2018; Fenwick et al., 2014; Littoz-Monnet, 2017; Peters et al., 2009).

One approach to grasping “travelling reforms” and the use of references to foreign educational systems is, as already mentioned, the externalization thesis. According to Schriewer (1988), references to external systems (i.e., externalization) are mobilized to add meaning, weight, or legitimacy to domestic reforms. Renowned scholars have used the systematic theoretical concept of externalization to explain the selective borrowing of global education policies, the popularization of international best practices, and the transfer of reforms from one country to another (Schriewer & Martinez, 2004; Steiner-Khamsi, 2003, 2009, 2012, 2021; Waldow, 2012). Likewise, Nordic researchers have contributed to research on policy transfer in the field of education policy and applied semantic analysis that draws on sociological system theory (Luhmann, 1990, 1997). They show how national public inquiry reports create their own projections of the future school as a contemporary way to manage co-existing expectations (Hansen et al., 2021) and how national curricula reconfigure the scientific logics of comparative assessment designs that characterize transnational policies (Sivesind et al., 2016).

Research on Evidence-Based Policy

Third, we discuss how evidence-based policy is anchored in particular forms of knowledge that legitimize school reforms, education policies, and practices. As a field in political science, the politics of knowledge focuses on the legitimacy of knowledge forms and institutional logics that are shaping, for example, the evaluation–knowledge nexus (Segerholm et al., 2019). Simultaneously, theories of the policy process more generally focus on various features of evidence-based policies and practices. Together, these fields reflect a rather weak interest in knowledge use (Daviter, 2015). Moreover, several studies have examined what can be used as evidence, usually as the foundation of knowledge-based policies and regulations. Kvernbekk (2011) recommended a deep look into the question of the nature of evidence and how it is viewed and conceptualized in academic scholarships; however, both previous and recent literature has acknowledged the use of multiple types of evidence in the policy process beyond what is considered validated by scientific procedures (Boaz et al., 2019; Weiss, 1979).

Despite doubts about how policy synchronizes its projected solutions and scientific evidence accordingly, a considerable number of knowledge providers have written books, articles, reports, and policy briefs to aid the development of evidence-based policy. Some of these knowledge providers have sought to single out “paths forward, toward bringing about more/better evidence use in education” (Malin et al., 2020, p. 11); as a result, evidence-based policy is regarded as a campaign-like adventure, or at least a master idea or a trend (Røvik, 2016). By pointing to various challenges that policy experts often experience, researchers have also raised doubts about the scientific validity of the knowledge sources produced and used throughout policy processes (Holst & Molander, 2018). Although academic scholars are more frequently involved in public inquiries (Christensen & Hesstvedt, 2019), the institutional logics and contextual conditions that regulate policymaking processes are not necessarily resulting in scientific practices as they normally conducted in universities or at research institutes. Policymaking processes follow their own institutional logics and can be seen as more pragmatic in terms of how expertise apply scientific procedures and standards. Moreover, as Cairney (2016) emphasized, the environments of decision processes are characterized by complex systems where connections between actors and agencies are many and varied. Due to a lack of centralized control that follows from the way society develops, there is no guarantee that evidence will actually be used even when it is provided by experts (Cairney, 2016).

Thus, the definition of evidence is broad and highly contested; consequently, researchers and policy analysts frequently discuss what constitutes “good” evidence. Moreover, evidence cannot be researched without insights into the contexts where it is used. In Chap. 2, Steiner-Khamsi presents a theoretical overview of the relevant definitions and literature that enlighten the various features of evidence-based policy in the education sector. In particular, she outlines how the definition of evidence (depending on the situation and the stage of the policy process in our case) relates to the kind of reform policies we have explored in the POLNET project. In the next sections of this chapter, we will provide a brief overview of the data and methods we draw on in the following national and comparative chapters and give a brief overview of the Chaps. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, and 13.

Significance of the Data and Methods

To examine evidence-based policy, the POLNET study applies bibliometric and network analysis to trace the reception of knowledge sources and their formation into reference networks. A core purpose has been to explore differences in reference patterns extracted for a collection of source documents that legitimized the most recent school reforms for compulsory education in five Nordic countries. In this method section, we will provide an overview of the database we have searched, including the search strategy and the date we collected the data, the selection process, and the number of records retrieved in order to make our visualizations reproducible. Against this backdrop, each empirical chapter covers the analytical strategies we have pursued in more depth.

We consider both bibliometric and network analysis to be useful and versatile tools that provide insights into what Eyal (2019, p. 33) labeled “distributed cognitions of expertise” (p. 33); in our case, this refers to how expertise outside individuals is visualized through bibliometric reference patterns. In order to interpret and make sense of the patterns visualized in terms of quantitative measures and graphs, we have also interviewed policymakers and experts as well as synthesized and examined the content of core documents. Moreover, in all chapters, we draw on contextual information to analyze the reception and translation of knowledge sources within our source documents. We combine quantitative and qualitative approaches to provide a rich and comprehensive understanding of the cases we look into and to explore differences in citation patterns.

The origins of this book lie in the joint research project POLNET, which enabled six research teams to systematize references in policy documents to create a shared bibliometric network database. Based on a joint protocol developed by the research team at Teacher College, Columbia University, New York, each team selected a comparable set of policy documents and associated sources that were produced in conjunction with the last national school reforms in the five Nordic countries. These reforms addressed broad themes, such as renewing the national curricula, redesigning assessment systems, and amending the legislative system.

An overall goal was to measure the same phenomenon, specifically official/state policy knowledge used to set reform agendas, develop new or modified policy options, or issue reforms for basic education (Years 1–9/10), and to focus on the same variables, that is to enter relevant information on the references (e.g., year of publication, location of publication). We used sampling plans and a shared template for how to enter data to guarantee consistency. In the Nordic group meetings, we agreed that the comparative study would use a sampling strategy that approximates the one used for the Norwegian study (see Baek et al., 2018). In particular, the selected source documents should reflect official/state policy knowledge that the government had used in preparation for a particular school reform; in other words, these were white papers, green papers, or functional equivalents of these Norwegian documents. Once all the data from the five research teams were entered, the Teacher College, Columbia University-based research team cross-checked the entered data in collaboration with the researchers. In addition, all the researchers participated in joint discussions during Nordic research meetings about challenges and solutions related to data entry, cleaning, and coding. In this way, we ensured that the data were comparable. For an overview of the source documents, see Chap. 3 (Table 3.2).

The joint protocol outlined the sampling method (i.e., the selection of official/state policy knowledge documents) and the data entry protocol (i.e., the identification of variables used in the databases). We regard the documents as reflecting both the artifacts and discourses produced by policy actors, whereas the references in those documents reflect the evidence base for these artifacts and discourses. Additionally, the references in policy documents show the linkages between policy documents; for example, if two policy documents draw on similar evidence, they share some kind of affinity. By applying network analysis, we analyzed how policymakers and experts recognized the same citations and references across thematic areas and contexts. This view helps us to interpret the various knowledge networks they build, being based on both proximity and distance, respectively. Furthermore, since our cases are located in the same geographical region of Europe, we explore to what extent school reforms in these countries draw on shared knowledge sources and, more generally, whether authorization of knowledge sources differs across contexts. Table 1.1 summarizes the reforms examined in the POLNET study, the number of source documents, and their references.

Table 1.1 Reform titles and data
Full size table

After extracting the bibliometric network data on the relationship between source documents and references from each source document, we entered a series of attributes for all documents in the database to allow for interpretation. These attributes are as follows: (a) year of publication; (b) publisher or institutional affiliation of the author/authoring organization; (c) location of publication (i.e., domestic, regional, and international); and (d) type of publication (i.e., report, book, journal article, government-published document, and other). We used the attribute information to comprehend the networks and reference utilization patterns within and across the countries.

In this project, we focused on one of the major network measures (i.e., in-degree centrality, which equals the total of incoming citations for a given document) to assess the importance of a reference in a bibliometric network. We have also used other measures, such as co-citations, to identify policy networks (i.e., authors within the ministries or public inquiry bodies that cite each other or that cite the same texts) and peripheral texts/discourses that bridge two or more networks. The software programs UCINET 6.289 and NetDraw 2.097 were used for network analysis and visualization, respectively (Borgatti et al., 2002).

In addition to bibliometric network analysis, several chapters in this book include additional data. Text analysis is essential for interpreting the quality of the relations between referenced knowledge sources at levels beyond what can be recognized by individuals within the policy context and what can be perceived and potentially developed. The content analysis of the white and green papers has enabled the research teams to trace and compare reception and translation processes—that is, to explore which concepts and statements of a national, regional, or international text resonate and how they are reframed or translated in a national setting.

The semi-structured interviews we have added to complement the bibliometric network analysis helped us make sense of the reference patterns. During in-person and Zoom interviews, a group of 20 policymakers and experts involved in reform-making processes in 4 of the given countries shared their experiences and knowledge about reference practices in and across policy areas and realms. These interviews covered the same themes for the purpose of synthesizing and analyzing findings based on the interview transcriptions. Examining their responses has allowed us to explore why knowledge is used differently between the Nordic countries. We collected data according to the General Data Protection Regulation, set by the European Union, and followed ethical guidelines.

Throughout our POLNET study, we have applied bibliometric analysis, which have been primarily utilized in the field of information science (Gingras, 2016). Policy analysts have used these instruments to evaluate education research and its impact, which is particularly known from research policy in the United Kingdom (Smith et al., 2020). However, few researchers have applied bibliometric analysis to conduct comparative and international research studies of policy knowledge in education reform projects and critically examined what these methods can be used for. This book contributes to the scholarship by applying bibliometric analysis to examine citation patterns in policy documents and theorize the policy process from a comparative perspective.

Chapter Overview

In Chap. 2, Gita Steiner-Khamsi provides a theoretical framework for how to examine knowledge use to understand how evidence-based policy evolves and changes within contemporary practices. She argues that the unit of analysis for bibliometric analyses is the reference, which may be listed conveniently at the end of a document in a separate bibliographical section, in a footnote or endnote, or, more inconveniently, only vaguely alluded to in a text. In either case, the reference communicates or conveys something to the reader. The question then becomes what exactly it communicates; in the context of the POLNET study, this raises the following question: How have we interpreted the reference in the larger corpus of the policy documents that we collected in Denmark, Iceland, Finland, Norway, and Sweden in order to understand the policy process?

For our bibliometric analysis of evidence-based policymaking in the Nordic region, Steiner-Khamsi found Paul Cairney’s (2016, p. 3) definition useful: “‘evidence’ is assertion backed by information” (p. 3). In concert with Cairney’s definition, Steiner-Khamsi argues that references are a construct or an aggregate of several pieces of information (e.g., authorship, year of publication, topic, or theme) that help position the author in a larger semantic space. According to Steiner-Khamsi, all these constitutive elements are essential in bibliometric analysis, as they are utilized as epistemological cues for understanding not only whose texts or whose knowledge the authors have selected to substantiate their points but also whose knowledge they cite as sources of expertise to reduce uncertainty or generate legitimacy about the validity of their own claims or assertions. Steiner-Khamsi recommends drawing on the systems-theoretical deliberations presented in Gil Eyal’s (2019) book The Crisis of Expertise, and what Baek (2020) labeled the “expertise-seeking arrangements” in the five Nordic countries. Finally, Steiner-Khamsi invites further research that helps to refine the framework and the method of inquiry, including investigating the hierarchization of evidence as reflected in the choice of references.

In Chap. 3, Oren Pizmony-Levy and Chanwoong Baek describe the methodological approach behind the project. The authors discuss how the research design is theoretically and methodologically inspired by existing literature on social network analysis and sociology of knowledge. They demonstrate how bibliometric network analysis that is grounded in these two lines of research allows us to examine the architecture of policy knowledge in the five Nordic countries. The authors also detail the procedures for collecting, analyzing, and presenting data for this project and provide the rationale for drawing conclusions based on bibliometric analysis.

Chapters 4, 5, 6, 7, and 8 represent national case studies from the Nordic countries where bibliometric data are used to shed light on the policy processes behind recent school reforms. They all examine the use of references in key documents, such as white papers and green papers, and address how school reforms are being legitimated in certain ways through alignments of evidence produced on domestic, regional, and international levels. To contextualize the reform, several of the studies combine data.

In Chap. 4, Trine Juul Reder and Christian Ydesen analyze the evidence base and policy context of the Danish public school reform of 2013. In addition to the bibliometric data, the study draws on a contextual reading of policy documents and interviews with informants who played a key role in the policymaking process. The findings indicate that the reform was based on a quite limited number of written sources and left a great role to be played by more informal knowledge. Overall, the chapter categorizes the evidence base into academic knowledge, strategic knowledge, and knowledge produced by international organizations, such as the OECD. Furthermore, the authors also identify stakeholder knowledge and practice-based evidence as additional categories.

In Chap. 5, Saija Volmari, Jaakko Kauko, Juho Anturaniemi, and Íris Santos examine what kind of evidence Finland draws on in the policymaking process of the 2014 national core curriculum. Based on bibliometric and content analysis, the authors show that the most influential international policy documents in terms of policy design came from the OECD; however, the references used were mostly domestic. Hence, while the global level has the power to produce evidence, the national or local level experts, who build their influence through networks, have the power to select the evidence and adjust it to meet national needs. The authors conclude that the type of evidence, merely consisting of empirical evaluation data, appeared to be more important than where the evidence originated from (local or global level). This means that a global policy space intervened with the national. 

In Chap. 6, Magnúsdóttir and Jónasson explore the formation of policy documents that were issued after the school reform the national school authorities in Iceland organized during 2008–2013. The main emphasis is on the first document issued as a white paper (2014) by Icelandic state educational authorities, while the two other policy documents were framed by international organizations. In addition to bibliometric analysis, a content and a interview study with with five experts, confirm a minimal use of academic references and unsystematic development of green papers. Apart from a shared focus on enhancing quality in education, there is a low interconnection between the documents in terms of content, bibliography, and semantics. Only one of these three source documents was written in Icelandic, and therefore the Icelandic case is exemplary in terms of externalization with robust references from the OECD. The reform’s short timeframe shaped by the aim of the Minister of Education was to react promptly to declining results in PISA. The different formulations manifested by these documents reveal an eclectic approach to knowledge usage shaped by ministerial governance.

In Chap. 7, Bernadette Hörmann and Kirsten Sivesind present a bibliometric and semantic analysis of the use of knowledge sources referenced in two white and eight green papers about the recent school reform for primary and secondary education in Norway (2016/2020). The authors identify the most often co-cited texts in the bibliographies and the most frequent in-text references in the two white papers. Based on Luhmann’s (2000) distinction between conditional and purposive programs, the authors examine how policymakers use the prominent references, classified into four types, as evidence to propose a set of policy options. The four groups are: (a) formal documents that are not primarily a result of research but serve as one type of evidence, (b) meta-analysis, (c) configurational reviews, and (d) empirical research studies. The chapter concludes that meta-analysis and configurational reviews support purposive arguments along with means-end reasoning. In addition, the authors find that formal documents, with some exceptions, serve a regulatory role in terms of their conditional orientation, while empirical research reports, originally developed for evaluative purposes, help policymakers to align systems of reasoning depending on the issues they address.

In Chap. 8, Andreas Nordin and Ninni Wahlström examine the selection and use of evidence in the most recent school reform in Sweden, the Knowledge Achievement Reform (2015/2018). The analyses of citation frequency show that, although the OECD had an important role in initiating the Swedish reform, the highest percentage of references was domestic and mainly governmental references. This shows the possibility for national politics to uphold a high level of self-referentiality even when, to a large extent, international organizations such as the OECD suggest the national political agenda. Another distinct feature the authors emphasize is the low number of academic references.

Unlike the individual country studies in the previous chapters, Chaps. 9, 10, 11, and 12 address four comparative topics and bring together researchers from the different national teams. Chapter 9 compares the knowledge governments use to inform their policy decisions. Chanwoong Baek, Dijana Tiplic, and Íris Santos show that all five Nordic countries actively utilized knowledge to support and legitimate their policy proposals. However, they did so in different ways and in different settings. Some Nordic countries sought evidence for policy proposals mainly through the policy advisory system within the bureaucracy, while others outsourced the production of policy advice. Furthermore, the analysis showed that reference utilization depends on the extent to which the policy system is self-referential or receptive to externalization. The authors also highlight the differences regarding whether the reform can be described as an incremental or a fundamental reform. A fundamental or controversial reform often utilizes more international references than an incremental or non-controversial reform.

In Chap. 10, Gita Steiner-Khamsi, Chanwoong Baek, Berit Karseth, and Andreas Nordin compare reference patterns between Norwegian and Swedish green papers (cross-national comparison), as well as between green papers and white papers (political translation). The authors conclude that the advisory commissions have been repurposed in ways that place greater emphasis on expertise rather than accountability and representation. Second, the multi-level analysis shows that the commissions merely represent one stage in a long sequence of evidence-based policymaking. Finally, by following a transnational perspective, the chapter shows how advisory commissions today are used as bridges between the global and the national.

Chapter 11 draws attention to the OECD’s role in the policy process in Nordic countries. Christian Ydesen, Jaakko Kauko, and Berglind Rós Magnúsdóttir investigate the extent to which the OECD and national institutions function as data-driven knowledge brokers in the shaping of education in the Nordic region. The chapter offers an in-depth analysis of Denmark, Finland, and Iceland to understand the field of knowledge brokers in general and the role of the OECD in particular. The chapter shows how policy flows via its illumination of the configuration and workings of the OECD-centered epistemic community, forming the modes of knowledge and governance woven into the fabric of Nordic education.

Chapter 12 considers how the regional level works as a reference in national education policy planning. By drawing on bibliometric analyses, content analysis, and interviews, Saija Volmari, Kirsten Sivesind, and Jón Torfi Jónasson examine the actual role and influence of Nordic knowledge and cooperation in recent school reforms. The bibliometric analysis uncovers an extremely low number of regional references in the actual policy documents. However, drawing on the interview data, the authors conclude that both global policy spaces and policy places in local settings provide regional evidence of relevance for policymaking processes. Nordic interrelations and interactions are experienced, meaningful, and important; however, not made visible using specific bibliometric sources but rather more indirectly by translating ideas about the “Nordic” or other Nordic countries.

In Chaps. 13 and 14, two renowned scholars, Kerstin Martens and Antoni Verger, contribute by commenting on the book’s overall theme and proposing what they regard as significant problems that research on evidence-based policy should address in the future. Finally, in the concluding chapter, Berit Karseth and Kirsten Sivesind synthesize and interpret findings from the Nordic POLNET study.