Keywords

In recent years, governments have greatly emphasized evidence-based policymaking. By referring to evidence, policymakers attempt to rationalize and scientificate their political claims when formulating and implementing education policy and practice (Steiner-Khamsi et al., 2020). As a result, references have become an essential part of the institutionalized practice of policymaking. It is now expected that political claims are supported by references to evidence. References indicate that policy statements and claims are not personal or political but scientific and technical.

Although national policymakers across the world commonly argue that their policy decisions are evidence based, there is little understanding of how they reach such decisions. How does a country practice evidence-based policymaking? Specifically, what are the mechanisms deployed to legitimize policy proposals and new legislation? What are the commonalities and differences across countries in terms of what they use as “evidence”? Despite the prevalent use of references in policy documents as evidence, the nature of these references has been underexplored.

In this chapter, we investigate the above questions, focusing on the most recent school reforms in five Nordic countries: Denmark, Finland, Iceland, Norway, and Sweden. There are two reasons to focus on the Nordic countries: first, because of the geographical proximity and strong historical ties among these five countries, their education models and policy systems are often commonly perceived to be similar; second, despite the similarities, there are contextual differences across these five countries. Thus, we discuss the policy process for an education reform in each Nordic country and examine how the existing mechanisms and systems may lead to different practices of evidence-based policymaking.

Theoretical Framework

The practice of evidence-based policymaking in each country differs by (1) institutionalized forms of policymaking system, (2) degree of self-referentiality, and (3) type of reform. Each country has various forms of institutionalized policy advisory systems that help the state authorize, validate, or legitimize its policy decisions. These arrangements connect the systems of politics and science by bridging policymakers and experts. Interestingly, as these two systems become more coupled, science has faced the crisis of legitimacy. The public started questioning legitimacy and credibility of science after witnessing how each political assertion could be supported by different scientific expertise and experts (Eyal, 2019; Maasen & Weingart, 2005).

Eyal (2019) introduced four strategies that the state adopts to respond to this legitimacy crisis: objectivity, inclusion, exclusion, and outsourcing. These strategies could be categorized by the ways that the state deals with the problem of trust (trust in transparent, objective, public procedures vs. trust in trained judgment of experts) and the problem of extension (technocratic vs. participatory). Inclusion and outsourcing approaches are similar in that they both strive for participatory decision making; however, they differ in that the former places emphasis on open and public procedures, inviting various groups of stakeholders into the process, whereas the latter contracts out evidence production to external groups. By contrast, both exclusion and objectivity strategies take a technocratic approach. The exclusion strategy establishes various gatekeeping mechanisms to generate an artificial scarcity of expertise, making it difficult for one to become an expert or create organizations in charge of setting regulations and managing credentials. The objectivity strategy excludes any potential involvement of human judgments and highlights quantitative and objective measures.

This typology of different strategies could be applied to examine the institutionalized process of evidence-based policymaking in the Nordic countries. Here, we hypothesize that there may be differences across the Nordic countries in terms of how policymakers produce and utilize evidence to claim the legitimacy of their policy proposals and recommendations. For example, policymakers can seek evidence within the bureaucracy, which is closer to the exclusion strategy, or can outsource evidence production outside the bureaucracy, which is more related to the outsourcing strategy. Nevertheless, it is important to note that because the strategies are not mutually exclusive, the institutionalized policy advisory systems may simultaneously adopt multiple strategies.

The characteristics of the institutionalized process are also closely related to the degree of “self-referentiality” in the system. The theory of self-referentiality states that sociological systems tend to make internal references and “externalize” when they cannot address the problem through communication within systems (Luhmann, 1990). Thus, we hypothesize that a policy system with a higher degree of self-referentiality may have a more exclusive evidence-based policymaking process than a system with a lesser degree of self-referentiality. Additionally, a self-referential policy system might make frequent references to government regulations and previous decrees rather than the knowledge produced in external systems.

In policymaking, references play instrumental and legitimizing functions. Policy actors could make a reference to a particular body of knowledge that helps address existing policy problems or signal legitimacy. A reform entailing more controversial ideas often needs references to first justify the problematization and then authorize and validate the solution (Baek et al., 2018; Howlett & Ramesh, 2003). Thus, fundamental reforms that call for substantial changes are more likely to use references for their legitimizing functions. In addition, there is a greater need for international references in fundamental or controversial reforms because “externalization” helps generate crisis talk and build coalitions among political entities (Steiner-Khamsi, 2004). Therefore, we hypothesize that fundamental or controversial reforms would have a greater number of references in the policy documents and make more frequent references to international sources.

The Cases, Data, and Methods

The comparability of the policy process has guided our case selection, data, and analytic methods. As illustrated in previous chapters in greater detail, we focus on the most recent school reforms in each Nordic country:

  • Denmark: the 2013 Public School Reform

  • Finland: the 2014 National Core Curriculum for Basic Education

  • Iceland: Reform of 2014/2018, the renewal of the Icelandic National Curriculum Guide for Compulsory Schools with Subjects Areas

  • Norway: Reform of 2016/2020, the renewal of the Knowledge Promotion Reform

  • Sweden: Reform of 2015/2018, the renewal entitled “A Gathering for School—National Strategy for Knowledge and Equivalence”

These five reforms were initiated around the same time: between 2013 and 2016. Some reforms had the characteristics of an incremental reform aiming to improve previous reforms implemented relatively recently. The Reform of 2014/2018 in Iceland builds on the comprehensive Education Act for all school levels in 2008. It laid out two main goals: (1) increasing compulsory school pupils’ attainment in reading standards and (2) improving upper secondary students’ on-time graduation rates. The Reform of 2016/2020 in Norway is the renewal of the Knowledge Promotion Reform implemented in 2006. It was designed to refine the Knowledge Promotion Reform of 2006, particularly focusing on the domains of curriculum and quality monitoring. Furthermore, the Reform of 2015/2018 in Sweden is the renewal of a series of fundamental reforms in 2011, including a new education law and new national curriculum for compulsory schooling. Interestingly, this renewal was heavily influenced by the recommendations made in the OECD study “Improving Schools in Sweden” (OECD, 2015). Chapter 10 explains the incremental aspects of the reforms of Norway and Sweden in detail. Finally, the new National Core Curriculum for Basic Education of 2014 in Finland is the most recent renewal of the national core curriculum. The new core curriculum did not call for major changes but instead focused on providing more guidance on pedagogy, emphasizing the proactive role of schools in building a future-oriented school culture (Vitikka et al., 2015, p. 84).

By contrast, the 2013 public school reform in Denmark required more fundamental changes in conceptualization and structure. The 2013 public school reform was one of the most recent major reforms developed in response to Danish students’ mediocre performance in international large-scale assessments (ILSAs), introducing many controversial new changes. In particular, a proposal to extend school hours led to much heated debate among stakeholders regarding its scientific basis.

In an era of evidence-based policymaking when these five reforms are placed in their policy contexts in terms of national policy process and types of reforms (incremental vs. fundamental), questions arise regarding the similarities and differences across the countries in relation to what knowledge each government drew on to inform their policy decisions. In this chapter, we examine a set of key policy documents prepared for the selected reforms in each country, along with their references. Regardless of whether the author(s) made a reference to support or reject an assertion, the act of referencing is intended to provide legitimacy and credibility to the information, claim, or evidence presented in the document. Thus, by examining the references used in policy documents, we not only explore what kinds of knowledge are used to inform or justify education reforms, but we also speculate why particular references are utilized more frequently than others.

The key policy documents were identified by the research team to best reflect the policy mechanisms and institutions in each country. These documents were put forward by a group of experts appointed or funded by the government or the government ministry responsible for education policy and practice in each country, respectively (Ministry of Children and Education in Denmark; Ministry of Education and Culture in Finland; Ministry of Education, Science and Culture in Iceland; Ministry of Education and Research in Norway and Sweden).Footnote 1 In total, there were five source documents in Denmark, ten in Finland, four in Iceland, ten in Norway, and nine in Sweden (please see Chap. 3 and previous national chapters for the full list of these documents). The final analytic database includes 5443 separate data entries of references extracted from the source documents. We examine these references by paying special attention to the type of references and location of publication across the five Nordic countries. We used STATA 14.2 for the descriptive and inferential statistical analyses.

Education Policymaking Process in Nordic Countries

Often, the Nordic countries have been externally and internally seen as a coherent group, with similar public policies representing the Nordic model of welfare states (Esping-Andersen, 1996; Hallsén & Nordin, 2020). In relation to education policy, the “Nordic way” of making policy in these five countries is frequently used as a benchmark that goes well beyond its geographic limits (Dovemark et al., 2018; Ringarp, 2016; Ringarp & Rothland, 2010). However, one may ask the following: are Nordic countries as similar as they are commonly perceived?

Research has shown that despite the existence of some similarities, the differences among the Nordic countries are evident in terms of governance, policy mechanisms, and institutions (see e.g., Arnesen & Lundahl, 2006; Dovemark et al., 2018). For instance, although all Nordic countries are highly decentralized in their comprehensive education services, with regional and local authorities being the main providers and owning a great level of autonomy in the management of structures, teaching, and even curriculum, the role of private schools and the levels of school choice vary significantly among the Nordic countries. Indeed, while in Sweden, Denmark, and Iceland there is a greater number of schools to choose from, in Finland and Norway, school choice is rather limited (Dovemark et al., 2018). Moreover, although the basic governance model for nonpublic actors within primary and lower secondary education in Denmark, Norway, and Sweden appears to be collaborative, different policies and legal frameworks are applied to the mix of public, nonprofit, and for-profit services within education in each country (Segaard & Saglie, 2017). For example, the extent of private nonprofit providers is relatively large in Denmark, while for-profit service providers play a more significant role in Sweden. In Finland, although there are a few nonprofit private providers, for-profit service providers are practically nonexistent (Lundahl, 2017; Dovemark et al., 2018). In Norway, the roles of both of these types of private actors are comparatively modest (Segaard & Saglie, 2017). When it comes to the local governing of Nordic schools, two trends prevail: first, the degree of municipal control is much lower when it comes to private schools than public schools; and second, there is a higher degree of collaboration between municipalities and public schools than nonprofit schools (Thøgersen, 2017). Compared with Denmark and Norway, Sweden is the most marketized of the three Nordic countries and has the largest degree of control and regulation over private schools (Thøgersen, 2017).

Specifically regarding the institutionalized policy development process, policymaking in Nordic countries has often been perceived as technocratic in that policy actors seek to identify the most effective solutions to address policy problems through scientific or technical knowledge (Arter, 2008; Christensen & Hesstvedt, 2019; Heclo, 1974). In particular, Nordic countries have a long tradition of developing policies based on the ideas and recommendations presented in Green Papers, which are written by policy advisory commissions. Furthermore, Nordic countries have been commonly characterized as societal corporatist systems in which various interest groups work closely with bureaucracies. Across the five countries, the involvement of different stakeholders in the policy process is noticeable. In addition to interest group participation in advisory commissions, public hearings and stakeholder consultations before formulating legislation to be passed in parliament are institutionalized processes of evidence-based policymaking found in most Nordic countries.

Nevertheless, there are also national variations in their institutionalized policy process. In Norway and Sweden, for example, Green Papers (NOUs/SOUs) are prepared at the beginning stage of policy development by an ad hoc commission appointed by the government when the government is seeking to collect policy ideas and recommendations. Furthermore, Green Papers stimulate debate on a particular policy issue among various stakeholders, organizations, and the public. Based on the collected knowledge and suggestions, the Ministry of Education and Research issues White Papers that outline the policy proposal. Although Green Papers may not necessarily lead to policy formulation or change, White Papers signify the government’s intention to pass the policy.

In Denmark, on occasion, Green Papers are released to explore policy issues and initiate the reform process. In the case of the 2013 public school reform, there were no Green Papers specifically prepared for the policy formulation. Instead, the reform proposal, Gør en god skole bedre—et fagligt løft af folkeskolen [Make a good school better—improving the academic level of the public school], made references to four documents that were authored by the government or institutions funded by the government. Two of the references were authored by the Agency for the School Council, an independent body that consists of representatives from various interest groups. The other two were authored by the institute sector, one by Danmarks Evalueringsinstitut [the Danish Evaluation Institute (EVA)] and the other by Socialforskningsinstituttet [the Danish National Center for Social Research (SFI)].

Although Finland also has a similar policy development process when it comes to Green Papers preceding White Papers for major legislative changes, neither White Papers nor Green Papers are necessarily required for curriculum reforms. For the reform that was carried out in 2014, however, there was the White Paper, “Future Basic Education” and several government-published reports were closely linked to the policy change. Chapter 5 explains in detail how key documents of Finland, equivalent to the White Papers and Green Papers of other countries, were identified. Indeed, although the curriculum reform was meant to be developed in parliament and to have the participation of different stakeholders, it tends to be much more centralized than it appears (see Chap. 5 for details). The Green and White Papers constitute proposals by the Ministry of Education, which are then sent to obtain the government’s approval. Finally, the Finnish National Agency decides on the content of the curriculum using the framework set by the White Paper as a basis.

The Icelandic policy process is more decentralized. The concepts of White Papers and Green Papers have not been institutionalized in Icelandic policymaking, except for the recent White Paper published in 2014, Hvítbók um umbætur í menntun [White Paper on Education Reform]. Besides this White Paper, there was an audit report prepared by the European Agency for Special Needs and Inclusive Education on behalf of the Icelandic Ministry of Education, Science, and Culture. Policy recommendations from this report were then adopted as education policy and practice, thereby functioning as a draft of the new legislation like a White Paper. A national policy report that provided background information for the Reform of 2014/2018—as a Green Paper would do in other countries—was the “Review of Policies to Improve the Effectiveness of Resource Use in Schools: Country Background Report Iceland.” This report was prepared by the Iceland Ministry of Education, Science, and Culture as a part of an OECD report. Although the final OECD report was never produced, not only was this report cited by the European Agency Report, but also the comprehensive information included in this report often served as a reference point for policymakers in Iceland.

An examination of the institutionalized policymaking process for the most recent education reform in each country shows that although preparation of Green Papers and White Papers for education reforms was not as formally institutionalized across the five countries, every country had its own ways of seeking expertise and information both from inside and outside the government. Indeed, previous literature has discussed that although policy advisory commissions have played a significant role in the Nordic corporatist policymaking context, their numbers, forms, and membership compositions have changed over time. Denmark, Finland, Norway, and Sweden had experienced a decrease in the number of advisory commissions appointed for policy preparation over the past few decades (Ekholm & Moos, 2012; Rommetvedt, 2017). In Sweden and Finland, starting in the 1990s, these commissions were increasingly replaced by one-person commissions led by a special investigator (see Chap. 10; Rommetvedt, 2017). Finland even abolished the commission system in 2003 (Erkkilä, 2012), and the role that the commissions used to play is now played by broad-based working groups (Holli & Turkka, 2019). The abolition process of commissions in Finland was complex, and there were various reasons behind the decision to remove the advisory commission. Since the 1990s, there had already been a growing criticism that the advisory commission system was ineffective (Holli & Turkka, 2019). Furthermore, the government intended to obtain control of the policy process by abolishing the advisory commission institution, which had a fair share of autonomy (Holli & Turkka, 2019). Interestingly, it is also suggested that the government emphasized receiving knowledge beyond the selected group of individuals and actively advocated for public hearings to stress openness and transparency in governance (Erkkilä, 2012).

Under the advisory commission system, commission reports were a formalized part of the policy mechanism that provided information on debates and legislative history regarding policy issues. However, the new system, which includes ad hoc networks and working groups, tends to produce less documentary evidence regarding its deliberations (Erkkilä, 2012). Although much information is now publicly available online, the available information is mostly performance and administrative management (Erkkilä, 2012). This also reflects the shift in modes of governance: the government now governs by monitoring and controlling the outputs (managerial accountability) instead of managing the inputs.

Furthermore, previous literature has found changes in the composition of advisory commissions in the five countries. Denmark, Sweden, and Norway have involved more academic researchers in advisory commissions over time (Christensen & Hesstvedt, 2019; Christensen & Holst, 2017; Ekholm & Moos, 2012). In Finland, by contrast, the share of researchers in broad-based working groups as well as their participation in the commissions as chairs, secretaries, and permanent experts have decreased (Holli & Turkka, 2019). Concurrently, interest group representation has dropped in Denmark, Finland, Norway, and Sweden in recent years (Binderkrantz & Christiansen, 2015; Christensen & Hesstvedt, 2019; Rommetvedt, 2017).

The changes described above signal the overall decline of corporatism in the Nordic countries. The sources of knowledge have been diversified, and it has become difficult for governments to develop policies in collaboration with only a few interest groups. As the number of interest groups has increased, the representativeness of the selected groups has become increasingly questionable. Furthermore, because today’s policy interests are complex, interest groups cannot promise the support of their members for political exchange, which discourages governments from cooperating and negotiating with interest groups in the policy process (Rommetvedt, 2017).

Strikingly, what is observed in Iceland differs from the rest of the Nordic countries. Óskarsdóttir (2018) examined the number and composition of public commissions in Iceland between 1970 and 2017, finding that although the number of advisory commissions has declined over time in other Nordic countries, it has significantly increased in Iceland. The results from the study showed that Iceland now has the highest number of preparatory corporatist commissions among the Nordic countries. Interestingly, when looking into the number by policy areas, the Ministry of Education and Culture was the agency with the highest number of commissions, by a wide margin. Óskarsdóttir (2018) explained that perhaps the reason behind why Iceland has demonstrated robust corporatism in recent years compared with other Nordic countries is because the state has undivided control over legislation because of its parliamentary majority and because a number of cohesive interest groups hold representational monopoly. This meets the properties required for a corporatist exchange between the state and interest groups (Öberg et al., 2011). The small size of bureaucracy and limited administrative capacities are other potential contributing factors to Iceland’s relatively strong corporatism (Óskarsdóttir, 2018).

Reference Patterns in Nordic Policy Documents

This study found varying reference patterns in policy documents across the countries in terms of (1) style and number of references, (2) type of references, and (3) location of publication. Each pattern reflects country-specific policy processes and reform contexts discussed in the previous sections.

Institutionalized Practice of Evidence-Based Policymaking

The frequency and style of reference could serve as an indication of the degree of institutionalized evidence-based policymaking practice. Table 9.1 shows that overall, the governments of the five countries made frequent references to support their policy proposals, ranging from 50 to 264 references per policy document. This high number of references may not be surprising given the recent shift toward evidence-based policymaking. Governments are now expected to be transparent about on what they are basing their policy decisions. Indeed, the number of references in national policy documents has increased over time. For example, policy documents prepared for the 1997 reform in Norway seldom made references in their reports, and most of the references were either embedded in the text or listed as footnotes. However, later reforms, such as the Knowledge Promotion Reform of 2006 and the renewal of the Knowledge Promotion Reform (2016/2020) had separate reference lists and made thousands of references. Even between the Knowledge Promotion Reform and the renewal of the Knowledge Promotion Reform (2016/2020), there was a significant increase in the number of references (Baek et al., 2018; Steiner-Khamsi et al., 2020).

Table 9.1 Number of references in the policy documents
Full size table

Nevertheless, each country appears to engage in different levels of the institutionalization of reference practice. Table 9.1 shows that Norway made the largest number of references in a policy document on average (265 references per source document), followed by Sweden (179 references per source). By contrast, Denmark only had about 50 references per policy document on average. Of course, the varying length of policy documents may influence their average number of references because a longer policy document has a higher chance of having more references. Regardless, the low frequency of references in Danish policy documents is worth highlighting. For example, the reform proposal Gør en god skole bedre—et fagligt løft af folkeskolen did not have a separate reference section and had only ten references in the footnotes.

Furthermore, Iceland has the highest variability in the number of references among source documents, followed by Finland. Perhaps this suggests that the reference practice in Iceland and Finland has not become as standardized as in other countries. Both Norway and Sweden did not have much variation across policy documents regarding their average number of references, signifying a greater level of institutionalization of frequent reference utilization. The patterns in the frequency of references in the five Nordic countries are also consistent with the institutionalized policy process described in the previous section in this chapter. Although Norway and Sweden had institutionalized the “standard model of bureaucracy” through Green Papers and White Papers prior to issuing a reform (see Steiner-Khamsi et al., 2020; Tullock, 2005), Denmark, Finland, and Iceland did not mandate such a process.

Despite the varying degrees and formats, each country has its own stakeholder review process where interest groups and the public can contribute their expertise to policy formulation. Furthermore, there are government-funded research institutes or groups of academics asked to conduct research for education reforms. In fact, the curriculum reform process in Finland seeks evidence by working closely with a variety of stakeholders throughout the process. Instead of the pyramid structure of bureaucracy, where the government is placed at the top, experts appointed by the government or other stakeholders work with the government in the middle, and the public is at the bottom, all these actors are perceived to coproduce an education reform. Scholars have observed that the democratization of political systems has led to the participation of numerous nongovernment policy actors and organizations in policymaking (Maasen & Weingart, 2005). In addition, the public now monitors and participates in policy knowledge production, democratizing the system of expertise (Weingart, 2003). In other words, the democratization of political systems and systems of expertise has prompted governments to seek legitimacy and credibility beyond the traditional policy process.

Types of References

What constitutes evidence varies over time and is highly context-related (Baek et al., 2018), which translates into diverse patterns of references. Among the Nordic countries, not only do they vary in their styles and numbers of references in policy documents, but there are also variations in what is used as evidence. Table 9.2 shows the distribution of references by type of document. Overall, government-published documents are the most commonly cited type of reference (37.55%). However, there was a statistically significant difference in the types of references utilized in each country (χ2 = 823.93, p < 0.001). Although government-published documents are the most cited type of reference in Denmark, Iceland, and Sweden, books and reports are the most cited type in Finland and Norway, respectively. Furthermore, it is noteworthy that journal articles are not frequently cited in the Nordic countries.

Table 9.2 Distribution of references by type
Full size table

Government-Published Documents

Our analysis reveals that the vast majority of the references in Nordic policy documents were produced by the government. The high presence of government-published knowledge in Swedish policy documents is particularly striking considering the substantial influence of international organizations on Sweden’s education policy agenda. In Chap. 8, Nordin and Wahlström interpret this as the government’s tendency to uphold a great level of self-referentiality (Luhmann, 1990). The Swedish government favors making references to its institutionalized norms, traditions, and own logic. Following Sweden, policy documents in Iceland made many references to government-published documents. More than half (55.21%) of the references were prepared by the government. When looking into these references more closely, however, it turns out that most of the government-published references were the data and statistics on the education system produced by Hagstofa Íslands [Statistics Iceland]. This is an interesting contrast to the government-published documents in other countries, which are mostly reports or proposals published by the respective Ministry of Education or the executive agency of the Ministry (e.g., Utdanningsdirektoratet [Norwegian Directorate for Education and Training] and Skolverket [Swedish National Agency for Education]).

Furthermore, in Finland, only 27.92% of the references used are government productions; however, a deeper analysis into the country context reveals that the influence of the government might be greater than that (see Chap. 5). When looking at the publishers and the authors, a larger amount of referenced documents are directly related to the government search for knowledge and evidence. The most cited publishers in Finland are the National Agency of Education (cited 170 times), followed by the Ministry of Education and Culture (57), the University of Jyväskylä (55), and, in the fourth position, the University of Helsinki (32). Of the top four publishers, the first two are government agencies, and the other two are universities that hold two important and publicly funded research institutes within them, Koulutuksen tutkimuslaitoksessa [the Finnish Institute of Educational Research (FIER)] and Helsingin yliopiston Koulutuksen arviointikeskus HEA [the Center of Educational Assessment (CEA) in the University of Helsinki], which perform research on the assessment and evaluation of the Finnish education system and are responsible for the implementation of the Programme for International Student Assessment (PISA) surveys, for example. In addition, when we analyzed the authors of the most referenced books, reports, and academic research, many of the authors work for the above institutions, and often, their publications are the result of government-funded projects. This phenomenon leads to the understanding that academic and government productions are now often tangled, suggesting the tighter coupling between the systems of politics and science.

Reports

Although Norway had the lowest percentage of government-published documents as references among the five Nordic countries, it had the greatest percentage of reports. Indeed, reports were the second most cited type of reference in the Nordic countries after government-published documents (see Table 9.2). However, there were significant differences among the Nordic countries regarding the use of reports as the reference type. As previously stated, in Norway, reports represented the most cited reference type (31.01%). In Denmark, the share of reports as the reference type was very close to that of government-published documents (29.87% and 33.77%, respectively). In Iceland, reports were the second most used reference type (25%) after government-published documents (55.21%). By contrast, reports were not as often used in Sweden (11.19%) and Finland (11.08%).

A possible explanation for why reports are the most represented reference type in Norway may be that the sector research in Norway has increased with the expansion of funding and the evaluation of educational research programs (Zapp et al., 2018). Consistent with this speculation, Denmark, another country with a growing influence of state-funded independent research institutions such as EVA and SFI (which is now VIVE after its merger with the Danish Institute for Local and Regional Government Research [KORA] in 2017), drew many of its references from reports (29.87%). Finally, many references to reports may also reflect the infrastructural and epistemological influence of international organizations on education policymaking (see Addey, 2017; Sellar & Lingard, 2013). For example, in Denmark, about 20% of the references to reports were produced by international organizations. In the following section, we discuss the role of international references in greater depth.

Location of Publication

With the increasing number of international organizations focusing on education, assessment, and improvement, education policymaking is no longer limited by the borders of each nation-state (e.g., Rizvi & Lingard, 2010). For example, the educational reforms in many countries consider twenty-first-century skills for pupils’ learning and well-being as critically important at both individual and societal levels. In addition, in most Nordic countries, the national political, legal, and institutional framework within which education and other local welfare services are provided has been regulated by an EU Directive (Segaard & Saglie, 2017). The relationships between the global and local identities working on education reform are increasingly tight and diverse, and they are not neccessarily one-directional: although international organizations do have an influence on local policymaking, national policymakers utilize international organizations and their instruments in the national policymaking process for their own agenda. One such utilization is the reference to the international instruments as authoritative tools to legitimize certain reforms (Steiner-Khamsi, 2004). Our bibliometric analysis reveals active utilization of international references in the Nordic policy documents, with Denmark having the most international references (36.36%) and Sweden presenting the least (18.93%). By contrast, national policy documents made fewer references to other Nordic countries, which remain less than 8% of the references across the five countries (Table 9.3).

Denmark’s frequent utilization of international references could also be broken down into the five types of documents applied in the previous section: report, book, journal article, government-published document, and other. The analysis informed us that most of the international references in the Danish policy documents were academic literature (42.8%) on the topic of educational leadership (see Chap. 4). Another interesting finding is that many international references (16.7%) were reports produced by the OECD. In the case of the 2013 public school reform, the OECD reports were referenced to identify the existing problem in the system and to present outsider perspectives. However, the influence of the OECD was not limited to providing bodies of knowledge and insights; it also contributed to the formulation of coalitions by ideology regarding educational debates, making the policy environment divided and antagonistic (see Chap. 11).

Indeed, the significant influence of the OECD on educational agenda and policy has been discussed by many scholars (e.g., Addey, 2017; Grek, 2009; Hansen & Rieper, 2010; Martens, 2007; Sellar & Lingard, 2013; Takayama, 2013), and this is not only the case in Denmark. Not only had each country used its PISA ranking to diagnose its education system or create reform pressure, but the reforms in Finland, Iceland, Norway, and Sweden examined in this study highlighted global education policies promoted by the OECD, such as competency-based education, twenty-first-century skills, and accountability reforms. Despite its strong subject-based tradition, the new core curriculum in Finland introduced and defined seven competence areas related to twenty-first-century skills (Vitikka et al., 2015). The Reform of 2014/2018 in Iceland sought to strengthen the competence required in society and economy of the twenty-first century, which had been emphasized in the 2008 education policy and the National Curriculum Guides. Similarly, the Reform of 2016/2020 in Norway built on the competency-based education introduced in the Knowledge Promotion Reform of 2006. It was also influenced by the OECD report, titled OECD Reviews of Evaluation and Assessment in Education: Norway 2011, which recommended assessments take place through learning goals and quality criteria (Baek et al., 2018; OECD, 2011).

It is important to highlight that the influence of international organizations such as the OECD is not only direct but also discreet or even silent (see Kallo, 2009; Kauko & Varjo, 2008; Moisio, 2014; Waldow, 2009). For example, a deeper content analysis of the Finnish White Papers and Green Papers found more embedded references to PISA and the OECD and the performative and competitive ideas expressed in their reports than our bibliometric analysis initially identified. Furthermore, although Sweden demonstrates a lower percentage of international references in its policy documents than that of other Nordic countries, this number does not exactly reflect the international effects on the development of the Swedish 2015/2018 Reform. Our additional analysis reveals that the Swedish Green Paper, “Gathering for school—national strategy for knowledge and equality,”Footnote 2 which had significantly shaped the 2015/2018 reform, was specifically commissioned to review the OECD recommendations in “Improving Schools in Sweden” (OECD, 2015). Steiner-Khamsi et al. call this national adaptation of the OECD “OECD-reviews-in-national-disguise” (see Chap. 10). In his study of Swedish educational policymaking, Waldow (2009) documented a trend of “silent borrowing,” by which international imports often remain invisible. Waldow (2009) explained that Sweden’s silence of borrowing may be because of its leading position in education and welfare systems after World War II. As a pioneer in the field, referencing external education systems, policies, and bodies of knowledge did not have any rationale for legitimization. Sweden instead relied heavily on obtaining authorization from scientific rationality (Ringarp & Waldow, 2016; Waldow, 2009). Ringarp and Waldow (2016) found that this culture shifted in the early 2000s as Sweden lost self-confidence in educational performance because of its declining results in PISA; hence, it started to make more international references around 2007. Despite this new orientation, the culture of silent borrowing may still be ingrained in the Swedish policy system compared with other Nordic countries, which could explain Sweden having the lowest number of international references.

When it comes to referencing one’s neighboring countries, Norway made the most regional references (see Table 9.3), suggesting its policy borrowing within and across the Nordic region (Steiner-Khamsi, 2004). Indeed, in her study of Norwegian governmental papers, Sivesind (2019) showed that Norwegian policymakers perceive Finland as a country of emulation, particularly in the areas of curriculum, quality and development, and student achievement (Sivesind, 2019). However, considering the common (mis)conception of the Nordic system as one, it is striking that there were not many references to regional documents. The different paths taken by the five Nordic countries in terms of the organization of the school system and its structures and management (Dovemark et al., 2018) might explain the small usage of regional references. Alternatively, it is possible that the Nordic countries borrow each other’s ideas, policies, and practices without referencing them both internationally and unintentionally (e.g., Waldow, 2009).

Table 9.3 Distribution of references by location
Full size table

Discussion and Conclusion

In this chapter, we have examined the policy mechanisms for evidence-based policymaking in the five Nordic countries and the references that national policymakers have utilized in their policy documents to evidentiate policy ideas and recommendations. In particular, we were interested in whether there are similarities or differences across the five countries regarding what knowledge the government used to inform their policy decisions. The results illustrate that all five Nordic countries included in this study actively utilize knowledge to support and legitimate their policy proposals; however, they do so in different ways and in different settings. The findings support most of our hypotheses that the practice of evidence-based policymaking varies by (1) institutionalized forms of policymaking system, (2) degree of self-referentiality, and (3) type of reform.

First, by comparing the policy development process of the most recent school reforms, we found that some countries sought evidence for policy proposals mainly through the policy advisory system within the bureaucracy (e.g., Green Papers in Norway and Sweden), while others outsourced the production of policy advice (e.g., EVA in Denmark and FIER in Finland). Our results also have shown that countries where evidence production is generated within the state bureaucracy (i.e., exclusion strategy) had more references than countries that produced policy knowledge outside the bureaucracy by outsourcing to think-tanks or sponsoring policy research (i.e., outsourcing strategy). This may suggest that in countries with an internal reference system, the bureaucracy is more proactive in producing and utilizing evidence or that it at least tries to demonstrate that its policy is evidence-based.

National policy contexts regarding how policymakers seek policy knowledge (e.g., internal or external commissions, public hearings, and stakeholder reviews) signify each nation’s political orientation and perception toward democratic and technocratic policymaking. Furthermore, the change in political models, such as the prosperity of corporatism in Iceland and the decline of corporatism in other Nordic countries, shapes who participates in the policy process and what their roles are. This information contributes to a more complete understanding of the boundaries of rationality and knowledge, which consequently influences the reference utilization in each country.

Second, our analysis of reference utilization in the five Nordic countries showed that reference utilization depends on the extent the policy system is self-referential or receptive to externalization. The frequent utilization of non-government-published documents such as reports, books, and journal articles in Norway indicates that policymakers were open to external sources of knowledge beyond the system of politics. Indeed, the Norwegian government extensively made use of the knowledge produced by institute-sector organizations such as the Nordic Institute for Studies in Innovation, Research and Education (NIFU) and Norwegian Social Research (NOVA), both of which bridge the systems of politics, science, and practice. Norway also demonstrated a relatively high percentage of international references that were produced across geographical boundaries. This is interesting considering that Norway demonstrates a higher level of externalization despite having an exclusive policy advisory system. On the contrary, Sweden, another country where the evidence-seeking process is centered in the state bureaucracy, demonstrated a greater tendency toward self-referentiality or was at least less explicit about its externalization. Sweden had the highest percentage of government-published documents and the lowest percentage of international references compared with the other Nordic countries. Despite this concentration on internally produced knowledge, many of these internal references turned out to be local translations of externally produced knowledge (see Chap. 10; Waldow, 2009). In other words, it is possible that externalization was disguised in the form of internal references. When discussing externalization and translation to understand reference patterns, an additional factor to consider is that there could be certain bodies of knowledge that had become common knowledge within the system and did not require any formal reference. Thus, the absence of references to particular knowledge does not necessarily mean the disregard of the knowledge.

Third, although our hypothesis that a fundamental or controversial reform would have more references in an attempt to obtain legitimacy and scientific base did not hold, the results confirmed another hypothesis: a fundamental or controversial reform would utilize more international references than an incremental or noncontroversial reform. An example from our analysis could be the Danish case—a highly controversial reform—which has the most active international reference utilization. In-depth analyses of international references in Denmark showed that policy actors used international references, particularly the ones produced by the OECD, to legitimize the need for the 2013 public school reform by problematizing and diagnosing the existing system (see Chaps. 4 and 11). By contrast, for the 2016/2020 Reform in Norway, which was an incremental reform of the 2006 Knowledge Promotion Reform, policy experts who served on advisory commissions shared that they often referred to international references to collect knowledge about a policy topic that had not been explored in-depth domestically (Baek, 2020).

In conclusion, our findings show that there are similar and different patterns in institutionalized policy processes and reference utilization across the Nordic countries. These differences could be understood by a combination of three factors: institutionalized policymaking system, self-referentiality, and type of reform. For future studies, we suggest a more in-depth analysis of the interplay between the three factors to better understand particularity of the national approaches to evidence-based policymaking.