Policy Borrowing and Evidence in Danish Education Policy Preparation: The Case of the Public School Reform of 2013

Abstract

Reder and Ydesen explore the evidence behind the 2013 Danish public school reform using a mixed-methods design as well as the political context surrounding the reform preparations. By combining quantitative bibliometric analysis and qualitative interviews of the key policymakers behind the reform, they identify different types of policy knowledge. Reder and Ydesen focus particularly on the role of international large-scale assessments and international organizations such as the OECD in legitimizing the need of the reform and identifying reference societies, such as the Canadian province of Ontario, which heavily influenced the reform contents. The chapter concludes with a discussion on the findings and presents a very uneven evidence landscape consisting mainly of what-works types of evidence and a distinct absence of broader types of evidence.

Since the beginning of the 1990s, Danish education policy has increasingly aligned with transnational trends. Danish student performance in international large-scale assessments (ILSAs) has sparked public and political debates concerning the Danish education system (Holm-Larsen, 2010). Given the students’ mediocre performance and relatively high public education spending, Danish policymakers have focused on reforming the education system (Imsen et al., 2017), resulting in the initiation of several educational reforms in Denmark since 2000. These reforms include the 2006 public school reform, several reforms addressing primary school curriculum objectives (2003, 2009 and 2015), the 2007 introduction of national assessment tests in primary and lower secondary schools, and finally the 2013 public school reform (Danish Evaluation Institute, 2012, 2015).

This chapter focuses on the 2013¹ public school reform, which is the latest major reform of the Danish public school system and one of the most controversial and hotly debated political reforms in recent years. The reform had three overall goals (Danish Government, 2012; Ministry of Education, 2014)²:

• 1. The Public School must challenge all pupils to reach their fullest potential.

• 2. The Public School must reduce the significance of pupils’ social background for academic results.

• 3. The trust in the Public School and pupil well-being must be enhanced by showing respect for professional knowledge and practice.

These goals were to be accomplished through measures such as longer school days, earlier foreign language learning, 45 minutes of exercise daily, homework assistance in “homework cafés,” and more lessons in Danish and math (Ministry of Education, 2014).

The reform proposal sparked heated public debate concerning the reform’s content and the scientific evidence behind its elements. Longer school hours were a central topic because the change interfered in the balance between school and leisure activities. A main argument for longer school hours was to improve students’ academic level; however, Danish reports on the Progress in International Reading Literacy Study (PIRLS) and the Trends in International Mathematics and Science Study (TIMSS) results published a mere week after the reform’s presentation stated there was no scientific evidence indicating a relation between school hours and academic achievement (Allerup, 2011; Mejding & Rønberg, 2012). Merete Riisager, of the right-wing opposition, consequently called for a consultation concerning the knowledge sources underpinning the reform, and the debates continued in the media and in Parliament during the spring of 2013.

Another striking point in the reform’s presentation was reference to ILSAs and foreign school systems’ experiences, particularly the reform of Ontario’s school system, which markedly improved its PISA rankings (Levin et al., 2008; Coninck-Smith et al., 2015). In 2012, Christine Antorini, then Minister of Education, expressed that the upcoming reform was largely based on an “inspirational” trip made by Danish Parliament representatives to Ontario (Aisinger, 2012) and later repeated that Denmark could learn much from Ontario’s success (Larsen, 2012; Møller, 2012). The reform was introduced following over a decade of discussion and negotiations around public school reform, justified mainly by the Danish students’ mediocre results in the PISA surveys from 2000 to 2011 (Cort & Larson, 2015; Egelund, 2008). Regarding ILSAs, only 5% of Danish pupils were “strong readers,” and 17% lacked both functional math and science skills (Danish Government, 2012). When the Social Democrats won the October 2011 election, the new government committed to reforming public school. In December 2012, the government presented a proposal for public school reform, entitled Make a Good School Better—Improving the Academic Level of the Public School (Danish Government, 2012).

Given this background, this chapter connects with the theme of this volume and contributes to understanding the interplay between international policy trends and national education policy development in general and the role of evidence in the process behind Denmark’s 2013 public school reform in particular. However, such an analysis calls for initial conceptual reflections because discussion about what constitutes real evidence often steals the focus (Christensen & Krejsler, 2015).

As Steiner-Khamsi (Chap. 2, this volume) argues, a reference can be understood as “validation of evidence.” Thus, a clear connection is established between references and evidence. Following Steiner-Khamsi, we employ a pragmatic approach to evidence, meaning we consider all kinds of knowledge sources and information used to inform policy processes and create a basis for decision making to be evidence (Cairney, 2015). These initial reflections allow for the following overall research questions:

• 1. What evidence base underpins the 2013 Danish public school reform?

• 2. In what policy context was the evidence base formed and used?

Methodological Considerations and Chapter Structure

Steiner-Khamsi et al. (2020) emphasized that evidence behind education reforms differs vastly. Therefore, we need to remain exploratory and open in terms of investigating “which evidence governments actually use when they formulate the policy” (p. 138). We also connect with Ozga’s (2019) argument about taking the political dimension of policy seriously: “Seeing policy as politics focuses on how state policy in its design and attempted delivery involves politics, through interests, conflicts, power and control—so that politics is an essential element of policy” (p. 21).

To realize these insights, we employ a mixed-methods design that creates an abductive platform between sources of official policy knowledge found in a bibliometric analysis of the policy documents and sources of unofficial policy knowledge emerging from research, media articles, and a contextual reading of policy documents. Additionally, we conducted four qualitative interviews with key informants in the spring of 2019 (Appendix 1) to uncover other sources of knowledge than those explicitly cited in the policy documents.

We start with a bibliometric analysis based on 231 references in the preparatory school reform policy documents. This quantitative analysis sheds light on the types of policy evidence used in the proposed reform and patterns emerging from the reference network of knowledge sources. The second part analyzes the policy context framing the formation and use of the evidence base. The conclusion summarizes the main findings and connects to the overall research questions.

Source Documents

For the basis of the bibliometric analysis, we include the reform proposal itself (Source Document 1) and the four references of the reform proposal authored by a government institution: two reports by the Agency for the School Council (Source Documents 2 and 3)³ and two reports by the national sector research agencies The Danish Evaluation Institute (EVA) and The Danish National Centre for Social Research (SFI) (Source Documents 4 and 5). Unlike the Official Norwegian Reports (Steiner-Khamsi et al., 2020), these four reports were not specifically prepared as background papers to formulate the school reform policy; however, we argue they constitute the best possible data for our bibliometric analysis. Table 4.1 displays the ten references of the school reform, where the first four constitute the source documents of this analysis.

Table 4.1 Ten references of the reform proposal (Source Document 1)

Source Documents 2 and 3 were authored by the Agency for the School Council, which was established in 2006 to “follow and assess the quality of the primary and lower secondary school and advise the Minister of Education” (School Council, 2007, p. 3). It was independent of the Minister of Education, and its members were representatives of different interests in the elementary school area. Although the council reports were produced by a secretariat at the Ministry, its content was decided by the council.

The two sector research reports were authored by SFI (Source Document 4) and EVA (Source Document 5), and both institutions were partly funded by the government. An independent state institution established under the Ministry of Education in 1999, EVA, is one of the central research institutions working in education policy (Bjerre & Reimer, 2014). EVA evaluates and researches within the education field at the request of other branches (e.g., ministries, local authorities), as well as on their own initiative (Danish Evaluation Institute, 2019). SFI was a sector research institution under the Ministry for Economic Affairs and the Interior. This institution mainly conducted research in welfare state policies (e.g., concerning the labor market and family-related issues). In January 2017 SFI merged with another public research institution and was renamed The Danish Center for Social Science Research (VIVE).

Qualitative Data

The qualitative analysis relies on four semi-structured interviews with key policymakers behind the reform: a former ministerial official who worked on the reform proposal (INF1), a former Parliament member (INF2), and two former members of the School Council (INF3 and INF4). The informants were directly involved in the reform preparation or in the work of the School Council, and the interviews were coded in a data-driven approach (Schreier, 2014). The codes are presented in Appendix 1. The interviews offer insights into the preparatory work in meetings and seminars with stakeholders for which no records or summaries are publicly available. The informants were anonymized, since the reform sparked heavy public debate regarding the evidence behind the reform. Therefore, it was an ethical and strategic choice to give the informants the opportunity to speak freely without fearing consequences in the form of media critique or otherwise.

To verify the reliability of the interview data, we cross-checked the interview information, arguments, and claims through other interviews and weighed it against open-source information using source criticism procedures found in historical research (Ifversen, 2003). We therefore include public statements from key policy officials to connect to the ways the reform was presented and justified to the public. We located the quotes and articles using Infomedia, a Danish media surveillance company that covers all Danish newspapers.

Mapping the Evidence Base of the School Reform

In this section, we present the main findings of the bibliometric analysis in terms of the reference locations, document types, network of references, and publishers. Table 4.2 presents the reference distribution between the five Danish source documents and the distribution between the document types and location.

Table 4.2 Reference distribution

The School Council reports and SFI reports have the most extensive citation practices, explained by the nature of these institutions: The School Council’s (2007) purpose is to “provide documentation for the initiatives and actions that contribute to increasing the quality of the primary and lower secondary school” (p. 14). Thus, its purpose is to provide evidence and disseminate research to the Ministry of Education in yearly reports, focusing on specific topics. SFI is generally considered a research-heavy institution compared to EVA, which mainly conducts empirical research and evaluates specific national initiatives in education.

We split the references into five document types: reports, books, academic, government, and others. The reference distribution reveals that the source documents mainly cite government documents (33.77%) or reports (29.87%; see Table 4.2). “Government” evidence includes documents and reports produced by the government, a ministry, or an agency of a ministry. “Reports” can be produced by research agencies, international organizations (IOs), or private consultancy firms. The “academic” literature accounts for 16.02% of all references, mainly distributed in the 2012 School Council report (Source Document 3) and the 2011 SFI report (source Document 5), both of which concern school leadership. Furthermore, the SFI report was commissioned by the School Council (Danish National Centre for Social Research, 2011, p. 7). It is therefore not surprising that the reports are similar in terms of references.

Network of References

Figure 4.1 illustrates the complete network structure and co-citations between the five source documents. The references are displayed as squares, and Source Documents 1–5 as circles. The size of each note reflects its in-degree centrality; the larger the note, the more source documents have cited it. The bibliometric analysis shows a relatively low frequency of co-citations between the source documents: only 16 references are shared by a minimum of 2 source documents, and no reference is shared by more than 3. The report cited by three source documents and thus at the center of Fig. 4.1 is The High-Performing School—How Can Schools Improve the Proficiency Level of Students with a Weak Social Background? (Mehlbye, 2010). It was published by the Danish Institute of Government Research (AKF) and commissioned by the School Council to examine so-called high-performing schools, focusing on schools that improved the academic performance of students with weak social backgrounds. The report is thus a classic example of a “what went right” approach, where policymakers attempt to find the best practice by studying successful examples. Qualitatively analyzing 12 schools, the report seeks to deduce what characterizes high-performing schools and concludes that, for instance, clear goals, academically competent teachers, and homework assistance were important for the pupils to succeed (Mehlbye, 2010). Since reducing the influence of students’ social background on academic results was one of the three goals of the school reform, it is not surprising that a report addressing these issues appears significant.

Fig. 4.1

Complete network structure. (Note: Circles represent source documents. Squares represent citations. The colors determine whether the citation is regional (gray), domestic (white), or international (black). Node size reflects in-degree centrality)

The shared references in the source documents mainly comprise research by the government or connected institutions⁴ (11 of 16 references) or international evidence from either IOs or international edu-businesses (3 of 16 references; see Appendix 2 for a list of all 16 co-citations). The low frequency of co-citations in Source Documents 2–5 reveals that the 4 reports behind the school proposal represent specialized knowledge: they each focus on specific subjects used to substantiate the different reform elements. The 2011 School Council report focuses on transfer between primary and secondary education, schools’ collaboration with municipalities, and after-school classes, whereas the 2012 School Council report is concerned with school management. The EVA report (commissioned by the School Council) concentrates on differentiated teaching, evaluation, and teacher professionalism, and the SFI report (commissioned by the Ministry of Education) centers on school leadership. This picture of specialized knowledge is similar to Norway’s case, in which the Norwegian Official Commissions (NOUs) generate “highly specialized knowledge” that allows the government to “selectively transfer what was produced at the commission level to the political level” (Steiner-Khamsi et al., 2020, p. 128).

International Inspirations: Academic Research and Large-Scale Assessments

A remarkable pattern appears when we look at the international and domestic references individually, as in Table 4.3. Namely, the academic references consist almost exclusively of international research. This finding is in line with a case study of Norwegian policy advisory commissions by Christensen and Holst (2017), who found growing reliance on what can be categorized as academic knowledge in general and (in the Norwegian context) on international academic knowledge in particular. Although Denmark has a long tradition of educational research going back to World War I (Gjerløff & Jacobsen, 2014), this research surprisingly does not seem to appear in the preparation of the 2013 school reform. The international academic references are mainly centered on the topic of educational leadership, and the majority are referenced in Source Documents 3 and 5, both reports concentrating on school leadership at different levels (school, municipal, and national). The most cited international journals are Educational Administration Quarterly and Journal of Public Administration Research and Theory, both of which focus on educational leadership and management, as well as public administration (Oxford Academic, 2020; Sage, 2020).

Table 4.3 International, regional, and domestic references distributed by types of knowledge

The remaining international references are distributed mainly as reports (21.43%) and books (25%). Of all international reports (n = 23), the majority come from IOs (14) and consultancy firms (4).⁵ The OECD (2010b, 2010c, 2010d, 2011a, 2011c) is the most cited IO, and the citations mainly refer to PISA.⁶ This emphasizes the importance of IO evidence, especially concerning PISA, in Danish school policymaking in preparing and legitimizing the reform. We detail this in the subsequent qualitative analysis, focusing on the use of international experience and lessons in the 2013 school reform.

Reliance on Government-Commissioned Evidence

While the international citations mainly refer to academic and IO evidence, domestic citations largely refer to government and public institution reports. Figure 4.2 shows the 11 most cited publishers, 6 of which are the following public institutions: Ministry of Education, EVA, SFI, the Danish Government, the School Council, and AKF.⁷ The Ministry of Education is by far the most cited.

Fig. 4.2

Most cited publishers

As seen, the field of social policy (including education) in Denmark contains a range of institutions aimed at providing evidence for policymakers. The bibliometric analysis reveals three of these institutions (EVA, SFI, and AKF) are among the most commonly cited in the preparations for the public school reform. One explanation for this finding is that it is common practice for institutions to cite themselves where possible. SFI and EVA are the authors of Source Documents 4 and 5, respectively, and the EVA evaluations on compulsory education were all assigned by the School Council, author of Source Documents 2 and 3 (OECD, 2011b; School Council, 2007).

Research from public agencies thus constitutes a large part of the references.⁸ The evidence produced by these institutes is mostly commissioned by public institutions, such as ministries and municipalities. Bjerre and Reimer (2014) explained,

the research institutes have established their own system of peer review and quality check […] and thus the reports move directly from the research agencies into the political, administrative, and public process, without the intermediation of a scientific community of critical readers and the general peer review system […]. (p. 85)

The majority of the evidence produced by these institutions can be categorized as strategic evidence, which is evidence commissioned by the government (or a government institution) or carried out by a public institution. However, there are two reservations. First, not all research by public agencies is strategic; for example, institutions such as EVA and SFI also conduct research for external parties, such as private funds (e.g., Danish Evaluation Institute, 2020). These agencies also conduct what they call independent⁹ research, which is research funded by their annual budget. Second, strategic evidence is sometimes produced by private organizations, companies, or universities. To establish the extent of strategic evidence in the 2013 public school reform, we therefore take a closer look at the references.

To identify how strongly the school proposal relies on strategic evidence, we thoroughly examine the references. First, we exclude all references to academic articles. We then select all references that could be categorized as strategic evidence based on the publisher. Finally, we examine each reference individually to establish the report’s funding and commissioning (see Appendix 3 for selection criteria).

Our results show that 63 of the 231 references count as strategic evidence produced in Denmark.¹⁰ The reports were mainly commissioned by the Ministry of Education or the School Council and carried out by various actors and organizations, mostly EVA, the Agency for the School Council, or SFI. A picture can be drawn of the remarkable evidence composition where strategic and politically influenced evidence accounts for as much as 27.27%.

Summary: Types of Policy Evidence

According to the reference analysis, we found that the Danish public school reform relies on at least three types of evidence, which we categorize as academic, strategic, and IO evidence. The first two categories are inspired by Bjerre and Reimer (2014), who distinguish between strategic and academic evidence in their analysis of Danish teacher training programs.

Our analysis of the references in the five source documents shows that strategic evidence constitutes a large part of the references. Research agencies such as EVA and SFI are behind many of them. Academic evidence constitutes a significant percentage (16.02%) of the references, almost all of which are published in international journals. The international academic references focus mainly on leadership, a central part of the reform. The IO evidence has been a large source of inspiration and legitimization of the school reform. The analysis shows the OECD is the fourth most cited publisher. Furthermore, Aarhus University, which produces the Danish TIMSS, PIRLS, and OECD reports on education, is also among the most frequently cited publishers. The reform proposal references OECD data to justify the need for reform: Danish students do not have proficient reading skills, “have problems in mathematics,” and are not “good enough” in science (Danish Government, 2012, p. 7).

The bibliometric analysis reveals reference patterns that we explore further in the following two qualitative sections. The first research question (RQ1) relates to the knowledge sources underpinning the reform. We have shown that international academic evidence, IO evidence, and strategic evidence constitute a large part of the references. However, although the analysis enables us to identify reference patterns, it does not tell us much about the context or reasons for the patterns. Moreover, it reveals only evidence explicitly referenced in the policy papers. As shown, in Denmark’s case, the number of explicit references is significantly lower than for other Nordic countries, which raises the question of whether the 2013 Danish public school reform relied on other types of evidence than what appears in the policy paper references.

Our analysis also shows that international references constitute a large part of the references (36.36%) and that these are partly academic and partly stem from IOs, particularly the OECD. In the second qualitative section, we analyze the policy context in which the evidence base was formed and used (RQ2).

Evidence Beyond the References

Scholars have argued that commissions play an important role in policy preparation in Scandinavia today, especially in Norway and Sweden (Christensen & Holst, 2017; Holst & Molander, 2018; Steiner-Khamsi et al., 2020). In contrast, such a practice is seldom seen in Denmark (Christensen et al., 2009). With the 2013 public school reform, no commission was established to prepare the bill, and there were no reports or documentation of the evidence, except for the references in the proposal itself (INF1, INF2, INF4).

It is worth noting that the new government’s memorandum of understanding between the three participating parties expressed the need for public school reform some two years before the reform (Danish Government, 2011). This indicates that, even before the government won the election, the reform’s main contents had already been negotiated between the Social Democratic Party, the Socialist People’s Party, The Red-Green Alliance, and the Danish Social-Liberal Party. A former Parliament member explained that preparing the reform proposal “has a long history before the government was established. In the years leading up to the election in 2011 […] there was close coordination between the spokespeople of the opposition parties” (INF2) and discussions about the reform’s content, particularly the idea of a comprehensive school,¹¹ inclusive of all children, regardless of wealth, social background, or abilities (Imsen et al., 2017). If anything, these preparatory tracks indicate the ideological priorities of the new government. These priorities are not rooted in evidence as such but testify to the politics of policymaking. One informant went so far as to say,

In my opinion, an important point to make is that the Danish school reform is not characterized so much by research evidence. Its main elements are based on political and ideological thinking rather than a scientific foundation. And that is probably one of the reasons there are so few references. In addition, and especially in the pedagogical area, there has not been a strong tradition of assembling a common platform of research evidence. […]. So that is also part of the explanation. It is different in Norway and Sweden. (INF4)

The quantitative analysis of references is based on official policy evidence and an assumption that evidence constitutes the core building blocks of policy development, that is, that policy development is a fairly rational process. Instead, we argue that, in Denmark’s case, we need to adopt a more nuanced view of what constitutes policy evidence and to reflect on the role of ideology (i.e., politics) in the processes surrounding all applications of evidence in the preparation of the 2013 reform. To start, we argue that, in addition to academic, strategic, and IO evidence, two other types of evidence are fundamental in the preparations of the public school reform: stakeholder evidence and practice-based evidence.

Stakeholder Evidence

After the 2011 election, the school reform proposal work took three different paths. The first was a partnership with the stakeholders of public school, who met in three seminars between October 2011 and December 2012 to discuss the main topics of the reform:

The public school reform was described quite in detail in the government program, but it had to be translated into an actual bill. And […] a partnership with the main stakeholders was established. […] There were three seminars which were theme-based on the main topics […]—it was like inspiration for us to see if we could create—well, we didn’t call it a consensus conference, but it was building on the idea: If everybody was there and we were talking about the content [of the reform]—if we could say, “there are these five points that we agree on.” (INF2)

The seminars were organized by the Ministry of Education, and the six stakeholders were the teachers’ union, the school principals’ union, the union for early childhood teachers and youth education, the interest organization of the municipalities called Local Government Denmark, the students’ union, and the parents’ organization (INF2). Second, there was a working group across several ministries, including the Ministry of Education, the Ministry of Higher Education and Science, the Ministry of Economy and the Interior, and the Ministry of Finance. Third, an office dedicated to the reform work was established in the Ministry of Education (INF1).

According to an informant from the Ministry of Education, the culture of negotiation is typical for Danish education policymaking:

It’s funny with Denmark and Sweden—we [the Danish people] have a culture of being merchants and traders and adjusting things along the way, and we have flexibility and close collaborations, whereas Sweden is like a big industrial nation—they produce cars—they take the commission work and then they lead it into the government and decide something. … So, it’s like an industrial nation way of thinking, which they also adopt in their policy development. And in that case, we are more like merchants … we negotiate something that everybody has a share in. And during the process, we collect some knowledge and try to establish a broad ownership. (INF1)

On a more objective note, a recurring observation in the Danish history of education is that reforms of the public school system have involved broad cross-party compromises to secure political continuity and broad support from stakeholders to secure involvement (Coninck-Smith et al., 2015; Gjerløff & Jacobsen, 2014). The observations in this section are indicative of the political culture surrounding the reform process. Stakeholder evidence is found to play a role, even if the framing of stakeholder evidence can be considered an expression of political and ideological priorities. These observations gain currency when we consider the role of practice-based evidence.

Practice-Based Evidence: Bottom-Up and Experimental Projects

In August 2012, the Minister of Education Antorini launched the project called New Nordic School, inviting institutions from the educational sector (from pre-primary education to upper secondary education) to participate. The New Nordic School project was part of the ideological basis of the school reform and constitutes another type of evidence groundwork for the reform, which we denote practice-based evidence. The project’s three overall objectives—identical to those of the school reform—had been determined beforehand. In an annual meeting held in the town of Sorø, practitioners and experts discussed the challenges for public school, focusing on different themes, and determined ten objectives. The 2012 Sorø meeting centered around the New Nordic School project; later, a manifest was released with these ten points, as well as three overall objectives (Ministry of Education, 2012). By November 2012, over 350 institutions had applied to participate in the project (Olsen, 2012). The institutions committed themselves to initiate change processes inspired by the project’s manifest and objectives. Across the country, “networking days” were held where institutions discussed and developed their individual projects.

Another example of practice-based evidence involves the tradition for experimental project work in Danish public schools (Andreasen & Ydesen, 2015; Coninck-Smith et al., 2015; Gjerløff & Jacobsen, 2014). Since the 1920s, Danish school policy development has been characterized by experimental projects in schools and teacher training colleges, a practice later known as the Danish Model (Coninck-Smith et al., 2015, pp. 112–115). In the case of the 2013 public school reform, an informant explained that “in the content of the school reform, there wasn’t a single new thing which hadn’t been experimented in schools” (INF2). Examples of experimental projects with elements of the reform include a 2005 project about homework cafés in 15 Copenhagen schools (Vogt-Nielsen & Hansen, 2005) and a project about extended school hours (heldagsskole), tested in 12 schools across the country and evaluated in October 2012 (Rambøll, 2012).

As seen, increasing schooling hours was one of the most radical and problematized elements of the reform. Interestingly, extending the school day is not referred to as heldagsskole, and Rambøll’s (2012) evaluation is not mentioned in either of the source documents. The reason could be that the report concludes there is no significant evidence supporting the statement that extended school hours improve academic performance. Furthermore, a former member of the School Council was skeptical about Rambøll’s evaluation (INF4), suggesting the reform was based on the idea that

[…] if you wanted to help students with a weak social background, then the school would have to—to a larger extent—deal with all learning activities. And you had to leave fewer things up to the parents—less homework …. […] This is not based on any scientific evidence—this was definitely driven by ideology and visions […]. (INF4)

So far, our analysis has shown that practice-based evidence is an important part of the evidence base of the school reform. Other research has demonstrated that the role and involvement of practice-based evidence in education reforms has a long history in Denmark. However, our analysis has also revealed a strong tendency toward cherry-picking. One informant said, “The politicians are very interested in evidence—if it matches their opinions” (INF4). This point takes us back to the ideological and political elements in the reform process.

Using International Evidence in the Preparation of the School Reform

More than in the other Nordic countries, Denmark’s public school reform draws on international evidence sources, namely international academic articles and IO reports. The bibliometric analysis concludes that, in spite of a long tradition of Danish educational research, the source documents cite mainly international academic research. This could be because Danish researchers often publish in international journals, in which case the data are still considered international research. While this observation may hint at an error in the analysis, the qualitative interviews indicated there could be another reason for the “missing” Danish education research:

I still think there is something missing [in Danish education research]. There are a lot of things that have not been properly examined. … I thought it was a big challenge that we didn’t have any study saying, “Ok, what will the effect be if we do A or B in some area?” … And some things are just not properly scientifically examined and others are but […] in studies where it is difficult to conclude anything about causality and effect. (INF3)

Another informant expressed that there was just not enough Danish education research (INF1), or at least not the kind requested by decision-makers.

The evidence provided by IOs has impacted the preparation of the public school reform both directly and indirectly. Since the first ILSA in which Denmark took part was published, international experience has played an important role in Danish education policymaking (Andreasen, 2019). First, as Addey and Sellar (2018) noted, ILSAs often serve to legitimize the need for reform, as was also the case of the public school reform (INF2), where PISA was used to “diagnose” the problems (INF1). The public school reform came after more than a decade of debates about the state of the public school system and even an earlier attempt to propose school reform by the former Danish Government (2010). The ground was thus laid for the reform, a point that lends support to what Dobbin et al. (2007) refer to as the constructivist mechanism of policy diffusion, emphasizing the importance of public policies becoming socially accepted for their diffusion ability. However, the ILSAs themselves usually do not indicate any specific solution:

As soon as you need to find solutions, then you have to render it probable that you causally will get the effect that you claim that you will get. And how can you get that? You need to have some studies where you have looked at some changes, done some intervention, and documented that this intervention had the planned effect. That is causality. And [those studies] are typically carried out in a different context [than the Danish one]. Then you look at Ontario—they did this reform and had that result. And then we add this idea about causality—but you have to consider whether you can have the same effect in a Danish context. And you have to be extremely careful. (INF1)

Nevertheless, the OECD’s PISA program set the agenda for which countries to look to when preparing school reforms. This holds true not only for the 2013 public reform, but also for the school policy reforms and development in the years leading up to the reform:

Under the Anders Fogh¹² government, there was a lot of focus on Singapore—that was where everybody was going, and you had to learn from the miracle of Singapore. Then the wind was blowing towards west, and it was Ontario that we all were looking towards. (INF2)

Thus, although Singapore functioned as a positive reference society during the right-wing government, it was replaced by Ontario after the election, with some criticism of the past focus on Asia. As explained by a former Parliament member,

The Asian school system is very far from the Danish one, which makes it really difficult to translate. With the Canadian model … it was easier to be inspired by the way they have built their school reform—and transfer it into the Danish context. (INF2)

Policy Borrowing: Inspiration from Ontario

The inspiration from the reform of Ontario’s school system in 2003 is reflected in media reports, as well as in the interviews (Fuglsang, 2012; Møller, 2012; Søndergaard, 2012). In September 2011, the School Council (2012) visited Ontario to study the reasons behind its successful school reform in 2003. One year later, members of the Danish Parliament’s Committee for Children and Education (consisting of the spokespersons for education from each political party) went on a similar field trip to Ontario to meet with politicians, researchers, and teachers (Ministry of Foreign Affairs, 2012). Furthermore, Mary Jean Gallagher, Chief Student Achievement Officer in Ontario, was invited to the Danish Ministry of Education during the reform’s preparation, as well as to a 2012 New Nordic School meeting (INF2; INF4). Because the representatives of all political parties had visited Ontario, it became a common reference point for the negotiations, with politicians referring to examples from Ontario when arguing (INF2). Not only did Parliament members and the School Council go to Ontario, but individual schools and municipalities interested in the “miracle of Ontario” also went on study trips to discover the factors for success (Fuglsang, 2012; Jacobsen, 2012). Through this engagement with stakeholders, municipalities, and the public, general social acceptance of the need for school reform was established.

The 2013 public school reform is a classic example of policy borrowing, in which reference (to Ontario) and transfer (of educational policies) occur together, lending authority to the reform as a package of best practice policies (Phillips & Ochs, 2004; Waldow, 2017). The three goals of the Danish public school reform were nearly identical to the goals of the 2003 education strategy of Ontario: “raising the bar for all students, reducing achievement gaps and restoring public confidence in the publicly-funded school system” (Ontario Ministry of Education, 2010). However, while policy borrowing, in the terminology of Phillips and Ochs (2004, p. 778), begins with the stage of “cross-national attraction,” how such “attraction” comes into play must be addressed. As Simons and Voß (2018) have argued, policy solutions do not always follow policy problems. The concept of instrument constituencies accounts for the fact that “instrumental options and their consideration as viable solutions to certain problems does not necessarily […] follow the diagnosis of problems—neither chronologically, nor in terms of stages in a sequenced process of rational analysis and problem-solving” (p. 15). While the qualitative interviews pointed to a common understanding of the policy solution (the school reform) following a problem (Danish students’ poor performance) (INF1; INF2), the role of IOs, especially the OECD, has clearly had a large impact on both defining the problems and providing possible solutions. Hence, it was the performance of students in PISA that sparked the need for education reform in both Ontario and Denmark.

Concluding Discussion

In this chapter, we have explored the landscape of evidence behind the 2013 Danish school reform by combining bibliometric and qualitative analysis. This particular reform has been the subject of intense debates concerning whether it was based on any evidence or whether it was a purely political project, with no documentation of the effect of the major changes it induced in Danish public school. Asking what evidence underpinned the reform could be interpreted as support for one side of this debate. However, as we argued, we have chosen to employ a rather broad and inclusive definition of the concept of evidence. We therefore reviewed all types of knowledge sources as part of a landscape of evidence in the reform process.

Combining the bibliometric and qualitative analysis has enabled us to unravel multiple dimensions of the preparatory work behind the 2013 Danish school reform. The bibliometric analysis allowed us to examine the reference patterns of the official policy knowledge behind the school reform. A key finding pertains to the academic evidence, which played a significant role in the bibliometric composition of evidence. Our analysis clearly reveals a particular clinical type of academic evidence considered by decision-makers to be useful to education reform. The prevalent type of academic evidence can be characterized as evidence-based and “what-works.” This evidence provides hard-core data and/or is concerned with revealing best practices. Moreover, we saw how academic evidence does not stem from Danish education research in general, but from a very particular string of research. Theoretically, this valorization of a particular kind of academic evidence could be considered an expression of competition in policy borrowing (Dobbin et al., 2007). The 2013 school reform seems to revolve around an idea of international competition, living up to international standards, and importing best practices to quench a fear of falling behind (Krejsler, 2019). From a broader perspective, the thesis about the competitive state could help further understand the implications. According to Pedersen (2010), the competitive state mostly views education as an investment in the state’s competitive performance on the global level.

Furthermore, the analysis reveals that the reform largely references international sources and strategic evidence. Categories such as domestic academic literature or references to documents and data from Nordic countries are almost non-existent in the policy evidence explicitly referenced in the policy papers behind the school reform. The strategic evidence is provided by public research institutes and ministry units (e.g., EVA, SFI, AKF, the School Council) and private actors and universities (e.g., Rambøll and Aarhus University).

The number of explicit references in the policy papers behind the reform is significantly lower than in the other Nordic countries. However, as our analysis shows, this does not necessarily mean that the reform did not rely on any type of evidence. In the qualitative analysis, we constructed additional analytical categories to account for two types of policy evidence (i.e., stakeholder evidence and practice-based evidence) not apparent in the bibliometric analysis, but important to the reform’s preparation. The New Nordic School project allowed more than 350 institutions (e.g., schools, kindergartens) to work individually with the reform’s goals. Meanwhile, meetings with the key public school stakeholders took place at the Ministry of Education, where the reform was discussed and developed. However, the reform elements were also largely based on ideology. Work on the reform began even before the government took office in 2011, and the key elements were determined before the official work began that same year. This suggests that the official portrait of evidence behind the reform has a distinct cherry-picking flavor, what Pawson (2006, p. 7) called “policy-based evidence” to describe research that “travels straight from ideology to policy recommendations via the cherry-picking of evidence.”

The case of the 2013 Danish school reform is also a clear example of policy borrowing (Phillips & Ochs, 2004), and we saw how a constructivist mechanism of policy diffusion supports the strong inspiration and lessons drawn from Ontario. In defining the constructivist mechanism, Dobbin et al. (2007) contends how “policy makers play follow the leader by mimicking the countries that appear to be doing best” (p. 452). It is remarkable how politicians, civil servants (officials), and researchers associated with the general reform process and the specific bodies conducting the reform’s preparatory work were all swayed by the results and methods of the Ontario education system. Theoretically, the reform was underpinned by a powerful narrative constructed by key agents about the wonders possible in education following the Ontario recipe. But the reform process also highlights the OECD’s influence in both legitimizing the need for reform (through PISA) and indirectly pointing to the solution in terms of Ontario as a positive reference society.

Thus, the evidence landscape emerging from this analysis is very uneven. It consists of distinct plateaus of evidence-based and what-works types of evidence underpinned by significant policy diffusion from Ontario, OECD framings of what counts in education, strategic evidence, and a good portion of political ideology. The absence of broader—and often critical—Danish education research constitutes the valleys of the landscape. While stakeholder evidence and practice-based evidence were allowed voices in the reform process, the precise impact and significance of this type of evidence remain obscure.

Appendices

Appendix 1: Coding of Qualitative Interviews

Table 4.4 Coding of qualitative interviews

Table 4.5 Informants of qualitative interviews

Appendix 2: References Cited in at Least Two Source Documents

Table 4.6 16 references cited in at least two source documents

Appendix 3: Methods for Counting Strategic Evidence

Step 1

First, we selected all references identified as reports, books, government-issued decrees or guidelines, or Other. References categorized as academic articles were excluded since they cannot be strategic.

Step 2

Of these, we identified and isolated the references from publishers likely to have produced research commissioned by the government or a government institution.

Public research institutions, councils, or ministries

• EVA

• SFI

• AKF

• Municipal and Regional Evaluation Institute (KREVI)

• Danish Centre for Teaching Environment (DCUM)

• Skolerådet [School Council] or Agency for Skolerådets formandsskab [the School Council]

• Ministry of Education (Undervisningsministeriet)

• Ministry of Health (Ministeriet for Sundhed og forebyggelse)

• Ministry for Children and Social Affairs (Børne- og socialministeriet)

• Danish Government (Regeringen)

• Ministry of Higher Education and Science (Uddannelses- og forskningsministeriet)

• Ministry for Taxation (Skatteministeriet)

Private research institutions

• Rambøll

• TNS Gallup

• Radius Kommunikation

Universities and higher education institutions

• Aarhus University

• Copenhagen Business School

• University of Southern Denmark

• University College Nordjylland

Unions and interest organizations

• Lederne

Publishing houses

• Dafolo

• Academia

• Fremad

• Jurist- og Økonomforbundets Forlag

• Odense Universitetsforlag

Other

• Aarhus University Hospital

• Folkeskolen.dk

We thus have 138 references.

Step 3

Then we checked each of the 138 references individually to establish whether that particular report or book would be categorized as strategic evidence, for example, as follows:

• 1. All reports commissioned by the School Council are counted as strategic evidence. These are reports from EVA, SFI, and AKF.

• 2. Three reports from Rambøll were counted as strategic evidence. Two were commissioned by the Ministry of Education, and the third was commissioned by the Ministry of Finance.

• 3. Thirteen SFI reports were counted as strategic evidence, commissioned by a ministry, a commission, the School Council, or SFI itself. Since the independent research SFI conducts is provided for by the annual Finance Acts, the knowledge production is counted as strategy.

• 4. One SFI report was not counted as strategic because it was commissioned by Bikubenfonden, an independent, commercially operating foundation.

Result

Among the 231 references, we identified 63 strategic evidence references. References cited by multiple sources were counted only once.