Keywords

Evidence-based policy making (EBPM) is currently considered the most appropriate approach to public policy formulation. EBPM implies that both policy decisions and policy changes are increasingly grounded on scientific reasoning and research findings. The most enthusiastic advocates of the evidence shift in policy applaud the fact that policy decisions are increasingly informed by science in contrast to anecdotal information, tacit knowledge, public sentiment, and other forms of support. EBPM is also considered suitable for addressing ideological biases in decision making and for generating broader political consensus on the strategies that should guide policy formulation in different domains. In the 1990s, both policy and academic circles in the field of education began to embrace EBPM to address a legitimacy crisis faced by educational research (Head, 2008). To mitigate this crisis, educational research borrowed rationales and methods from research in medicine and other experimental sciences, with the expectation of promoting a more cumulative and transferable type of knowledge. Scholars and universities around the world have welcomed the EBPM transition because it makes academic research more socially valuable and visible and, importantly, because it generates new venues for research funding.

To a great extent, evidence-based policy creates opportunities for high quality, knowledgeable policy debates—that is to say, policy debates informed by causal beliefs, sound policy evaluations, and theoretically informed case studies. Politically speaking, EBPM is expected to be embraced in an international environment characterized by the resurgence of populist movements for which mistrust in science pays off politically and in an environment in which policy decisions on sensitive issues, such as global pandemics or climate change, can put population and the future of the planet at risk when those decisions are based on “alternative facts” (see Marten’s commentary chapter in this volume).

Nevertheless, despite the many ways that paying more attention to scientific evidence benefits policy making, EBPM still must be approached with caution, as it has its own limitations and risks for both policy research and practice. Furthermore, some assumptions behind enthusiastic versions of EBPM may not work as expected in real educational policy settings. EBPM discourses assume that, with the emergence of this policy approach, political values, ideologies, and normative beliefs are being sidelined from policy processes in favor of scientific knowledge and causal beliefs. They also assume that EBPM can make policy processes more open to external participation, especially to academic voices and sources. In this respect, EBPM implies that the locus of decision-making shifts from political authorities to networks of experts. Additionally, EBPM embraces a rationalistic ontology to policy transfer according to which policy learning is the main mechanism behind the traveling and selection of “best practices” in different policy contexts. This implies, to a degree, acknowledging that policy learning has gained centrality as a mechanism of policy transfer against other well-known transfer mechanisms (such as competition, coercion, or emulation).

The book you have in your hands problematizes these and other assumptions about EBPM through comparative research conducted in the Nordic education policy space. The book is part of a five-year research program called POLNET (Policy Knowledge and Lesson Drawing in Nordic School Reform in an Era of International Comparison) investigating the implementation of the EBPM approach in contemporary educational reforms in Nordic European countries. The POLNET study has brought together an international team of outstanding researchers from Nordic European universities and the Teachers College (University of Columbia), under the leadership of Kirsten Sivesind, Berit Karseth, and Gita Steiner-Khamsi. One of the main originalities of this trans-Atlantic research effort is the application of a bibliometric methodology to conduct comparative analyses of educational country reforms. That the chapters are grounded on this same methodology contributes to making the volume a cohesive piece of research and strengthens the comparability of the country cases. In the following sections, I reflect on the most important contributions of the study and their implications for future research.

Science and Politics in Educational Reform: Does Science Rule?

One of the most persistent critiques of EBPM is that it runs the risk of expertizing policy processes and of generating new forms of social closure within policy networks. According to Biesta (2007, p. 1), through the technification of certain policy debates, EBPM “restricts the opportunities for participation in educational decision making.” The POLNET study shows that this would not be the case in most Nordic countries, where EBPM challenges a corporatist form of governance and promotes the transition toward forms of network governance in which new actors—such as experts, interest groups, and civil society organizations—are considered in policy formulation processes. In fact, since Nordic governments have embraced EBPM in educational reform, parliamentary debates on education policy have intensified. Thus, the policy process is becoming more transparent, open, and political, while being configured through the involvement of a wider nebulous of actors, spaces, and devices for participation (including conferences, commissions, social media interactions, position papers, public blogs, and so on).

Pro-EBPM discourses celebrate the increasing influence of science over politics in policy making and the scientization of policy debates that were considered too political. However, the POLNET study demonstrates that, in policy processes, the relationship between science and politics is not necessarily a zero-sum game. The educational reforms analyzed in the book, in fact, show that the bigger presence of science has, indeed, evolved in parallel to more political participation and to the politicization of some stages of the reform processes.

Certainly, in real-life situations the separation between politics and science is precarious and far from linear. Chapter 10 in this text develops this idea eloquently; it graphically depicts the recursive and constant iteration between the political and scientific domains in education policy making (see Fig. 10.8). In the words of its authors, “information-gathering and consensus-building occur in practice at each and every step of the policy making process, blurring the line between science and politics.”

These findings confirm that scientific evidence is not ontologically distinguishable from political ideologies, normative beliefs, and bureaucratic control in policy-making processes. As illustrated by the POLNET study, the institutionalized forms of EBPM in different Nordic countries make clear that politics are at play in decisions that alter the influence of science over policy at many levels. To start with, by designing, regulating, and funding the EBPM architecture, political actors can dynamically condition the policy outcomes of evidence-based deliberation processes. Decisions regarding the configuration and funding of the agencies in charge of drafting green papers, literature reviews, and position papers are key to understanding the dialectal relationship between science and politics in policy processes, as are decisions about who can integrate the agencies or advisory commissions (including the identification of criteria regarding disciplinary background or research experience). As principal-agent models would predict, these decisions have important implications for the outcomes of EBPM schemes, as well as for the actual role of independent research in educational reform.

However, politics also condition and, to an extent, restrict scientific influence over policy in more indirect ways. EBPM assumes that policy makers, especially in periods of uncertainty and crisis, are receptive to scientific evidence on best practices. Thus, policy makers increasingly welcome the role of knowledge brokers and experts when selecting new policy instruments or calibrating those that already exist. Yet, they are also aware of the type of knowledge that is more useful to them in both political and policy terms and have at least notions of who can provide them with this kind of knowledge more effectively. Specifically, policy makers tend to resort to research sources where they can obtain straight-forward answers to frequently complex policy problems—sources that fit within what Roger Dale (1994) defined as a “problem-solving” type of research. Policy makers usually rely on technical knowledge to address “what works” questions and are more inclined to base their assessments on quantitative sources rather than on qualitative research and data, with the latter often seen as more interpretative and biased. In doing so, they reproduce ideational frameworks of what type of knowledge counts as policy-actionable evidence. Discursive selectivity, hence, becomes a subtle mechanism for understanding how politics shapes EBPM processes and privileges certain research approaches over others.

Not surprisingly, many of the references cited in the green papers published in the Nordic countries analyzed fit within a mainstream “school effectiveness research” approach—an approach that pays more attention to measure the school effects on student learning than to multiple causal and structural explanations of educational outcomes (Parra, 2018). International scholars with a pragmatic approach to educational research who prescribe clear policy guidelines on how to organize effective schools and classrooms, such as John Hatti and Michael Fullan, are more often cited in the policy papers reviewed in the POLNET study than scholars with a more critical or theoretical understanding of education policy matters.

Nonetheless, a finding as unexpected as it is important from the POLNET study is that references to international and national educational researchers are relatively scarce in the policy documents used to back educational reform. Even though the green and white papers published in the Nordic countries analyzed cite numerous pieces of evidence, academic evidence in the form of publications in peer-reviewed journals is rather marginal. In contrast, other types of domestic publications (gray literature, ad hoc literature reviews, blog posts, and so on) are much more present. The response to the question “whose knowledge is used in educational reform in Nordic countries?” is not straightforward; indeed, it seems easier to respond to the question of whose knowledge is not used. Educational research produced in national universities is not strongly represented in most of the country cases.

Externalization and the Role of International Organizations

In contrast to the marginal presence of academic publications produced by national universities, reports and papers published by the Organization for Economic Co-operation and Development (OECD) are widely cited in the policy documents produced by state agencies and research institutes in all Nordic countries. The OECD is not only the most cited international source in the corpus of policy reports gathered: the bibliometric analysis conducted by the POLNET team shows that OECD reports enjoy a high level of in-degree centrality in the networks of publications produced in relation to the different country contexts. This confirms the OECD’s authority in the education policy realm, and in Nordic European education in particular (Grek, 2017; Ydesen, 2019). Still, as acknowledged in Chap. 11, the bibliometric analysis helps to test the reputation of the OECD in the Nordic region but does not capture, on its own, the level of penetration of the OECD in the Nordic education policy space neither the nature of this international organization’s influence in national policies.

OECD reports can be cited in green or white papers for multiple reasons, not all of which are related to the ideational influence of this international organization—or to how much national policy actors have learned from or within OECD initiatives. In the context of educational reforms, governments may cite OECD sources to legitimize their policy options, and impregnate these options with international status. Moreover, as the authors of Chap. 6 wittily observe, referring to external sources such as the OECD in a country like Iceland, where policy networks are rather compact, is simply a way to “avoid being accused of nepotism.” Countries can also cite OECD papers to technify political debates considered too controversial in the domestic policy arena (Rizvi & Lingard, 2009; Browes & Verger, 2020). In fact, as the authors of Chap. 12 observe, the number of references to the OECD increases in those countries in which education reforms have been more contentious. To a degree, externalization intensifies when more sensitive or controversial reforms are being debated in national policy spaces. All of this corroborates what Martin Marcussen observed some time ago: the fact that countries increasingly resort to and cite OECD knowledge products says more about the increasing international legitimacy of this international organization than about its policy influence (Marcussen, 2004).

The OECD has a longstanding legitimacy in Nordic countries. This international organization has been present in education policy deliberations in the region since the 1990s; it has conducted education policy reviews in several countries repeatedly; and has involved them in different training and research initiatives. However, knowledge dissemination and policy evaluations are not the only, or the most important, ways in which the OECD has affected national policy debate in Nordic Europe. As several chapters in this volume reflect, the OECD has influenced substantive policy change through the “scandalization” that came with the Programme for International Student Assessment (PISA) (cf. Steiner-Khamsi & Waldow, 2018). Specifically, because of the PISA effect, many Nordic countries have adopted performance-based accountability, standardized testing instruments, and curricular standards that were not in place before (see, for instance, Camphuijsen et al., 2020). International performance data has had a significant affective type of impact as a catalyst of educational reform (see Sellar & Lingard, 2018). In other words, the adoption of the reforms as described resulted more from governments’ political and, to some extent, “emotional” reactions to poor (or, rather, lower than expected) international performance on PISA than to scientific-based deliberations about policy.

These policy dynamics do not neglect that EBPM may play a role in decisions on how to calibrate “governance by numbers” instruments. What they highlight is that the substantive decision on whether to adopt the “governance by numbers” approach and related policy instruments in Nordic countries is more related to the between-countries competition and to the anxieties over educational performance that international large-scale assessments such as PISA have triggered.

Is There a Nordic Model of EBPM?

The POLNET study also examines education policy transfer within the Nordic region. The study findings do not support that policy transfer dynamics within the region have shaped country reforms. Specifically, the number of references to other countries’ sources in educational reform processes is quite low in the documents analyzed: between 1%–7% of all references (see Chap. 12 in this volume). Even references to Finland, a reference educational system at the global level, are very low in the Nordic region. Nevertheless, again, these results need to be considered with caution because policy transfer mechanisms may operate via more subtle forms and informal relations than through explicit mention in policy documents.

At the same time, this book has much to say about the Nordic model of education as such, as well as about the predominant model of EBPM that is being enacted within the region. For external observers, the image of educational systems in Nordic countries includes policy principles such as equity, comprehensiveness, decentralization, publicness, and teachers’ professional autonomy. However, as several chapters in this book highlight, the Nordic education model is no longer as cohesive. Growing marketization, school choice demands, and performance-based accountability are transforming the Nordic educational model (with Sweden as the regional outlier in advancing these trends). The reforms analyzed in this book provide good examples of the tensions that co-exist within the Nordic model. These reforms tense educational systems by including policy measures that, on one hand, strengthen the knowledge base of teachers but that, on the other hand, challenge teachers as the main source of educational expertise. They also include measures that promote equity and inclusion and, at the same time, strengthen competitive and performative attitudes within schools. Overall, contemporary reforms in Nordic countries have stressed educational systems by reinforcing a post-bureaucratic governance approach that is re-scaling power in education upwards (from teachers to local and national authorities).

Nordic countries are also known internationally for their well-functioning democracies and respect for political rights. Despite the limitations of EBPM highlighted in the book, the education policy-making processes described reflect and attempt to advance more transparent educational reforms that are open to external voices. As an educational policy researcher from a different world region with a more immature democracy, I find remarkable how transparent and, to some extent, Cartesian policy formulation processes in the Nordic region seem to be, and the accessibility of the sources of knowledge on which reforms are to be grounded. Citations may be selective and used for legitimation purposes. State agencies and research institutes, in their role as knowledge brokers, can act as gatekeepers and exclude certain voices. But explicit effort to back policy change with evidence is evident. Thus, the opportunity exists to tell policy makers whether they have misinterpreted research findings or that they have obviated important publications on specific topics. In contrast, in contexts where policy making seems to more closely follow the garbage-can model than the EBPM model, the opportunities to mobilize knowledge for policy change purposes are much more restricted.

The POLNET study finds some regional citation patterns in the reports produced by state agencies and research institutes in preparation for educational reform. The first is the above-mentioned marginal presence of academic research produced by national universities; the second is the predominant presence of national sources (in contrast to regional sources); and the third is the important presence of OECD reports and other knowledge products from this international organization. The contributors to the book also identified differences regarding the EBPM approach in the Nordic countries. Bibliometric analysis is a useful tool to make sense of the particularities of the EBPM process that each country has followed and identify variation regarding the number of written sources used in policy papers, the concrete percentage of academic and international sources cited, and the level of externalization on international organizations.

The case of Denmark is interesting in this regard. The POLNET study shows that education reform in this country has been backed by a limited number of written sources but has been heavily influenced by the OECD (see Chap. 4 in this volume). However, what makes this case somewhat exceptional in the Nordic context is that not only has EBPM contributed to the internationalization of the policy process, but the EBPM infrastructure itself has become highly influenced by international sources, specifically, through the direct involvement of UK players, such as the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI) of the University College London-Institute of Education.

To Conclude: Future Research Directions

Advocates of EBPM assume that this policy approach favors more open and participatory policy processes in which deliberation based on academic sources intensifies. They also assume policy learning is the main mechanism of policy transfer. In this chapter, I explained how the POLNET study challenges these and other assumptions through original comparative research conducted in the Nordic education policy space. The study shows that the enactment of EBPM has not flattened the terrain of policy making, nor is it a synonym for the increasing influence of science over politics. In the Nordic region, EBPM translates into a complex policy process in which science and politics interact at different levels but in such a way that the political domain retains bureaucratic control over the policy process and its main outcomes. The study also demonstrates that, more than policy learning, dynamics of legitimation and competition are key to understanding some of the most substantive policy changes Nordic countries have undergone in the educational realm in the last two decades.

The POLNET study addresses numerous research questions, but as happens with good research, it also generates new questions and ideas about future lines of inquiry. In numerous chapters of the book, the authors show that scientific knowledge (in particular, the education research produced in national universities) is not a main source of political authority in national education reforms. Overall, the research products cited in policy documents seem to play a bigger role in legitimizing policy decisions than in shaping them. However, under what conditions would policy makers be more genuinely receptive to scientific evidence? Can changes in the public regulation of state agencies and research institutes contribute to promote more pluralistic approaches to evidence use? Nonetheless, to be fair, the challenges of EBPM implementation do not only originate in the field of politics. Indeed, the very knowledge base of educational policy research challenges the comprehensive use of research for policy making. The many themes on educational policy in which research evidence is inconclusive (such as the costs and benefits of pedagogic innovation, school choice, performance-based accountability, public-private partnerships, etc.) is conducive to both knowledge selection biases and the instrumentalization of research for political purposes. To test this statement, future research could analyze whether EBPM is more genuine and rigorous in relation to those reform domains with more cumulative and conclusive research results.

Another question to unpack EBPM processes is: at what stage of the policy process can research evidence become more influential? As we have seen, in mainstream EBPM frameworks, the soundest and most welcome scientific evidence comes from “problem-solving” research that mainly informs policy formulation at the policy design stage; in particular, this is research that focuses on school-level effects over learning and packages and sells policy solutions in a rather prescriptive way. However, other types of research can also play a role in policy processes at different stages, although this role tends to be less acknowledged. Numerous research initiatives in education are better equipped to problematize existing situations in the educational realm than to prescribe straight-forward solutions. This, for instance, is the case of research that focuses on identifying the problems that policy makers need to address, or on constructing policy priorities and preferences that are not central in public agendas yet. Future studies on knowledge uses in policy processes could, thus, pay more attention to the role of research evidence, not only at the policy formulation stage, but also in terms of agenda-setting and problematization.

The fact that the boundaries between the sites of knowledge production and policy making are being blurred, as the POLNET study reveals, means that the corporatist state may not have been totally left behind in Nordic countries. To some extent, more than entirely over, the corporative tradition has mutated, and the interest groups to which the state resorts to in policy processes have been transformed or, in some cases, replaced. Something I find intriguing in this regard is the role of teachers’ unions and the political representation of teachers more broadly speaking. Have teachers’ unions and other teachers’ representatives lost political centrality in the current scenario? Have they been sidelined by the growing centrality of research institutes, state agencies, or new interest groups? More research on the changing role of teachers’ unions within EBPM frameworks, and on how and whether EBPM has transformed the functions, power, and nature of collective action within unions, would be welcome as well.

Overall, publications, citations, and references constitute a good entry point for the study of the knowledge base of educational reform. Nonetheless, as the POLNET study makes clear, publications and references are only the tip of the iceberg—the most visible and empirically tangible resource—of more profound and determining political and knowledge mobilization dynamics over policy processes. The interest in bibliometric analysis is obvious; however, combining it with other methods can boost the potential of this methodology. The most informative chapters in the book are, in fact, those that combine bibliometric data with interview data or that situate bibliometric findings within broader research frameworks. Future research can combine different forms of social network analysis (including those drawing on bibliometric analysis) with a more qualitative understanding of the EBPM phenomenon. In-depth interviews or observational methods along the lines of political ethnography approaches (see Papanastasiou, 2020) can provide insights into how policy makers make sense of scientific evidence, the type of evidence they find more approachable, and the forms of knowledge to which they resort to construct notions of best practice in public policy.