Abstract
Evidence-informed change (EIC) has gained attention recently because it is seen as a lever to enhance the effectiveness and sustainability of educational change. Important questions are: how is EIC conceptualized, what counts as evidence, and what factors can support EIC in practice? Because of the complexity of EIC, we aimed to understand these factors from a systems perspective. Different parts of the educational system (e.g. policy, practice of teachers’ and school leaders, research) are interrelated and need to be aligned for effective and sustainable change. Based on our scoping review we propose a model that conceptualizes EIC, identifies and defines different sources of evidence, and discusses influencing factors describing a system's readiness and capacity for EIC. Our results are an important step forward in understanding and supporting EIC in practice and developing targeted policy. This article also defines a common ground for future research, bringing together insights in an integrated framework of evidence-informed change.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Educational organizations and educators around the world are constantly seeking to improve and advance education, so as to provide students with the best education possible. That effort itself takes place in a changing world—from global warming, pandemics, and wars to shifting social realities. Educational change is therefore more a daily reality than a choice. Or, as Stoll (2020) stated, this “isn’t a time for cruise control (…) we have to shift gear” (p. 428) to achieve deep and meaningful change for students.
Unfortunately, educational change is often not sustainable (Askel-Williams & Koh, 2020; Cohen & Mehta, 2017). Changes to education are complex, dynamic, and emerge through interactions in context (Spillane, 2012). Rapid digital, economic, and societal changes lead to complex problems that are influenced by many interdependent factors and are hard to disentangle. Finding effective and sustainable solutions requires a deep understanding of problems and their causes (Ramley, 2014).
In today’s world, educators need to find solutions for these complex problems in a rapidly changing context. These are not easy processes for schools to navigate. Using evidence to untangle and manage complex problems has shown promise, as it is crucial for developing targeted approaches in a complex and dynamic environment such as education (Hargreaves & Shirley, 2009). More specifically, using evidence helps with critically analyzing assumptions and identifying possible blind spots related to change initiatives (Vanlommel, 2022), for example, by identifying the problem within the school organization—instead of relying on a gut feeling—and showing whether the focus of the change matches the problem that needs to be managed. In that way, evidence use can help break existing patterns and routines by providing educational organizations with new insights to drive their complex change processes.
However, different interrelated factors influence the complexity of educational change. Effective educational change must attend to structure and culture, as well as to capacities in different parts of the system, in alignment with the outside world (Bates, 2013). Problems cannot be solved by fixing one element. Inequality in education, for example, cannot be solved by developing a new vision or setting up a new curriculum if the underlying norms and beliefs are not addressed. Finegood (2012), amongst others, warned against reductionist approaches that focus on changing one part of the system. Educational change should be studied and developed based on the interdependencies “between individuals, organizations, or levels in the system” (Finegood, 2012, p. 125), and solutions should target improvements to processes instead of outcomes. In complex environments, change initiatives that are approached in terms of the interdependencies between parts of the system are more likely to lead to effective and sustainable change (Finegood, 2012; Fullan, 2006). To understand the different elements in schools and the broader educational system that influence complex change initiatives, a systems theory lens thus seems inevitable.
From a systems perspective, schools can be seen as dynamic collections of tightly or loosely coupled subsystems or of people who constantly interact and co-evolve (Behrens & Foster-Fishman, 2007; DiMaggio & Powell, 1983; Monahan et al., 1994). Accordingly, when this perspective is applied to educational change, different stakeholders and different elements in schools mutually influence the change process. That means that teachers or school leaders hold important roles in initiating, diffusing, and adopting educational change, which will be affected by their competences and the extent to which they use their energies, passion, and time to promote that change (Barrenechea et al., 2023). Moreover, teachers or leaders may influence change beyond the boundaries of their teams or schools and positively affect the larger system through “building bridges” (Barrenechea et al., 2023). Teachers or leaders can act as change agents and cross boundaries within and across schools. This is important, because the complexity of educational change makes it hard for individual teachers, leaders, and schools to address complex problems and challenges individually (Stoll, 2020).
In sum, the changes education faces are growing more and more complex, challenging educators to gain a deep understanding of problems and solutions and how different elements in their context are interrelated. Using evidence to inform change—or evidence-informed change (EIC), as we will call it from here on—can help untangle the wickedness and interrelatedness of problems. Thus far, however, there seems to be a lack of conceptual clarity regarding what counts as evidence. Moreover, scholars have shown that schools still find it challenging to use multiple sources of evidence in their change processes (e.g., Austin & Claasen, 2013; Grol & Wensing, 2004; Vanlommel, 2022; Wubbels & Van Tartwijk, 2017). To enhance the effectiveness of educational change that aims to enhance teaching and learning, in this review we want to develop a conceptualization of EIC (RQ 1) and to identify factors that stimulate EIC (RQ 2). To do so, we will review multidisciplinary literature on this topic, as other sectors such as health and management also have longstanding traditions of working with evidence. As sustainable change requires a systems perspective (Finegood, 2012; Fullan, 2006), the potentially influential factors will not be investigated as independent elements. In this study, we focus on explaining the interdependencies of these factors in relation to the system’s readiness, capacity, and alignment regarding EIC.
Theoretical framework
Evidence: What is it and how can it be used to support educational change?
Attention to the use of evidence to drive practice in schools and other organizations has been steadily increasing. The main reason for this trend is the need to adapt to the rapidly changing environment we live in, and to adapt organizational outcomes accordingly (e.g., Austin & Claasen, 2013; Grol & Wensing, 2004). Evidence is used in this way across the globe—from Australia to the USA, and from the UK to South Africa—and across different sectors—such as health, social work, management, and education.
“Evidence” is a term that authors use easily, but its meaning is often left unclear. From an etymological perspective, “evidence” can be traced back to its Latin root, “evidentia”, which is a compound of “ex-”, meaning out of, or from, and “videre”, meaning to see. Based on that etymology, evidence started out as referring to something that you have seen (clearly) yourself. According to frequently used dictionaries, evidence is now seen as “facts, information, documents, etc.” (Cambridge Dictionary) or “the facts, signs, or objects” (Oxford Dictionary) that give you reason to believe that something is true (both dictionaries). When talking about evidence, one thus needs to keep in mind that evidence has a dual role; it informs and supports drawing a conclusion. Additionally, evidence needs to have a certain “quality” for it to be called evidence. The quality of evidence depends on one’s confidence in the evidence and the evidence’s validity for the use that is being made of it. Validity in (qualitative) research requires careful collection and documentation of evidence that allows transparent insight into its source. Validity also refers to the extent to which the data collected are appropriate for answering the guiding research question (FitzPatrick, 2019). Searching for criteria that help us define the quality of evidence, we say that evidence needs to be suited for its purpose (Does the evidence to be used match its intended aim?) and needs to be transparent (Can the evidence be investigated and discussed by other people?). Integrating the etymology of the word evidence and the idea of the quality of evidence, in this article we define evidence as an information source that is transparent, appropriate for its intended purpose, and permits drawing a conclusion.
When investigating evidence use in education, a shift over time can be observed. At first, organizations were focused on having their work be evidence-based. This indicated that research, and more specifically scientific or formal research, was used as the source of evidence to base practices on. Research was long considered as the best available evidence, as it was systematically collected, empirically grounded, and clearly showed what works for improving practice (e.g., Davies, 1999). A recent study by Groß Ophoff et al. (2023) showed that schools that reported more research-informed activities demonstrated better results in the average student performance assessment at the end of the school year.
However, implementing scientific evidence in practice has been considered problematic; practitioners seldom use formal research (Grol & Wensing, 2004). This has to do with the intended users’ skills as far as meaningfully engaging with research, but also with the way research is produced, executed, and communicated (Biesta, 2007; Graves & Moore, 2018). Research “rarely translates into simple, linear changes in practice” (Browns & Greany, 2018, p. 2). Both the context in which one works and the context in which the research is conducted influence how research might be applicable to practice. Simply stated, “what works” does not necessarily work for each school (Biesta, 2007; Bryk et al., 2015; Graves & Moore, 2018). An approach is needed in which the technical question of “what works” could be addressed in close connection with normative and contextual questions about what is desirable (Biesta, 2007).
Attention to the use of other evidence sources increased. The importance of using data was stressed more often (cf. Schildkamp, 2019). Data are pieces of information that are systematically collected and organized to represent some aspect of the organization (Wayman et al., 2012). Data are collected within the context of the organization, and therefore contextualized. An example is feedback from stakeholders, such as students. A third source of evidence that was found to be important is expertise or personal knowledge, defined as the knowledge and competences needed to perform a certain job, gathered over the years that people perform their jobs (Smith, 2005). Sometimes, personal knowledge is so strongly grounded in experience that it cannot be fully expressed (Tschannen-Moran & Nestor-Baker, 2004). This type of evidence helps professionals understand what data and research mean within their context (Vanlommel et al., 2018). However, the use of data or personal knowledge as evidence has disadvantages as well. Using only personal knowledge often leads to biased judgements and stereotyping (Brookhart, 2003; Feinberg & Shapiro, 2009; Vanlommel & Schildkamp, 2019). Moreover, decisions based on personal knowledge often provide superficial, practical solutions for complex situation (Schildkamp & Kuiper, 2010). Data alone do not provide solutions, either. Without personal knowledge to understand what data mean and what data are needed, data can remain meaningless (Brown et al., 2017) or educators can be overwhelmed by all the data that surround them (Vanlommel, 2022).
Because the use of any one source of evidence has disadvantages, the evidence-informed movement emerged. To overcome the obstacles associated with individual sources of evidence, the importance of combining evidence sources is stressed in the evidence-informed movement (Brown et al., 2017; Vanlommel, 2022). The rapid and complex changes in education call for deeper questioning of the assumptions and beliefs that frame both the problems and the solutions pertaining to educational change. Using different sources of evidence can help bring actual causes to the surface and interrupt routines and patterns that hinder the effectiveness and sustainability of change (Vanlommel, 2022). Evidence-informed in this article therefore refers to the use of a combination of multiple sources of evidence to inform organizational processes.
Scholars mainly focus on evidence-informed practice, decision-making, or policy, but it is important to gain a better insight in the vital role evidence can play in change processes in organizations Evidence-informed change might strengthen the effectiveness, sustainability and innovative capacity of change initiatives in education. Up till now educational change often fails to break existing patterns and routines (Bates, 2013). A reason may be that educational change is ideologically based (Verbiest, 2021) or mostly implemented based on intuition or "gut feeling," without a clear vision or strategy (Vanlommel et al., 2018). Due to a common lack of time and money, innovations are also rarely evaluated and not further institutionalized (Kirschner et al., 2004). Wubbels and Van Tartwijk (2017, p. 9) found that “evidence on the potential effects of innovations (…) was not used at all”. As a result, the outcomes of educational change disappear and the impact remains unclear (Verbiest, 2021). Using evidence to monitor and evaluate processes of change can contribute to the effectiveness, and the systematic approach of EIC might also support the sustainability.
It thus seems crucial that educational organizations are willing and able to use evidence suitable for their context and combine it with other evidence sources to support their change processes. If they can to do that, schools can adapt to our fast-changing society and focus on managing complex problems in a sustainable way. Evidence-informed change (EIC) thus has a lot of potential. But how can EIC be understood and supported in (educational) organizations?
A systems approach to supporting evidence-informed change
Educational change is complex and dynamic (Hargreaves, 2003). Processes of educational change are influenced by many interrelated factors, such as the socio-economic context of students and parents, the resources of schools, relations between teachers and students, and growing ethnic diversity. Fast-paced changes in the environment, for example, and the growing influence of artificial intelligence, also make it impossible to follow blueprints and linear change models. The dynamic and complex nature of educational change requires more complex and adaptive approaches to change.
Complexity science offers insights into the challenges associated with understanding such complex system dynamics (Castellani & Hafferty, 2009) that need to be adaptive and responsive (Paina & Peters, 2012). The interrelatedness between different elements in schools, universities, and the broader educational system requires an understanding of the interdependencies of different parts of the system (Lanham et al., 2013). Berta et al. (2014) used a whole-system approach to change to investigate what factors contribute to the development and best use of evidence and how evidence-informed innovations could diffuse beyond the boundaries of the subsystems in which they originate. They found three key elements that strengthen the effectiveness and sustainability of change in social systems: the system’s readiness, capacity, and alignment. Below, we will discuss these elements in relation to (evidence-informed) change, as they will be used as part of the organizational framework in our literature review.
The system’s readiness for change
A system’s readiness for change is an important precursor of effective and sustainable change (Jones et al., 2005). Translated to the educational context, this concept refers to the overall readiness of schools, teachers, and leaders to make changes in an evidence-informed way. The levels of readiness can vary across people, teams, or subsystems, and can change over time (Wang et al., 2023). Readiness for change is a precondition for accepting and adopting a particular plan (e.g., to make evidence-informed change), purposefully interrupting the status quo and moving forward (Wang et al., 2023). Readiness does not just refer to how ready educators are ready to implement EIC developed by others. An important aspect is educators’ readiness to read research, collect data, combine and weigh evidence, and seek new ways of and ideas for adapting their practices. Creating readiness for change might require “unfreezing” existing mind-sets and creating motivation and energy for EIC (Cummings et al., 2016).
Schools’ and educators’ readiness for change are influenced simultaneously by different factors (Holt et al., 2007). At the individual level, readiness reflects the extent to which teachers and leaders are cognitively and emotionally ready to change. It is influenced by beliefs among educators that (a) They are capable of implementing a proposed change (i.e., change-specific efficacy); (b) The proposed change is appropriate in their school context (i.e., appropriateness); (c) School leaders are committed to the proposed change;, and (d) The proposed change is beneficial (i.e., personal valence; Holt et al., 2007). Emotions that come with change, such as enjoyment, anxiety, or anger, impact an educator’s readiness for EIC (Ittner et al., 2019).
At the organizational level, the research evidence about system readiness for change in schools is still limited. Agnew and Van Balkom (2009) found that cultural readiness for change is influenced by the congruency between espoused and enacted values, between the values of (evidence-informed) change and the broader (school) vision, and by the political forces that articulate EIC as a priority. Further, trust in schools supports teachers’ readiness for EIC by reducing change-related resistance and stress (Zayim & Kondakci, 2015).
It is difficult to implement changes if the people who are most affected are not involved (Armenakis & Harris, 2002). Teacher agency in change processes can contribute to a greater commitment to the change and will help teachers make meaning of change (Datnow, 2020; Monahan et al., 1994). However, educational change does not result from individual actions, but through interactions within an organizational context (Datnow & Stringfield, 2014). Readiness for change in schools is often characterized by processes of negotiation, interaction, resistance, and compromise seeking and is strongly determined by the level of participation in the change process (Wang et al., 2023). Participation and collaboration do not always have a positive effect on teachers’ readiness for change, however. If teachers have the feeling that they are asked to participate or collaborate without having any real influence on the change process, they are likely to resist the change (Hargreaves & Shirley, 2020). This might also affect future readiness, as teachers might feel cynical about subsequent change processes—also known as aversion to future change.
The system’s capacity for change
Even though (school) organizations are faced with similar forces for change, their responses can differ (Hargreaves, 2023). The extent to which teachers, teams, and schools are able to enact evidence-informed change largely depends on the system’s capacity for change. This refers to a combination of attributes at the individual, collective and organizational levels that allow (school) organizations to implement changes without compromising daily practice or subsequent change processes (Meyer & Stensaker, 2006). Capacity building is an essential component of any successful change process or improvement strategy (Fullan, 2006).
At the organizational level, the presence or absence of resources, structural or cultural elements, may hinder or support change. Fox example, structures can sustain disciplinary silos and inhibit co-construction that is based on a broad array of evidence from different perspectives and stakeholders. A culture of openness toward interdisciplinary and transdisciplinary approaches and toward other systems of thought and sources of knowledge is a driver for change (O'Brien et al. 2013).
The system’s capacity for EIC also calls upon the collective capacity of professionals working together to make evidence-informed changes to improve practice through mutual support, accountability, and challenge (Fullan, 2006; Harris, 2011). Collective dispositions, shared feelings, emotions, and relations among team members define a social system’s capacity for change (Fullan, 2010). For example, teachers’ collective efficacy (a team’s feeling that together they can handle change) greatly influences the acceptance, implementation, and sustainability of change initiatives (Monahan et al., 1994; Vanlommel et al., 2023). Collective capacity may also be understood from a social capital perspective (Hargreaves & Fullan, 2015), insofar as it includes collaboration, shared decision-making, collective responsibility for teaching and learning, collaborative inquiry, and mutual trust.
Individual capacity for EIC demands a range of knowledge and dispositions. For example, educators need to be able to make sense of evidence and use the evidence to improve teaching and learning in meaningful ways (Datnow & Hubbard, 2016). Educators often need new knowledge and skills to collect, combine, and use evidence such as data (Gummer & Mandinach, 2015).
The system’s capacity for change is more than just the capability to change important elements of teaching and learning that need to be improved; insight into what is already good and needs to be protected is also necessary (Monahan et al., 1994). Capacity for change requires an understanding of strengths, weaknesses, and the balance between change and stability that the context requires.
The system’s alignment for change
The system’s alignment for change refers to the extent to which the strategy, structure, and culture of a system are aligned with the environment and the change initiative to create a synergetic whole that makes change possible. System alignment is not an absolute measure. It can range from complete opposition or misalignment to perfect synergy and alignment (Semler, 1997). The alignment or misalignment of a system greatly influences the diffusion, adaption, and acceptance of change. (Mis)alignment is a consequence of similarities or differences in understandings, expectations, goals, beliefs, values, and opinions among stakeholders engaged in various stages of educational change.
The broad concept of organizational alignment is comprised of distinct but interrelated aspects. For one thing, there needs to be alignment between the strategy, goals, and different activities related to educational change within an organizational structure. This is also called structural alignment. When processes and activities are well-aligned, the output of each process in the subsystem contributes to goal attainment for the larger system (Swanson, 1994). In education, the achievement of the goals of the individual teacher should contribute to the goals of the team to which the teacher belongs, and the realization of team goals should in turn contribute to improvement of the school. With the aim of enhancing EIC in schools, for example the goals of evidence use need to be aligned between teachers, teams and the school organization.
Further, there should be a match between the norms, beliefs and routines that define the school culture. This is what is called cultural alignment. Cultural values have a strong effect on the behavior of people in a change process (Hargreaves, 2003) and act as a filter through which change is interpreted (Kelchtermans, 2009). Agreement between the cultural values of the school and the (implicit) values that underly (evidence-informed) change greatly influences the success or failure of change initiatives. Finally, there needs to be alignment between the inside and the outside world (i.e., environmental alignment). Change initiatives should be aligned with both the internal goals of schools and the needs and demands of the broader societal, economic and political environment (Sahlberg, 2021). The effectiveness and sustainability of EIC depends on the degree of congruence, consistency, or fit both within and between each of these components of alignment (Moss et al., 2022).
Present study
The use of a combination of evidence sources is valuable for managing the complex problems education faces, as it helps with developing targeted approaches to educational change in a complex and dynamic environment (Hargreaves & Shirley, 2009). Using evidence (information sources that are transparent, appropriate for their intended purpose, and that permit drawing a conclusion) can inform and support change processes in education. Schools that aim to change programs or practices of teaching and learning need a clear understanding of context, problems and solutions. Further evidence can help monitor and evaluate (lack of) progress and results in processes of change. Evidence-informed change (EIC) can help school organizations with realizing sustainable change. While EIC is an emergent phenomenon, there seems to be a lack of conceptual clarity about what counts as evidence. Moreover, scholars have shown that schools still find it challenging to use multiple sources of evidence in their change processes (e.g., Austin & Claasen, 2013; Grol & Wensing, 2004; Vanlommel, 2022; Wubbels & Van Tartwijk, 2017). To provide clarity around the concept of EIC and insight into factors that can help schools stimulate EIC, our literature review focuses on the following questions:
-
1.
How can evidence-informed change (EIC) be conceptualized? (RQ 1)
-
2.
What factors stimulate evidence-informed change (EIC)? (RQ 2)
Because change initiatives that take an approach based on the interdependencies between parts of the system are more likely to lead to sustainable change in complex systems (Finegood, 2012; Fullan, 2006), a whole-system approach will be used in our literature review. Based on our theoretical framework, we created a research model (see Fig. 1) that will be used as a theoretical framework to organize our findings.
Method
As our study aims to clarify a key concept in the literature (RQ 1) as well as characteristics influencing EIC (RQ 2), and aims to identify and analyze knowledge gaps, we conducted a scoping review (Munn et al., 2018). A scoping review is “a form of knowledge synthesis that addresses an exploratory research question aimed at mapping key concepts, types of evidence, and gaps in research related to a defined area or field by systematically searching, selecting, and synthesizing existing knowledge” (Colquhoun et al., 2014, pp. 1294). Therefore, databases were searched to identify relevant studies, studies were selected based on inclusion criteria, experts were consulted, and the selected articles were then analyzed and synthesized with a data-extraction form (Arksey & O’Malley, 2005; Colquhoun et al., 2014; Levac et al., 2010; Tricco et al., 2018). The steps will be discussed in detail below.
Literature search
The literature was searched and publications collected for this review in October 2022. A library professional was consulted to advise on our literature search. We used a combination of two search strings. The first search string included the search terms “evidence-informed” and “evidence informed”. The second search string included the search terms “change”, “reform”, “improvement”, and “innovation”. The combination of these search strings was used in four online databases: ERIC, PsycINFO, Scopus, and Web of Science. These databases were chosen to include articles from several fields and sciences. The search was limited to sources published from 2000 onwards. Table 1 presents an overview of the number of studies obtained for each search string and the combination of search strings per database. The total number of studies retrieved was 7168 (Step 1 in the Flow Chart of the Search Process in Fig. 2). After removing duplicates, 5769 articles remained (Step 2 in Fig. 2).
Literature selection
To ensure the relevance of the selected literature, we applied four criteria during the selection process. First, the article had to be published in a peer-reviewed journal. Second, the article had to be empirical. Third, the article had to be about using more than one source of evidence (e.g., articles only about stimulating research use in organizations were not included). Fourth, the article was about (an) evidence-informed change (process), not just about implementing an evidence-informed tool, program, intervention, or the like (e.g., articles about implementing evidence-informed programs against bullying in schools were not included).
We reviewed the total of 5769 articles manually by reading the titles and abstracts (Step 3 in Fig. 2). After this step, 45 articles remained. We read the full texts of these remaining articles to determine whether they matched our criteria (Step 4 in Fig. 2). After this step, three articles remained. It was quite striking that just three empirical articles focused on evidence-informed change as such.
Because literature selection in a scoping review is not linear, but rather an iterative process that involves searching the literature, refining the search strategy, and reviewing articles for inclusion (Levac et al., 2010), we discussed our search strategy again. This discussion led to a refinement in two areas. First, we decided that we were too strict in the exclusion of some of the articles. More specifically, we excluded some articles because they were about the content of an innovation, not about the process of change. Carefully reading the articles, we concluded that they were about stimulating the use of evidence within organizations. This could also be seen as an evidence-informed change process: organizations were moving toward using evidence to inform their work, which calls for changes on different levels (Austin & Claassen, 2008). We therefore decided to re-analyze the excluded articles and include those about stimulating evidence use in organizations. Second, we only wanted to include empirical articles to have a sound foundation for our findings related to RQ 2. For the conceptualization of evidence-informed change, this was not a prerequisite. We therefore decided to include conceptual articles as well, but to use them only to answer RQ 1.
Based on this discussion, we re-reviewed our collected articles by reading titles and if necessary abstracts. We read 52 additional full texts, resulting in the inclusion of 16 additional articles (Step 5 in Fig. 2).
Additional literature selection
The set of articles that remained after reading the full texts (Step 5 in Fig. 2) was presented to an expert peer group for consultation (cf. Arksey & O’Malley, 2005; Colquhoun et al., 2014). The group consisted of three experts in both the field of innovation and taking an evidence-informed approach. We invited them in a digital setting and discussed the final set of articles with them. We asked them for additional articles that were missing in this set and for insights beyond those in the literature. Additionally, all reference lists from the articles were analyzed (snowballing). Based on these two steps, we added four additional articles (Step 6 in Fig. 2). This led us to our final set of 23 articles.
Analysis and synthesis of the literature
Our full final set of articles was used for answering RQ 1, which focused on conceptualizing evidence-informed change processes. As the final set included both conceptual and empirical articles, we decided to use only the 14 empirical studies to answer RQ 2, which focused on the factors influencing to which change processes were evidence-informed change processes. The main reason for this decision was that we wanted to present an empirical foundation for our findings related to what influences evidence-informed change.
To analyze data from these 23 articles and gain in-depth insight into evidence-informed change conceptualizations and processes, we designed a data-extraction form. The use of such a form ensured that comparable data could be gathered from the selected publications (Arksey & O’Malley, 2005; Petticrew & Roberts, 2006). The form contained 22 questions (Questions 1–14 focused on general characteristics of the studies, Questions 15–18 focused on the evidence sources used, Question 19 focused on the definition of change, and Questions 20–22 on the influential factors). We used direct quotes from the articles to fill in the data-extraction form to stay faithful to what was written by the authors, for objectivity purposes.
After filling in the data-extraction form, we coded the data per research question, using sensitizing concepts derived from our conceptual model. This provided us with a general sense of reference and guidance in approaching our data (e.g., G. Bowen, 2006). Both authors coded part of the data and discussed the codes together until consensus was reached.
Results
Below we present the general characteristics of the included studies, and then address the research questions on the conceptualization of evidence-informed change (RQ 1) as well as factors stimulating evidence-informed change (RQ 2).
General characteristics of the included articles
In Table 2, an overview of the included articles is presented. Three initial findings could be reached by looking at their general characteristics. First, 14 empirical and 9 conceptual articles were included, indicating that information about evidence-informed change in general is scarce, but empirical data on the matter is scarcer. Second, most studies were conducted in Europe (n = 10), followed by North America (n = 9). Only a few studies (n = 4) were conducted elsewhere, in Australia and New Zealand. Third, the bulk of the studies were conducted in the health sector (n = 9), followed by education (n = 7) and social work (n = 4). Three studies were conducted in other sectors. Fourth, of the 23 included studies, 12 studies were published after 2015, four studies between 2010 and 2015, and six studies before 2005—with the oldest publication coming from 2004. This indicates that evidence-informed change is a relative newcomer on the scene.
Conceptualization of evidence-informed change (RQ 1)
To analyze the way in which the included articles conceptualized EIC, we used our full final set of articles (n = 23) and analyzed them in two steps. First, we looked at how change was conceptualized. Second, we looked at how evidence that could inform change was conceptualized. The results are described below.
Conceptualization of change in the included articles
All but two articles in this review focused on the realization of organizational change, as the aim was to stimulate all actors in the organization to use evidence for change. The other two articles (Brown et al., 2017; Klinner et al., 2015) mainly focused on change at the individual (practitioner) level. Six studies specifically focused on evidence-informed change (Austin, 2008; Bamber, 2015; Brown & Greany, 2018; Brown et al., 2017, 2018; Evans, 2020). Others, for example, mentioned that they wanted to stimulate the use of evidence-informed practice in making changes (Bowen & Zwi, 2005; Brown, 2017; Hodson & Cooke, 2004; Hole et al., 2016; Klinner et al., 2015; Lwin & Beltrano, 2022; Lwin et al., 2022; McEwen et al., 2008; Newhouse et al., 2007; Peters et al., 2016), stimulate evidence-informed (organizational) decision-making for change (Bowen et al., 2009; Conway et al., 2019; Hardy et al., 2015; Rousseau, 2020; Semenic et al., 2015), or evidence-informed inquiry for change (Timperley & Parr, 2007). Nevertheless, all articles aimed at or advocated for achieving change (also described as transformation or improvement) through stimulating the use of evidence. Improving practice was the goal of change in all articles, independent of the sector concerned.
Conceptualization of evidence in the included articles
In Table 3, the conceptualization of evidence per article is given. The concept of evidence was rarely defined. Only the study by Bowen and Zwi (2005) did so. The authors mostly used examples to describe what they meant by evidence that could inform the change process that was central in their studies. Hardy et al., (2015, p. 1) mentioned “existing evidence” as an example of evidence. As this description was too vague, we could not code this in a specific category. All other examples could be categorized into the following categories: research, data, expertise, and other.
Research
In 21 out of the 23 studies, research was used as a source of evidence. Most authors (n = 12) were very brief in describing what they meant by research (Austin, 2008; Bamber, 2015; Bowen et al., 2009; Conway et al., 2019; Evans, 2020; Jansson & Forsberg, 2016; Klinner et al., 2015; Lwin & Beltrano, 2022; Lwin et al., 2022; McEwen et al., 2008; Rousseau, 2020; Semenic et al., 2015; Timperley & Parr, 2007). Others (n = 3) added descriptive but broad terms such as “soundly conducted” or “the best available, current, valid, and relevant” to describe what they saw as research evidence (Hodson & Cooke, 2004; Hole et al., 2016; Peters et al., 2016). In four studies, authors described what types of research they considered as evidence, including both quantitative and qualitative research findings, as well as findings from systematic research conducted by researchers and practitioner inquiry (Bowen & Zwi, 2005; Brown, 2017; Brown & Greany, 2018; Brown et al., 2017; Newhouse et al., 2007).
Data
In 18 out of the 23 studies, data were mentioned as a source of evidence. As was the case with research, most authors (n = 12) were very brief in describing what they considered as data, for example, by describing them as “feedback”, “preferences”, “sources” or “views” from different stakeholders (Hodson & Cooke, 2004; Jansson & Forsberg, 2016; Lwin & Beltrano, 2022; Lwin et al., 2022; McEwen et al., 2008; Peters et al., 2016; Semenic et al., 2015), or mentioning specific types of data, such as administrative, evaluation, financial, organizational, or student achievement data (Austin, 2008; Bamber, 2015; Brown & Greany, 2018; Rousseau, 2020; Timperley & Parr, 2007). Others (n = 6) were brief, too, but used a combination of both descriptions (Bowen & Zwi, 2005; S. Bowen et al., 2009; Brown et al., 2017; Brown et al., 2018; Evans, 2020; Newhouse et al., 2007).
Expertise
In 21 out of the 23 studies, expertise was discussed as a source of evidence. This was described as “expertise” (Brown, 2017; Brown & Greany, 2018; Hardy et al., 2015; Hodson & Cooke, 2004; Hole et al., 2016; Jansson & Forsberg, 2016; Newhouse et al., 2007), “expert knowledge” (Bowen & Zwi, 2005), “expert opinion” (Rousseau, 2020), “practitioner wisdom” or “practitioner sources” (Bamber, 2015; Evans, 2020). Others (n = 8) used a combination of those terms and other descriptions such as “competencies”, “concerns”, “tacit knowledge”, “experiences”, and “professional judgment” (Austin, 2008; Bowen et al., 2009; Brown et al., 2017; Lwin & Beltrano, 2022; Lwin et al., 2022; McEwen et al., 2008; Peters et al., 2016; Semenic et al., 2015). In two articles (Brown et al., 2018; Conway et al., 2019), expertise was not defined, but they described how it was gained and used in the process. Although different concepts were used, all authors focused on evidence derived from experience gained by an individual in their studies or career.
Other
In 11 articles, other sources than research, data, or expertise were mentioned as evidence. These were case context (Conway et al., 2019; Lwin & Beltrano, 2022; Lwin et al., 2022), feasibility (Semenic et al., 2015), politics (Bowen & Zwi, 2005), and stakeholder values (Bowen et al., 2009; Brown et al., 2017; Hole et al., 2016; Jansson & Forsberg, 2016; Lwin & Beltrano, 2022; Lwin et al., 2022; McEwen et al., 2008; Peters et al., 2016).
Factors stimulating evidence-informed change (EIC) (RQ 2)
Our second research question aims at understanding factors that can stimulate EIC. For this purpose, we used only the empirical articles included in our final set (n = 14). Our research model was based on a whole-system approach, arguing that the factors influencing EIC are interdependent and contribute to the readiness, capacity, and alignment of the whole system for EIC. We used these concepts to organize our results and have clustered all identified factors under these three main categories. An overview can be found in Table 4. In the subsequent section, we discuss each category with its identified factors in detail.
Factors supporting the system’s readiness for EIC
System readiness refers to the overall readiness of people, processes, and resources in systems and is a critical precursor to successful EIC. In 11 studies, factors related to system readiness were mentioned. First, our literature review identified factors describing people’s readiness for change. This starts with people's awareness that EIC is valuable and necessary, and that multiple sources of evidence are needed to provide a clear and complete picture of performance and outcomes at all levels of the system (Brown, 2017; Conway et al., 2019; Hole et al., 2016; Lwin et al., 2022; McEwen et al., 2008; Semenic et al., 2015). Individuals must also understand the relevance of EIC to their work and feel motivated to use evidence (Hardy et al., 2015; McEwen et al., 2008). Finally, the readiness of the system for EIC depends on the shared understanding among individuals that all stakeholders have a role in EIC (Brown, 2017; Conway et al., 2019; Semenic et al., 2015).
Second, factors related to organizational processes contribute to the system’s readiness for EIC. Implementation needs to be carefully sequenced and paced (Conway et al., 2019). Moreover, systems (e.g., an organization’s intranet) must be set up to provide (advice on) evidence in general and EIC in particular (McEwen et al., 2008). Guidelines need to be written or updated as well (Conway et al., 2019). Additionally, organizational processes for training and education for practitioners related to evidence use and EIC need to be set up (Newhouse et al., 2007). Finally, adding expectations related to the use of evidence and EIC to job descriptions would also contribute to system readiness (Newhouse et al., 2007).
Third, certain resources also define a system’s readiness for EIC. In addition to commonly described resources such as time (Bowen et al., 2009; Brown, 2017; Conway et al., 2019; Hardy et al., 2015; McEwen et al., 2008) and adequate finances to get started with EIC (Brown, 2017; Hardy et al., 2015), other resources were also identified. It is important for professionals to have access to various sources of evidence (Bowen et al., 2009; Brown, 2017; Conway et al., 2019; McEwen et al., 2008; Newhouse et al., 2007). Resources in the form of tools to support the use of evidence contribute to system readiness (Bowen et al., 2009; Newhouse et al., 2007), as do professionals who can support and assist in obtaining and using evidence (Bowen et al., 2009).
Finally, two other, more general factors were found to contribute to system readiness for EIC: supportive organizational structures (i.e., clear responsibilities and the existence of networks; Bowen et al., 2009) and an organizational policy and climate that supports the use of evidence and EIC (Bowen et al., 2009; Brown & Greany, 2018; Conway et al., 2019; Timperley & Parr, 2007).
Factors contributing to the system’s capacity for EIC
System capacity refers to individual, collective, and organizational capacity for EIC. In all 14 empirical studies, factors related to system capacity were mentioned as important influences for successful EIC. First, factors related to individuals' capacity for EIC were identified. It seems important to have sufficient knowledge and analytical competencies to use evidence for EIC (Conway et al., 2019; Hole et al., 2016; McEwen et al., 2008). Individuals must also feel confident to engage with evidence, as well as believe they are capable of applying EIC in their work (Conway et al., 2019; Hole et al., 2016; Lwin et al., 2022; McEwen et al., 2008). Moreover, when individuals act as change agents for EIC by driving EIC in their organization, this would in turn benefit the organization's capacity for EIC (Hole et al., 2016).
Second, our literature review revealed factors related to the organizational capacity for EIC. For example, it is important for organizations to collect, offer and disseminate relevant evidence to professionals in a timely manner (Bowen et al., 2009). Here, establishing networks or professional learning communities in the organization could play an important role (Brown & Greany, 2018; Hole et al., 2016). Clear communication about EIC is also a factor that contributes to organizational capacity (Bowen et al., 2009; Conway et al., 2019; Hardy et al., 2015; Semenic et al., 2015). Communication should be about not only the use and availability of evidence, but also the process of change: what is the current state of affairs, what problems are being encountered and what is the next step? Finally, a system’s capacity for EIC depends on the organization's ability to prioritize and focus (Bowen et al., 2009; Hardy et al., 2015; McEwen et al., 2008); otherwise people are "too busy dealing with the urgent that they can’t get to the important" (Bowen et al., 2009, p. 95). This can lead to a general feeling among staff that there are too many organizational issues to address, and that EIC is therefore not a priority.
We found little evidence on collective factors that influenced a system’s capacity for EIC. Our review did identify factors specifically related to leadership capacity for EIC. Therefore, we dropped ‘collective and added ‘leadership’. Leaders must not only have the necessary knowledge and skills to implement EIC (Hodson & Cooke, 2004; Lwin et al., 2022; Timperley & Parr, 2007), they must also have the competencies to communicate clearly about EIC (Bowen et al., 2009; Hardy et al., 2015; Hodson & Cooke, 2004; McEwen et al., 2008). In addition, their professional credibility, and the importance they place on using evidence also contribute to the system's capacity for EIC (Hodson & Cooke, 2004). Furthermore, it is important that leaders act as role models for EIC and behave accordingly (Conway et al., 2019; Hardy et al., 2015; Hodson & Cooke, 2004; Lwin et al., 2022), for example, by applying EIC themselves, being (closely) involved in it, or by acting as an EIC “opinion leader” or champion. Leaders who provide positive feedback encouraging EIC also make a positive contribution to EIC in their organization (Brown, 2017; Hardy et al., 2015; Hodson & Cooke, 2004; Lwin et al., 2022). When leaders direct, by specifically asking professionals to use evidence (Jansson & Forsberg, 2016) and providing opportunities for professionals to expand their knowledge and skills related to evidence-informed change (McEwen et al., 2008), this also influences EIC in the organization. In addition, it is important that leaders properly manage and provide the resources needed for EIC (Newhouse et al., 2007).
Factors contributing to the system’s alignment for EIC
System alignment refers to the similarities or differences in understandings, expectations, goals, beliefs, values, and opinions among stakeholders or subsystems engaged in various stages of EIC. In eight studies, factors related to a system’s alignment for EIC were mentioned. First, studies showed the importance of structural alignment. This involves connection between the goals of different change initiatives within the organizational structure. The goals of EIC need to be linked with the broader organizational goals (Brown & Greany, 2018; Hardy et al., 2015; McEwen et al., 2008; Timperley & Parr, 2007). Initiatives should be taken to bridge differences regarding EIC between different parts of the systems (e.g., teams, departments). Furthermore, organizing interactions between leaders and practitioners could contribute to structural alignment within the system (Hodson & Cooke, 2004; McEwen et al., 2008). Scheduling moments, such as during a team meeting, to regularly discuss ideas related to and the progress of EIC also contributes to structural alignment (Hardy et al., 2015).
Second, factors related to cultural alignment within the system influence EIC. To this end, the included articles mainly mentioned the importance of a shared vision regarding EIC (Hodson & Cooke, 2004; Lwin et al., 2022).
Third, factors related to environmental alignment are important for EIC. Organizing interactions between users and "providers" of evidence, and between researchers and practitioners can improve this type of alignment (Hodson & Cooke, 2004; McEwen et al., 2008; Peters et al., 2016; Semenic et al., 2015). Moreover, encouraging interprofessional collaboration across disciplines (Conway et al., 2019) and establishing and participating in networks that focus on the use of evidence and EIC (Hodson & Cooke, 2004; McEwen et al., 2008) can also help with environmental alignment.
Overview
An overview of the results of our literature review as incorporated in our research model is presented in Fig. 3. This figure provides a conceptualization of EIC as an outcome of our review study. The influencing factor ‘collective’ from our initial research model is replaced by ‘leadership’ based on our results.
Discussion
The use of evidence can be a strong lever enhancing the effectiveness and sustainability of complex educational change, yet there has been little shared understanding on what it means to change in an evidence-informed way, what counts as evidence and how using it can be supported in practice. A striking first finding of our review is the limited conceptualization of what counts as evidence-informed and what sources of evidence can be used to guide change in schools. Our study showed that many sources of evidence can be used to support educational change, although clear conceptualizations were often lacking in the articles. Data were sometimes described very broadly, such as sources from different stakeholders, where other others delimited it to specific sources such as administrative or evaluative data. Although the definitions of expertise were often vague, all authors in our study saw experience as a form of tacit or expert knowledge that is gained by an individual throughout their studies or career. Results showed different types of research that can be taken into account, including both quantitative and qualitative research findings, as well as findings from systematic research conducted by researchers and practitioner inquiry as long as it is soundly conducted, valid, relevant and the best research available. We also found few sources of evidence that could not be fit in the categories data, research or expertise, for example stakeholder values or politics. Depending on the context, other sources of information might be relevant to use in the decision process. We started from a broad view on evidence, defining it as an information source that is transparent, appropriate for its intended purpose, and permits drawing a conclusion. Differing decision processes in differing contexts might start from a critical reflection on what counts as evidence.
Our results have led to a refined model that enabled us to clarify EIC as the combination of evidence sources (research, data, expertise) that are suitable for the intended purpose and transparent to inform a change process that is focused on improving practices for its stakeholders (e.g. students, teachers). Stimulating EIC in education should be approached from a systems perspective, focusing on the system’s capacity and readiness for change and alignment.
An educational system’s capacity to make evidence-informed change largely depends on people’s awareness, ability and motivation to use different sorts of evidence to get a rich, fine-grained and unbiased picture of the question and context. Given the complexity and dynamics of educational change, we argue that it is impossible to rely on only one source of evidence, such as research. Choosing and using research that is appropriate for the context requires data, for example, indicators of the socio-economic status of students in the educational system at hand. Transforming data into useable knowledge requires expertise to understand what data mean in the context (Vanlommel & Schildkamp, 2019). Expertise does not stand alone either, but needs to be understood in the light of local data, for example, about the history of the school. This has led to a definition of EIC as the combination of evidence sources (research, data, expertise), that are suitable for the intended purpose and transparent to inform a change process that is focused on improving practice for its stakeholders (e.g., students, teachers).
A system’s readiness for change largely depends on individuals’ awareness of the merits of EIC and the pitfalls of decision bias when relying on one-sided evidence. Educational leaders should model critical thinking and support strategic partnerships between researchers and practitioners. Educational systems also need the capacity to change in an evidence-informed manner. A school’s capacity depends not only on individual skills and dispositions, but also on a team’s collective efficacy regarding EIC and a culture of support, reflection and constructive conflict. Leaders need resilience and persistence, because EIC breaches the DNA of education. Promoting EIC asks educators to challenge their assumptions, become aware of their implicit knowledge and inferences and change their routines. It is likely to make people feel insecure or questioned as a professional. Asking educators to change habits can make them feel as if they have been doing things wrong for years (Kelchtermans, 2009). Evidence-informed change requires a shift in mind-set and culture, which is likely to bring insecurity and resistance. Leaders have the important task of providing supportive feedback and showing consideration for teachers’ feelings. In times of change, leaders’ influence on teacher change is largely exerted through the relational path (Vanlommel et al., 2023).
Organizations need the capacity to collect, provide and disseminate relevant evidence among staff in a timely manner, but most important, they need to prioritize. Otherwise, people are so busy dealing with the urgent that they cannot get to the important. Enhancing evidence-informed change in education is not easy and requires deliberate attention to and alignment between the interrelated elements of the educational system, for example, between practice, research and policy, between structure and culture or between teachers, students and parents.
In systems such as education these interrelated elements often exist only to a limited extent. Enhancing evidence-informed change in education will require targeted interventions and a stronger alignment between research, policy and practice. Policymakers and educational leaders need to be very clear about the reasoning and rationale behind changes that create understanding and motivation. Clear communication can create a feeling of urgency and insight into the need to change and why evidence is important. Understanding the risk of relying on one-sided or unsubstantiated evidence helps to create motivation for EIC. In many educational systems around the world, educators have little insight into human judgement and decision-making theory. Starting with high commitment toward their students, building on long-standing experience, and being absorbed by everyday practice’s fast and intuitive decision-making is often more of a habit that a deliberate choice. When educators learn to understand the strengths and pitfalls of information sources and how evidence can help improve practice, intrinsic motivation for EIC is likely to grow. Good examples and collegial support can also help in that regard. Innovation champions for example with high individual capacity for EIC can support and stimulate EIC in their schools. They can disseminate knowledge about EIC, facilitate understanding and provide targeted support to peers. Most importantly, they can also serve as boundary-crossers between different parts of the system, by facilitating knowledge brokerage and alignment of values and meaning (Vanlommel, 2022). Networks within and between different parts of the educational system might provide the supportive infrastructure needed for this (Poortman et al., 2022).
We must mention some limitations of this study. Despite the broad search, including studies from different fields, only 14 articles could be used to identify factors influencing EIC. Most of these were qualitative case studies. The recent publication dates show that EIC is an emerging field in which most insights are built on conceptual articles. These findings stress the need for more empirical research to understand and explain how EIC can be enhanced in practice. Future research should include more quantitative empirical studies with broader and more generalizable insights on EIC. Despite our search for articles that combined more then one source of evidence, although many articles promised a broad view in the title or abstract, evidence-informed change was often limited to use of research evidence. Given that different sources of evidence are needed and intertwined in practice, future research should start from a broader view on EIC to move the field forward. Our conceptualization can be a valuable starting point in that regard.
For practice, our findings highlight the need to invest in schools’ and educators’ readiness and capacity for evidence-informed change. Stronger alignment between different parts of the system, between policy and practice and between researchers and practitioners is therefore needed. For policy, the supporting factors we identified from a systems perspective can guide targeted policy development.
Change processes in education are complex and dynamic. The motivation and capacity to use a broad array of evidence can help intentionally interrupt routines and overcome individual and/or cultural decision bias, opening the door for effective and sustainable educational change.
References
References included in the analysis are marked with an asterisk (*).
Agnew, M., & VanBalkom, W. D. (2009). Internationalization of the university: Factors impacting cultural readiness for organizational change. Intercultural Education, 20(5), 451–462.
Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32.
Armenakis, A. A., & Harris, S. G. (2002). Crafting a change message to create transformational readiness. Journal of Organizational Change Management, 15(2), 169–183.
Askell-Williams, H., & Koh, G. A. (2020). Enhancing the sustainability of school improvement initiatives. School Effectiveness and School Improvement, 31(4), 660–678.
*Austin, M. J. (2008). Strategies for transforming human service organizations into learning organizations: Knowledge management and the transfer of learning. Journal of Evidence-Based Social Work, 5(3–4), 569–596.
Austin, M. J., & Claassen, J. (2008). Implementing evidence-based practice in human service organizations: Preliminary lessons from the frontlines. Journal of Evidence-Based Social Work, 5(1–2), 271–293.
Austin, M. J., & Claassen, J. (2013). Implementing evidence-based practice in human service organizations: Preliminary lessons from the frontlines. Evidence for child welfare practice, 271–293.
*Bamber, V. (2015). Mixing decks: Frameworks for master’s scholarship. Higher Education Quarterly, 69(3), 221–236.
Barrenechea, I., Beech, J., & Rivas, A. (2023). How can education systems improve? A systematic literature review. Journal of Educational Change, 24, 479–499.
Bates, A. (2013). Transcending systems thinking in education reform: Implications for policy-makers and school leaders. Journal of Education Policy, 28(1), 38–54.
Behrens, T. R., & Foster-Fishman, P. G. (2007). Developing operating principles for systems change. American Journal of Community Psychology, 39, 411–414.
Berta, W., Virani, T., Bajnok, I., Edwards, N., & Rowan, M. (2014). Understanding whole systems change in health care: Insights into system level diffusion from nursing service delivery innovations–a multiple case study. Evidence & Policy, 10(3), 313–336.
Biesta, G. (2007). Why “what works” won’t work: Evidence-based practice and the democratic deficit in educational research. Educational Theory, 57(1), 1–22.
Bowen, G. A. (2006). Grounded theory and sensitizing concepts. International Journal of Qualitative Methods, 5, 12–23. https://doi.org/10.1177/160940690600500304
*Bowen, S., Erickson, T., Martens, P. J., & Crockett, S. (2009). More than “using research”: The real challenges in promoting evidence-informed decision-making. Healthcare Policy, 4(3), 87–102.
*Bowen, S., & Zwi, A. B. (2005). Pathways to “evidence-informed” policy and practice: A framework for action. PLoS Medicine, 2(7), 600–605.
Brookhart, S. M. (2003). Developing measurement theory for classroom assessment purposes and uses. Educational Measurement: Issues and Practice, 22(4), 5–12.
*Brown, C. (2017). Further exploring the rationality of evidence informed practice: A semiotic analysis of the perspectives of a school federation. International Journal of Educational Research, 82, 28–39.
*Brown, C., & Greany, T. (2018). The evidence-informed school system in England: Where should school leaders be focusing their efforts? Leadership and Policy in Schools, 17(1), 115–137.
*Brown, C., Schildkamp, K., & Hubers, M. D. (2017). Combining the best of two worlds: A conceptual proposal for evidence-informed school improvement. Educational Research, 59(2), 154–172.
*Brown, S. T., McGreevy, J., & Berigan, N. (2018). Evidence-informed improvement through collaborative professional integration. New Directions for Teaching and Learning, 2018(155), 55–64.
Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Harvard Education Press.
Castellani, B., & Hafferty, F. W. (2009). Sociology and complexity science: A new field of inquiry (p. Springer)
Cohen, D. K., & Mehta, J. D. (2017). Why reform sometimes succeeds: Understanding the conditions that produce reforms that last. American Educational Research Journal, 54(4), 1–47. https://doi.org/10.3102/0002831217700078
Colquhoun, H. L., Levac, D., O’Brien, K. K., Straus, S., Tricco, A. C., Perrier, L., Kastner, M., & Moher, D. (2014). Scoping reviews: Time for clarity in definition, methods, and reporting. Journal of Clinical Epidemiology, 67(12), 1291–1294.
*Conway, A., Dowling, M., & Devane, D. (2019). Implementing an initiative promote evidence-informed practice: Part 2—Healthcare professionals’ perspectives of the Evidence Rounds Programme. BMC Medical Education, 19(1), 1–17.
Cummings, S., Bridgman, T., & Brown, K. G. (2016). Unfreezing change as three steps: Rethinking Kurt Lewin’s legacy for change management. Human Relations, 69(1), 33–60.
Datnow, A. (2020). The role of teachers in educational reform: A 20-year perspective. Journal of Educational Change, 21(3), 431–441.
Datnow, A., & Hubbard, L. (2016). Teacher capacity for and beliefs about data-driven decision making: A literature review of international research. Journal of Educational Change, 17, 7–28.
Datnow, A., & Stringfield, S. (2014). Working together for reliable school reform. In Crespar findings (1994–1999) (pp. 183–204). Psychology Press.
Davies, P. (1999). What is evidence-based education? British Journal of Educational Studies, 47(2), 108–121.
DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48, 147–160.
*Evans, T. R. (2020). Improving evidence quality for organisational change management through open science. Journal of Organizational Change Management, 33(2), 367–378.
Feinberg, A. B., & Shapiro, E. S. (2009). Teacher accuracy: An examination of teacher-based judgments of students’ reading with differing achievement levels. The Journal of Educational Research, 102(6), 453–462.
Finegood, D. T. (2012). The importance of systems thinking to address obesity. In A. Drewnowski & B. J. Rolls (Eds.), Obesity treatment and prevention: New directions (Vol. 73, pp. 123–137). Karger Publishers.
FitzPatrick, B. (2019). Validity in qualitative health education research. Currents in Pharmacy Teaching and Learning, 11(2), 211–217.
Fullan, M. (2006). The future of educational change: System thinkers in action. Journal of Educational Change, 7(3), 113–122.
Fullan, M. (2010). All systems go: The change imperative for whole system reform. Corwin Press.
Graves, S., & Moore, A. (2018). How do you know what works, works for you? An investigation into the attitudes of senior leaders to using research evidence to inform teaching and learning in schools. School Leadership & Management, 38(3), 259–277.
Grol, R., & Wensing, M. (2004). What drives change? Barriers to and incentives for achieving evidence-based practice. Medical Journal of Australia, 180, S57–S60.
Groß Ophoff, J., Brown, C., & Helm, C. (2023). Do pupils at research-informed schools actually perform better? Findings from a study at English schools. Frontiers in Education, 7, 1–18.
Gummer, E. S., & Mandinach, E. B. (2015). Building a conceptual framework for data literacy. Teachers College Record, 117(4), 1–22.
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. Bmj, 336(7650), 924–926.
*Hardy, A. K., Nevin-Woods, C., Proud, S., & Brownson, R. C. (2015). Promoting evidence-based decision making in a local health department, Pueblo City-County, Colorado. Preventing Chronic Disease, 12(100), 1–10.
Hargreaves, A. (2003). Teaching in the knowledge society: Education in the age of insecurity. Teachers College Press.
Hargreaves, A. (2023). Leadership from the middle: The beating heart of educational transformation. Taylor & Francis.
Hargreaves, A., & Fullan, M. (2015). Professional capital: Transforming teaching in every school. Teachers College Press.
Hargreaves, A. P., & Shirley, D. L. (Eds.). (2009). The fourth way: The inspiring future for educational change. Corwin Press.
Hargreaves, A., & Shirley, D. (2020). Leading from the middle: Its nature, origins and importance. Journal of Professional Capital and Community, 5(1), 92–114.
Harris, A. (2011). System improvement through collective capacity building. Journal of Educational Administration, 49(6), 624–636.
*Hodson, R., & Cooke, E. (2004). Leading the drive for evidence-informed practice. Journal of Integrated Care, 12(1), 12–18.
*Hole, G. O., Brenna, S. J., Graverholt, B., Ciliska, D., & Nortvedt, M. W. (2016). Educating change agents: A qualitative descriptive study of graduates of a master’s program in evidence-based practice. BMC Medical Education, 16(1), 1–9.
Holt, D. T., Armenakis, A. A., Feild, H. S., & Harris, S. G. (2007). Readiness for organizational change: The systematic development of a scale. The Journal of Applied Behavioral Science, 43(2), 232–255.
Ittner, D., Hagenauer, G., & Hascher, T. (2019). Swiss principals’ emotions, basic needs satisfaction and readiness for change during curriculum reform. Journal of Educational Change, 20, 165–192.
*Jansson, I., & Forsberg, A. (2016). How do nurses and ward managers perceive that evidence-based sources are obtained to inform relevant nursing interventions?–An exploratory study. Journal of Clinical Nursing, 25(5–6), 769–776.
Jones, R. A., Jimmieson, N. L., & Griffiths, A. (2005). The impact of organizational culture and reshaping capabilities on change implementation success: The mediating role of readiness for change. Journal of Management Studies, 42(2), 361–386.
Kelchtermans, G. (2009). Who I am in how I teach is the message: Self-understanding, vulnerability and reflection. Teachers and Teaching: Theory and Practice, 15(2), 257–272.
Kirschner, P. A., Hendricks, M., Paas, F., Wopereis, I., & Cordewener, B. (2004). Determinants for failure and success of innovation projects: The road to sustainable educational innovation. Association for Educational Communications and Technology.
*Klinner, C., Carter, S. M., Rychetnik, L., Li, V., Daley, M., Zask, A., & Lloyd, B. (2015). Integrating relationship-and research-based approaches in Australian health promotion practice. Health Promotion International, 30(4), 891–902.
Lanham, H. J., Leykum, L. K., Taylor, B. S., McCannon, C. J., Lindberg, C., & Lester, R. T. (2013). How complexity science can inform scale-up and spread in health care: Understanding the role of self-organization in variation across local contexts. Social Science & Medicine, 93, 194–202.
Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5, 1–9.
*Lwin, K., & Beltrano, N. (2022). Rethinking evidence-based and evidence-informed practice: A call for evidence-informed decision making in social work education and child welfare practice. Social Work Education, 41(2), 166–174.
*Lwin, K., Laryea, A., Walker, H., Elgie, C., Head, S., & Gizzarelli, R. (2022). Evidence-informed decision-making training in child welfare: Evaluation of a proof of concept. Journal of Evidence-Based Social Work, 19(4), 475–492.
*McEwen, J., Crawshaw, M., Liversedge, A., & Bradley, G. (2008). Promoting change through research and evidence-informed practice: A Knowledge Transfer Partnership project between a university and a local authority. Evidence & Policy, 4(4), 391–403.
Meyer, C. B., & Stensaker, I. G. (2006). Developing capacity for change. Journal of change management, 6(2), 217–231.
Monahan, S. C., Meyer, J. W., & Scott, W. R. (1994). Employee training: The expansion of organizational citizenship. In W. R. Scott & J. W. Meyer (Eds.), Institutional environments and organizations: Structural complexity and individualism (pp. 255–271). Sage.
Moss, J., Hicks, S., Lassiter, S., & Miller, A. (2022). Implementing organizational change in a multidimensional community college district: A case study. Seattle University.
Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18, 1–7.
*Newhouse, R. P., Dearholt, S., Poe, S., Pugh, L. C., & White, K. M. (2007). Organizational change strategies for evidence-based practice. JONA: the Journal of Nursing Administration, 37(12), 552–557.
O’Brien, L., Marzano, M., & White, R. M. (2013). ‘Participatory interdisciplinarity’: Towards the integration of disciplinary diversity with stakeholder engagement for new models of knowledge production. Science and Public Policy, 40(1), 51–61.
Paina, L., & Peters, D. H. (2012). Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health Policy and Planning, 27(5), 365–373.
*Peters, S., Hopkins, L., & Barnett, T. (2016). Education for children with a chronic health condition: An evidence-informed approach to policy and practice decision making. British Journal of Special Education, 43(2), 142–158.
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences. Wiley Blackwell.
Poortman, C. L., Brown, C., & Schildkamp, K. (2022). Professional learning networks: A conceptual model and research opportunities. Educational Research, 64(1), 95–112.
Ramley, J. A. (2014). The changing role of higher education: Learning to deal with wicked problems. Journal of Higher Education Outreach and Engagement, 18(3), 7–22.
Resnick, L. B., Spillane, J. P., Goldman, P., & Rangel, E. S. (2010). Implementing innovation: From visionary models to everyday practice. In H. Dumont, D. Istance, & F. Benavides (Eds.), The nature of learning: Using research to inspire practice (pp. 285–315). OECD Publishing.
*Rousseau, D. M. (2020). Making evidence-based organizational decisions in an uncertain world. Organizational Dynamics, 49(1), 1–12.
Sahlberg, P. (2021). Finnish lessons 3.0: What can the world learn from educational change in Finland? Teachers College Press.
Schildkamp, K. (2019). Data-based decision-making for school improvement: Research insights and gaps. Educational Research, 61(3), 257–273.
Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26(3), 482–496.
*Semenic, S., Edwards, N., Premji, S., Olson, J., Williams, B., & Montgomery, P. (2015). Decision-making and evidence use during the process of prenatal record review in Canada: A multiphase qualitative study. BMC Pregnancy and Childbirth, 15, 1–16.
Semler, S. W. (1997). Systematic agreement: A theory of organizational alignment. Human Resource Development Quarterly, 8(1), 23–40.
Smith, K. (2005). Teacher educators’ expertise: What do novice teachers and teacher educators say? Teaching and Teacher Education, 21(2), 177–192.
Spillane, J. P. (2012). Data in practice: Conceptualizing the data-based decision-making phenomena. American Journal of Education, 118(2), 113–141.
Stoll, L. (2020). Creating capacity for learning: Are we there yet? Journal of Educational Change, 21(3), 421–430.
Swanson, E. B. (1994). Information systems innovation among organizations. Management Science, 40(9), 1069–1092.
*Timperley, H. S., & Parr, J. M. (2007). Closing the achievement gap through evidence-based inquiry at multiple levels of the education system. Journal of Advanced Academics, 19(1), 90–115.
Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., … Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473.
Tschannen-Moran, M., & Nestor-Baker, N. (2004). The tacit knowledge of productive scholars in education. Teachers College Record, 106(7), 1484–1511.
Vanlommel, K. (accepted). Evidence-Informed Educational Change: Conceptual Proposal for a Capacity-Building Model. BERA-Sage Handbook on Research-informed Education Practice and Policy.
Vanlommel, K., & Schildkamp, K. (2019). How do teachers make sense of data in the context of high-stakes decision making? American educational research journal, 56(3), 792–821.
Vanlommel, K. (2022). Drivers and obstacles for evidence-informed practice in an autonomous and decentralized educational system: Belgium. In C. Brown & J. R. Malin (Eds.), The Emerald handbook of evidence-informed practice in education: Learning from international contexts (pp. 259–273). Emerald Publishing Limited.
Vanlommel, K., Van Gasse, R., Vanhoof, J., & Van Petegem, P. (2018). Teachers’ high-stakes decision making. How teaching approaches affect rational and intuitive data collection. Teaching and teacher education, 71, 108–119.
Vanlommel, K., van den Boom-Muilenburg, S. N., Thesingh, J., & Kikken, È. (2023). How a sense of collective efficacy influences teacher learning during change: the role of academic optimism and shared vision. Journal of Professional Capital and Community, 8(1), 1−16.
Verbiest, E. (2021). Perspectieven op Onderwijsvernieuwing (Perspectives on Educational Change). Gompel&Svacina.
Wang, T., Olivier, D. F., & Chen, P. (2023). Creating individual and organizational readiness for change: Conceptualization of system readiness for change in school education. International Journal of Leadership in Education, 26(6), 1037–1061.
Wayman, J. C., Jimerson, J. B., & Cho, V. (2012). Organizational considerations in establishing the data-informed district. School Effectiveness and School Improvement, 23(2), 159–178. https://doi.org/10.1080/09243453.2011.652124
Wubbels, T., & van Tartwijk, J. (2017). Evidence informed innovation of education in the Netherlands: Learning from reforms. In M. Y. Eryaman & B. Schneider (Eds.), Evidence and public good in educational policy, research and practice (pp. 125–142). Springer.
Zayim, M., & Kondakci, Y. (2015). An exploration of the relationship between readiness for change and organizational trust in Turkish public schools. Educational Management Administration & Leadership, 43(4), 610–625.
Funding
This study was supported by the Netherlands Initiative for Education Research (NRO) (Grant No. 40.5.22945.052.).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Vanlommel, K., van den Boom-Muilenburg, S.N. How can we understand and stimulate evidence-informed educational change? A scoping review from a systems perspective. J Educ Change (2024). https://doi.org/10.1007/s10833-024-09506-z
Accepted:
Published:
DOI: https://doi.org/10.1007/s10833-024-09506-z