1 Introduction

In recent years, there has been a growing recognition of the importance of teachers’ data literacy for educational policy, research, and practice. This trend was ignited in 2009 when Arne Duncan, the former Secretary of Education of the United States, advocated evidence-driven practices in schools to enhance student performance (Mandinach & Gummer, 2016). Since then, there has been an increasing expectation for teachers to engage in data-informed practices to guide teaching and decision-making in schools. Following this trend, educational researchers have also increasingly directed their attention towards offering conceptual and theoretical foundations for teachers’ data literacy.

Various organizations and researchers have provided the definitions of teachers’ data literacy. For example, drawing on the opinions of diverse stakeholder groups, Data Quality Campaign (2014) defined teachers’ data literacy as teachers’ capabilities to “continuously, effectively, and ethically access, interpret, act on, and communicate multiple types of data from state, local, classroom, and other sources to improve outcomes for students in a manner appropriate to educators' professional roles and responsibilities” (p. 1). Kippers et al. (2018) defined teachers’ data literacy as “educators’ ability to set a purpose, collect, analyze, and interpret data and take instructional action” (p. 21). Similarly, teachers’ data literacy has been defined as “one’s ability, or the broad knowledge and skills, needed to engage in data use or implement a data use inquiry process (Abrams et al., 2021, p. 100,868).

The data literacy for teachers (DLFT) framework proposed by Mandinach and Gummer defined teachers’ data literacy as “… the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data to help determine instructional steps” (Gummer & Mandinach, 2015, p. 2). In recent years, much of the research efforts to provide a theoretical framework on teachers’ data literacy has been led by Mandinach and Gummer (Gummer & Mandinach, 2015; Mandinach & Gummer, 2012, 2013a, 2016; Mandinach et al., 2015). As far as we can ascertain, their work presents the most comprehensive framework of teachers’ data literacy in the current literature. The primary sources of Mandinach and Gummer’s DLFT framework were their previous works, Mandinach and Gummer (2012) and Mandinach et al. (2015). Their DLFT framework was developed as the results of the analysis of the teacher licensure documents across the US states (Mandinach et al., 2015) and the text analysis of the perspectives and definitions provided by 55 researchers and professional development providers during a braining storming at the conference held in 2012 (cf. Mandinach & Gummer, 2012). There are five components in the framework: (a) identifying problems and framing questions, (b) using data, (c) transforming data into information, (d) transforming information into decisions, and (e) evaluating outcomes. Their framework aimed to identify “the specific knowledge, skills, and dispositions teachers need to use data effectively and responsibly” (Mandinach & Gummer, 2016, p. 366). However, a potential sixth dimension, “dispositions, habits of mind, or factors that influence data use” (Mandinach & Gummer, 2016, p. 372) was mentioned but not included in the framework.

2 The present study

In the present study, we conducted a systematic review of the empirical studies on teachers’ data literacy and data use published in academic journals between 1990 and 2021. Our primary purpose was to enhance the conceptual clarity of teachers’ data literacy by providing its updated definition, indicators, and dimensions.

We argue that there are several reasons to justify the need for this systematic review. Firstly, we update, complement, and compare our review outcomes and the DLFT framework in Mandinach and Gummer (2016). A systematic review of research studies on teachers’ data use was conducted by Mandinach and Gummer (2013b), but the study selection was limited to years between 2001 and 2009. Therefore, one of the aims of the present study is to compare our systematic review outcomes against the dimensions and specific indicators identified in the DLFT framework (Mandinach & Gummer, 2016). The present literature search spans a period from 1990 to 2021. We have set 1990 as the lower-boundary year because “during the 1990s, a new hypothesis – that the quality of teaching would provide a high-leverage policy target – began to gain currency” (Darling-Hammond et al., 2003, p. 5).

Secondly, it appears that much work on teachers’ data literacy, including that of Mandinach and Gummer, has tended to focus on teachers’ data use in relation to teaching (e.g., Beck et al., 2020; Datnow et al., 2012) and instructional improvement (e.g., Datnow et al., 2021; Kerr et al., 2006; Wachen et al., 2018) or in relation to student academic performance (e.g., Poortman & Schildkamp, 2016; Staman et al., 2017). However, we argue that classroom teachers’ tasks and responsibilities go beyond teaching itself and include many other tasks such as advising/counselling, organising excursions, and administrative work (e.g., Albiladi et al., 2020; Kallemeyn, 2014). Our review, therefore, examines how teachers’ data use practices may be manifested across a range of teacher responsibilities beyond teaching and teaching-related tasks.

Thirdly, there has been a relative lack of attention to teachers’ personal dispositions in data literacy research. Dispositions refer to a person's inherent tendencies, attitudes, approaches, and inclinations towards ways of thinking, behaving, and believing (Lee & Stankov, 2018; Mischel & Shoda, 1995). According to Katz (1993), a disposition can be defined as “a tendency to exhibit frequently, consciously, and voluntarily a pattern of behavior that is directed to a broad goal” (p. 2). In the context of education, disposition refers to the attitudes, beliefs, and values that influence a teacher’s actions, decision-making, and interactions with various stakeholders including students, colleagues, and school leaders (Darling-Hammond et al., 2003). While teachers’ dispositions were mentioned in Mandinach and Gummer (2016), dispositions were not included in their DLFT framework. Teacher educators have long emphasized that accomplished teachers need to possess extensive knowledge, skills, and a range of dispositions to support the learning of all students in the classroom, engage in on-going professional development, and continuously strive to enhance their own learning throughout their careers (Darling-Hammond et al., 2003; Sykes, 1999). Therefore, we aim to identify a range of teachers’ dispositions in relation to data literacy and data use in the school contexts.

Fourthly, we argue that teachers’ data literacy may be more important in the current context of the rapidly evolving data and digital landscape influenced by the technical advancements in artificial intelligence. Teachers may encounter significant challenges in comprehending and addressing a wide array of issues, both anticipated and unforeseen, as well as observed and unobserved situations, stemming from various artificial intelligence tools and automated machines. In this sense, comprehending the nature, types, and functions of data is crucial for teachers. Without such understanding, the educational community and teaching workforce may soon find themselves in an increasingly worrisome situation when it comes to evaluating data and information.

Finally, we argue that there is a need to update conceptual clarity regarding teachers’ data literacy in the current literature. Several systematic review studies have focused on features in professional development interventions (PDIs) aimed at improving teachers’ data use in schools (e.g., Ansyari et al., 2020; 2022; Espin et al., 2021), emphasizing the need to understand data literacy as a continuum spanning from pre-service to in-service teachers and from novice to veteran educators (Beck & Nunnaley, 2021). Other systematic review studies have given substantial attention to data-based decision-making (DBDM) in the schools (e.g., Espin et al., 2021; Filderman et al., 2018; Gesel et al., 2021; Hoogland et al., 2016). For example, Hoogland et al. (2016) investigated the prerequisites for data-based decision-making (DBDM) in the classroom, highlighting nine themes that influence DBDM, such as collaboration, leadership, culture, time, and resources. These systematic reviews are highly relevant to the current review, as the PDIs, understanding the continuum, or data-based decision-making would require a clear and updated understanding of what teachers’ data literacy should be. We hope that the current study’s definition, indicators, and dimensions of teachers’ data literacy may be useful in conjunction with other systematic review studies on teachers’ data use and factors influencing teachers’ data use.

3 Method

3.1 Data sources and selection of the studies

Our strategies for literature search were based on the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), a framework for reporting and synthesising literature review (Moher et al., 2009). In accordance with PRISMA suggestions, we followed the four steps in locating and reviewing the relevant studies. First, we conducted initial searches to identify relevant studies, using three databases: Scopus, ProQuest, and Web of Science. Keywords in our search were teacher, school, data, data use, data literacy, evidence-based, and decision-making (see Table 1 for the detailed search strategy syntax). This initial search, using the combination of the identified keywords, yielded 2,414 journal articles (see Fig. 1). After removing duplicates, 1,976 articles remained.

Table 1 Search strategy syntax
Full size table
Fig. 1
figure 1

Study selection flow using PRISMA guidelines

Full size image

Secondly, we set and applied the inclusion criteria to screen the studies. The inclusion criteria were: (a) topics relating to the key words, (b) school context of primary or secondary school settings (i.e., excluding studies focusing on university, vocational education, and adult learning), (c) the full text written in English (excluding studies if the full text is presented in another language or if only the abstract was presented in English), (d) peer-reviewed empirical studies (across quantitative, qualitative, and mixed-methods) published in academic journals (excluding book chapters, conference papers, thesis) to ensure the inclusion of the published work that has undergone peer-review process, and finally, (e) published studies from 1990 onwards. The titles and abstracts of the studies were reviewed to assess their eligibility based on the inclusion criteria. As a result of applying these criteria, 117 articles were selected for the next step, full-text review.

Thirdly, we evaluated the eligibility of the full-text versions of the published studies. This full-text review resulted in a further exclusion of 34 studies as they were found to not meet all the inclusion criteria. We also examined whether the studies included data literacy or data-driven decision-making. Following these assessments, we identified 83 articles that met all the inclusion criteria.

Finally, we reviewed, coded, and analyzed the final set of the selected studies. The analysis approaches are described below.

3.2 Approach to analysis

We employed a thematic synthesis methodology, following the framework outlined by Thomas & Harden (2008). The coding and analysis process consisted of three main stages: (a) conducting a line-by-line reading and coding of the text, (b) identifying specific descriptive codes, and (c) generating analytical themes by grouping conceptually inter-related descriptive codes. The final analytic process was, therefore, categorizing and naming related descriptive codes to produce analytical themes. During the development of the analytic themes, we utilized an inductive approach, organizing conceptually interconnected codes into broader themes.

The first author developed the descriptive and analytical themes, which were then reviewed by another two authors. To ensure coding rigor and consistency, three authors independently coded the same two articles, and then compared the coding to address any inconsistencies and reach a consensus. This process was repeated in four iterations. Once the three authors who were involved in the initial coding reached the consensus, the remaining authors double-checked the final outputs of the thematic analysis (i.e., codes, and themes). We have labelled descriptive codes as ‘indicators’ of teachers' data literacy, while the broader groups of descriptive codes, referred to as analytic themes, represent ‘dimensions’ of teachers’ data literacy.

4 Results

4.1 Characteristics of the reviewed studies

The main purpose of the present study was to examine the conceptualization of teachers’ data literacy from 83 peer-reviewed empirical studies. Table 2 presents the studies included in our systematic review, along with the summary of the study characteristics such as country, school-level, study focus (i.e., main constructs), study purposes/objectives, research method, data collection tools, and sample size. Figure 2 presents the number of the reviewed studies by publication year. We found that since 2015, there has been an increase in the number of published empirical studies on teachers' data literacy.

Table 2 Study characteristics: Country, school-level, study focus & purpose, research method, data collection tool, and sample size
Full size table
Fig. 2
figure 2

Number of the reviewed studies by publication year

Full size image

Out of 83 studies, 50 were conducted in the United States. Thirteen studies were from Netherlands, four from Belgium, three from Australia, two from each of Canada and the United Kingdom, and one study for each of the following ten countries: China, Denmark, Germany, Indonesia, Ireland, Kenya, Korea, Norway, South Africa, and Sweden. Therefore, more than half of the studies (i.e., 58 studies, 70%) were conducted in the English-speaking countries. In terms of school-settings, studies were mostly conducted in primary school settings or in combination with high school: 36 studies in primary school settings, 16 in secondary school settings, and 30 studies were in both primary and secondary school settings. The most common design was qualitative (n = 35 studies), followed by mixed methods (n = 30) and quantitative (n = 18). Multiple sources of data collection (e.g., interview and survey) were used in 22 studies. The most commonly used data collection tool was interview (n = 55), which was followed by surveys (n = 37) and observation (n = 25). A smaller set of studies used focus group discussion (n = 18) and document analysis (n = 19). A few studies used students’ standardised assessment data (n = 4), field notes (n = 4), and teacher performance on data literacy test (n = 4).

We also reviewed the study topics and found that there are seven foci among the reviewed studies: (a) factors influencing teachers’ data use (n = 29), (b) specific practices in teachers’ data use (n = 27), (c) teachers’ data use to enhance teaching practices (n = 25), (d) teachers’ data use for various purposes (n = 24), (e) approaches to improve teachers’ data literacy (n = 22), (f) approaches to improve teachers’ assessment literacy (n = 19), and (g) teachers’ data use to improve student learning outcomes (n = 19).

4.2 Dimensions and indicators of teaches’ data literacy

Our thematic analysis identified 95 descriptive codes (see Table 3). Careful review of the identified descriptive codes suggested that they can be viewed as indicators of teachers’ knowledge, attitudes, behaviors, and dispositions in data use. These indicators were further organized into inter-related concepts, which formed analytic themes; we refer to these as ‘dimensions’ (see Table 3). There were five broad dimensions that emerged from the indicators: knowledge about data (Dimension 1), skills in using data (Dimension 2), dispositions towards data use (Dimension 3), data application for various purposes (Dimension 4), and data-related behaviors (Dimension 5).

Table 3 Dimensions and indicators of teachers’ data literacy
Full size table

It is necessary to point out that Dimension 1 pertains to understanding the nature of data itself, focusing on knowledge about data. On the other hand, Dimension 2 revolves around data-related skills in the actual use of data, encompassing a spectrum of sequences including data generation, processing, and production. These two dimensions, i.e., knowledge and skills, are highly interconnected and complement each other. Proficiency in data-use skills (Dimension 2) may not be developed without a solid understanding of how data can be utilised, for instance, in teaching practices or school improvement in data use (Dimension 1). Conversely, teachers' understanding of how data can enhance teaching practices (Dimension 1) can guide them in determining specific approaches to analysing particular datasets (Dimension 2). While we acknowledge the complementary nature of knowledge and skills, it is important to note that certain aspects of knowledge and skills may not completely overlap. For instance, a teacher who understands the process of creating state-level assessment data may not necessarily possess the technical expertise required to analyze state-level data, taking into account measurement errors. Therefore, we maintain knowledge and skills as two distinct dimensions to highlight both as the core components of teachers’ data literacy.

Within each of the five broad dimensions, we also uncovered sub-themes to illuminate the constituent elements of those dimensions. Under Dimension 1, four sub-themes emerged: “knowledge about data”, knowledge about data for “teaching practices”, understanding “data culture in the school”, and understanding the use of “external assessment”. Dimension 2 featured sub-themes highlighting the sequential stages of data utilization: “data generation & collection”, “data analysis”, “data interpretation”, “data integration”, “evaluation”, and “reporting”. Within Dimension 3, we identified dispositions towards data use, encompassing sub-themes such as confidence, values/beliefs, trust/respect, and anxiety. Dimension 4 revealed various purposes of data applications, categorized into three sub-themes: “teaching,” “student learning,” and “school improvement.” Lastly, Dimension 5 delineated teachers’ behaviors related to data into two sub-themes: “communication & discussion” and “participation & engagement.”

In the following passages we provide detailed descriptions of the indicators and their associated dimensions. Figure 3 presents a visual a summary of these indicators and dimensions.

Fig. 3
figure 3

A summary of the dimensions and indicators of teachers’ data literacy

Full size image

4.2.1 Dimension 1. Knowledge about data

The first dimension of teachers’ data literacy pertains to teachers’ knowledge about data. Many studies recognized the importance of data-related knowledge to be utilized in the schools (e.g., Jacobs et al., 2009; Omoso et al., 2019; Schildkamp et al., 2017). Our review revealed four major ways that teachers' data-related knowledge can be manifested. Firstly, teachers’ knowledge about data involves their understanding of the necessary steps in data analysis procedures (Ebbeler et al., 2016; Snodgrass Rangel et al., 2016; Vanlommel et al., 2021; Wardrip & Herman, 2018) and understanding of different data types to be used for varying purposes (Abdusyakur & Poortman, 2019; Beck et al., 2020; Howley et al., 2013; Reeves et al., 2016).

Secondly, teachers’ knowledge about data involves their capability to relate the insights gleaned from data to inform their teaching practices (Abrams et al., 2016; Jimerson et al., 2016). Specifically, data-literate teachers leverage student assessment data to evaluate learning progress (Abrams et al., 2016; Jimerson, 2014; Jimerson & Wayman, 2015; Jimerson et al., 2016; Snodgrass Rangel et al., 2016), to tailor classroom instruction based on data insights (Mokhtari et al., 2009; Poortman & Schildkamp, 2016; Staman et al., 2017; van der Scheer & Visscher, 2018), and to ensure alignment between instructional approaches and appropriate assessment methods (Howley et al., 2013; Marsh & Farrell, 2015; van der Scheer & Visscher, 2018).

Thirdly, teachers’ data literacy extends to understanding of the school culture surrounding data utilization (e.g., Andersen, 2020; Schildkamp, 2019; Wachen et al., 2018). This encompasses recognizing the conditions that may facilitate or hinder teachers’ data use (Abdusyakur & Poortman, 2019; Anderson et al., 2010; Keuning et al., 2017) and navigating various challenges associated with using assessment data in the school (Datnow et al., 2012; Ford, 2018; Kanjee & Moloi, 2014; Thomas & Huffman, 2011).

Lastly, teachers’ knowledge about data includes understanding of externally administered assessment data and data system, such as state-level assessment policies related to data use (Copp, 2017; Hardy, 2019; Reed, 2015) and understanding the broader state-level contexts that impact data utilization within the school (Datnow et al., 2013; Dunn et al., 2013a; Ford, 2018; Omoso et al., 2019; Powell et al., 2021). Teachers may need to have thorough knowledge of educational government policies to ensure alignment between state-level curriculum initiatives and school-level assessment policies (Anderson et al., 2010; Copp, 2017; Gelderblom et al., 2016; Hardy, 2015).

In summary, existing literature highlights that data-literate teachers would have a comprehensive understanding of a diverse range of data sources and purposes, regularly reviewing and evaluating student outcomes from various channels. Consequently, if teachers face excessive pressure to meet accountability measures and improve standardized testing results, it could potentially hinder their overall development and growth in a broad spectrum of data-related knowledge.

4.2.2 Dimension 2. Skills in using data

Skills in using data is the second key dimension in teachers’ data literacy. There were a wide range of specific data-skills mentioned in the literature, spanning from data generation and collection (Farley-Ripple et al., 2019; Jimerson & Wayman, 2015) to data analysis (Farley-Ripple et al., 2019; Jimerson & Wayman, 2015; Marsh et al., 2010), data interpretation and integration (Jimerson & Wayman, 2015; Marsh et al., 2010), evaluation (Andersen, 2020; Dunn et al., 2013b; Thomas & Huffman, 2011), and report writing (Farley-Ripple et al., 2019; Jimerson & Wayman, 2015). These indicators (see Table 3) emphasize that teachers’ data literacy requires proficiency across the entire sequence, across different stages of data generation, processing, and production.

Teachers’ skills in data use also involve selecting specific data types appropriate for different purposes (Anderson et al., 2010; Jimerson et al., 2016; Kanjee & Moloi, 2014), analysing multiple sources of data on student learning outcomes (Datnow et al., 2012; Vanlommel et al., 2021; von der Embse et al., 2021), and integrating multiple data sources to arrive at a holistic assessment of student progress (Brunner et al., 2005; Farley-Ripple et al., 2019; Ford, 2018; Jacobs et al., 2009; Mausethagen et al., 2018). For example, teachers may need to apply different data analytic approaches when evaluating student outcomes based on school-based versus externally administered standardized assessments (Copp, 2017; Curry et al., 2016; Omoso et al., 2019; Wardrip & Herman, 2018; Zeuch et al., 2017). Data-literate teachers may also plan data analysis for targeted purposes, such as analyzing students’ social-emotional outcomes (Abrams et al., 2021; Jimerson et al., 2021; von der Embse et al., 2021; Wardrip & Herman, 2018), identifying individual students’ learning needs, making recommendations for curriculum revisions, or evaluating pedagogical approaches (Dunn et al., 2013a; Snodgrass Rangel et al., 2016; Wolff et al., 2019; Young, 2006).

In summary, this “skills” dimension highlights the importance of teachers possessing a diverse array of competencies to leverage data effectively. The literature reviewed identified various aspects of teachers’ data use, spanning the spectrum from data collection and generation to analysis, interpretation, integration across multiple sources, evaluation, and reporting.

4.2.3 Dimension 3. Dispositions towards data use

While somewhat overlooked in data literacy literature, teachers’ disposition is a crucial component of their data literacy. Our review identified four major types of such dispositions in the context of teachers’ data literacy (see Table 3). Firstly, studies have underscored that teachers’ confidence in using data may be necessary when making data-driven school-level decisions, for example, to design intervention programs (Andersen, 2020; Keuning et al., 2017; Staman et al., 2017; Thompson, 2012), or to develop strategic plans for school improvement (Dunn et al., 2013b; Poortman & Schildkamp, 2016). Researchers also claimed that teachers may need to feel confident in many steps of data processes, across accessing, analyzing, interpreting, evaluating, and discussing data within the school environment (Abrams et al., 2021; Dunn et al., 2013a; von der Embse et al., 2021).

The second disposition pertains to teachers valuing and believing in the importance of data use in schools. Data-literate teachers would recognize the usefulness of data in informing school improvement and enhancing student performance (Howley et al., 2013; Poortman & Schildkamp, 2016; Prenger & Schildkamp, 2018). They would also place value on collaboration among colleagues and actively seek institutional support for effective data use (Kallemeyn, 2014; Marsh & Farrell, 2015; Nicholson et al., 2017; Poortman & Schildkamp, 2016). Furthermore, they would appreciate the pivotal role of school leaders in supporting and promoting teachers’ data use within the school (Albiladi et al., 2020; Curry et al., 2016; Joo, 2020; Young, 2006).

A third type of teacher disposition that our review identified is trust in and respect towards colleagues and school leaders. Teachers often work collaboratively in the school environment when they learn about and utilise school-level data. In this sense, teacher collaboration and sustaining trusting relationships are fundamental in fostering a school culture that appreciates data-driven decision-making, as well as for encouraging teachers to further develop their own data knowledge and skills (Abrams et al., 2021; Andersen, 2020; Keuning et al., 2017). Mutual trust and respect among teachers can allow them to have open and honest conversations about their experiences and share any concerns arising from data use in the school context (Andersen, 2020; Datnow et al., 2013; Ford, 2018; Wachen et al., 2018).

Lastly, data anxiety may play a role when teachers use or are expected to use data in the school (Abrams et al., 2021; Dunn et al., 2013b; Reeves et al., 2016). Teachers may experience data anxiety when they are expected to effectively analyze student assessment outcomes (Dunn et al., 2013b; Powell et al., 2021), when they are introduced to new data management systems in the school, when they feel pressured to quickly grasp the school’s data management system (Andersen, 2020; Dunn et al., 2013a), or when they are tasked with developing specific strategies to assess and enhance student learning outcomes (Dunn et al., 2013a, b; Jimerson et al., 2019). These types of teacher responsibilities demand proficient data skills and knowledge, which not all teachers may possess, and thus, anxiety may hinder their ability to further develop their data literacy.

In summary, teacher dispositions towards data use can impact their effective utilization of data or impede the capacity to further develop their own data literacy. Our review also illuminated that it is not just individual teachers’ confidence or anxiety towards data use, but also the social dynamics within the school environment, including teacher collaboration, trust and respect, and relationships with the school management team, that can influence teachers’ data literacy. Therefore, fostering a collaborative climate within the school community and creating more opportunities for data use may strengthen a data-driven culture within the school.

4.2.4 Dimension 4. Data applications for various purposes

Our review suggests that teachers' data literacy can be manifested in their use of data for multiple purposes, primarily in three areas: (a) to enhance teaching practices (e.g., Datnow et al., 2012, 2021; Farrell, 2015; Gelderblom et al., 2016; Wachen et al., 2018), (b) to support student learning (e.g., Joo, 2020; Lockton et al., 2020; Staman et al., 2017; Vanlommel et al., 2021; van der Scheer & Visscher, 2018), and (c) to make plans and strategies for school improvement (e.g., Abdusyakur & Poortman, 2019; Jimerson et al., 2021; Kallemeyn, 2014).

With respect to teaching enhancement purposes, teachers use data to inform their lesson plans (Ford, 2018; Gelderblom et al., 2016; Snodgrass Rangel et al., 2016; Reeves et al., 2016), set lesson objectives (Kallemeyn, 2014; Snodgrass Rangel et al., 2016; Reeves et al., 2016), develop differentiated instructions (Beck et al., 2020; Datnow et al., 2012; Farley-Ripple et al., 2019), and provide feedback to students (Gelderblom et al., 2016; Andersen, 2020; Jimerson et al., 2019; Marsh & Farrell, 2015). Furthermore, teachers use data to reflect on their own teaching practices (Datnow et al., 2021; Ford, 2018; Jimerson et al., 2019; Snodgrass Rangel et al., 2016) and evaluate the impact of using data on teaching and learning outcomes (Gelderblom et al., 2016; Marsh & Farrell, 2015).

In relation to supporting student learning, teachers use data to recognize individual students’ learning needs (Curry et al., 2016; Gelderblom et al., 2016), guide students to learning new or challenging concepts (Abrams et al., 2021; Keuning et al., 2017; Marsh et al., 2010; Reeves et al., 2016), set learning goals (Abdusyakur & Poortman, 2019; Curry et al., 2016), and monitor learning progress (Curry et al., 2016; Gelderblom et al., 2016; Marsh et al., 2010).

In terms of guiding school improvement strategies, teachers use data to develop school-based intervention programs (Abdusyakur & Poortman, 2019; Jimerson et al., 2021; Kallemeyn, 2014; Thompson, 2012), make decisions about school directions (Huffman & Kalnin, 2003; Prenger & Schildkamp, 2018; Schildkamp, 2019), and evaluate school performance for meeting the accountability requirements (Hardy, 2015; Jacobs et al., 2009; Jimerson & Wayman, 2015; Marsh et al., 2010; Omoso et al., 2019; Snodgrass Rangel et al., 2019).

In summary, the literature indicates that data-literate teachers use data for multiple purposes and consider it essential in fulfilling their various roles and responsibilities within the school. Teachers’ data use for supporting student learning tends to focus primarily on helping students achieve better learning outcomes; in contrast, teachers’ data use for teaching enhancement includes a broader range of data processes and practices.

4.2.5 Dimension 5. Data-related behavior

The fifth and final dimension we identified pertains to teachers' data-related behaviors within and outside the school context. Within this dimension, there appear to be two distinctive sets of teacher behaviors: (a) teachers’ data use to enhance communication and discussion with various stakeholders such as colleagues (Datnow et al., 2013; Van Gasse et al., 2017), school leaders (Jimerson, 2014; Marsh & Farrell, 2015; Nicholson et al., 2017), and parents (Jimerson & Wayman, 2015; Jimerson et al., 2019); and (b) teachers’ participation in and engagement with learning about data use (Schildkamp et al., 2019; Wardrip & Herman, 2018) and data culture in schools (Datnow et al., 2021; Keuning et al., 2016). These behaviors were found to be integral aspects of teachers' data literacy. Teacher engagement with data is manifested in multiple ways, such as involvement in team-based approaches to data utilization (Michaud, 2016; Schildkamp et al., 2017; Wardrip & Herman, 2018; Young, 2006), active participation in creating a school culture of data use (Abrams et al., 2021; Albiladi et al., 2020), evaluation of the organizational culture and conditions pertaining to data use (Andersen, 2020; Datnow et al., 2021; Lockton et al., 2020), and participation in professional development opportunities focused on data literacy (Ebbeler et al., 2016; O’Brien et al., 2022; Schildkamp et al., 2017).

In summary, this dimension highlights that teachers’ data literacy includes various forms of their active engagement and behavior to enhance the effective use and understanding of data. Our findings also indicate that teacher communication and discussions regarding data primarily focus on student assessment data with various stakeholder groups including colleagues, school leaders, and parents.

5 Discussion

The present study reviews 83 empirical studies on teachers' data literacy published in peer-reviewed journals from 1990 to 2021, and we identified 95 specific indicators categorized across five dimensions: (a) knowledge about data, (b) skills in using data, (c) dispositions towards data use, (d) data applications for various purposes, and (e) data-related behaviors in the school. Our review of the identified indicators of this study has led to the following definition of teachers’ data literacy:

A set of knowledge, skills, and dispositions that empower teachers to utilize data for various purposes, including generating, collecting, analyzing, interpreting, integrating, evaluating, reporting, and communicating, aimed at enhancing teaching, supporting student learning, engaging in school improvement, and fostering self-reflection. Teachers’ data literacy also involves the appreciation for working together with colleagues and school leaders to (a) assess organizational conditions for data use, (b) foster a supportive school culture, and (c) engage in ongoing learning to optimize the effective utilization of data.

Our analysis also revealed several noteworthy findings that are presented in the following sections.

5.1 Teachers’ data literacy and assessment literacy

There have been concerns expressed by scholars about conceptual fuzziness in teachers’ data literacy and assessment literacy (cf. Brookhart, 2011; Ebbeler et al., 2016; Mandinach, 2014; Mandinach & Gummer, 2016). Indeed, student assessment data are the most salient form of data in the school (Mandinach & Schildkamp, 2021). The research trend of recognising the importance of teachers’ data literacy is often based on the premise that teachers’ data literacy would enhance teaching and ultimately improve student outcomes (cf. Ebbeler et al., 2016; Mandinach & Gummer, 2016; Poortman & Schildkamp, 2016; Thompson, 2012; Van Gasse et al., 2018; Zhao et al., 2016). Furthermore, the systemic pressure on schools to meet accountability requirements has also impacted their endeavors to utilize, assess, and demonstrate school performance based on student assessment data in recent years (Abdusyakur & Poortman, 2019; Farrell, 2015; Schildkamp et al., 2017; Weiss, 2012). In these contexts, it is not surprising that educational practitioners would think about student assessment data when they are expected to improve their data skills.

In this light, we have tallied the teacher data literacy indicators that directly relate to student assessment or about students’ learning outcomes. In Table 3, the symbol “⁑” is used for the indicators related to student assessment, and “ξ” is used for the indicators related to students’ learning outcomes. We found that there were only 19 out of 95 indicators that directly related to student assessment (e.g., knowledge about different purposes of assessment, understanding the alignment between instruction and assessment, understanding state-level assessment policies on data use). Similarly, there were only 13 out of 95 indicators that directly related to students’ learning outcomes (e.g., identifying evidence of student learning outcomes, understanding student learning outcomes using multiple sources).

Our review demonstrates that teachers regularly interact with a diverse array of data and undertake various tasks closely associated with its utilization. Therefore, teachers' data literacy encompasses more than just its use in student assessment and learning outcomes; it extends to understanding students’ social-emotional learning and higher-order thinking skills, assessing school conditions for data use, reflecting on teaching practices, and communicating with colleagues. Consequently, limiting the perspective of teachers’ data literacy solely to assessment literacy may impede their full utilization and appreciation of data applications essential to their multifaceted work in supporting and enhancing student and school outcomes.

5.2 Teachers’ data literacy and data-related dispositions

We found that one of the key aspects of teachers’ data literacy is teachers’ dispositions towards data use. As noted by Mandinach and Gummer (2012, 2016), this aspect of teacher characteristics has not received as much research attention as data knowledge or data skills. It is perhaps due to ‘literacy’ being traditionally linked to knowledge and skills (Shavelson et al., 2005; also see Mandinach & Gummer, 2012) or due to the research trend of unpacking teachers’ needs and pedagogical approaches in specific subject/learning domains (Sykes, 1999; see Mandinach & Gummer, 2016). However, our review suggests that teacher dispositions towards data use are required in virtually all aspects of data use and data analyses processes. We also found that the most important data-related teacher disposition was confidence. The data literacy literature recognized the importance of teacher confidence, with respect to accessing, collecting, analysing, integrating, evaluating, discussing, and making decisions, suggesting that for teachers to be data literate, confidence may be required in every step of data use. There has been extensive research that has demonstrated a strong link between confidence and learning motivation, indicating that individuals tend to gravitate towards domains in which they feel comfortable and confident (e.g., Lee & Durksen, 2018; Lee & Stankov, 2018; Stankov & Lee, 2008). Our review findings contribute to this existing body of research, emphasizing the importance of confidence in teachers’ data utilization. This underscores the necessity for policies and professional development initiatives aimed at enhancing teachers’ data use to also prioritize strategies for building teachers’ confidence in this area.

Our findings also indicate that teachers’ data literacy is associated with their trust in colleagues and school leaders, as well as their respect for the leadership team's role in leading data use and school improvement (Andersen, 2020; Ford, 2018; Wachen et al., 2018). This suggests that for teachers to be effective data users, they need to feel empowered to voice concerns and express frustrations with colleagues (Andersen, 2020; Ford, 2018; Wachen et al., 2018), seek help when necessary (Wardrip & Herman, 2018; Young, 2006), and collaboratively develop strategies for effective collaboration within the school (Datnow et al., 2013; Huffman & Kalnin, 2003; Michaud, 2016; Van Gasse et al., 2021).

Many teacher tasks are deeply intertwined with human relationships (Lee, 2021) and often completed through collaborative efforts (Li et al., 2022). Therefore, school leaders and policymakers may recognize that fostering teachers’ data literacy may necessitate cultivating open, honest, and trusting school environments conducive to collaboration. Notably, the social aspect of data literacy was not prominently evident in dimensions related to teachers' knowledge and skills, which suggests that teachers may enhance their knowledge and skills independently from others in the school environment. However, fostering teacher dispositions, such as active engagement in effective data use within the school, appears to be influenced by collaborative relationships with colleagues, as well as the supportive roles of school leaders.

5.3 Teachers’ data literacy and data-related behaviors

Our review showed that teachers’ data literacy goes beyond the knowledge, skills, and dispositions that are required to effectively use data; it also involves a range of behaviors that enhance their ways of using and learning about data. Within this dimension, we noted two sub-categories, communication/discussion and participation/engagement. Therefore, one core aspect of teacher behaviors related to data was found to be communicating with various stakeholders such as colleagues, parents, and school leaders to discuss instructional approaches (e.g., Datnow et al., 2013; Militello et al., 2013; van der Scheer & Visscher, 2018) and assessment results (e.g., Curry et al., 2016; Howley et al., 2013). The other aspect—participation and engagement—underscores the importance of teacher involvement in team-based learning regarding data use (e.g., Andersen, 2020; Young, 2006), active engagement in establishing conducive school conditions and fostering a culture of data use within the school community (e.g., Datnow et al., 2021; Keuning et al., 2016), and proactive participation in professional development to enhance knowledge and skills (e.g., Ebbeler et al., 2016; van der Scheer & Visscher, 2018). Existing studies on data literacy have not given substantial attention to the importance of teachers' behaviors related to data. However, we argue that teachers’ behaviors related to data deserve recognition as a distinct category within the concept of teachers’ data literacy.

Dimension 4 (about teachers’ disposition) and Dimension 5 (about teachers’ behaviors) would be correlated. For example, teachers who are confident in data use may be more inclined to lead the discussions with other colleagues about data use in the school, and they may pursue additional learning opportunities to become an effective leader in school data use. Trust and respect within the school communities mentioned above would also influence how teachers behave in order to collectively enhance data literacy within the school. Studies (e.g., Ebbeler et al., 2016; van der Scheer & Visscher, 2018) have highlighted teacher participation in professional development, but there has been a relative lack of research attention to examine the collaborative nature of teacher engagement and learning within the professional settings. With the rapid evolution of educational tools and applications driven by learning analytics and artificial intelligence, the influx of data generated in this new era poses a significant challenge for teachers and school leaders. Accordingly, teacher collaboration in learning and addressing data-related challenges in schools will increasingly become a paramount concern, more so than ever before. In this regard, future policies concerning data use may prioritize the expansion of teacher collaboration and mutual learning as essential components of in-school professional development activities.

5.4 Reflections on Mandinach and Gummer’s (2016) DLFT framework

We have compared the indicators and dimensions arising from the present study and those in Mandinach and Gummer's (2016) “data literacy for teachers” (DLFT) framework. For this purpose, the conceptually similar indicators of Mandinach and Gummer (2016) are included in Table 3 alongside the corresponding indicators identified in this study. As can be seen in Table 3, some indicators were identified in both studies, but there were also notable differences between the two sets of indicators.

Firstly, it appears that there were more fine-grained indicators across the five dimensions arising from the present study, compared to those included in Mandinach and Gummer’s (2016) DLFT framework. For instance, our study identified the importance of teacher knowledge about externally administered assessments and associated policies to guide teacher use of data, which were not a part of Mandinach and Gummer’s (2016) DLFT framework. Overall, 95 indicators of the present study, compared to 59 indicators within Mandinach and Gummer’s (2016) DLFT framework, indicates the level of details incorporated in our framework.

Secondly, perhaps the most important discrepancy is articulated in our Dimension 3 “Dispositions towards Data Use”. We have identified 25 specific indicators under this dimension, which were clustered into confidence, values/belief, trust/respect, and anxiety. These four constructs were identified as the most prominently featured psychological dispositions when teachers deal with data in the school. In Mandinach and Gummer (2016), “Dispositions, habits of mind, or factors that include data use” is mentioned, but they “chose not to include them in the conceptual framework… [due to the nature of] these positions as general to effective teaching, rather than specific to data use. They are likely to influence data literacy but are seen as more general habits of mind of good teaching” (p. 372). As such, their framework did not include dispositions as integral part of teachers’ data literacy. We argue that teacher dispositions are an essential component of teachers’ data literacy. Perhaps this discrepancy may have arisen from the views that the teacher dispositions identified in Mandinach and Gummer (2016) are general teacher qualities – such as “belief that all students can learn”, “belief in data/think critically” and “belief that improvement in education requires a continuous inquiry cycle” (p. 372). On the other hand, teachers’ dispositions in our framework were all specific to data use – such as “confidence in integrating data from multiple sources”, “confidence in discussing data with colleagues”, “trust in principals’ leadership in data use”, “trust in open and honest discussions about data use with colleagues”, and “anxiety in using data to make decision”.

On a related point, and thirdly, our framework has two separate dimensions, one focusing on individuals’ psychological dispositions under “Dimension 3: Dispositions towards Data Use”, and the other centered on behaviors “Dimension 5: Data-Related Behavior”. Most of the indicators under the behavioral dimensions were found to be social interactions, communication, discussion, participation, and engagement, as mentioned above. In Mandinach and Gummer (2016), psychological dispositions (such as belief) and behavioral tendencies (such as ethical use of data, collaboration, and communication skills) were grouped into one dimension of “Dispositions, habits of mind, or factors that include data use”. Considering these, it appears that there was less emphasis on the dispositions and behavioral tendencies in Mandinach and Gummer (2016).

On the other hand, Mandinach and Gummer (2016) offered a fine-grained description of skill-related indicators within their DLFT framework. For example, our indicator of “selecting data appropriate for different purposes” was described with more granularity in the DLFT framework: “understand what data are appropriate”, “use qualitative and quantitative data”, “prioritize data”, and “understand specificity of data to question/problem”. Likewise, our indicator of “describing data patterns” was further divided into “understand data properties”, “access patterns and trends”, “drill down into data” and “examine data” in the DLFT framework. Additionally, two indicators within the Mandinach and Gummer’ (2016) framework—“understand student privacy” and “ethics use of data, including the protection of privacy and confidentiality of data”—did not fit into any of the indicators or dimensions of the present study. This is because we were unable to locate empirical studies that directly examined ethical data management and data use among teachers. Therefore, data ethics issues, which we believe to be an important aspect of teachers’ data literacy, were omitted from our framework.

Finally, we also note the differences between the broad dimensions proposed by Mandinach and Gummer's (2016) DLFT framework and our framework. The DLFT framework consisted of: (a) identifying problems and framing questions, (b) using data, (c) transforming data into information, (d) transforming information into decisions, and (e) evaluating outcomes. These five dimensions are primarily about data skills, which was just one dimension of our framework. Furthermore, their indicator descriptions suggest heavy emphasis on data use to inform teaching and learning. In contrast, our dimensions and indicators illustrate the broader purposes and contexts of teachers' data use, highlighting the significance of fostering teacher dispositions and data-related behaviors through effective leadership and a collaborative school environment. In particular, the detailed descriptors for each of the indicators under Dimensions 3, 4, and 5 of the present study are the strengths of our framework, as they illustrate a wide range of varied and specific purposes and data-related dispositions and behaviors related to teachers’ data literacy; these descriptions are relatively sparse in Mandinach and Gummer (2016).

5.5 Limitations of the present study and future research directions

We acknowledge several limitations of the present study. First, our review focused on empirical studies published in journal articles, and omitted government documents, books, and book chapters and publications by professional organizations. Second, we did not differentiate the studies based on in-service teachers vs. pre-service teachers. Future studies may look into potential differences between these two groups and suggest policy directions and strategies for teacher preparation. Third, teachers may possess discipline-unique capabilities and inclinations, and thus it may be worthwhile to identify teacher characteristics across different disciplines (e.g., Science vs. English) and examine the influences of discipline contexts on teachers’ data use and data literacy. Fourth, exploring teachers’ data literacy required for students at different levels of schooling (e.g., early childhood, primary, and secondary) and for students with diverse needs (e.g., learning difficulties, dyslexia) may provide further insights into the specific expectations within the daily tasks and responsibilities of teachers. Fifth, most of the reviewed studies were conducted in Western or English-speaking countries, and thus our findings may have limited relevance to teacher data literacy in different world regions. Future studies may investigate cross-country characteristics in teachers’ data literacy. Sixth, our research also reveals that current studies of teachers’ data literacy have not explored the possible connections between technological advancements, particularly in AI-based systems, and teachers’ data literacy. This suggests a need to investigate the link between teachers’ data literacy and their proficiency in understanding emerging technologies such as AI-based systems. It is anticipated that discussions on data ethics will emerge as a crucial aspect of teachers’ data literacy in the era of artificial intelligence (AI). Finally, our review did not include, and thus future reviews may examine, system-level contextual factors (e.g., digital technology infrastructure, schools’ socio-economic standing) and their influences on teacher practices in data use.

6 Conclusion

Our review of 83 empirical studies published between 1990 and 2021 produced 95 specific indicators of teachers’ data literacy. The indicators were further categorised into five dimensions: (a) knowledge about data, (b) skills in using data, (c) dispositions towards data use, (d) data applications for various purposes, and (e) data-related behaviors. Our findings suggest that teachers' data literacy encompasses more than just knowledge and skills; it also includes a wide range of dispositions and behaviors. Additionally, teacher data literacy extends beyond assessing student learning outcomes and meeting accountability requirements and includes teachers’ reflection and engagement in professional development.