Keywords

1 Introduction

School closures as a result of the coronavirus disease (COVID-19) pandemic have caused unprecedented education disruption for 1.6 billion children (or 91.3% of enrolled learners worldwide, according to the United Nations Educational, Scientific and Cultural Organization (UNESCO 2020a). Lessons from the Ebola crisis show that impacts on education from school closures have the potential to be far-reaching and long-lasting, including loss of learning gains and an increase in school drop-outs, particularly for girls. But COVID-19 comes on top of a preexisting crisis: before the pandemic, there were already 258 million children and adolescents around the world out of school (UNESCO 2019). In other words, we currently have the very opposite of a learning society in this current age of disruption.

Technology has been heralded as a powerful complement to education systems, seen as one of the most viable means of quickly reaching out-of-school children. There are already good examples of this happening. In India, a study of a free, after-school program that introduced Mindspark, a digital personalized learning software, showed improvements in mathematics assessment scores of up to 38% in less than 5 months, thereby dramatically reducing students’ learning gaps (Muralidharan et al. 2019).

However, the history of education technology or edtech is also littered with failures. This chapter will argue that technology will only deliver enhanced learning outcomes—including for marginalized children—if there are effective data and content systems in place. Actionable data in the hands of the right people in the system (teachers and education ministries) can make for vastly improved decision making and accountability. Careful and deliberate low-cost data collection will make it possible to continuously monitor and improve at every level. Good data can also allow personalized lessons and content. These are the kinds of positive disruptions that governments want if they are looking to transform their education outcomes.

Currently however, the opposite is more likely to be the case: many countries are working from a low baseline in effectively capturing and using data. Decision makers rarely have useful data on whether learning outcomes are improving. Even if data is collected, it is rarely acted upon. For example, Edo State in Nigeria has conducted annual teacher performance evaluations, but these evaluations did not affect decisions about workforce management and improvement (Reboot 2013).

Unleashing the full potential of data, in turn, will also require a focus on creating the right enabling digital foundations, including data governance and privacy rules. These data frameworks will enable system managers to learn from experience, compare interventions, and set standardized criteria for success, boosting their ability to understand their system and implement solutions at scale.

2 Managing Education Systems

“Technology for education” has traditionally materialized in large-scale investments in infrastructure. A well-known example is the “One Laptop Per Child” (OLPC) program that saw 2 million children and teachers in 42 countries equipped with laptops (OLPC 2020). Although this program has now closed, countries are still heavily investing in hardware (for instance, Japan in 2019; see Japan News2019).

3 The Problem of Focusing Investment on Digital Hardware

If anything, the focus on digital hardware has increased during the COVID-19 school closures, and for good reason: when education moves online, students need a computer and internet access. Nearly all (87%) of educational systems surveyed by UNESCO during school closures (UNESCO 2020b) reported that unequal access to digital infrastructure at home was the main barrier toward continued learning. At that time, many countries were investing in a rapid response to boost digital infrastructure, focusing on connectivity rather than hardware: Sri Lanka provided free internet access to the Ministry of Education website, and Maldives provided mobile data to parents.

In the past, large-scale investments in digital infrastructure have tended to fail to improve learning, unveiling a key lesson for policy making, which is that systems are complex. Programs such as OLPC have had little effects on test scores in math or reading, as an evaluation in Peru revealed (Cristia et al. 2017). These programs failed for a diverse range of reasons, from lack of electricity to lack of skills, or disinterest among the teachers. In effect, the real lesson is one step further: that learning is the product of an educational system, in all its complexities, and not of a textbook or a computer alone (Pritchett 2015).

3.1 Systems Approach to Investing in Education

Academic research and policy research in education is increasingly taking a “systems approach” (Mansoor and Williams 2018), but this sort of thinking is only just beginning to permeate actual policymaking (Pathways for Prosperity Commission 2019). Two key lessons emerge from the literature on systems: one, that any singular intervention will only produce results if it directly responds to a specific binding constraint within the system (e.g., Mansoor and Williams 2018; Pritchett 2015); and two, that as a principle, improving isolated components will be insufficient without concerted changes across the system.

One potential starting point for policy makers is to identify key binding constraints within their educational systems, and analyze how these may be addressed with technology. In developing countries, common binding constraints include low motivation or skills of teachers or poor involvement from students. Digital technologies can help address some of these gaps more efficiently than other policies, but only when well-integrated within the system. For instance, if attendance is a binding constraint, digital systems can monitor and report on teacher and student attendance levels to help principals and other decision makers manage their schools better (Muralihdaran et al. 2019; Duflo et al. 2012; Kisakye et al. 2016; Centre for Education Innovation 2018). However, such interventions should not just be about creating digital attendance police: they work better if complemented with other interventions, such as personal feedback to teachers (World Bank 2016; Pathways for Prosperity Commission 2019).

In order to manage complex educational systems, data is essential. Without data, it is difficult to even begin identifying binding constraints. In the aftermath of a rapidly evolving situation such as pandemic school closures, data becomes indispensable for answering the most pressing questions (David et al. 2020), including which children have fallen behind, whether they are catching up, or whether the divides are deepening. Good data systems can uncover patterns; for instance, a school that started off poorly on teacher preparedness during COVID-19 might consequently be less able to produce the necessary materials, and eventually fall behind on multiple aspects of learning in the aftermath. The only way to capture this complexity and use it to inform system management is through data.

3.2 Using Data to Manage Public Systems

3.2.1 Identifying Potential Data Sources

As described earlier, one of the biggest opportunities to improve education systems lies in better systems management and decision making. This is not just true for education, but really for all national-scale public systems—from health to social services. Better data and information can power better decision making, and sometimes even automate it. Policy makers often approach data as a new entity: something around which grand systems of collection and specification need to be built. But it can be more productive to start by looking at what data already exists, or could easily be recorded, within existing systems. Only with strong foundations of basic data can countries work on more powerful analytical tools. In an education system, “basic data” might be simple objective truths: a register of schools and students, records of attendance, and exams taken.

In many systems (including education) there are a variety of different actors who might have useful data already. In addition to government service providers (such as schools), there are also private providers (e.g., private tutors); third-party digital tools (e.g., edtech apps); and completely separate sources of data that could be relevant (e.g., information about childhood nutrition). Many of these actors will already collect data—such as a staff management system for an NGO, or student progress database for an edtech app. The challenge is to put this data to use. For governments, this means focusing on standardization and aggregation.

3.2.2 Standardizing Data

The first thing this requires is simple data standards and registries. Without standardized data, much of the information generated by different actors in a system would be completely incompatible (Fry 2018; Cvitkovic 2019). Imagine a health system: if every NGO has a unique way of recording hospital districts, then their pooled data will not be able to generate useful data about each district. At its most basic, governments can build standard registries (for example, of students, or schools, or curriculum modules) that can be used by others. The second component would be to promulgate basic specifications for how data should be formatted and transmitted for aggregation. If the data from each school, or each community health worker, had the same structure and meaning, it would enable powerful analytical tools. But no one in the system wants to spend hours submitting government reporting forms. Data standards allow each exam board, hospital administrator, edtech app, or social services district office to structure their digital systems however they like, according to their own needs, provided they are able to transmit data that fits the standard specifications.

3.2.3 Data Protection and Governance

Finally, such a system needs appropriate governance and safeguards. Citizens need to be assured that their personal and sensitive data is safe from an impulsive bureaucrat who wants to snoop. Simple anonymization such as encrypting identifying information may not be enough. Policy makers will need to build robust legal and technical safeguards to ensure that data is only accessed to certain users in certain ways. For instance, perhaps only a student and their teacher should be able to see the student’s individual scores, whereas the district administrators are only ever presented with aggregate data. These protections need to be in place early, as in India, which had to change significant parts of their digital identification system, at huge cost, when the courts ruled that their initial system did not have satisfactory protections (Privacy International 2018).

3.3 Data in Education Systems

3.3.1 Aggregate Data and Metrics

Managing and improving educational systems with the use of data starts at the level of deciding what to track; and individual student progress is one of the most important missing data points. As policy makers start to set up their systems to tap into the power of data, this should be a priority.

Educational progress is mostly tracked at a nationally aggregate level. Recent innovations such as the metric learning-adjusted years of schooling (LAYS) have helped improve our ability to compare across countries, by seeing how much students are learning, on average, compared to how much they were expected to learn given their years of schooling (Filmer et al. 2020; Angrist et al. 2019). This sort of data can highlight the global learning crisis and mobilize resources. (Pritchett 2015); however, it is not enough. As highlighted by the Pathways for Prosperity Commission (2019), there is one more fundamental problem: that measuring learning only at a nationally aggregate level is only reflective of outcomes, and does not give any insight into the individual causes of the learning crisis. Rather, to be informative for decision making, data needs to be collected at the individual student level, where the correlates of poor performance can shed light on what needs to be improved.

3.3.2 Tracking Individual Student Progress

Collecting data on individual students’ learning, and combining this with insights into systemic feedback loops can help improve learning. The literature on evaluation of digital educational technologies is sparse, although some though digital platforms used in resource-constrained contexts are already helping to bridge the gap between expected years of schooling and LAYS, as Tusome did in Kenya. Tusome involves multiple components: tracking of individual student progress; serving as a repository of digital content and materials; training of teachers to use it; and maintaining a centralized data management system that monitors schools and counties’ improvements in student literacy. Early analysis suggests Tusome has been a success, helping 7 million Kenyan children (Piper et al. 2018).

Tools such as Mindspark (mentioned in the introduction) have similarly used their data to track and respond to individual student progress; while VISHWAS, a data collection system in India, uses devices with global positioning system or GPS capability to collect data and improve the performance of not just teachers but also district-level administrators (Centre for Education Innovations 2018). To make such promising outcomes possible, these programs often address multiple components of a system concomitantly, focusing on the information and the specific use case, for a machine, rather than investing in hardware for hardware’s sake. Moreover, they enable feedback loops, not just collect data; and provide this back to users in accessible ways (tailored lesson plans for students, specific feedback for teachers, and statistical dashboards for administrators).

3.3.3 Making Connections and Putting Information to Use

A first step toward a data-driven educational system management requires effective “digital plumbing” with teachers at the heart of it. The COVID-19 pandemic has unveiled two key facts about educational systems and technology: (i) educational systems worldwide lacked the foundations to become digital—skills, infrastructure, and procedures; and (ii) the success of any transition to digital education is entirely at the hands of teachers (World Bank 2020). There was also another insight: that some “champion” teachers emerge, and if their know-how can be used to support the teaching community, a whole system may benefit.

There are examples of digital programs put in place with teachers at their center, enabled by the right “digital plumbing”. One such example is Sunbird, a platform developed in India to bring information to teachers, pulling together courses, lesson plans, assessment materials, teacher training, and dashboards for progress (Sunbird 2020). The platform’s most important feature is interoperability, allowing the integration of digital infrastructure across the educational system and embedding data collection all throughout. The good news is that, with such a platform, if one digital product works well, or if a “champion teacher’s” performance is outstanding, lessons can be learned rapidly, and benefits can be transferred throughout the system.

3.4 Disrupting Education

3.4.1 Data Can Power a Future Learning Society

When done right, technology does have the potential to positively disrupt education and lead to the learning societies of the future. There are several key visions for how digital technologies can revolutionize the provision of educational services (Pathways for Prosperity Commission 2019).

First and foremost is by enabling proactive systems. Fueled by real-time data and feedback loops from teachers all the way to policy makers—and everyone in between—a system is able to adapt in real time to the needs of the students. Equipped with data insights, teachers and school managers can actively identify at-risk students in order to ensure their inclusion.

Second, virtual systems will become ever more important to reach all children. Indeed, because of the COVID-19 pandemic, almost every child in the world has had their learning disrupted. Virtual delivery, if done well, can ensure a more equal quality of service delivery throughout a country, for instance through videoconference-based lessons with the best teachers.

Third, adaptive software or apps based on machine learning algorithms are already revealing the vision of personalized systems, where lessons are tailored for each individual student’s needs, helping to fill their particular learning gaps and build on their individual strengths.

3.4.2 Make Sure Learning Societies are Inclusive

Educational systems were set in place to serve all children, and with current digital technologies, inclusive systems may finally become a reality. Inequalities are inherent to educational systems: not all schools are equally resourced, not all teachers are equally prepared, not all students have the same enabling environment at home. The use of data, however, can identify these gaps in real time, enabling policy makers to proactively target resource allocation.

Data can help identify not only which schools or students lag behind, but also precisely on which subjects (e.g., math more than reading) and which specific topics (e.g., multiplication). Such software exists already in pockets. These apps assess a students’ current level on a range of topics and give targeted lessons and practice exercises, and the teacher can access each students’ data to guide them. For a traditional educational system, such a precise level of personalized care would have been an impossible task. Technology may be the only conceivable pathway to inclusive education, and it may also become an increasingly realistic pathway: research currently shows that once scaled up, personalized software can become relatively inexpensive.

Last but not least, educational technologies hold a new promise for closing gender gaps in learning. Worldwide, one of the widest gender gaps in employment is in science, technology, engineering, and mathematics (STEM) careers. For instance, in Malaysia, in 2015, only 10% of engineers were female, and in 2020 only around 20% of STEM researchers across Asia were female (UNESCO 2015; Ugwuegbula 2020). The gender gap in math is notoriously difficult to bridge. However, very recently, the personalized learning software “onebillion” prevented the gap from surfacing among first-graders in Malawi, compared to traditional classroom teaching where the gap did emerge (Pitchford et al. 2019).

4 Conclusions

This chapter has explored the implications of rapidly increasing the use of data in education systems to enable the promise of edtech to be realized. When solutions are built around meaningful systems and information, then data can power transformational applications, from personalized content for students to advanced analytics and insights for decision makers. This is more than simply investing in hardware and scattering computers across schools. Building a learning society requires deliberate and thoughtful investment in systems.

4.1 Focus on Data and “Digital Plumbing”

There are a clear set of policy reforms and initiatives that education leaders can pursue. They need to identify sources of data, invest in “digital plumbing”, standardize the information, and make it available for decision makers and teachers. This may sound trivial, but it can enable better teaching, better resource allocation between schools, and better systems management by government officials. It also builds the digital foundations for a learning society of the future.

4.2 Ensure Inclusiveness

However, there will need to be a specific effort made to reach those marginalized groups. Edtech will not automatically get children into learning, and indeed may increase educational (and other) inequalities rather than close them. If the same social norms that prohibit girls from walking longer distances to attend secondary school also limit their access to mobile technology, inequalities will not merely remain but may even be exacerbated. Instead, to maximize benefits from digital transformation, governments will need to make explicit efforts to engage with and build solutions for people who are typically left behind. When conducting pilots, a focus on easy-to-reach groups (such as the urban or middle class population) could at times be defended, for example, to offer a proof of concept or collect evidence on possible impact. New technological solutions will need to be designed to scale to reach all citizens, not just those with higher incomes who enjoy a personal computer, English literacy, and high-speed internet access.

Indeed, if delivered well, the disruptive use of data in education systems can power learning societies and bring hope for inclusion.