Skip to main content

Learning loss during Covid-19: An early systematic review

Abstract

With Covid-19 having caused significant disruption to the global education system, researchers are beginning to become concerned with the impact that this has had on student learning progress and, in particular, whether learning loss has been experienced. To evaluate this, the authors conducted a thorough analysis of recorded learning loss evidence documented between March 2020 and March 2021. This systematic review aims to consolidate available data and to document what has been reported in the literature. Given the novelty of the subject, eight studies were identified; seven of these found evidence of student learning loss among at least some of the participants while one of the seven also found instances of learning gains in a particular subgroup. The remaining study found increased learning gains in their participants. Additionally, four of the studies observed increases in inequality where certain demographics of students experienced learning losses more significant than others. It is determined that further research is needed to increase the quantity of studies produced, their geographical focus, and the numbers of students they observe.

Due to the emergency nature of the Covid-19 pandemic, education systems around the world are facing extreme disruption. At its peak, UNESCO (2020) reported that nearly 1.6 billion learners in more than 190 countries, or 94 percent of the world’s student population, were impacted by educational institution closures. Given the abruptness of the situation, teachers and administrations were unprepared for this transition and were forced to build emergency remote-learning systems almost immediately. In response to this disruption, education researchers are beginning to analyze the impact of these school closures on student learning progress or lack thereof.

The term “learning loss” is commonly used in the literature to describe declines in student knowledge and skills (Pier, Hough, Christian, Bookman, Wilkenfeld, & Miller, 2021). Historic data provides researchers with information regarding where student learning should be year over year and is often measured through regular testing. Learning loss occurs when educational progress does not occur at the same rate at which it has historically compared to previous years (Pier et al., 2021).

Outside of the classroom, these losses may translate to greater long-term challenges. Currie and Thomas (2001) put this into perspective because they observe that a 0.20 standard deviation (SD) decrease in standardized test scores could decrease future employment probability by 0.86 percent. Additionally, Chetty et al. (2014) observe that increasing student achievement by 0.20 SD results on average in a 2.6 percent increase in annual lifetime earnings (Maldonado & De Witte, 2020). Likewise, another year of schooling is, on average, associated with an 8–9 percent gain in future earnings (Psacharopoulos & Patrinos, 2018).

While many researchers have established learning-loss prediction models (Azevedo et al., 2021), formal research and documentation of the actual impact that Covid-19 has had on student learning progress is just beginning to emerge. As the global education system continues to face pandemic-related disruption, a strong understanding of how Covid-19 school closures are impacting student learning progress can better equip educators, policy-makers, and researchers going forward.

Our focus in this article is on an important research question: Have Covid-19 education closures resulted in recorded student learning losses? To answer this, we conducted a thorough analysis of recorded learning-loss evidence documented between March 1, 2020, and March 18, 2021. This systematic review aims to consolidate such data and document what has currently been reported in the literature. To the best of our knowledge, this is the first systematic review of its kind.

Our article makes three important contributions. First, we develop a comprehensive review that consolidates the research that has been presented related to the impact of Covid-19 on student learning progress. Second, we answer the research question, “Has pandemic-related learning loss been recorded in the literature?” Third, based on our review, we identify significant gaps in the literature and provide relevant guidance for further research.

We begin by describing the methods used in this review to identify and collect the articles analyzed. We then present an analytical review where each article was categorized by its geographical region studied, length of school closure, education level of students analyzed, subject analyzed, documented learning impact, presence of increased inequality, and sample size. Next, we move on to our discussion where we review the findings of the analytical review. We then end by discussing areas for future research and summarizing the main ideas of this paper.

Methodology

The initial search was performed using English-language articles published between March 1, 2020, and March 18, 2021. To ensure a comprehensive, multidisciplinary search, the electronic databases included were ECONLIT, Google Scholar, PubMed, Education Resources Information Center, and Cochrane Library. To conduct the search, the keywords “Covid-19”, “coronavirus”, “2019-ncov”, “sars-cov-2”, or “cov-19”, were used in combination with “learning loss”, “learning slide”, “education gap”, or “achievement gap”. Along with this, some studies were identified by reaching out to colleagues and researchers.

When conducting the search analysis, thousands of articles were identified; however, the majority of these pertained to hypothesized or predicted learning loss. To narrow this down to studies with recorded results, article abstracts were then screened. Studies that conducted student analyses and reported impacts on learning progress (either positive, negative, or insignificant) as a result of Covid-19 school disruptions were included. After this screening process, eight articles remained (Table 1). Factors for rejecting studies from our review included the absence of student analyses or recorded impacts on learning progress, analyses occurring before the onset of Covid-19, or hypothesized results.

Table 1 Studies included

Analytical review

Once articles were selected, they were then coded using the classifications in Table 2.

Table 2 Classifications used to analyze studies

As indicated in Table 3, we find that seven out of the eight studies identified learning loss among at least some of the students analyzed. For example, Maldonado and De Witte (2020) found Grade 6 students in Belgium experienced losses of 0.19 SD in math and 0.29 SD in Dutch. Engzell, Frey, and Verhagen (2021) find that, overall, Grade 4–7 students in the Netherlands have encountered an average 0.08 SD learning loss in math, spelling, and reading. Tomasik, Helbling, and Moser (2020) found learning progress of primary-school students in Switzerland during in-person learning to be more than twice as high compared to the progress made during the eight-week school closure. Orlov et al. (2020) determined that economics students at four USA universities were 0.19 SD behind. Kuhfeld, Tarasawa, Johnson, Ruzek, and Lewis (2020) found that Grade 3–8 students in the USA scored 5–10 percentile points below historic levels in math. Gore et al. (2021) found Year 3 students studying math in low ICSEA (Index of Community Socio-Educational Advantage) schools in Australia to be two months behind the progress students made in 2019. Last, Schult, Mahler, Fauth, and Lindner (2021) find learning losses of 0.07 SD in reading comprehension, 0.09 in operations, and 0.03 in numbers for Grade 5 students in Germany. At the university level, in a single university, for 458 students in STEM faculties, learning outcomes actually improved (Gonzalez et al., 2020).

Table 3 Results of literature classification.

Discussion

Learning loss is being experienced. As Table 3 indicates, the early findings of seven studies provide evidence of experienced learning losses among students. These observed losses are occurring across a range of subjects, grade levels, and geographical regions. This signals that although robust and empirical research on Covid-19-related student learning loss is limited, learning loss itself may not be.

Not all students are experiencing learning loss. While the majority of the literature analyzed indicates that students are experiencing some level of learning loss, there were also instances where this was not the case. For example, both Maldonado and De Witte (2020) as well as Kuhfeld et al. (2020) found learning losses in certain subjects but insignificant impacts in others. Likewise, while Tomasik et al. (2020) found primary students to be impacted, they found no impact on secondary students. This is consistent with the literature showing that students in the early grades may be more vulnerable than secondary students because of their inability to seek learning on their own, due to the differences in developmental and cognitive abilities. In their Australian study, Gore et al. (2021) found there to be no evidence of overall learning loss in Year 3 and 4 students in math and reading with the exceptions being Year 3 students in math in low-ICSEA schools who experienced losses, while mid-ICSEA students experienced small gains. Last, in the case of Gonzalez et al. (2020) who studied university students in Spain, it was determined that student learning progress actually improved rather than declined during the Covid-19 learning disruption period, but this was for university students in STEM subjects at one university.

Some students are experiencing more learning loss than others. Of the eight studies, four found instances of inequality, while only one found demographics exclusively to have no impact on learning loss. Gore et al. (2021) found instances of increased inequality as well as instances of no change. The other studies did not specify in this area or in the case of Kuhfeld et al. (2020) found inconclusive and minor differences between ethnic or racial groups. In the four studies where increases in inequality were observed, certain demographics of students experienced losses more significant than others. Maldonado and De Witte (2020) observed inequality within schools rise by 17 percent for math and 20 percent for Dutch. Engzell et al. (2021) determined that losses were up to 60 percent larger among students from uneducated homes. Gore et al. (2021) found the only losses to be among students from low-ICSEA schools where the lower the ICSEA level the lower the educational advantage attending students have due to their parents’ occupation and education, their geographical location, and the school’s proportion of indigenous students (Australian Curriculum, Assessment and Reporting Authority, 2014). Schult et al. (2021) found losses in math among Grade 5 students to be more severe in low achieving students. In reading comprehension, Schult et al. (2021) found more severe losses among middle- to high-achieving students.

More research is needed. In general, the literature representing the impact that Covid-19 has had on student learning progress is limited in the quantity of studies available, geographical regions analyzed, and number of participating students. Given the novelty of the subject, it is understandable why education researchers are only just beginning to analyze the learning losses that students have experienced. However, a stronger understanding of how Covid-19 school disruptions have impacted student learning is still needed. To support this, more studies are needed.

Along with this, the current studies that are available are limited in their geographical span. The only limited information that is currently available is from Belgium, The Netherlands, Switzerland, Spain, the United States, and Australia. Given the differences in educational institutions between countries, in terms of quality, length of school closures, and remote-learning strategies, it is crucial that researchers continue to investigate Covid-19-related learning loss in countries where limited research exists.

Last, many of the studies themselves who were analyzed in this systematic review had limited numbers of participants. For example, Gonzalez et al. (2020) analyzed just 458 students at one university. Similarly, Orlov et al. (2020) observed economics students in just seven classes across four universities. While the information these studies presented remains relevant to their observed samples, research that can more accurately represent larger groups of students remains crucial to policy-makers. As such, there is a demand for studies that analyze representative groups of students.

Conclusion

Through conducting a thorough analysis of recorded learning-loss evidence documented between March 2020 and March 2021, this systematic review provided a consolidated audit of available research on Covid-19-related learning loss. Given the novelty of the subject, eight studies were identified. Seven of the eight found evidence of student learning loss among participants while one of these found instances of learning gains in a particular subgroup. The remaining study observed learning gains among university students. Along with this, four of the studies observed increases in inequality where certain demographics of students experienced learning losses more significant than others. Further research is needed to increase the quantity of studies produced, their geographical focus, and the numbers of students they observe.

References

  • Australian Curriculum, Assessment and Reporting Authority (2014). What does the ICSEA value mean? https://docs.acara.edu.au/resources/About_icsea_2014.pdf

  • Azevedo, J. P., Hasan, A., Goldemberg, D., Geven, K., & Iqbal, S. A. (2021). Simulating the potential impacts of Covid-19 school closures on schooling and learning outcomes: A set of global estimates. World Bank Research Observer, 36, 1–40.

    Google Scholar 

  • Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104, 2633–2679.

    Article  Google Scholar 

  • Currie, J., & Thomas, D. (2001). Early test scores, school quality and SES: Longrun effects on wage and employment outcomes. Research in Labor Economics, 20, 103–132.

    Article  Google Scholar 

  • Engzell, P., Frey, A., & Verhagen, M. (2021). Learning inequality during the Covid-19 pandemic. Proceedings of the National Academy of Sciences, 118(17).

  • Gonzalez, T., de la Rubia, M. A., Hincz, K. P., Comas-Lopez, M., Subirats, L., Fort, S., & Sacha, G. M. (2020). Influence of Covid-19 confinement on students’ performance in higher education. PLoS ONE, 15(10), e0239490.

    Article  Google Scholar 

  • Gore, J., Fray, L., Miller, D., Harris, J., & Taggart, W. (2021). The impact of Covid-19 on student learning in New South Wales primary schools: An empirical study. The Australian Educational Researcher, 48, 605–637.

    Article  Google Scholar 

  • Kuhfeld, M., Tarasawa, B., Johnson, A., Ruzek, E., & Lewis, K. (2020). Learning during Covid-19: Initial findings on students’ reading and math achievement and growth. NWEA Research. https://www.nwea.org/content/uploads/2020/11/Collaborative-brief-Learning-during-COVID-19.NOV2020.pdf

  • Maldonado, J., & De Witte, K. (2020). The effect of school closures on standardised student test outcomes. KU Leuven Faculty of Economics and Business discussion paper series DPS20, 17. https://feb.kuleuven.be/research/economics/ces/documents/DPS/2020/dps2017.pdf

  • Orlov, G., McKee, D., Berry, J., Boyle, A., DiCiccio, T., Ransom, T., Reese-Jones, A., & Stoye, J. (2020). Learning during the Covid-19 pandemic: It is not who you teach, but how you teach. NBER working paper 28022.

  • Pier, L., Hough, H. J., Christian, M., Bookman, N., Wilkenfeld, B., & Miller, R. (2021). Covid-19 and the educational equity crisis: Evidence on learning loss from the CORE data collaborative. Policy Analysis for California Education. https://edpolicyinca.org/newsroom/covid-19-and-educational-equity-crisis

  • Psacharopoulos, G., & Patrinos, H. (2018). Returns to investment in education: A decennial review of the global literature. Education Economics, 26, 445–458.

    Article  Google Scholar 

  • Schult, J., Mahler, N., Fauth, B., & Lindner, M. (2021). Did students learn less during the Covid-19 pandemic? Reading and math competencies before and after the first pandemic wave. PsyArXiv, 11. https://doi.org/10.31234/osf.io/pqtgf

  • Tomasik, M., Helbling, L., & Moser, U. (2020). Educational gains of in-person vs. distance learning in primary and secondary schools: A natural experiment during the Covid-19 pandemic school closures in Switzerland. International Journal of Psychology, 56(4), 566–576.

    Article  Google Scholar 

  • UNESCO (2020). UN Secretary-General warns of education catastrophe, pointing to UNESCO estimate of 24 million learners at risk of dropping out. https://en.unesco.org/news/secretary-general-warns-education-catastrophe-pointing-unesco-estimate-24-million-learners-0

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Harry Anthony Patrinos.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Verify currency and authenticity via CrossMark

Cite this article

Donnelly, R., Patrinos, H.A. Learning loss during Covid-19: An early systematic review. Prospects (2021). https://doi.org/10.1007/s11125-021-09582-6

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11125-021-09582-6

Keywords

  • Covid-19
  • Education
  • Learning loss
  • Systematic review