The COVID-19 pandemic disrupted schooling throughout the world on a scale never seen before (UNESCO 2020a). In NSW government schools, the disruption was relatively short; it took the form of an 8–10 week ‘learning from home’ period in which most students engaged in schooling remotely. In this paper, we examined the effects of the COVID-19 pandemic and learning from home on student achievement in mathematics and reading. Effects on student and teacher well-being, which were substantial, will be the focus of separate papers in order to do justice to the important issues raised.
Although ‘learning loss’ is now part of the 2020 lexicon, together with ‘unprecedented’, ‘pivot’ and ‘you’re on mute’, we have deliberately avoided the expression throughout this paper to guard against literal readings and causing undue worry among parents and the wider community. Students learned and achieved during 2020. They did not go backward or lose what they had learned. Rather, some did not achieve the same level of growth as students in the previous cohort. Most affected, according to our analysis, were Year 3 students in lower ICSEA schools in mathematics. We return to these findings shortly.
The importance of context
Speculation about the impact of COVID-19 and learning from home on student academic achievement has been widespread, relying heavily on evidence and modelling from previous crisis situations. However, the size and scale of disruption caused by COVID-19 is truly unprecedented and cannot directly be compared with these earlier accounts. Our study provides rigorous empirical evidence of what happened to student achievement in Years 3 and 4, in NSW, during the pandemic. While the analysis has implications for countries around the world, we note that extrapolation even within Australia should be approached with care. In the state of Victoria, for example, schools were closed for around 18–20 weeks while schools in the Northern Territory were closed for just four days at the end of Term 1 (Storen and Corrigan 2020). Such contextual differences require vigilance when interpreting research findings.
To date, with the exception of the Dorn et al. (2020) report from the United States and the Engzell et al. (2020) report using data from the Netherlands, we have found no quantitative evidence of the impact of COVID-19 on student academic achievement. Interpreting the results of these (any) studies must take important contextual differences into account. For example, the Dorn et al. (2020) report is based on a secondary analysis of data collected by Curriculum Associates (2020). The data were collected from more than 250,000 students across 28 states in the United States, each with different ‘closedown’ or ‘learning from home’ periods’. In addition, they compare test scores to the average achievement of students in the previous three testing cycles.
The Engzell et al. (2020) analysis shares more similarities with our own, given that both studies are based on data collected before and after an eight-week period of school closure and a relevant comparison group, but the follow-up data in the Netherlands were collected straight after the return to school. Such immediate measures were not possible in our study, given the exclusion of non-essential personnel from schools. Nor did we want to burden teachers or students with additional testing when many were under great stress already.
In our study, students attended school for most of Term 1 and were (mostly) back by Term 3. The follow-up data collection a full term after the return to school therefore represents achievement growth over the entire year, not just during the learning from home period. Before closedown, students and teachers in our study had established relationships and ways of working that would have helped in the shift to learning from home. By contrast, the new school year in the United States, framed by astonishing levels of COVID-19 (at the time of writing, 27.4 million cases and 470,000 deaths) compared with Australia, could be expected to negatively affect student testing. These differences in research design and local circumstances are critical to meaningful comparison of findings.
Predicted versus actual impact on student learning
While it was broadly predicted that students would face some ‘learning loss’ during the COVID-19 learning from home period (Brown et al. 2020; OECD 2020; Pedro Azevedo et al. 2020; Sawchuk 2020; United Nations 2020), our study indicates that growth in student achievement during the 2020 school year varied minimally from growth in achievement in 2019. This result might partly be accounted for by the relatively short closedown period and by the timing of our achievement growth measures, one term after the return to school for most students.
Reading achievement was not significantly different for either Year 3 or Year 4 students. Additional time spent reading, supported by family members, during the learning from home period may have been a factor in these results. Furthermore, there was no apparent effect on mathematics achievement for Year 4 students. The only significant effects were for Year 3 students in mathematics whereby those in mid-ICSEA schools showed an additional two months’ growth and those in low-ICSEA schools showed two months less growth than the comparison schools.
If students fell behind in their learning during closedown, as the Check In assessments in NSW government schools suggested (Baker, 2020), our study indicates that teachers have done an outstanding job in helping students draw level with and even overtake (in the case of students in mid-ICSEA schools in mathematics) expected achievement levels. They have ensured that achievement, at least in maths and reading, is as strong as usual (taking the 2019 cohort to be indicative of student growth in a typical year). Our results also signal the capacity of students to learn despite serious disruption to ‘schooling as usual’. Teacher reports of students’ increased facility with technology as a result of learning from home may have been a factor in the varying achievement growth by ICSEA. Instructional volume might also have contributed to these results. That is, teachers reported spending more time in mathematics and reading during Term 3 and Term 4 than in Term 1, of 2020, and more time than teachers reported in Term 4 of 2019. This increase in subject-specific instructional time is likely to have played a role in students ‘catching up’.
Concern for the most vulnerable
However, as predicted by many commentators (Brown et al. 2020; Schleicher 2020; Sonnemann and Goss 2020), there were some negative effects on student achievement in lower ICSEA (disadvantaged) schools, particularly for younger students. The lower growth in mathematics for Year 3 students in these schools might be explained by the greater challenges faced by families in disadvantaged circumstances who are likely to have been disproportionately impacted by the pandemic (Institute for Social Science Research [ISSR] 2020). It is worth highlighting that our finding of two months' less growth in mathematics in less advantaged schools was associated with a remote learning period of around two months. In contexts where schools were closed for much longer periods (such as in Victoria, the United States and many European nations), research is urgently needed to understand and ameliorate the effects of COVID-19 on the learning of vulnerable students.
The results we obtained for students in regional locations, which follow a similar pattern of extra growth for students in mid-ICSEA schools are noteworthy but less robust given the smaller samples. Stories we heard from teachers of some country kids spending the learning from home period working and playing on the family farm may have been a factor for some.
The result of no significant differences for Indigenous students between 2019 and 2020 is cause for celebration, given that lower growth might have been predicted given, on average, their over-representation in lower ICSEA schools. It is a testament to their families and teachers that no negative effects of COVID-19 and learning from home were evident in their academic achievement. On the other hand, achievement levels for Indigenous students in Australia have consistently been significantly below those of their non-Indigenous peers which means there is still much to do in working towards more equitable outcomes.
In all disadvantaged contexts, ameliorating lower growth in academic achievement is likely to require significant investment in the form of additional support for teachers and students. The recently announced $ 337 million tutoring scheme (NSW Government 2020) has a critical role to play here. It represents a unique opportunity to address longstanding inequities as well as those exacerbated by the pandemic, if done well (Slavin 2020).