In this section, we summarize the main results according to the four research sub-questions on teaching practices, teacher beliefs, didactics, and assessment. In Sect. 4.5, we present the results across these perspectives to answer our initial hypotheses.
Teaching practices at a distance
To describe the teaching practices in times of school lockdown, we distinguish how the teachers set up their teaching (cf. the didactical configuration in the IO model) and the ways in which they used this setup (cf. the IO exploitation mode).
To get more insight into how teachers set up their distance teaching, we asked them how they delivered their math teaching before and since the schools were closed (questionnaire items T3 and T4). Of the German teachers, 3% indicated they did not teach anymore since the school lockdown. In Flanders and the Netherlands, all respondents were still teaching. Table 2 shows which tools were used for delivering the teaching in each country before and since school closure. For each item, the percentage of teachers who use the tool is reported.
Table 2 Delivery tools before and since school lockdown in the participating countries (items T3 and T4, N = 1719) In each of the three countries, math teachers indicate a strong increase in the use of video conferencing software, but to a lesser extent in Germany than in Flanders and the Netherlands. In Flanders, and to a lesser extent in Germany and the Netherlands, there is also a big increase in the use of homemade video clips. After the schools were closed, online learning environments (e.g., Desmos, DWO, GeoGebra Books, GeoGebra Tube, Math4All) and audience response systems (e.g., Socrative, Mentimeter, Kahoot!) have been used less in the three countries.
When it comes to the use of online exercise platforms, online video clips, social media, and learning management systems, some discrepancies between the countries can be noted.
-
The use of online exercise platforms (e.g., AlgebraKit, Bettermarks, DWO, Sowieso, software from the textbook publishing company) decreased remarkably in the Netherlands and Flanders while it slightly increased in Germany.
-
The use of already available online video clips increased the most in Flanders, where online video clips had been used only rarely before the pandemic. An increase can also be observed for Germany, while no important change was observed for the Netherlands.
-
Social media use for mathematical learning decreased in Flanders and the Netherlands whereas it slightly increased in Germany. The use of learning management systems increased in Flanders and Germany while it dropped in the Netherlands, probably because learning management systems had been used by a wide group of teachers in the Netherlands already before the crisis (94%).
When it comes to delivery through asynchronous teaching formats (e.g., forum, sending out exercises via mail), the majority of the teachers (FL: 84%, GE: 92%, NL: 72%) used asynchronous formats weekly after the schools closed. However, the use of synchronous formats (e.g., video conferencing, simultaneous working with students in a shared document, live chats) differed between the countries (see Fig. 2). Remarkably, almost one-third of the German teachers (31%) did not use synchronous formats, compared to 9% in Flanders and 3% in the Netherlands. Only 47% of the German teachers used them every week, compared to 82% of the Flemish and 94% of the Dutch teachers. The use of asynchronous practices per country is shown in Fig. 3.
Most Flemish mathematics teachers (65%) used Smartschool Live, local software available to nearly all schools (item T6) as a video conferencing system to connect with students. In the Netherlands, almost two-thirds (65%) of the mathematics teachers used Microsoft Teams. In Germany, various systems were used such as Microsoft Teams (14%), Zoom (13%), and Jitsi (10%).
For item T7, the majority of the teachers in the Netherlands prepared mathematics lessons at distance by setting up groups for video conferencing (73%), setting up rules for behavior (66%), and giving instructions on how to use the platform (61%). In Flanders, most teachers gave instructions (64%) and set up rules for behavior (53%), and 50% of the Flemish mathematics teachers set up groups for video conferencing. As 40% of the German teachers did not have distant mathematics lessons yet, the overall percentages are lower here: 43% set up rules for behavior, 42% gave instructions on how to use the platform, and 41% set up groups for video conferencing.
As a next step, we investigated how teachers used the video conferencing environment during teaching. Figure 4 summarizes the results of item T8. In the three countries, video conferencing lessons are mainly used for answering questions (FL: 91%, GE: 58%, NL: 96%), giving lectures to explain mathematical topics (75%, 39%, 90%, resp.), showing solutions to tasks (59%, 42%, 73%, resp.), asking to use online content (41%, 26%, 66%, resp.), and speaking with students about their progress and their way of working (61%, 35%, 62%, resp.). Activities such as students showing and presenting their work (8%, 21%, 18%) and engaging students in group work (6%, 8%, 10%) are sparse. Once more, the overall percentages for Germany are low here due to the number of German teachers who did not have video conferences so far.
To summarize the findings on teacher practices, the main result is a drastic increase of the use of synchronous formats such as video conferencing (in Germany to a more limited extent), and a decrease in the use of mathematical tools embedded in online exercise platforms and learning environments, as well as of social media.
Teacher beliefs
As a starting point to consider teachers’ beliefs, item T1 in the questionnaire asks to what extent teachers like working with technology. Item T14 concerns their beliefs on how mathematics education at a distance provides opportunities for specific learning practices. We focused on the following practices: teaching algorithms (T14_1), focusing on mathematical concepts (T14_2), focusing on argumentation and reasoning (T14_3), working with complex mathematical tasks (T14_4), letting students discover mathematics on their own (T14_5), and letting students learn from their own mistakes (T14_6). In addition, one item referred to teachers’ beliefs about digital assessment (T18) and how much these beliefs changed (T19). Furthermore, teachers were asked about their confidence in using digital technology before the closing of schools (T9), their current confidence (T10) and their confidence for using digital formative assessment formats (T20), and the respective change of this confidence (T21). Table 3 gives an overview of the results on these items.
Table 3 Mean results on items related to teacher beliefs In general, teachers stated that they liked working with technology (T1), and this opinion correlated positively with their confidence (T9): τ = 0.42. Teachers’ confidence before and during lockdown increased from 58 (T9) to 73 (T10) on a scale ranging from 0 to 100 and the two correlated (τ = 0.60). In particular, of the teachers who were not confident before the schools closed (30% of the sample, scoring less than 50 on T9), 69% indicated being confident during the school closures (T10 ≥ 50). Furthermore, teachers were quite positive on distance learning supporting teaching algorithms (T14_1) and providing means to make students discover mathematics on their own (T14_5). However, teachers were more negative about the opportunities of distance learning for teaching complex mathematical tasks (T14_4). Teachers’ confidence in using digital technology clearly increased during the closing of the schools (T9-T10). Their confidence on using digital formative assessment formats, however, was not very high (T20), and—even more important—increased only to a limited extent (T21).
Country differences appeared: German teachers were less positive about distance learning, particularly for teaching mathematical concepts (T14_2) and argumentation and reasoning (T14_3). In addition, the confidence for using digital technology (T9, T10) of German teachers did not increase as much as in Flanders and the Netherlands. However, German teachers were particularly confident in using digital formative assessment formats (T20). To summarize, the findings suggest positive changes in teachers’ beliefs and confidences.
Didactical approaches to distance mathematics education
Questionnaire items T13, T16, and T17 concerned the teachers’ didactical approaches. Item T13 asks if mathematics teaching in times of school closure focused on rehearsing and practicing (T13_1), new topics (T13_2), conceptual understanding (T13_3), procedures and algorithms (T13_4), argumentation and reasoning (T13_5), and authentic complex tasks and modeling (T13_6). The results (see Table 4) show that teachers report paying attention to rehearsing, practicing, procedures, and algorithms (T13_1 and T13_4). In the meantime, they also claim a focus on new topics and conceptual understanding (T13_2 and T13_3). Finally, attention to argumentation and reasoning (T13_5) and authentic complex tasks (T13_6) is limited. However, we do not know whether this limited attention is a change compared to regular practices before school lockdown.
Table 4 Mean results on the focus in mathematics teaching during lockdown per country (item T13) The results on T13_1 show a remarkable difference between Germany and the other two countries (see Fig. 5). German teachers report more focus on rehearsing and practicing than their colleagues in other countries. An explanation might be the guidelines provided by some German ministries to do so (see Sect. 3.1).
Item T16 investigates interactive formats used in the distant mathematics lessons, and in particular learning intentions and success criteria (T16_1), discussions and tasks (T16_2), feedback (T16_3), peer instruction (T16_4), self-checking (T16_5), and adapting teaching to formative assessment (T16_6). The mean values in Table 5 suggest that the interactive practices did occur on a more or less regular basis, but that giving peers a role in the learning process was only exploited to a limited extent.
Table 5 Mean results on interaction formats in mathematics teaching during lockdown (item T16) These findings hold for all countries, with the practice of having discussions and tasks to foster conceptual understanding as an exception (T16_2): Fig. 6 shows that German teachers paid little attention to this, probably caused by the limited use of synchronous video conferencing in Germany (see Sect. 4.1).
Quite some attention is spent on activating students as responsible for their learning (Wiliam & Thompson, 2008), especially in the Netherlands (T16_5). However, teachers relatively seldom adapt their own teaching based on the results of formative assessments (T16_6).
To summarize the results on the didactical approach to distance mathematics teaching, the initial hypothesis that teachers would focus on procedures and algorithms and on rehearsing and practicing skills is not confirmed, even if there are some indicators in this direction, in particular for the case of assessment. There is little attention for argumentation and reasoning, and for authentic, complex tasks. Some results suggest that the limited means for interaction through digital technology may hinder didactical approaches, probably mostly in countries where video conferencing is not used frequently, like Germany.
Assessment
Item T15 asks for the methods the teachers used to keep track of the mathematical learning and as such refers to formative assessment. The following methods are included: gathering written results (T15_1), live questions during video conferences (T15_2), live chat during video conferences (T15_3), assessment questions in the schools’ learning management system (T15_4), online learning environments (T15_5), commercial tutorial systems (T15_6), and audience response systems (T15_7). Table 6 summarizes the results.
Table 6 Mean results of methods of formative assessment (item T15) The most common method of assessment is gathering results from students (e.g., via email or a cloud system like Dropbox). A less frequently used approach was to keep track of students’ mathematical learning using assessment items in the schools’ learning management system or using live questions during a video conference (either verbally or using the chat window). Even more rare was the use of online learning environments, commercial tutorial systems, or audience response systems. Not surprisingly, formative assessment through video conferencing was used much less by German teachers, compared to their Flemish and Dutch colleagues.
Item T17 investigates didactical aspects of assessment, and specifically addresses procedures and algorithms (T17_1), conceptual understanding (T17_2), argumentation and reasoning (T17_3), and authentic, complex tasks and modeling (T17_4). The results, shown in Table 7, are in line with the findings for item T13 on similar sub-items for teaching: teachers report paying attention to both procedures and algorithms, and to conceptual understanding in their assessment, even if the first seems to gain some more attention than the latter, and in assessment less than in teaching (item T13). This once more nuances our hypothesis that mathematics education at distance might focus on procedural skills at the cost of conceptual understanding. Also, in line with the results of T13 is the limited attention to argumentation and reasoning and to authentic, complex tasks (e.g., modeling tasks) in assessment. All in all, these results are very similar in all three countries.
Table 7 Mean results of didactical approaches to assessment (item T17) To summarize, the most-used formative assessment practices in distance mathematics include gathering student materials and video conferencing (with an exception for Germany). Other options, as suggested in the questionnaire, were rarely used. Assessment focused on procedural skills and a little less on conceptual understanding. There is little emphasis on argumentation and reasoning or other demanding activities such as modeling. Taking into account the limited confidence teachers have in their formative assessment skills through digital means (see Sect. 4.2), the overall picture is that formative assessment is an issue in distance mathematics education.
Since we mainly focused on formative assessment, we know little about summative assessment practices. However, summative assessment in the form of high-stake tests was not recommended by the administration in most cases (see Sect. 3.1).
Crossing boundaries between the four perspectives
In this section, we explore the interplay between the four perspectives that impact teaching practice, that is, instrumental orchestrations, teacher beliefs, didactical approaches, and assessment. To do so, we focus on variables that seem to be the most relevant ones in terms of our hypotheses. For instrumental orchestrations, we include school technical facilities and support (T2) and the use of synchronous and asynchronous teaching formats (T5). For teacher beliefs and confidence, we concentrate on the corresponding items (T1, T9, T10, T14) and include teachers’ previous experience with digital tools (T3). For didactics, we focus on procedural versus conceptual approaches, and practicing old topics versus introducing new topics (T13). Concerning assessment, the items T18 and T20 are included. The overall Kendall’s Tau-b correlation matrix for these variables can be found in Appendix 3 (Table 10). In the matrix, values of 0.2 or higher are printed in bold, and the ones of − 0.2 or below in italic.
Let us first consider the interplay between school technical facilities and support on the one hand, and synchronous and asynchronous orchestrations on the other. We would expect the correlation between school facilities and support on the one hand (T2_1 and T2_2) with synchronous distant orchestrations on the other (T5_1) to be important. However, for the three countries together, these correlations are small (Kendall’s Tau-b < 0.2). For Flanders and the Netherlands, a possible explanation might lie in the widespread availability of video conferencing software such as Smartschool Live (Flanders) and Microsoft Teams (the Netherlands). For Germany, this correlation is 0.2, which we interpret as modest support for the conjecture that school support did play a role in Germany in facilitating video conferencing.
Second, we wonder how teachers’ beliefs and confidence relate to their (a)synchronous orchestrations and didactical approaches. Teachers’ beliefs are expressed in their appreciation of using digital technology (T1), their prior experience with digital tools (T3), and their prior confidence (T9). Despite our expectations, however, Appendix 3 shows positive but low correlations with synchronous orchestrations (T5_1), and with didactical approaches such as treating new topics under challenging circumstances or focusing on conceptual understanding (T13). As a possible explanation, it might be that teachers, no matter what their previous experience was, were under so much pressure in the lockdown situation that they faced the new challenges, despite their possible doubts or limited experience. Overall, we found little evidence that teachers’ preparation for the new situation in terms of prior views and experiences was decisive in their orchestrations and didactics.
Third, we consider teachers’ beliefs and confidence before school closure and their views on digital assessment. As shown in Table 8, teachers’ general opinion and prior confidence (T1, T9) positively correlated with their views on the opportunities for distance formative assessment (T18) and on their confidence in the ability to use them (T20). We interpret this as a support for the conjecture that teachers’ views on using digital technology in general coincide with their views on distant formative assessment in particular.
Table 8 Kendall’s Tau-b correlations for teachers’ general opinion (T1) and prior confidence (T9), and opinion on (T18) and confidence in (T20) formative assessment (in bold if > 0.20)