Keywords

1 Introduction

In March 2020 state officials declared an end to on-site schooling throughout Germany as a measure to contain the COVID-19 pandemic. As a result, schools were forced to switch to distance learning methods. In schools of secondary education teaching was supposed to be conducted with digital tools, such as learning management systems and live video classes. Most schools did not have the technical and organizational infrastructure to support such a rapid change (Delcker & Ifenthaler, 2020). Further, teachers and students did not possess the necessary competencies to learn and teach online (Howard et al., 2020; Huber & Helm, 2020). The situation changed gradually as stakeholders adapted to the new situation under the ongoing pandemic.

In response to the on-going crises situation at German schools, this project was conducted to examine how school development may facilitate online learning for students and teachers, focusing on the technological, personal, educational, organizational and cooperative domains of school development (Eickelmann & Gericke, 2018). One of the main issues for school leaders during the COVID-19 pandemic was a decision-making process which could hardly be based on past experiences at the respective school or at similar schools. While school leaders implemented rules for the usage of technology, set up guidelines for communication and tried to provide information for the different stakeholders to enable distance learning at their schools, their options to evaluate their managerial decisions with regard to digitalization in school development were very limited. At that time, school officials in Germany could not provide an appropriate tool yet. The resources of school leaders, especially the time and staff necessary to design, test and implement such a tool, were needed in other domains of the school to keep day-to-day operations running during the COVID-19 pandemic.

One of the main goals pursued by project was to develop and provide a tool for decision makers and school leaders at vocational schools to evaluate the status of continuous school development. School leaders are defined as the school managers of the participating schools. At most vocational schools these school managers are a single principal and a small team of deputies. The evaluation helps school leaders to use empirical evidence to reflect on managerial decisions which focus on the facilitation of distance learning. As a consequence, changes in already implemented rules, guidelines and processes can be made. In addition, school leaders are enabled to include data-driven considerations into their decision-making processes (Schildkamp, 2019).

The second objective of the project was to identify factors of school development which facilitate distance learning processes, shifting the perspective from a single school view to the group of vocational schools as a whole. Vocational schools in Germany are one of the two parts of the German dual vocational training system. While training companies provide practical competencies, vocational schools facilitate mostly theoretical knowledge to support students in completing their vocational training. Practical and theoretical training in the dual system always happens in conjunction: school attendance at a vocational school and a trainee position is mandatory to acquire a professional qualification for all professions, apart from the qualification through a university programme. The increasing demands of the workplace towards digital competencies require changes in teachers and students’ digital competencies (Roll & Ifenthaler, 2021), and therefore further strategies to implement digitalization into school development (Delcker & Ifenthaler, 2020).

2 Theoretical Framework

Brindley et al. (2004) define Distance Learning (DL) as a superior construct, which includes various forms of media-based learning. The main characteristic of DL is the geographical separation between learners and educators, making it a major challenge for the involved stakeholder (Moore & Kearsley, 2011). The requirements for the integration of DL into schools and school development processes are very diverse, which makes a multidimensional perspective on school development necessary (Ames et al., 2021; Bellin-Mularski et al., 2016). This multitude of perspectives is reflected by the different stakeholders involved in school development processes, namely school leaders, teachers, students and parents (Harris, 2010; Ilomäki & Lakkala, 2018). At vocational schools, training companies have to be included as a relevant stakeholder (Delcker & Ifenthaler, 2020). Following the definition by Rolff (1995), school development is defined as processes within a single school and not the whole school system. The aim of school development from within the school as a single organizational unit is the improvement of students’ subject-specific and interdisciplinary competencies. In his model, the multidimensional composition of school development is represented by the three different dimensions inside a school. These dimensions can be summarized under teaching (activities in the classroom), personnel (mentoring, teacher training) and organization (school agendas, school management). With regard to digitalization processes, the technical infrastructure of a school has to be added to the conceptual considerations of school development (Fraillon et al., 2020). Eickelmann and Gericke (2018) expand the model by Rolff (1995) by adding a technological dimension. Furthermore, cooperation is added as a fifth dimension in their model of school development. Namely, these five dimensions are Organizational Development (OD), Personnel Development (PD), Educational Development (ED), Technological Development (TD) and Cooperation Development (CD).

OD includes a school’s agenda, it’s mentality and beliefs towards communication and digitalization. The dimension PD covers teacher training and the onboarding of new teachers. ED subsumes activities in the classroom, such as the usage of learning tools and methods. TD consists of requirements regarding technological infrastructure and administration of systems. The fifth development dimension CD, which describes cooperation processes between the internal and external stakeholder of school development. The five dimensions (OD, PD, ED, TD, CD) are developed towards two goals. The first goal is the ongoing facilitation of students’ digital competencies, and the second goals is teaching and learning with digital media.

A number of studies has shown the influence of the development fields on students learning as well as educators’ roles and teaching competencies (Dirk Ifenthaler & Schweinbenz, 2013, 2016) The participation of students in the classroom and their ability to reach their educational goals can be increased through the organizational structure of schools (Alinsunurin, 2020; Maxwell et al., 2017; Sebastian et al., 2014). The cooperation between teachers with regard to curricular alignment supports academic improvement (Bryk, 2010). If a school is well-structured and organized, it produces an academic climate that is “conducive to learning and high student performance” (Wang & Degol, 2016), while methods and tools used for teaching influence the learning experience of students (Stefanou et al., 2004).

The way school leaders manage and structure their schools impacts teachers’ satisfaction and performance, which in return enhances classroom practices and school effectiveness (Mulford, 2003). Teachers are less likely to leave the schools when they perceive the school administration as effective leaders (Nguyen, 2021). The professional development of teachers is positively influenced by a school’s agenda and the creation of learning opportunities within the organization (Huang et al., 2020).

Due to requirements of the modern working world, the integration of digital teaching methods and tools into school development is a necessity. Key digital competencies cannot be facilitated without them (Fraillon et al., 2020; Roll & Ifenthaler, 2020, 2021). The concept of media expansion plans (MEP) has been deployed in the German school system to help schools transitioning towards digitalization in school development. The MEP is a written document that contains important steps towards this goal. Within the MEP, a school can formulate different digitalization goals, as well as how and when they want to reach those goals. The MEP should include an analysis of the digital status quo at the school (Ifenthaler, 2019; Obermöller, 2019). Most importantly, schools are required to specify the financial resource they need to meet MEP-specific goals to be applicable for the biggest funding program for digitalization in schools in Germany, called “digital pact” (in German Digital Pakt) (km-bw., 2021). The MEP can be an important managerial tool for school leaders to analyze, plan and implement digital tools and methods into their schools. Currently, no published studies about the effects of the MEP exists, due to the novelty of the MEP and the relative short implementation time.

Two research questions emerge from the described problems at vocational schools and theoretical assumptions about school development.

RQ 1: What role do organizational factors play in the perceived learning success of students during distance learning in times of crisis?

RQ 2: Does the implementation of a media expansion plan influence the perceived learning success of students?

3 Method

3.1 Participants and Data Collection

A convenience sample of 14 vocational schools in the federal state of Baden-Württemberg, Germany took part in the project from November 2020 until March 2021. Each school could choose the starting date of the survey, to avoid conflicts with internal school constraints. In addition, schools could choose which stakeholder groups they wanted to survey, with students, teachers and school leaders being mandatory choices. The schools were provided with hyperlinks to the online questionnaire, which were distributed by the schools through internal email addresses. At each school the data collection was conducted over a period of 4 weeks. The data collected from the three mandatory stakeholder groups consists of 2827 students, 444 teachers and 37 school leaders (N = 3872). After the data collection, each school was provided with an individual report that summarized the results of the schools.

3.2 Instrument and Analysis

The online questionnaire “Evaluation of Distance Learning” by (Balzer & Schorn, 2021) has been adapted to collect data from students, teachers, school leaders, parents and training companies at vocational schools. The items can be allocated to five scales, namely organization (orga), class activity (clac), teaching & learning (tl), social interaction (soci) and personal resources (perr) and are assessed on a Likert scale from 1 to 5 (totally disagree, partially disagree, neither nor, partially agree, completely agree). A small number of items to collect demographic data has been added to the questionnaire. The variable for the MEP (mep) contains different stages of the MEP. Schools currently either do not have a MEP (stage 0), the MEP is currently worked on (level 1) or the MEP is fully planned and integrated into the school (stage2). Additionally, schools might have already applied for funding (stage 3) or they have been provided funding based on their MEP (stage4). The questionnaires differ between the stakeholders to allow data collection from multiple perspectives. The longest questionnaire (66 items) was provided for the teachers, the shortest one (23 items) for the parents. Only a few of the questions were mandatory to answer to decrease the likelihood of dropouts. The Cronbach’s alpha values of the five scales for the three main stakeholder groups are shown in Table 1.

Table 1 Reliability scores (Cronbach’s alpha) for the five main scales, the number in brackets shows the number of items per scale
Full size table

Hierarchical linear modelling was used to examine the relation between the variables stated in the research question. Different variables have consequently been added to model to identify the one with the best fit. The conditions for HLM have been met (F. L. Huang, 2018). Student learner success (lsuc) is defined by a set of items which include questions towards their perceived effectiveness and their perceived learning progress while practicing distance learning (α = .71). During the first and second lockdown, grading was not allowed except for final-year classes, so learner success could not be measured by grades. The students’ perception of their schools’ organizational structure (orga_st) was measured with 11 items, which are described in Table 2.

Table 2 Items for students’ perception of schools’ organizational structure (orga_st), N = 1763
Full size table

In addition, the age of the students is used as a possible predictor variable on level 1. The mep variable was used as a predictor on level 2.

4 Results

Initial analysis on level 1 variables showed a significant medium effect of orga_st on students perceived learning success (d = 0.69). The age of students did not show a significant effect, therefor it was dropped from further analysis. To increase the accuracy of the model, the items of the orga_st scale where subsequently added to the model. In the model with the highest fit, 8 of the 11 items showed a significant effect on lsuc, ranging between 0.1 and 0.15. Mep didn’t show a significant effect as a predictor on level 2. Although the low ICC values of the models (<0.1) across all the models indicates that students’ perceived organization at their schools does not vary between the schools, the regression estimates of the HLM model is presented in Table 3, because the approach is more sensible and represents the nested structure of schools within education systems (Alinsunurin, 2020).

Table 3 Regression estimates for the model with the highest fit (** < 0.01, *** < 0.001), N = 1763
Full size table

To answer Research Question 1, it can be stated that some parts of the perceived organization have an effect on students’ perceived learning success. The parts of the organization that had the biggest positive effect were the clearness of procedure instructions and the fairness of the assessment. Secondly, the adaptation of teaching to the crisis, students’ tool competency and the provided infrastructure positively influence student’s perceived learning success. The results indicate that these organizational parts play an important positive role in the improvement of students’ perceived learner success.

Regarding Research Question 2, the stage of the MEP does not have a significant effect on the learning success, and adding the variable mep as an explanatory variable on level 2 does not improve the explanatory strength of the HLM. It can be stated that the implementation of a MEP does not influence the perceived learners’ success.

5 Conclusion

The findings of the study show the importance of the organizational structure of schools for the learning success of students. Most importantly, changes to single parts within the development field of organization can help students to achieve their educational goals. Students rely on clear procedural instructions, more so when dealing with a crisis like the COVID-19 pandemic, because they create the necessary safety within the learning processes (Sebastian et al., 2014). Fair assessment of students’ works encourages students to spend time and effort on submissions and tasks, which increases their chance of succeeding. The teaching processes have to be adapted for the crisis to be feasible and plausible for the students. For the realization of digitalized teaching processes, the students need the competencies to work with the necessary tools such as video conferencing tools and the learning management system of the schools (Olszewski & Crompton, 2020). School leaders can support the students by supplying them with appropriate digital tools (Bond, 2020), which is strongly connected to the development field of technology (Eickelmann & Gericke, 2018). While the effects of the single parts of organizational structure seem to be small, the combination of the diverse perspectives including information flow, communicated rules and the provision of tools that are easy to access shape the characteristic of schools’ organizational structures as a facilitator for a successful school environment (Alinsunurin, 2020; Ames et al., 2021; Bryk, 2010; Mulford, 2003; Stefanou et al., 2004; Wang & Degol, 2016).

The analysis of the data suggests that the vocational schools and especially their students’ perception of organization and learner success are not very distinct. The similarity of the challenges school leaders face admits a common crisis and the requirement for digitalization in school development underline the importance of improving the collaboration between school leaders (Ilomäki & Lakkala, 2018). The existence of a MEP does not have a significant effect on student learner success. This is an important finding underlying the necessity to evaluate the creation and implementation of MEPs on a more detailed level (D. Ifenthaler, 2019). In addition, the expressiveness of a MEP for the authorization of government funds has to be criticized.

The introduced evaluation tool is currently evaluated in cooperation with the involved school leaders, to enhance its capability as a managerial tool. One of the goals of the evaluation process is the optimization of the sample size, especially on the school level. To further examine the integration of the MEP, a refinement of the survey instrument is being conducted. This will improve the collected data and increase the benefit for the stakeholders involved in the digital school development.