Introduction

The flipped classroom (FC) continues to receive considerable attention in K-12 and higher education research (Strelan et al., 2020). The FC is a pedagogical approach that combines online delivery of instruction prior to attending face-to-face sessions. The benefit of the FC is that by “flipping” the instruction and in-class activities, in-class time that is traditionally spent on passively receiving instruction is freed up for engaging in active learning. To benefit from the in-class sessions, it is essential that students prepare actively beforehand and the type and the amount of preparation students engage in is related to their course performance (Jovanović et al., 2017; O’Flaherty et al., 2015). Students are not homogeneous in the type and amount of preparation they engage in over the course, leading to the investigation of patterns (or trajectories) that students demonstrate over the duration of the course. Several studies have identified that students display different activity patterns (e.g., Jovanović et al., 2017; Pardo et al., 2018); however, why students differ in their preparatory activities is largely understudied. Students in the FC need to adjust from passive learning to an active preparation before each lecture by setting goals, finding appropriate learning strategies, and monitoring their behavior (Estes et al., 2014), which might be particularly important when they encounter the FC for the first time. Therefore, differences in motivation and self-regulation might explain why students differ in their activities during FC courses (Hew et al., 2021; Jovanović et al., 2017). These factors have been individually studied in the context of the flipped classroom but research investigating them simultaneously is scarce. Consequently, in this study we investigated how preparatory activity patterns relate to course performance, motivation, and self-regulation in students participating in a flipped course for the first time. The results are of theoretical value by offering more insight into the relation between these constructs. Furthermore, the results could be of practical value by informing the students and teachers of which activity patterns are most effective, in particular when implementing the FC for the first time.

Flipped classrooms

The FC restructures the traditional classroom by shifting activities that typically take place inside the classroom to outside of the classroom and vice versa (Lage et al., 2000). However, in practice, teachers in the FC do more than simply re-arranging in-class and homework activities (Bishop & Verleger, 2013). By assigning the content and instruction as homework, classroom time is freed-up for engaging in learner-centered activities, such as discussions, problem-solving (Mason et al., 2013), receiving feedback, and diagnosing learning by the teacher (Bergmann & Sams, 2012; Roehl et al., 2013). Teachers can choose in-class activities incorporating active learning strategies where students can analyze, synthesize, and apply their knowledge during small-group collaboration (Gilboy et al., 2015). The main rationale of the FC is that active preparation before class is thought to result in deeper engagement in the activities during the face-to-face meetings (Tune et al., 2013). The learner-centered approach of the FC shifts the responsibility of being prepared for class to the student which requires students to be, or become, self-regulated learners (Lai & Hwang, 2016). However, students not always engage in the required preparation before the in-class meetings (Jong et al., 2019).

Despite the promising features of the FC like devoting in-class time to active learning, findings on its effectiveness are mixed compared to traditional learning (Chen et al., 2017). On the one hand, there is evidence for a positive effect of the FC on students’ course performance in diverse courses in higher education (for meta-analyses, see Cheng et al., 2019; Hew et al., 2021; Strelan et al., 2020). Similar positive findings were obtained in earlier reviews concerning improved learning performance, enhanced motivation, attitude, and satisfaction with the flipped classroom (Akçayır & Akçayır, 2018; Zainuddin & Halili, 2016).

On the other hand, several studies found no difference in student outcomes between the flipped and the traditional classroom (e.g., Harrington et al., 2015; Molnar, 2017; Ziegelmeier & Topaz, 2015). In some studies, students in the traditional classroom outperformed students in the FC (e.g., Bossaer et al., 2016; Moffett & Mill, 2014). Some students reported feeling less satisfied with the organization of the FC due to the variety of learning activities in class (Strayer, 2012) and the increased responsibility for learning outside of class (Wilson, 2013; Yeung & O’Malley, 2014). To explain the mixed findings on the effectiveness of the FC, it is important to distinguish between different activities students engage in and how these activities change over time (Pardo et al., 2018).

Student activity clusters

The effectiveness of the FC is related to students’ preparation before the in-class meetings (O’Flaherty & Phillips, 2015). However, students differ in the type and amount of preparation activities they engage in before coming to the in-class meeting (Boevé et al., 2017; Jovanović et al., 2017; Lust et al., 2011). Some students do not adapt to the flipped format of the classroom and attend the in-person meetings without preparing in advance (Boevé et al., 2017). Therefore, simply having the possibility to engage with the online environment does not necessarily lead to students being engaged in preparation (Lust et al., 2012). Other students, after realizing the importance of being prepared for class, adapted to the flipped format and they had equal or better performance than students in the traditional classroom (Mason et al., 2013). These finding imply that differences in student preparatory activity and change in activity over time (e.g., adapting) could explain the mixed findings concerning the FC.

Prior research has demonstrated the benefits of clustering students based on their preparation and engagement with online activities in the FC (Jovanović et al., 2017), online courses (Del Valle & Duffy, 2009; Kovanović et al., 2015), and blended learning (Lust et al., 2013b). Three clusters of students based on their technology use have been consistently identified: (1) students showing high activity, deep approaches to learning, and the best exam performance, (2) students showing low activity and superficial approaches to learning, and (3) students showing selective strategies, aimed at high achievement using minimal effort (Kovanović et al., 2015). In the FC context, Jovanović et al. (2017) identified five distinct clusters which were partly consistent with the three types described by Kovanović et al. (2015) and successfully related cluster membership to course performance. Nevertheless, these studies (e.g., Jovanović et al., 2017; Lust, et al., 2013b) considered only online activities whereas the FC often also includes certain “offline” activities. Therefore, in the current study, we extend these findings by additionally including offline preparatory activities, such as reading the course book and working on a group project. Furthermore, we examine how students’ engagement in the types of preparatory activities changes over the course.

Temporality and activity patterns

Another way to explain the mixed findings of the FC is to look at the changes in students’ preparatory activities over time. Learning is a process that develops over time, and it therefore requires temporal investigation (Knight et al., 2017; Molenaar, 2014; Wise et al., 2012). Despite its importance, researchers tend to neglect the temporal aspect of learning and instead use aggregated measures of activity (e.g., Gašević et al., 2017). By using aggregated measures of the overall frequency of specific activities, information about the pattern of activity over time is lost, which reduces the explanatory power of the analysis and the validity of the conclusions (Reimann, 2009).

Learning management systems (LMS) used in flipped courses provide large amounts of logfile data incorporating temporal information about the timing or the sequence of activities, which makes these temporal analyses possible. Previous research has demonstrated the value of analyzing changes in students’ learning approaches based on logfile data (Jovanović et al., 2017; Pardo et al., 2018). Students tend to change their learning strategies throughout the course (Jovanović et al., 2017) and they tend to differ in how they use the online tools available (Lust et al., 2013b). Therefore, when investigating how students prepare for the face-to-face meeting in the FC, it is essential to consider the patterns of preparatory activities students engage in over time. While prior research has identified different patterns (e.g., Pardo et al., 2018), it is still unclear why students differ in their preparatory activity patterns. Students’ self-regulation and motivation have been proposed as potential explanatory factors (Hew et al., 2021); however, they have not been directly investigated in prior research (Jovanović et al., 2017; Pardo et al., 2018).

Student characteristics

The FC is a learner-centered approach whose effectiveness relies on students’ active preparation and engagement. Student characteristics like motivation and self-regulation are relevant for engagement and could explain differences in preparatory activities in the context of FCs. Students in the FC are responsible for scheduling time for learning and they need to adapt their learning strategies over time; in other words, students need to be motivated and self-regulated learners (Lai & Hwang, 2016). The need for the self-regulation and motivation could become even more pronounced for first-time students who have not yet developed appropriate learning strategies for the FC. In a review of 19 meta-analyses on the FC, Hew et al. (2021) identified the need to further investigate the role of motivation and self-regulation in relation to students’ preparation in the FC. During web-based distance instruction, motivated students outperformed less motivated students, and they used more deep learning strategies such as elaborating, linking, and integrating information (Sankaran & Bui, 2001). While the FC differs in certain regards from distance learning, motivation could be an important factor for successful preparation in the FC as well. Nevertheless, prior studies have only investigated how a FC course influences teacher education students’ motivation (e.g., Debbağ & Yıldız, 2021), but not how students’ motivation relates to preparation before class.

Self-regulated learning in the FC

Self-regulated learners are active, motivated, persistent, and adaptive in their learning strategies (Järvenoja et al., 2015; Zimmerman, 2000). Self-regulation is particularly important in the FC because students are responsible for actively preparing before class to be able to benefit from the face-to-face activities (Tune et al., 2013). Students need to regulate their effort and time management to ensure sufficient time for performing the pre-class preparatory activities (Ahmad Uzir et al., 2020; Sletten, 2017).

Students’ self-regulation is also related to course performance in the FC (Sletten, 2017; Sun et al., 2018; Zheng & Zhang, 2020). For instance, elementary students with high self-regulation had better performance, higher self-efficacy, and showed different learning approaches than students with low self-regulation abilities (Lai & Hwang, 2016). Nevertheless, these studies did not include information on students’ preparation and how it relates to their self-regulation and learning outcomes. As preparation and self-regulation are both influential factors for learning success in the FC, it is essential to address them simultaneously. Therefore, this study bridges this gap by investigating preparation in relation to both learning outcomes and differences in self-regulated learning in university students enrolled in a FC course for the first time.

Motivation

Motivation is defined as the driving force that directs students’ choices and approach to learning (Wlodkowski, 1985). Motivational theories distinguish between intrinsic and extrinsic motivation (e.g., Deci & Ryan, 2000). Extrinsic motivation is directed at performance in order to achieve a certain goal or receive an external reward (Walker et al., 2006). In contrast, intrinsic motivation originates within the individual and results in enjoyment from increasing one’s competency. Intrinsically motivated students are interested in learning, they value education, and they are confident in their abilities (Deci et al., 1991). Furthermore, intrinsically motivated students are more likely to engage in metacognitive strategies such as planning and monitoring (Pintrich & DeGroot, 1990), they are more persistent (Vallerand & Bissonnette, 1992), and they spend more time and effort on learning (Salili et al., 2001) than students with low motivation. Despite the importance of motivation on learning outcomes, motivation has not received much attention in the FC (Abeysekera & Dawson, 2015). Moreover, the few studies that consider motivation tend to examine how the FC influences students’ motivation (e.g., Abeysekera & Dawson, 2015; Huang et al., 2023), but not the other way around. Motivation plays an important role in engagement (Ainley, 2012); therefore, it is important to consider how students’ motivation relates to their preparatory activities in the FC.

The present study

The present study provides an ecologically valid investigation of students’ approaches to preparation when experiencing the flipped classroom for the first time. This study extends prior research on students’ approaches to preparation in the flipped classroom by looking at (1) the type and the amount of preparatory activities students engage in, (2) the patterns of preparatory activities students exhibit over time, and (3) the relationship between students’ characteristics (self-regulation and motivation) and their activity patterns. We had the following research questions and hypotheses:

RQ1. What types of activity clusters can be identified based on students’ preparatory activities?

Based on previous findings (Jovanović et al., 2017; Kovanović et al., 2015), we expect to find between three and five distinct clusters, mainly distinguishing between the type and the amount of preparatory activities students engage in.

RQ2. What preparatory activity patterns can be identified based on students’ change in activity cluster membership over time?

To investigate activity patterns, we used repeated measures of students’ preparatory activities and explored changes in students’ activity cluster membership over time.

RQ3. Do students with different preparatory activity patterns differ in their course performance?

Based on previous research demonstrating that students showing high activity during preparation had higher grades at the end of the course (Jovanović et al., 2017; Lust et al., 2013b), we expected that students who remain in high activity clusters over time would have higher course performance than students in low activity clusters, and that moving to a high activity cluster would benefit students’ course performance.

RQ4. Do students with different preparatory activity patterns differ in their self-regulation and motivation?

We expected that students with high levels of self-regulation would show high activity during preparation over time, indicating that these students are scheduling time for their learning and they are monitoring their preparation. Further, we expected that intrinsically motivated students choose to engage in out-of-class preparation whereas extrinsically motivated students prepare because they have to comply with the course requirements (Ryan & Deci, 2000). In summary, we expect that intrinsic and extrinsic motivation will both be associated with active preparation, albeit for different reasons.

Method

Participants and educational context

Participants

The sample comprised 174 undergraduate students (Mage = 23.04, SD = 5.77, 17.2% male) enrolled in a flipped classroom course about educational design at a Dutch University. All students participated in a flipped classroom course for the first time. Students followed different majors in educational sciences (i.e., teacher training for primary or secondary education or educational sciences) and the majority (87.9%) were studying full-time. The sample is representative of educational sciences and teacher training programs in the Netherlands. All students provided their consent for use of their data for research purposes.

Educational context

The course subject was about designing instructional systems for complex cognitive skills and provided students with theoretical knowledge about the design process and an opportunity to practice their knowledge by designing a realistic practice-oriented project for a school or a company. Students worked in groups of four to complete the project (group assessment) and they took a test at the end of the course (individual assessment). The course had a duration of 10 weeks with several weekly activities. Preparation comprised reading chapters of the course book and/or watching web lectures (24 web lectures in total, three for each of the eight subjects covered in the course) that further elaborated the material in the book. For each meeting, students were asked to construct one formative multiple-choice question and to upload it to the online platform PeerWise (Denny et al., 2008). This activity was meant to stimulate processing of the materials and served to build a shared database of questions to practice the materials with. The aim of the plenary lectures was to help students design and theoretically frame their group assignment. The plenary lectures consisted of two parts and provided opportunities for questions and discussion. The first part provided in-depth theoretical and practice-relevant information, building on the material discussed in the corresponding web-lecture. The second part discussed the weekly assigned reading in detail to help students structure the theoretical aspects of their group assignments.

In the days following the lecture, students had working group sessions where they answered and discussed the questions posted in PeerWise. In the remainder of the working group, students worked collaboratively on their group projects under supervision of a teacher. Students were also expected to work on the assignment outside of the working groups.

Measures

Preparatory activities

Preparatory activities comprised watching web-lectures, creating and answering questions in PeerWise, reading course materials, and working on the group project. Watching web-lectures was captured automatically by the LMS and reflected the duration (in minutes). Creating and answering questions was captured automatically by the PeerWise system and providing the frequency of both activities per week. The hours spent reading the course materials and the hours working on the group project were measured by a weekly questionnaire during the working group. The short questionnaire contained questions about these activities as well as students’ intrinsic and extrinsic motivation and their perception of the collaboration with their project group members. Students could fill in the questionnaire online or on paper. Because we focus on individual preparation, students’ perceptions of group collaboration were not used in this study.

Course performance

Students’ course grade comprised students’ exam grade (individual assessment) and the group project grade (group assessment). Students could receive a grade from 1 (lowest grade) to 10 (highest grade) and a minimum of 5.5 on both assessments was required to pass the course. The individual assessment accounted for 40% of the final grade and the group project accounted for 60% of the final grade. Since we were interested in how individual activity patterns relate to course performance, we focused our analyses on students’ exam grade.

Student characteristics

At the beginning of the course, students’ demographic information (age, gender, and major) and motivation (intrinsic motivation, 4 items, α = 0.90; extrinsic motivation, 4 items, α = 0.87) were assessed. The items in the scales were based on the Situational Motivation Scale (SIMS; Guay et al., 2000) and were fitted to the context of this study. All items were assessed on a seven-point Likert scale, ranging from 1 (strongly disagree) to 7 (strongly agree). Additionally, students’ intrinsic (2 items, “This week I work with interest on this subject.” and “This week I follow this course to further develop myself as an educational expert”) and extrinsic motivation (2 items, “This week I mainly participate in the activities of this course because it has to be done.” and “This week I would like to score better in this course than my fellow students.”) was assessed weekly during the working group sessions on a five-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). In the demographics’ questionnaire, task value (6 items, α = 0.86) and self-efficacy (8 items, α = 0.91) were also measured, and used for a different research study.

Students’ self-regulation (36 items) was measured midway through the course using the Self-Regulated Online Learning Questionnaire (SOL-Q; Jansen et al., 2017). The SOL-Q measures self-regulation based on five constructs—metacognitive strategies (18 items, α = 0.88), persistence (5 items, α = 0.84), help seeking (5 items, α = 0.82), environmental structuring (5 items, α = 0.66), and time management (3 items, α = 0.71). All items were assessed on a seven-point Likert scale, ranging from 1 (not at all true for me) to 7 (very true for me).

Data analysis

To identify student activity clusters (RQ1), k-means cluster analyses (Han et al., 2011) were performed in SPSS using the measures of preparatory activities at three time points. To obtain a stable cluster solution and to ease the comparison between indicators, all indicators were standardized. Since the first week of the course required no preparation, we used measurements from week 2 to reflect students’ preparation at the beginning of the course (T1). We chose week 5 as the measurement of preparation in the middle of the course (T2) and week 7 as the measurement of preparation at the end of the course (T3).

The number of questions created in PeerWise showed no contribution to the separation of the clusters. Therefore, k-means cluster analyses with 2 to 6 clusters were conducted using the remaining four indicators (reading hours, project hours, duration, and questions answered in PeerWise). To select the optimal number of clusters in each time point, the scree-plot was inspected together with the interpretability of the clusters and their differences on the indicator variables (Kovanović et al., 2015; Lust et al., 2013a, 2013b). The scree-plots indicated that either the three or the four-cluster solutions were best fitted (see Figures -S1, S2, and S3 in the supplemental material). The clusters differed significantly on the indicators only in the three-cluster solutions; therefore, we chose the three-cluster solution.

To investigate how students changed their cluster membership over time (RQ2), we computed a new categorical variable showing students’ cluster membership at the three time points, displaying their activity pattern. To compute the activity patterns, we used students’ cluster membership at each time point that was previously identified by the k-means cluster analysis and created a new overarching variable indicating cluster membership at each time point. Some patterns were followed by very few students; therefore, we analyzed patterns which contained at least 10 students. To compare students with different activity patterns in their course performance (RQ3) and student characteristics (RQ4), we used the categorical variable “activity pattern” in a one-way ANOVA with Tukey HSD post-hoc corrections.

Results

Student activity clusters

Overall, the same three clusters were found and their relative size was similar at each time point (see Table 1)—Low Activity was the largest cluster, followed by Intensive which was around half the size of Low Activity, and Strategic was the smallest cluster. Strategic was most underrepresented at T3 (2.9%).

Table 1 The proportion and number of students within clusters at each time point
Full size table

The Strategic cluster focused predominantly on one activity during each time point. At T1, students showed very high activity in answering questions in PeerWise, indicating that students were focused on formative assessment. At T2, students worked intensively on their group project. At T3, students again focused on answering questions, reading the course materials, and watching the web lectures. Descriptive statistics (see Table 2) indicated that Strategic students had high self-regulation, high extrinsic, and high intrinsic motivation. The Low Activity cluster had below average engagement with all preparatory activities at each time point. Time invested slightly increased toward the end of the course but was still below average and the lowest compared to the other two clusters, see Fig. 1. Low Activity students had the lowest self-regulation, high extrinsic and low intrinsic motivation.

Table 2 Descriptive statistics of self-regulation and motivation per cluster at each time point
Full size table
Fig. 1
figure 1

The frequency of each activity (z-scores) at the beginning, the middle, and the end of the course for Strategic (top), Low activity (middle), and Intensive (bottom) cluster. Error bars represent standard errors

Full size image

The Intensive cluster had the highest activity in three preparatory activities in each time point, see Fig. 1. At T1, students were reading the course materials, watching the web-lectures and working on the group project. At T2, instead of working on the group project, students were answering the questions in PeerWise, next to watching lectures and reading. At T3, students showed similar activity to T1 but with increase in time working on the group project. Intensive students had high self-regulation, low extrinsic, and high intrinsic motivation.

Preparatory activity patterns

Due to its small size, the Strategic cluster was not included in any of the activity patterns (the patterns including the Strategic cluster had between 1 and 6 students). We included patterns demonstrated by at least 10 students in our analyses, resulting in a sub-sample of 128 students and five distinct activity patterns. Table 3 summarizes the five activity patterns along with students’ mean grade and self-regulation. Students who belonged to the Low Activity cluster at each time point, were categorized in the Low Activity pattern. Students showing this pattern had low exam grades and the lowest self-regulation scores, although self-regulation scores were quite similar to almost all other patterns. Students in the Middle Peak pattern started in the Low Activity cluster, then moved to Intensive at T2 and then moved back to Low Activity at T3. These students had high grades and similar self-regulation scores to the other patterns. Students in the Initial Peak pattern started in the Intensive cluster and then remained in the Low Activity cluster at T2 and T3; they had the lowest average exam grade and average self-regulation scores. Students in the End Drop pattern were in the Intensive cluster at T1 and T2 and moved to Low Activity at T3 which was associated with average grades and self-regulation score. Last, students in the Intensive pattern belonged to the Intensive cluster at each time point and they had the highest grade and highest self-regulation score.

Table 3 Descriptive statistics of exam grade and self-regulation per activity pattern
Full size table

Course performance

The patterns differed significantly in grade, F(4, 123) = 10.36, p < 0.001, η2 = 0.26, see Table 4 for the significant differences between patterns. Students with the Low Activity pattern had significantly lower grades than students in the End Drop, Middle Peak, and Intensive patterns, indicating that preparation even in only one phase of the course could still be beneficial for course performance. Furthermore, students with the Middle Peak pattern had significantly higher grades than students with the Initial Peak pattern, indicating that preparation in the middle of the course resulted in better course performance than preparation at the beginning of the course. Students with the Initial Peak pattern had significantly lower grades than students with the End drop and Intensive patterns, indicating that initial preparation at the beginning of the course was not sufficient for successful course performance. Students in the Initial Peak pattern also had the lowest exam grades overall, but not significantly lower than students in the Low Activity pattern. Students in the Low Activity and Initial Peak patterns scored below the passing grade in the exam.

Table 4 Tukey HSD comparison between patterns in exam grade
Full size table

Student characteristics

The patterns differed significantly in self-regulation, F(4, 54) = 3.73, p = 0.01, η2 = 0.23. Post-hoc comparisons showed that students in the Low Activity pattern (M = 4.47, SD = 0.64) had significantly lower self-regulation than students in the Intensive pattern (M = 5.23, SD = 0.53). No significant differences were found between other patterns. When looking at the different components of self-regulation, the only difference was in time management, F(4, 53) = 3.73, p = 0.01, η2 = 0.23. The Low Activity pattern (M = 4.01, SD = 1.29) scored significantly lower in time management than the Intensive pattern (M = 5.51, SD = 1.18), p = 0.008.

At the beginning of the course, the patterns did not differ significantly in intrinsic motivation, F(4, 119) = 1.57, p = 0.19, η2 = 0.05, and extrinsic motivation, F(4, 119) = 2.23, p = 0.07, η2 = 0.07; therefore, no post-hoc comparisons were conducted. We additionally compared the patterns in the weekly measures for intrinsic and extrinsic motivation. At T1, the patterns differed significantly in intrinsic motivation at T1, F(4, 122) = 5.33, p < 0.001, η2 = 0.15. In particular, students in the Intensive pattern (M = 4.53, SD = 0.41) had significantly higher intrinsic motivation than students in the Low Activity (M = 3.80, SD = 0.64, p < 0.001), the Middle Peak (M = 3.77, SD = 0.47, p = 0.01), and the End Drop (M = 3.89, SD = 0.72, p = 0.02) patterns. The patterns did not differ significantly in motivation at T2 and T3.

Discussion

To further understand how students engage in preparation in the FC, this study first identified distinct clusters of students based on their preparatory activities (RQ1). Second, we investigated how students change in their preparation during the course by identifying preparatory activity patterns (RQ2). Last, we related preparatory activity patterns to students’ course performance (RQ3) and self-regulation and motivation (RQ4).

Student activity clusters

At each measurement point, a comparable three-cluster solution was obtained (RQ1), characterized by a cluster focused primarily on one activity (Strategic), a cluster characterized by below-average engagement with all preparatory activities (Low Activity), and a cluster characterized by high engagement with almost all preparatory activities (Intensive). These findings align with prior research on student activity clusters in the FC and blended learning, that typically describe clusters characterized by selective activity, low activity, and high activity (Jovanović et al., 2017; Lust et al., 2013b; Pardo et al., 2018). Our contribution to these existing studies is twofold. First, next to logfile data which is the usual primary data source (Jovanović et al., 2017; Pardo et al., 2018; Wang, 2017), we incorporated additional information from “offline” activities. By combining information about students’ online and offline preparatory activities like reading the course materials or working on the group project, this study extends prior research and provides a more complete investigation of students’ engagement during preparation in the FC.

Accordingly, we showed how students engaged in different types of activities at different time points. The three types of clusters we identified at the three time points were very similar (strategic, low activity, and intensive); however, the specific activities that students focused on differed over time. For instance, the Strategic cluster focused on one specific activity in the different phases of the course. At the beginning, these students were focused on formative assessment by answering questions. In the middle of the course, they were mostly working on their group project, where they applied their knowledge on a specific problem. At the end, Strategic students were more engaged with preparing for the exam by reading and focused on answering questions. Therefore, our findings contribute to the literature by showing that students have different approaches to preparation in the FC and these approaches tend to change over time.

Preparatory activity patterns and course performance

The main contribution of this study relates to the identification of distinct activity patterns that students exhibit over time and their relation to course performance (RQ2 and RQ3). Our findings further demonstrate the value of investigating students’ preparatory activity temporally and corroborate findings that students tend to change their activity over time (; Pardo et al., 2018; Jovanović et al., 2017). We found five distinct patterns showing how students moved between clusters characterized by low activity and high activity at different phases of the course and their differences in course performance. First, students who consistently prepared before class performed better than students who did not, thereby corroborating prior research (e.g., Gross et al., 2015; Pardo et al., 2018). This finding is not surprising since the main rationale of the FC is that students who come prepared to class have better course performance. However, not all students are homogeneous in their preparation which led to the identification of interesting preparatory activity patterns showing temporal changes of students’ engagement.

The Initial Peak pattern had the lowest course performance and students in this pattern were intensively preparing at the beginning of the course but they stopped preparing in the middle and at the end of the course. This pattern of preparation suggests that students tried initially to adapt to the FC model but that also changed during the course. These students had worse course performance, although not significantly, than students who were in the Low Activity pattern. Students in the Initial Peak and Low Activity patterns scored below the passing grade, indicating that preparation prior to attending the lecture is essential for successful course performance. In contrast, students who prepared intensively only in the middle of the course (Middle Peak pattern) had significantly higher course performance than those who prepared only in the beginning (Initial Peak) or not at all (Low Activity). The Intensive cluster in the middle of the course (see Fig. 1) was characterized by high activity in watching the videos, reading hours, and formative assessment. This could indicate that students in the Middle Peak pattern were trying to catch up on the material they skipped in the beginning of the course. Alternatively, students in this pattern have recognized the importance of preparation prior to the face-to-face meetings and adapted accordingly, in line with prior research (e.g., Mason et al., 2013).

Taking these findings together, it appears that it is not only the amount of preparation (time on task) that students undertake but also the timing of the preparation that can result in large differences in course performance. First, the preparatory activities themselves may have increased exam scores simply because students spent more time on the material than students who did not prepare or because the preparation incorporated more active types of activities (such as working on a project) that stimulated learning. However, the main implication of our findings is that adapting to the flipped model, particularly in the middle of the course, is related to better course performance.

Second, preparatory activities may indeed make the face-to-face meetings more effective, in turn leading to higher exam scores. It remains unclear what the role of the face-to-face meetings was in our sample and whether they mediated the effect of preparation on performance. Lecture attendance itself showed little variance in our sample. Prior research has indicated that students’ interaction during face-to-face meetings positively relates to the perceived usefulness of the online activities in the FC (Shih et al., 2019). However, their analysis did not include students’ preparatory activities. Therefore, an important direction for future research is to examine students’ activity patterns in relation to their engagement during face-to-face meetings.

A worrying but consistent finding concerning the patterns of student activities is that the majority of students were in the Low Activity pattern, displaying little to no preparation throughout the course. Multiple explanations for this finding are possible. First, as students in our sample had no prior experience with the FC, they might have relied on learning strategies from traditional classrooms that require no preparation before attending lectures. Although students were informed about the course structure and the underlying idea of the FC model, studies point out that a significant change in attitude and behavior is needed from students (Orton-Johnson, 2009). Therefore, when introducing the FC model for the first time, students might benefit from additional support to transition to the FC more effectively, for example using automated messages and feedback that adapts to students’ activity patterns. Another explanation concerns students’ motivation and self-regulation, which we further discuss in the next section.

Preparatory activity patterns and student characteristics

In line with our expectations, students who remained in the Intensive cluster over time (i.e., Intensive pattern) showed higher levels of self-regulation than students in lower activity patterns. We hypothesized that this could be due to a better ability to monitor and plan their learning, which are important for the autonomous learning required by the FC. Our findings further showed that the difference between the Intensive and Low Activity patterns was in time management. In the FC, time management is related to deliberately allocating time for preparation before the in-class meetings which is related to academic success (Ahmad Uzir et al., 2020). The Intensive and Low Activity patterns had large differences in course performance and time management, in line with Ahmad Uzir et al. (2020). In particular, students who reported higher time management skills prepared intensively and consistently also had the best course performance. Our results extend prior findings on the relationship between time management and course performance in the FC using trace data (Ahmad Uzir et al., 2020) by relating students’ self-reported time management to their activity patterns and course performance.

Returning to the finding of a large number of students in the Low Activity pattern, it could be that these students might benefit from additional time management support especially when they experience the FC for the first time. An important implication from our findings is to instructionally support students’ time management in the FC. Implementing self-regulated learning support in a LMS was shown to be effective for supporting students’ self-reflection and monitoring (Yoon et al., 2021). Our recommendation for future research and practice is to investigate how time management support relates to students’ preparatory activity patterns and whether such support can help students transition to more effective preparatory activity clusters.

We found no differences between the patterns in extrinsic and intrinsic motivation at the beginning of the course and before students have experienced the flipped format. When we investigated differences in intrinsic and extrinsic motivation using the weekly measures, we found that the Intensive pattern had significantly higher intrinsic motivation than the Low Activity, Middle Peak, and End Drop patterns at the second week of the course. Differences in intrinsic motivation could potentially explain why students in the Intensive pattern continuously prepared during the course. Students in the Middle Peak and End Drop patterns prepared at different phases of the course (only at the middle or at the beginning and at the middle, respectively) but they did not sustain their preparation at all phases of the course. Low intrinsic motivation suggests that students in the Low Activity pattern attempted to pass the course with exerting minimal effort (attending the lectures without preparing). The lack of differences between the patterns at the middle and at the end of the course could have two potential explanations. First, the small number of students within each pattern may have reduced our power to detect differences between patterns regarding intrinsic and extrinsic motivation at those time points.

Second, students’ motivation has within-day variations (Martin et al., 2015) and it might be more informative to measure motivation when students actually perform the learning activities (Gašević et al., 2017; Zhou & Winne, 2012). In our study, the weekly measurements of motivation were administered during the work groups. Given the scarcity of studies investigating how students’ motivation relates to their activity patterns in the FC, our findings provide initial insight into the role of intrinsic motivation for sustaining engagement during preparation over time. Nevertheless, more research is needed to fully capture how differences in extrinsic and intrinsic motivation relate to preparation in the FC, for example by administering motivation questionnaires when students start the weekly preparatory activities in the LMS.

Implications for research and practice

Several implications for future research and practice can be drawn from the findings of this study. First, our findings demonstrate that it is important to continue to examine students’ preparation temporally, as students tend to change how they prepare over time. Moreover, these changes in preparatory activities are directly related to course performance. For educational practitioners, it is important to explain to students the importance of preparing for the face-to-face meetings. Important directions for future research include investigating how students’ preparatory activity patterns relate to their engagement during the face-to-face meetings.

Likewise, it is important to consider other relevant constructs for students’ approaches to preparation in the FC. The present study focused on self-regulation and intrinsic and extrinsic motivation. Our findings suggest that it is necessary to incorporate support for time management, in particular for students experiencing the FC for the first time. To further uncover the role of motivation for students’ preparatory activity patterns, we recommend measuring motivation during preparation, for example by integrating these measures in the LMS. Last, there are other relevant factors that could explain why students differ in their activity patterns. For example, students’ achievement goal orientation (Elliot & McGregor, 2001) has been suggested as a prominent factor related to students’ learning strategies (Jovanović et al., 2017; Lust et al., 2013b) that requires further investigation.

Limitations

First, a consequence of the relatively small sample size is that only a small number of students belonged to the Strategic cluster which prevented us from investigating the activity patterns for these students. Nevertheless, students in our sample had no prior experience with the FC, which could explain why most students did not know how to prepare strategically. The Strategic cluster was particularly interesting as these students had a high focus on one specific activity over time. Therefore, relating this strategic activity to course performance, self-regulation, motivation could provide important insights regarding engaging in specific activities as opposed to engaging in as much activity as possible. In future research, interviews with first-time FC students who show strategic behavior and adaptation to the FC model, could provide valuable insight into their learning strategies and important implications for educational practice.

Second, a methodological limitation ingrained in all cluster analysis techniques relates to the subjectivity in the decision-making regarding the indicators used for clustering, the number of clusters, and their interpretation (Grieger et al., 2012). We attempted to diminish this subjectivity by combining information from the scree-plot with information about differences between the clusters and their interpretation. To fully validate the cluster solution, cross-validation of the solution is advisable with a similar dataset. Due to the low number of participants, we could not split the dataset and use part of it for training and the rest for cross-validation. However, given that the results of this study corroborate prior research on student clusters in the flipped classroom (Jovanović et al., 2017; Pardo et al., 2018), this cluster solution seems valid.

Third, we related individual activity patterns to individual course performance, which contributed partly to students’ course grade. Students’ group assignments were also graded but were not analyzed in this study. To pass the course, students had to divide their time between preparing for the individual and the group assessment. Nevertheless, we incorporated the group assignment in the activity patterns using the hours students reported working on the group assignment. Therefore, we considered students’ preparation for both assessments despite only analyzing the effects on individual course performance.

Last, in addition to trace data, we used self-reported measures of the hours spent reading the book and working on the group project. A drawback of self-reported measures is their subjectivity, suggesting that students might have over- or underestimated the hours they spent reading the book and working on the group project. By combining self-report measures with trace data, this study considers both offline and online preparation, thus extending prior FC research mainly focused on trace data. Another limitation of our measures is that the reliability of the sub-scale environmental structuring was slightly below the acceptable threshold of 0.70.

Conclusion

The present study contributes to existing literature on student activity clusters in the FC and the investigation of preparatory activity patterns provides important findings on how changes in activities over time relate to course performance, motivation, and self-regulation. Students with consistent intensive preparatory activity patterns had higher course performance, higher intrinsic motivation at the first time point, and higher time management at the middle of the course than students with patterns with lower activity. The temporal investigation of activity patterns provided important insights into how preparation (or lack thereof) at different phases of the course relates to course performance. Furthermore, we provide initial findings relating students’ intrinsic motivation and self-regulation to their activity patterns.