Introduction

The Association of Mathematics Teacher Educators (AMTE, 2017) advises that an effective elementary school teacher must have strong mathematics knowledge and skills as well as a positive attitude about mathematics. Unfortunately, many elementary school teachers experience high levels of mathematics anxiety and have suboptimal mathematics skills (Hughes et al., 2019; Jenßen, 2021; Olson & Stoehr, 2019). In turn, teachers’ negative beliefs and insufficient mathematical skills are related to elementary students’ negative attitudes toward mathematics and lower mathematics achievement (Levine & Pantoja, 2021; Richland et al., 2020; Szczygieł 2020), which may dissuade students from later pursuing careers in STEM fields (Acar et al., 2018). Given the pivotal role of teachers on elementary school students’ mathematics attitudes and achievement, it is imperative that the field seeks innovative methods to increase prospective teachers’ mathematics skill and confidence.

One means to enhance prospective teachers’ mathematics outcomes may be through blended learning. Across a multitude of academic domains, several meta-analyses indicate the positive effects of blended learning, which can also be referred to as flipped classrooms, on student learning and/or attitudes compared to traditional face-to-face (F2F) instruction (Chen et al., 2018; Cheng et al., 2019; Hew & Lo, 2018; Lo et al., 2017; Means et al., 2013; van Alten et al., 2019). Effect sizes have generally been small to verging on medium (g = 0.19–0.47), although not every study has shown positive effects. Of particular relevance to this study, Lo and colleagues’ (2017) meta-analysis of 21 studies on flipped mathematics classrooms in elementary through post-secondary education found a small effect size (g = 0.298). However, only a limited number of studies have specifically examined whether blended learning/flipped classrooms may promote prospective teachers’ mathematics achievement and/or attitudes more than a traditional F2F format (Yudt & Columba, 2017; Dove & Dove 2015; Wong-Ratcliff & Modesto, 2020). Findings have been inconsistent about the promise of blended learning, which may be related to the instructional design of the learning formats (Hew & Lo, 2018; Lo et al., 2017; van Alten et al., 2019).

Given the potential of blended learning/flipped classrooms to promote achievement in a multitude of domains and the limited research focused on prospective teachers, further research is needed to determine whether this method is positively related to prospective teachers’ mathematics achievement and attitudes. The current study is a conceptual replication of the Yudt & Columba (2017) earlier study of 57 elementary prospective teachers who completed their required mathematics course in either a blended learning or traditional F2F format. In alignment with AMTE (2017) teacher preparation standards, the aim of the mathematics course was for preservice elementary teachers to (a) learn the mathematical content and processes that they will ultimately teach to their elementary students and (b) develop a positive attitude toward mathematics such that they can present mathematics as a “useful, challenging, and interesting discipline” to their students (p. 33). Findings indicated that students in the blended learning format had significantly better attitudes toward mathematics, but no significant difference was found for mathematics achievement. In this conceptual replication, empirically-based enhancements were made to the blended learning condition in order to investigate whether these alterations would be associated with both higher attitudes and achievement for students in the blended learning condition compared to the traditional F2F condition. Further, the current study extends previous studies by employing a mixed-methods approach to examine what features of blended learning are perceived by preservice elementary mathematics teachers as a benefit or challenge to their learning.

Definition of blended learning

Although researchers use slightly different definitions, a general definition of blended learning is the integration of F2F and online experiences to optimize student learning (e.g., Means et al., 2013). Although the instructor exerts some degree of control of the students’ pace, place, time, and path of learning, blended learning allows students more flexibility and control over their learning (e.g., slow their pace to adjust to challenging content) than found in a traditional F2F learning environment (Christensen et al., 2013; Means et al., 2013). There are multiple modalities for blended learning. The flex model was used in the current study and is fully described in the Method. In general, the flex model is when significant content knowledge is delivered online outside of F2F class time, such as through recorded lectures, and application activities along with supplemental instructional support are provided during F2F meetings (Christensen et al., 2013). Our implementation of the flex model is identical to how flipped classrooms are operationalized in many studies, including the aforementioned meta-analyses (Chen et al., 2018; Cheng et al., 2019; Hew & Lo, 2018; Lo et al., 2017; van Alten et al., 2019), and as such, that literature is also highly relevant to the current study.

Theoretical framework

Ryan and Deci’s (2000) self-determination theory, which served as the study’s theoretical framework, posits that individuals all share a need for competence (i.e., need to be successful), autonomy (i.e., need to be in control and independent) and relatedness (i.e., need for belonging in a social group). In circumstances where these needs are met, individuals are more intrinsically motivated. Abeysekera & Dawson (2015) argue that blended learning/flipped classrooms may make a unique contribution to filling those needs for students. Clearly, autonomy is maximized in blended learning/flipped classrooms since the format is largely self-paced and allows students to engage with the course material on their own time and at a speed that makes them comfortable. That is, students have the control to manage their own cognitive load to maximize learning. Students indicate that the “on demand” availability of learning resources is valuable to their learning (Lo et al., 2017; Tomas et al., 2019; Varthis & Anderson, 2018; Wong-Ratcliff, 2020; Yilmaz & Malone, 2020). Additionally, in blended learning/flipped classrooms, students report improved learning because class time is devoted to more active engagement where they can apply the new concepts (Lo et al., 2017). As a result, students may be more successful and gain a greater sense of competence. Enhancement of relatedness is specific to how the blended learning condition is designed. Student’s relatedness needs can be met when blended learning/flipped classrooms include cooperative work or other activities to foster peer interaction as well as foster more active (versus passive) engagement of students to enhance instructor-student relationships.

Outcomes of blended learning/flipped classrooms

As previously mentioned, a growing body of research, including several meta-analyses, has shown the positive effects of blended learning/flipped classrooms on student achievement or attitudes (Chen et al., 2018; Cheng et al., 2019; Hew & Lo, 2018; Lo et al., 2017; Means et al., 2013; Tseng & Walsh, 2016; van Alten et al., 2019). Van Alten and colleagues (2019) recently conducted a rigorous meta-analysis of 114 studies that compared flipped classrooms to F2F instruction in the context of multiple academic domains in secondary and higher education. Findings indicated a significant small effect (g = 0.36) on student achievement. As another example, Cheng and colleagues (2019) conducted a meta-analysis on the effects of flipped classrooms on student achievement that included 55 studies across various academic domains. Again, flipped classrooms showed significant benefit with an overall small effect size (g = 0.193). In regard to student attitudes, Tseng & Walsh (2016) found that 52 undergraduate students in an English course self-reported greater motivation and learning in favor of the blended learning condition compared to a traditional F2F course.

Relatedly, Lindsay and Evans (2021)’s systematic review of the use of lecture captures in university mathematics courses recommended blended learning/flipped classrooms as a promising approach to promote student mathematics achievement. Their review found that lecture capture was not associated with improved learning since many students were likely to substitute watching the recorded class lectures rather than attending class. The researchers indicate that blended learning/flipped classrooms would provide the affordances of autonomous self-learning outside of the classroom as well as the active engagement experiences of in-person class (including the expectation of attendance).

Critically, not all studies demonstrate statistically significant effects in favor of blended learning/flipped classrooms (Chen et al., 2018; Cheng et al., 2019; Hew & Lo, 2018; Lo et al., 2017; Means et al., 2013; Olitsky & Cosgrove, 2014; van Alten et al., 2019; Wong-Ratcliff & Modesto, 2020). For instance, although the meta-analysis by Lo and colleagues (2017) found a significant effect for achievement, they found no significant effect on student attitudes. As another example, of the 55 studies included in Cheng et al.’s (2019) meta-analysis, 14 studies showed a negative effect for the flipped classroom condition; furthermore, only 14 of the 41 positive effect sizes were statistically significant.

Results of meta-analyses indicate that differences in effect sizes emerged based on instructional design, namely effects are stronger when instructors in flipped classrooms use formative assessments (Hew & Lo, 2018; Lo et al., 2017; van Alten et al., 2019). Differences may also emerge based on student characteristics. Joyce and colleagues (2014) conducted a randomized controlled trial with over 700 undergraduate students and found that undergraduate students on average do not perform statistically different in a blended learning compared to a traditional F2F course; however, a subsample analysis revealed that students who were not high performers had statistically better performance in the F2F course than in the blended format. In contrast, the meta-analysis by Means et al., (2013) found no achievement differences between online and F2F learning based on student characteristics.

Prospective teachers and mathematics

An emerging body of research has examined the effects of blended learning/flipped classrooms in teacher education (e.g., Yudt & Columba 2017; Demirer & Sahin 2013; Dove & Dove, 2015; Heba & Nouby, 2008; Kurt, 2017; Yilmaz & Malone, 2020). However, very few studies focus on mathematics. In addition to the earlier study conducted by the first author (Yudt & Columba 2017), only two known studies have specifically investigated the effect of blended learning/flipped classrooms compared to traditional F2F instruction on prospective teachers’ mathematics attitudes and/or achievement (Dove & Dove, 2015; Wong-Ratcliff & Modesto, 2020). Findings have been mixed. As context, the design of blended learning/flipped classroom in these studies is similarly, albeit generally, described as students reviewing instructional material (e.g., reading textbook chapters, completing chapter quizzes, watching online lectures) outside of and prior to F2F class time, with class time being devoted to application of concepts (e.g., hands-on cooperative activities, answering questions from independent learning, student presentations).

Two studies found that prospective teachers in the blended learning/flipped classroom condition had statistically significantly better mathematics attitudes compared to students in the traditional F2F condition (Yudt & Columba 2017; Dove & Dove 2015). However, Wong-Ratcliff & Modesto (2020) found that there were no statistically significant differences in the mathematics teaching self-efficacy of 92 U.S. prospective teachers based on their instructional condition.

Inconsistent results have also emerged on prospective teachers’ mathematics achievement. In the Yudt & Columba (2017) previous study, no significant condition difference was found for mathematics achievement. Given that instructional content and the instructor were the same across both conditions, it was not overly surprising that students learned equally well in both conditions. In their meta-analysis, Means and colleagues (2013) found that blended learning effect sizes were smaller when the F2F course was more similar to the blended learning course. In contrast, Dove & Dove (2015) found that students in the flipped classroom evidenced higher grades on a mathematics achievement test than students in the F2F condition. An important study limitation of the study by Dove & Dove (2015) is that students’ initial mathematics skills were not assessed, and it is thus unknown whether condition differences can be attributable to the class format or whether the baseline mathematics skills of students in flipped classroom condition were more advanced than students in the F2F course.

The mixed nature of the effectiveness of blended learning is likely related to how the course formats were designed, although insufficient details were provided to systematically compare the design of the learning environments across studies. Given the inconclusive results, additional research is needed that provides a thorough description of the learning environments as well as an examination of what features of blended learning students find to be most beneficial.

Features and challenges of blended learning

Empirical literature illuminates four salient features of blended learning that promote student attitudes and achievement. First, students routinely report the affordance of flexibility in a blended course format, including their ability to control the pace or dosage of instruction (e.g., Lo et al., 2017; Tomas et al., 2019; Varthis & Anderson, 2018; Wong-Ratcliff, 2020; Yilmaz & Malone, 2020). Second, various methods can be used to promote student active engagement with instructional materials, such as interactive videos and simulations and incorporation of periodic formative assessments (Hew & Lo, 2018; Lo et al., 2017; Means et al., 2013; Tomas et al., 2019; an Alten et al., 2019; Varthis & Anderson, 2018). Third, students report learning more effectively when the online and F2F components of the course systematically reinforce one another, such as when F2F discussion is facilitated about online learning activities or students have F2F time to transfer their declarative knowledge into procedural knowledge (e.g., Lo et al., 2017; Tomas et al., 2019; Varthis & Anderson, 2018). Fourth, creating connections and dialogue among peers, such as through online discussion boards, has a significant impact on students’ achievement and course satisfaction (e.g., Cheng & Cau, 2016; Lo et al., 2017; Means et al., 2013).

However, barriers do exist. Technology, including reliable access to needed hardware or software, can be a challenge (Means et al., 2013; Yilmaz & Malone, 2020). Students who are used to traditional F2F learning may initially find the change in the learning format to be difficult (Lo et al., 2017). Specifically, students indicate that they may (a) not fully prepare for the F2F component, such as not spending the time to fully engage in the online component (Lo et al., 2017; Tomas et al., 2019), and (b) desire for more direct interaction with the professor, such as the ability to ask questions when material is first presented (Lo et al., 2017; Wong-Ratcliff & Modesto, 2020).

Conceptual replication

The current mixed-methods study is a conceptual replication of a quasi-experimental study conducted by the first author that examined the effects of blended learning on elementary prospective teachers’ mathematical achievement and attitudes (Yudt & Columba 2017). This study is a conceptual rather than a direct replication because several aspects were changed between the two studies. In the earlier study (Yudt & Columba 2017), mathematics achievement at the end of the course was satisfactory, but not exceptional, for both conditions (average of 82% in the traditional F2F format and 84% in the blended learning format). As such in this study, lectures used in both conditions (in person for F2F condition and online for the blended learning condition) and F2F activities across both conditions were updated to improve understanding of knowledge and processes (described further in Method). Additionally, significant enhancements were made to the out-of-class activities in the blended learning condition to provide greater active student engagement through increasing interactivity of instructional materials and using formative assessments. In the earlier study, the asynchronous blended learning videos were less engaging and interactive. Initially, the instructor-created video lessons were made using a SMART Notebook that displayed only the instructor modeling concepts on a white board. In the current study, the video lessons were recreated using Screencast-o-matic, which allowed the instructor to incorporate notes from the interactive whiteboard along with an interactive calculator and other relevant mathematics resources, such as interactive websites and resources (e.g., NCTM Illuminations). Asynchronous videos also included formative assessments (e.g., answering embedded questions via EdPuzzle software), which was not a design feature in the previous study. [Other recommended elements, such as synchronicity between the online and F2F activities and development of peer connections, had been present in the earlier study and were not adjusted in this study.] More detail is provided in the Method.

The second change was the addition of qualitative methodology in order to examine prospective teachers’ perceptions of which features of blended learning were more or less effective in supporting their learning. Previous studies that examined whether blended learning/flipped classrooms improved the mathematical achievement and attitudes of prospective teachers have not employed a mixed-methods approach.

Study purpose

The primary aim of this quasi-experimental mixed-methods study was to extend research on the effectiveness of blended learning to promote prospective teachers’ positive mathematical achievement and attitudes as well as understand what features of blended learning may be conducive to their learning.

Specifically, two research questions guided this study:

  1. 1.

    Do prospective teachers in the blended learning condition demonstrate better mathematics achievement and attitudes compared to students in the traditional F2F learning condition?

  2. 2.

    What specific features in a blended learning course are noted by preservice mathematics teachers as helpful (or not) to their learning?

In regard to the first research question, we hypothesized that prospective teachers in the blended learning condition would have significantly higher scores in mathematical achievement and attitudes than prospective teachers in the traditional F2F condition. We grounded this hypothesis in the extant literature that shows the benefits of blended learning/flipped classrooms and the improvements that were made to the blended learning condition since the earlier study (Yudt & Columba 2017). In regard to the second research question, all blended learning features were selected based on empirical evidence, so we hypothesized that all would be valued but that course flexibility would be deemed the most valued affordance given that this is a very commonly cited benefit.

Method

Participants

The study participants were 63 undergraduate students in a teacher education program at a small liberal arts college located in the northeastern United States. Participants were preservice elementary teachers who were enrolled in a required mathematics course. Participants ranged in age from 17 to 42 years. Forty-two (67%) participants were female, and 21 (33%) were male. The majority of participants were Caucasian (71%). Other races/ethnicities included: Black (8%), Asian (5%), Latinx (5%), and Other (11%).

Procedures

The participants in the study were enrolled in a required mathematics content course for elementary education majors. In alignment with AMTE (2017) standards for teacher preparation, the primary aims of this course were to increase preservice elementary teachers’ understanding of mathematical content and processes and to promote positive attitudes toward mathematics. Teachers obviously cannot effectively teach their students if they do not have the foundational knowledge and skills themselves. At the time of the study, there was concern in the teacher education program at the focal institution that too many prospective teachers were not achieving a passing score on the state’s basic skills test, which is required for certification. As such, remediation of mathematical content and processes was a critical instructional need for these students. The course was also designed to present material in an engaging and accessible way such that prospective teachers would also develop positive attitudes toward mathematics.

Based on natural course sequencing variations, students enrolled in the required mathematics course during different semesters; there were no differences in students between semesters. The Fall course section (n = 27 students) was designated as a traditional F2F format. The course sections in the Spring (n = 31 students) and Summer (n = 5 students) were designated as a blended flex learning format. The same instructor taught both the blended learning and traditional face-to-face formats to control for between-instructor confounds. All of the students who were enrolled in the courses consented to participate in the study.

During class time, all students completed the same researcher-developed mathematics achievement test and Dutton’s Attitude Scale (Dutton & Blum, 1968) at pre- and post-course. The pre- and post-course achievement tests were departmental business as usual practices in the mathematics course. The pre-test allowed the instructor to gain a critical understanding of students’ baseline knowledge and skills. The post-test was used as a graded summative assessment. A researcher-developed blended learning questionnaire was only administered to the blended learning students at the conclusion of the course. The participants received this questionnaire upon the completion of the course and had one week to complete and return it.

Commonalities across conditions

Students in both conditions had the same curriculum, objectives, and studied the same units including: introduction to problem-solving; sets, whole numbers and numeration; operations and properties; mental, electronic, and written computation; number theory; fractions; decimals, ratios, proportions, and percentages; integers; rational numbers, real numbers, and algebra; statistics; probability; geometric shapes; and measurement. Students in both conditions used the same textbook and had access to the same textbook e-learning activities provided by the publisher that were used for homework. All homework (formative assessment) and exams (summative assessment) were the same across conditions. Additionally, students engaged in the course for approximately six hours/week regardless of course type. Table 1 provides a typical weekly breakdown of activities and their time allocation for both conditions.

Table 1 Typical Weekly Time Allotment of Traditional and Blended (Flex Model) Course Activities
Full size table

Students in the F2F learning condition physically attended class three hours per week, whereas students in the blended learning condition attended class for 90 min per week. During the F2F teaching in both conditions, the teacher provided guided and independent practice activities in order for students to actively apply the mathematical concepts. Activities were designed to follow the Concrete, Representational, Abstract (CRA) instructional process, which moves learners’ understanding from concrete objects (e.g., Cuisenaire rods and unit blocks) to representational units (i.e., diagrams) to an abstract understanding (Heddens, 1986; Ross & Kurtz, 1993). Activities varied in terms of whether students completed them cooperatively (peer to peer learning), alone, or a combination of both. For example, for the lesson focused on experimental probability, students were each given a pack of M&M’s candy. In the first part of the activity, each student was to find the probability of selecting each individual color for their own candy. In the second part of the activity, students formed into groups of four and found the probability of selecting each color for their entire group. During the activities, the instructor circulated to answer questions. Following completion of the activities, students shared their answers and reasoning and the instructor provided positive and corrective feedback.

Specific to the blended learning condition

The blended learning environment included sessions of course content delivery through digital devices issued to all college students (MacBook Pro laptop or an iPad) or another personally-owned computing device of the students’ choosing. Prior to attending F2F class, students viewed instructor-created asynchronous video lessons where the instructor explicitly taught content through modeling on an interactive white board and other relevant resources (e.g., calculator, online simulations). Students were provided guided note templates to take notes on the key concepts as well as record any questions that they had. The instructor embedded activities to actively engage students and formatively assess their learning. Examples included inserting quizzes into the video lessons with Edpuzzle software and embedding NCTM Illuminations Interactives into the video lessons that students were required to complete and submit with their video lesson notes. Learning also involved asynchronous self-study using e-learning opportunities from the textbook publisher and other sources, such as IXL practice activities and Khan Academy.

Once a week, students in the blended learning condition were also required to post questions and comments related to the mathematical concepts learned that week on a discussion board. The aim of the discussion board was to facilitate communication between the instructor and students as well as between students. Students were encouraged to use the discussion board to collectively work through areas of confusion with the mathematical content. Specifically, students were required to write one post per video lesson and provide two responses to their peers’ posts. Typically, students would write one of two types of posts. First, students would upload a picture of their attempt at solving a particular homework problem if they experienced difficulty, and peers would provide feedback. Second, students would also reflect back to their elementary school days by recalling certain lessons that stood out to them for positive or negative reasons and the implications that they saw for their own teaching future. Peers would provide responses that affirmed and related to those experiences. The instructor moderated the responses, such as indicating what solutions were correct/incorrect and responding to students’ reflections.

With the exception of the 10-minute weekly quiz, all of the F2F class time was dedicated to knowledge application. Class would begin with a brief orientation to the activities by the instructor and then 70 min of guided and independent practice.

Specific to F2F learning condition

The traditional F2F course also began with a 10-minute weekly quiz which was followed by the instructor providing approximately 90 min of explicit instruction (two segments of 45 min). Explicit instruction covered concepts using a didactic lecture format, including instructor modeling on a white board. This content mirrored the concepts that students in the blended learning condition experienced outside of class time. Additionally, F2F students were given the same guided notes template as students in the blended learning condition. The guided and independent practice activities were comparable in content to what students in the blended learning condition participated in, although they were slightly abbreviated (55 min compared to 70 min) due to more class time being dedicated to the lecture format.

Measures

Three measures were used as data sources for the study: a researcher-developed mathematics achievement test, Dutton’s Attitude Scale (Dutton & Blum, 1968), and a researcher-developed post-course questionnaire on blended learning.

Mathematics achievement

Students in both conditions completed a researcher-developed mathematics achievement test. The 24 items were determined according to the mathematics topics studied in this course and subsequently taught at the elementary level in alignment with the Pennsylvania Core Standards for elementary mathematics education. To gain a thorough understanding of their knowledge and processes, items were open-ended (not multiple choice), and students were expected to show their work (e.g., operations with rational numbers, determining prime factors, ordering fractions from least to greatest, long division). Additionally, prior to administration, the test was reviewed and approved by an expert panel of three elementary school teachers, one curriculum supervisor, three mathematics professors, and the mathematics department chairperson. The mathematics achievement test was graded on a scale of 0-100 points. The students completed the test during 75-minutes of allotted class time. Two mathematics professors scored the tests, and the average score was calculated and utilized in the data analysis. The results of the interrater analysis were Kappa = 0.76 (p < 0.001), which indicates substantial agreement (Landis & Koch, 1977).

Attitudes about mathematics

Students in both conditions completed Dutton’s Attitude Scale (Dutton & Blum, 1968) to measure overall attitudes toward mathematics, which has been used with elementary through secondary students as well as teachers (e.g., Dutton & Blum 1968; Philippou & Christou, 1998). Students rated 18 statements (e.g., “I have always been afraid of mathematics”; “Mathematics is something you have to do even though it is not enjoyable”) using a five-point Likert Scale (1 = strongly disagree to 5 = strongly agree). The measure took approximately five minutes to complete. The original published internal reliability coefficient was 0.9 (Aiken, 1970) and was similarly high at 0.84 in this study. Scores were averaged, and higher scores indicated more positive attitudes toward mathematics.

Attitudes about blended learning

A researcher-developed questionnaire was administered only to the students in the blended learning condition. Four open-ended questions queried students about their conceptualization of blended learning (e.g., “How does blended learning differ from traditional learning?, “In this course, did you view the instructor as a traditional teacher or facilitator; why?”). As shown in Table 2, eight items were Likert-scale items in which prospective teachers rated their agreement (1 = strongly agree to 5 = strongly disagree) with statements that compared blended learning to traditional F2F learning (e.g., “I have more control over my learning in blended learning than in traditional learning class”) or the degree to which blended learning supported their learning (e.g., “The blended learning experience has positively influenced my learning of mathematics and my approach to mathematical problems). Students then were provided open-ended prompts that asked students to explain why they provided their selected Likert rating. The final question asked students to rank seven features of blended learning in terms of what they liked the most to the least: (a) structure, (b) depth and breadth of learning, (c) pace of instruction, (d) in-class hands-on activities, (e) online interactive activities, (f) online discussion board, and the g) viewing of online video lessons. The survey took approximately ten minutes to complete. Prior to distribution, this questionnaire was reviewed by three teacher education faculty members.

Table 2 Descriptive Statistics for Post-course Questionnaire Questions (n = 25)
Full size table

Results

The purpose of this study was two-fold: (a) to evaluate whether students who participated in a blended learning course would demonstrate greater mathematics achievement and attitudes compared to students who participated in a traditional F2F course and (b) to determine what specific features of blended learning prospective teachers noted as supportive (or not) to their learning of mathematics.

Mathematics achievement and attitudes

To address the first research question that focused on achievement and attitude differences between conditions, we first ran descriptive statistics at pretest and posttest by condition (see Table 3). Achievement scores are represented in percentages. On average, students in the blended learning condition increased from 23.24 to 84.41%, and students in the traditional F2F format increased from 21.61 to 83.75%. Thus, students in both groups achieved satisfactory scores at posttest. Attitude scores are represented on a five-point scale, with one indicating very negative beliefs and five representing very positive beliefs about mathematics. On average, students in the blended learning condition increased from 2.80 to 3.00, which indicated they began with a slightly negative attitude toward mathematics and finished the course with a neutral attitude. Students in the traditional F2F format stayed static with a slightly negative attitude from pretest to posttest with scores of 2.82 and 2.80, respectively.

Table 3 Descriptive Statistics for Mathematics Achievement and Attitude in Blended and Traditional Learning
Full size table

Mathematics achievement

We ran a two-way analysis of covariance with mathematics achievement pretest scores as the covariate to examine whether posttest mathematics achievement was related to the instructional format. Prior to running the ANCOVA, several assumptions were tested. First, results from an ANOVA revealed no significant difference between the mathematics achievement pretest scores for students in the blended and traditional F2F conditions (F (2, 60) = 1.54, p = 0.22). Second, results of a two-way ANOVA revealed that the interaction term of type of instruction by mathematics achievement pretest scores was not a statistically significant predictor of mathematics achievement at posttest (F (2, 60) = 1.93, p = 0.18), therefore the homogeneity of regression assumption was met.

ANCOVA results indicated that there was no significant difference in mathematics achievement posttest scores between students in the blended learning condition and the traditional F2F learning condition after adjusting for pretest scores (F (1, 61) = 1.24, p = 0.27). Thus, the hypothesis that prospective teachers in the blended learning condition would have statistically significant higher scores in mathematics achievement at posttest than prospective teachers in the traditional condition was not supported.

Attitude toward mathematics

We ran a two-way analysis of covariance with Dutton’s Attitude Scale (Dutton & Blum, 1968) pretest scores as the covariate to examine whether mathematical attitudes at posttest were related to the instructional format. We again tested the two assumptions of the ANCOVA. First, attitude pretest scores were not significantly different between conditions (F (2, 60) = 1.03, p = 0.36). Second, the interaction term of type of instruction by attitude pretest scores was not a statistically significant predictor of attitude posttest scores (F (1, 61) = 5.05, p = 0.07), therefore meeting the homogeneity of regression assumption.

ANCOVA results indicated that there was a significant difference in attitude posttest scores between blended learning and traditional learning students after adjusting for pretest scores (F (1, 61) = 14.58, p < 0.01). Thus, results supported our hypothesis that participating in the blended learning condition was related to an improved attitude toward mathematics compared to traditional learning.

Features of blended learning

The second research question addressed what features of blended learning were valuable to prospective teachers in the blended learning course. Twenty-five of the 36 (69.4%) students in the blended learning condition completed the questionnaire.

One item on the questionnaire prompted students to rank seven features of blended learning in order what they liked most (1) to least (7): (a) structure, (b) depth and breadth of learning, (c) pace of instruction, (d) reinforcing in-class hands-on activities, (e) online interactive activities, (f) online discussion board, and (g) viewing of online video lessons. As shown in Table 4, students indicated that the flexibility of the course structure followed by control over the pace of instruction was the most liked component of the course while the online discussion board was least liked.

Table 4 Descriptive Statistics for Post-course Questionnaire (n = 25)
Full size table

prospective teachers rated their agreement with statements that compared blended learning to F2F learning or the degree to which blended learning supported their learning on eight Likert-scale items. As shown in Table 2, prospective teachers most highly agreed that they had greater control over their learning in blended learning as compared to F2F learning (and most disagreed with the reverse-worded item). In contrast to the mathematics achievement test results in which no significant improvement existed, prospective teachers perceived that blended learning positively impacted their mathematics learning and approach to problem-solving (i.e., this item had the second highest rating with 18 of 25 teachers agreeing or strongly agreeing). prospective teachers were fairly neutral in their other ratings about their overall learning experience, comfort, and use of discussion boards.

The open-ended items were analyzed using inductive coding by the first two authors. The first author initially used in vivo coding to identify what attributes of blended learning were found to be beneficial (or not) to participants’ learning (Saldaña, 2016). The second co-author reviewed the in vivo codes to determine their representativeness. No changes in coding were noted. Next, the first and second authors separately analyzed the in vivo codes to develop emergent themes (Saldaña, 2016). Discussions were held until consensus was reached on the themes. Three themes emerged from the blended learning participants’ open-ended responses on the post-course questionnaire: (1) control over environment, pace, and dosage, (2) time management, and (3) physical presence.

Theme 1: Control over environment, pace, and dosage

Control over environment, pace, and dosage refers to the prospective teachers’ feelings of choice over where they study and learn mathematics, how quickly or slowly they decide to learn, and the depth and breadth in which they study the content. All 25 prospective teachers discussed control over environment, dosage, and pace in their responses. The vast majority (86%) of the 126 statements positively indicated that blended learning provided enhanced control. Specifically, 28 comments (22%) focused on environmental control, 53 (42%) on control over pace, and 45 (36%) on control of dosage.

Twenty-one participants discussed how the choice of environment was a factor in their learning. Participants noted that being comfortable in their own environments helped them to concentrate. Specifically, many participants reported that the ability to work from home was a key benefit to blended learning. For instance, participant P stated, “I felt more comfortable viewing the lessons in my own time in my own environment” which allowed her to “block out the annoying things from class…and learn better laying down when I’m comfortable, which helps me focus.”

It is important to note that several participants found the blended learning environment to hold more challenges than traditional F2F learning. Despite free wireless internet service on campus, two students noted they experienced inconsistent internet access, which was a setback to learning. Additionally, two students commented that having choice of their environment created distractions. For instance, participant N noted, “I am used to a traditional classroom setting, and allowing me to study in an ‘environment of my choice’ only gives me the option to procrastinate.”

All 25 participants provided feedback that blended learning enhanced their control of the pace of instruction. Specifically, 21 prospective teachers positively commented on the issue of their ability to rewind, review, and pause the videos to ensure understanding of the content. For instance, participant P noted, “I am able to pause a lesson & practice the problem and work slower than others may want in a traditional classroom.“ Likewise, participant T agreed stating, “Learning at my own pace, where and when I want is easier than being forced to sit there and learn at the pace of the class.” Furthermore, two participants described feeling less anxious about learning course material because they had control over their pace of learning. For example, participant V stated, “Math is a relatively difficult course…but working at a pace that suits me allows me to better accustom to any other stress or anxiety I receive as a result of math.”

Control over dosage was operationalized differently than control over pace, although the two concepts are closely related. Whereas control over pace referred to how quickly one progresses through lessons, control over dosage referred to the number of exposures to a given lesson. Specifically, students described how the ability to rewatch lessons increased their understanding. For example, participant W reported: “The independence and constant availability of lessons allowed for me to revisit things I don’t understand.“

Theme 2: Time management

Time management referred to the preservice teacher’s ability to be productive and effectively manage their time in their learning. Ten prospective teachers discussed the impact of time management on their learning. The majority of comments (85%) were negative. prospective teachers indicated how affordances granted by increased control led to procrastination or feelings of anxiety related to poor time management skills. For example, participant Q explained, “I do have more control, but [that] could be an issue because I’m more likely to procrastinate and forget things.”

However, some comments (15%) indicated that the ability to work independently and at their own pace was a helpful asset to manage time effectively. For instance, participant K described, “I seemed to manage my time better and prepare more thoroughly for assignments and exams [in the blended learning environment].”

Theme 3: Decreased physical presence

Twenty-two prospective teachers reported that decreased physical presence, both their own and the instructor’s, influenced their learning and perception of the course. Slightly more than half of the comments (55%) reported it to be a hindrance to learning, whereas approximately one-third (38%) found it helpful and empowering to learn by themselves in physical separation from the instructor and classroom environment; the remaining 7% of comments were neutral.

Many prospective teachers believed that the decrease in instructor physical presence in the blended learning environment increased prospective teachers’ own workload and required self-teaching skills. Participant P commented, “asking questions online will never help me feel more comfortable…” Participant N said, “I was surprised by the immense workload and difficulty. I felt like I had to teach myself more of the material than I would have liked to,” and as a result this student reported feeling “anxiety… because it was not always easy to ask for direct help.”

Most prospective teachers who positively remarked about decreased physical presence commented on the lack of embarrassment when asking questions. For instance, participant B described that blended learning “eliminates the embarrassment of seeming dumb in front of a classroom of students.” Similarly, participant V noted that she is “able to ask as many questions as [she] want[s] since [she does] not need to worry about taking up the class time.” Another student commented on increased access to the instructor: “She helped me to learn math easier both online and during class. A traditional teacher only helps students learn during class for a set time period.”

Several students expressed neutrality on the subject of physical presence, indicating that learning did not have to be inherently different between blended learning and traditional F2F instruction. For instance, participant F said, “I found it [asking questions online] the same as asking questions in class,” and participant S noted that the preference for blended versus traditional learning ultimately depends upon the instructor’s teaching efficacy and course material.

Discussion

This study, which was a conceptual replication of Yudt & Columba (2017), investigated the promise of blended learning in a mathematics content course for preservice elementary teachers with respect to mathematics achievement and attitudes as well as understanding what features prospective teachers valued in blended learning. There is a call in the educational field for replication studies (e.g., Travers et al., 2016) since they are critical to understanding the generalizability of interventions’ findings and are encouraged by funders, such as by the Institute of Education Sciences. Three main findings emerged from this study that require discussion. First, although students in the blended learning course did not have statistically significant greater achievement compared to students in the traditional F2F course, they evidenced statistically significantly better attitudes about mathematics. Second, although a key reported benefit of blended learning was a general sense of flexibility and control throughout the course, some students were concerned with their lack of adequate time management. Third, more than half of students in the blended learning condition reported that the decreased physical presence of the instructor was a barrier to their learning.

Mathematics achievement and attitudes

As was found in the former study (Yudt & Columba 2017) and partially consistent with our study hypotheses, findings indicated prospective teachers in the blended learning condition had statistically better attitudes toward mathematics than prospective teachers in the traditional F2F, but no significant difference between conditions was found for prospective teachers’ mathematics achievement. While the basis for this investigation were the meta-analyses and other studies that have demonstrated that blended learning results in increased achievement and/or attitudes compared to traditional F2F formats (Chen et al., 2018; Cheng et al., 2019; Hew & Lo, 2018; Lo et al., 2017; Means et al., 2013; Tseng & Walsh, 2016; van Alten et al., 2019), other studies have found no or mixed differences between conditions (e.g., Joyce et al., 2014; Olitsky & Cosgrove, 2014; Wong-Ratcliff & Modesto, 2020). While we endeavored to maximize the online (out-of-class) learning resources based on empirical recommendations (Hew & Lo, 208; Lo et al., 2017; van Alten et al., 2019), students in both the F2F course and the blended learning course performed similarly and to a satisfactory degree (grade average of B on the achievement test). As such, students likely met their need for competence as described in the Self Determinism Theory (Ryan & Deci, 2000) in both course formats. Further, both course formats had numerous features in common, including the instructor, which likely minimized the effects of blended learning (Means et al., 2013).

Although prospective teachers’ mathematics achievement was not associated with their learning condition, the improvement in participants’ attitudes about mathematics is an important finding. Preservice elementary mathematics teachers will soon be responsible for teaching mathematics to elementary students. If they have poor attitudes toward mathematics as teachers, it can be reasonably expected that they will convey these poor attitudes to their students (Levine & Pantoja, 2021; Richland et al., 2020; Szczygieł 2020). Ensuring that teachers have positive dispositions toward mathematics is critical (AMTE, 2017), especially in the subject of mathematics for which there is extensive evidence of common and widespread anxiety among both teachers and students (Hughes et al., 2019; Jenßen, 2021; Olson & Stoehr, 2019). We clearly acknowledge that students in the blended learning condition ended the course with only a neutral attitude toward mathematics, so more work needs to be done to improve prospective teachers’ mathematics attitudes. Grounded in the Self Determinism Theory (Ryan & Deci, 2000), learning environments that meet individual’s needs for autonomy, competence, and relatedness lead to increased intrinsic motivation for learning in those contexts.

One other important finding to discuss is that the results did not differ based on students’ baseline achievement or attitudes. For both achievement and attitudes, no statistically significant interaction between pretest scores and instructional condition was found for posttest scores. This indicates that students who began the course with lower or higher attitudes or achievement did not fare differently based on whether they participated in the blended learning or traditional F2F course. Although this result diverges with the finding of Joyce and colleagues (2014), the meta-analysis by Means et al., (2013) found no relation between student characteristics and differences in students’ performance in online versus F2F formats.

Flexibility and control versus time management

A theme in the qualitative data was that the increased flexibility and control afforded by the blended learning course allowed prospective teachers to proceed at their own pace, revisit unclear lessons, and otherwise engage with the course material in a more dynamic way than would have been possible within the traditional F2F learning context. This finding aligns with the affordance that blended learning promotes students’ autonomy (Abseyekera and Dawson, 2015), which is a fundamental human need per the Self Determinism Theory (Ryan & Deci, 2000). Extant empirical research on blended learning/flipped classrooms also documents students’ appreciation of blended learning’s flexibility to allow for more control over their own learning (Lo et al., 2017; Tomas et al., 2019; Varthis & Anderson, 2018; Wong-Ratcliff, 2020; Yilmaz & Malone, 2020). Furthermore, this form of engagement may be responsible for the improved attitudes toward mathematics and self-reported perceptions of improved achievement (albeit no corresponding statistically significant effect on the achievement tests was found). The need to engage in independent learning may have bolstered the prospective teachers’ sense of confidence and competence, also a fundamental human need per Self Determinism Theory, as they realized that they actually were able to master the course material through independent learning, without needing the instructor to walk them through every step. Increasing teachers’ self-perceptions has been shown to translate to higher quality teaching (e.g., Guo et al., 2012)​​, so this finding is promising in regard to prospective teachers’ future students.

Although all participants appreciated the flexibility and control of blended learning, a downside to this aspect appeared in the open-ended responses. The increased flexibility of blended learning placed more responsibility on students to manage their own time in an effective manner, rather than counting on the instructor to fulfill this function for them. For ten of the 25 participants, this time management responsibility could be a daunting challenge. Difficulties managing time is echoed in the existing empirical literature (Lo et al., 2017; Tomas et al., 2019) as a known challenge to engagement in blended learning.

Comments specifically mentioned procrastination, which by definition, delays or postpones learning to a point when cramming is necessary to meet deadlines. Difficulties with time management may have resulted in limited engagement with the course material. In turn, this may have led to the null finding on mathematics achievement. Research on students who procrastinate experience significant difficulty in mastering course content and lower academic performance (Kennedy & Tuckman, 2013; Klassen et al., 2008). According to various theories of self-regulated learning (see Panadero 2017), before individuals act (in this case engage in learning activities), they must first engage in a task analysis to decide upon their performance goal and plan what strategies, including time management, are necessary to meet their goal. Future research could examine whether the incorporation of explicit instruction on self-regulated learning skills at the onset of the course would reduce time management concerns and positively impact mathematics achievement in blended learning.

Decreased physical presence of instructor

The most frequent blended learning challenge that students noted was the decreased physical presence of the instructor. Given that students in the blended learning condition were physically present with the instructor for half of the time that students in the F2F condition were, this concern is justified. This finding comports with extant literature that students who are less familiar with blended learning report wanting more direct interactions with the instructor, including being able to ask questions when material is first presented (Lo et al., 2017; Wong-Ratcliff & Modesto, 2020). According to the Self Determinism Theory (Ryan & Deci, 2000), individuals benefit when they experience strong feelings of relatedness. Although relatedness can be more difficult to achieve in a blended learning format than a traditional F2F format, blended learning approaches can be thoughtfully designed to incorporate interactions between teachers and students as well as peer interactions. For instance, instructor presence could be increased by virtual between-class meetings to address questions or common misconceptions found in homework or discussion board posts.

Although peer interactions were not explicitly noted as a concern, they obviously contribute to students’ feelings of relatedness and may somewhat offset students’ desire for more instructor presence. Instructors could encourage students to form study groups to develop peer interactions. The study group as a collective could post to the discussion board or directly contact the instructor in another way (e.g., email) to receive feedback. A different approach to the course discussion board may also be warranted, especially considering students reported that they did not find the discussion board to be a valuable learning tool. Thirty-one students were enrolled in the spring blended learning course. Given this class size, it is unlikely that students would deeply read all of their peers’ discussion board posts (Wise et al., 2013). Creating smaller parallel discussion boards that include fewer students may encourage students to more fully engage with the discussion board and develop closer relationships with their peers.

Lastly, incorporation of self-regulated learning supports may also diminish students’ reliance on the physical presence of the instructor. Guiding students on how to analyze tasks, set goals, and manage their time may lead to increased feelings of self-competence and less need for the instructor to serve as the physical expert.

Implications

Although blended learning promoted more positive, albeit still neutral, mathematical attitudes, no differences in mathematical achievement were found. The question remains whether more positive outcomes would emerge if blended learning had been designed differently. While enhancements to promote active engagement in the blended learning condition were made in this current study, this was not sufficient to create conditions where students in this condition outperformed students in the F2F condition or had overtly positive attitudes toward mathematics. prospective teachers reported that they appreciated the sense of autonomy that blended learning afforded (i.e., control over environment, pace, and dosage). As such, it is worthwhile to build on this benefit and explore further alterations to the blended learning approach based on preservice teacher feedback. Frontloading a module on self-regulated learning would be of utility to address student concerns about time management. Prior to students diving into the mathematical content, students could be taught how to analyze the tasks for the course, set performance goals, and plan how to meet these goals (Panadero, 2017). Additionally, the instructor could employ a gradual release of responsibility approach to scaffold students’ time management. For example, the instructor could use the announcement feature on the learning management system to remind students of due dates one week, three days, and one day in advance. Eventually, the instructor can lessen the frequency of deadline reminders. Through teaching and supporting students to manage the demands of more independent learning, students are likely to achieve higher achievement and have improved confidence since procrastination on coursework is associated with lower grades and lower self-efficacy for self-regulated learning (Klassen et al., 2008).

Additionally, prospective teachers also expressed a desire for more teacher-student interactions with some concerns that they felt the instructor was less present for them in the online learning component. This finding is echoed in literature describing the need for community formation in online learning environments (Lo et al., 2017; Means et al., 2013). Although a discussion board was provided and its use encouraged to create connections between and among the instructor and students, this forum was not seen as valuable to students. In the future, the instructor should (a) provide explicit instruction on how to maximize the learning potential of the discussion board which would align with recommendations of the task analysis step in self-regulated learning theory (Panadero, 2017) and (b) redesign the discussion board to provide more relationship building, such as having parallel discussion boards for smaller groups of users (Wise et al., 2017). Further, creation of collaborative small groups that meet between class sessions and virtual meetings with the instructor may be a more effective means of connection.

Access to technology is a critical concern when implementing blended learning. All students in this study were provided devices, but several students indicated that they did not readily have consistent access to the Internet, which made it difficult for them to effectively keep up with the coursework and study for exams. In this day and age, it may be tempting to assume that virtually everyone has convenient Internet access. However, this view reflects a privileged standpoint. Steps must be taken to first determine whether all students have full and equitable access to the online environment, and if not, it is necessary to provide this access as well as bolster in-class reinforcement activities to provide a legitimate alternative.

Limitations

Several study limitations require mention. First, this study involved a small number of students from only one college in the same elementary educational preparation program, which affects the generalizability of the findings. It is possible that studies with a larger sample drawn from a variety of teacher education programs would produce divergent results. Second, the course condition varied by - rather than within - semester. Although there was no obvious difference related to students between the cohorts (e.g., no students repeated the course), unknown contextual factors outside the scope of the study could have possibly influenced the results. Third, no one observed the instructor during the course to determine fidelity of implementation. Although course content and assessments were designed to be identical, no data was collected to verify that the courses were truly equivalent. Fourth, the self-reported data on the blended learning questionnaire may have suffered from social desirability. While it is possible that the prospective teachers wanted to be “nice” to their instructor and held back their true feelings about blended learning to some extent in their responses in the post-course questionnaire, this seems unlikely given that they provided criticisms about blended learning. Fifth, the questionnaire was heavy on probing students’ perceptions of the out-of-class components (since that is what was primarily redesigned) but the items do not allow for a discrete understanding of the various in-class and out-of-class elements of students’ experiences.

Recommendations for future study

Conducting this study with a larger sample would allow for subsample analyses. Although we did not find that initial student achievement or attitudes moderated the relation between post-test scores and learning condition, the sample was small. A larger sample size would provide a better test of this effect. As a second consideration for future research, researchers could enhance the blended learning condition to better meet students’ concerns, including embedding self-regulated learning instruction and providing more opportunities for teacher-student interactions, to determine whether those alterations would yield improvements in both achievement and attitudes. Finally, while this study assessed students’ perceptions about blended learning, the survey methodology yielded less detail than interviews or focus groups would provide. Future research should incorporate more rigorous qualitative methodology.

Conclusion

In order to encourage students to pursue critical careers in STEM fields, a strong mathematical foundation needs to be laid in elementary school, which includes having teachers who are skilled mathematics teachers and have positive attitudes about mathematics (Acar et al., 2018; AMTE, 2017; Levine & Pantoja 2021; Richland et al., 2020; Szczygieł 2020). Sadly, many elementary school teachers do not possess these characteristics (Hughes et al., 2019; Jenßen, 2021; Olson & Stoehr, 2019). This study was an attempt to examine how instructional design of a foundational mathematics course for future elementary teachers would impact their mathematics achievement and attitudes. While participation in blended learning was associated with improved mathematics attitudes, no similar improvement was found for achievement. prospective teachers’ feedback provided critical elements to consider for future design of effective and motivating learning environments.