Development of an instructional design model for flipped learning in higher education
- 14k Downloads
In response to pedagogical challenges in higher education, blended learning has become a prevalent practice in colleges and universities. Flipped learning (FL) represents a newly emerging form of blended learning, where students individually watch online lectures prior to class and then engage in classroom learning activities interacting with peers and instructors. Although the conceptual framework of FL may be intuitively appealing, its design and implementation involve considerable intricacy. The purpose of this study was to develop a FL design model for higher education that could systematically guide instructors or designers in creating an appropriate blend of individualized online lectures and collaborative face-to-face learning activities. Using an established method for model development research, a theoretically constructed initial model was iteratively improved and underwent internal and external validation through model usability tests, expert review, and field evaluation. The implementation of an algebra class in a Korean university designed by the final model resulted in significant increases in the maturity of mathematical views, quality of reflections, and satisfactions of students. Features of the model are discussed, along with theoretical and practical implications and suggestions for further research.
KeywordsFlipped learning Inverted learning Flipped learning design model Model development methodology Higher education
Colleges and universities have made ongoing calls for innovation in higher education, where for decades knowledge has been transmitted mainly through traditional lectures, despite continuous quality concerns (Arum et al. 2012). Numerous studies indicated that students do not learn critical, creative thinking, or complex reasoning skills during intellectually unstimulating classroom experiences (e.g. Arum and Roska 2011; Garrison and Kanuka 2004; McLaughlin et al. 2014). Since the early 2000s, blended learning (BL)—a convergence of face-to-face (F2F) classroom learning with technology-mediated online learning—has emerged in response to such challenges and become a prevalent pedagogical practice (Ginns and Ellis 2007; Graham et al. 2013). Typically implemented as a combination of synchronous classroom lecture and asynchronous text-based discussion forum (Graham 2006; Kerres and de Witt 2003), BL is intended to offer a more interactive and flexible learning environment for students that fosters core values of higher education such as advanced research and scholarship or critical and creative knowledge claims (Garrison and Vaughan 2008; Garrison and Kanuka 2004; Kerres and de Witt 2003; Lee and Jang 2014).
A crucial issue in the design of BL experiences is designing the right blend of learning online and F2F (Allen et al. 2007; Garrison and Kanuka 2004; Gedik et al. 2013; Hofmann 2002). Blending the two modes of learning does not mean simply combining the two or laying one on top of the other, as frequently occurs, but rather integrating the two by maximizing the advantages of each environment. This sort of complementary relationship requires recreating and reorganizing nearly the entire learning experience, however. The typical mix of blended learning involves adding supplementary online discussion activities to the classroom lecture. Furthermore, such F2F lectures have remained unchanged leaving learners passive recipients, and online activities have not engaged learners as intended (Arum and Roska 2011; Garrison and Kanuka 2004). Many researchers have critiqued that such a mix achieves neither the tremendous benefit of effective human interaction in the classroom nor the advantages of online systems and technological affordances for individualized pacing and flexible access to learning resources (Huon et al. 2007; López-Pérez et al. 2013; Lou et al. 2006; Orton-Johnson, 2009). The current improper mix of blended learning activities, then, calls for a better blend.
As a newly emerging type of BL, flipped learning (FL) represents a means to such a better blend. FL combines asynchronous online lectures that individual students study outside of class with F2F classroom learning activities in which students interact with peers and instructors (Bergmann and Sams 2013; Bishop and Verleger 2013; Goodwin and Miller 2013; Hamdan et al. 2013; Herreid and Schiller 2013; Mason et al. 2013; McLaughlin, et al., 2014; Milman 2013; Strayer, 2012). The driving questions to suggest FL were “when students are struggling and seeking help from their instructor or peers?”, and “how instructors can help students while optimizing the technological benefits of an online learning environment and the in-person benefits of human interactions in the classroom?” (Bergmann and Sams 2012). Classroom time is devoted to discovering and sharing ideas with one-to-one assistance, scaffolding, and inspiration, all made feasible by offloading content delivery onto the online lectures that are better at visual representations and self-pacing (Bush 2013; Goodwin and Miller 2013). In other words, FL saves for human instructors to do what only they can do best, and leaves to the technology to do what it can do best. This way of blending posits that neither component has a supplementary role, but rather each is a core complementary constituent of the learning experience as a coherent whole (Stannard 2012).
Yet, although the conceptual framework of FL may be intuitively appealing, there is considerable intricacy in its design and implementation, and practically infinite design variability in diverse contexts (Bergmann and Sams 2013; Garrison and Kanuka 2004; Overmyer 2012). Instructors and designers may find designing activities for a flipped learning classroom to be a daunting challenge for several reasons. First, online lectures must be designed so that their instructional and technical qualities are adequate enough to prepare students for more engaging classroom activities. Yet, even if the lectures are of good quality, students may not view or comprehend them, perhaps due to distractions online, a lack of self-regulation, or inadequate live support by the instructor (Bergmann and Sams 2012; Hamdan et al. 2013; Milman 2013). Second, the design of F2F requires greater preparation and deliberation in order to provide the interactive learning experiences that achieve the traditional values of higher education (Arum et al. 2012; McLaughlin et al. 2014). Most importantly, the design needs to create a tight link between what students do in class and what they do at home (Stannard 2012), a link that is not addressed by existing models that prescribe separate design processes for online lecture and classroom learning activities. Thus, though FL offers many advantages, its potential may be compromised by inadequate design and implementation, resulting in a corresponding lack of improvement in learning practices and outcomes.
The purpose of this study was to develop a FL design model to systematically and effectively guide designers or instructors throughout the design and implementation process of FL at the higher education level. This FL design model is intended to result in an appropriate blend of individualized online and collaborative F2F learning activities that coherently harmonize meaningful learning experiences for specific needs and contexts. Models make possible a systematic approach to design in that they intentionally lead designers to balance considerations of varied critical factors (Daly et al. 2012). Models incorporate both theoretical and empirical research in related fields (Dick 1997; Smith and Boling 2009). Further, the iterative process of evaluation and modification leads to improvement of practice, pointing to the potential of design to stimulated continued progress in a larger context (Daly et al. 2012; Smith and Boling 2009). The FL design model developed for this study is prescriptive by nature. It proposes when and how particular elements of FL should be selected and designed in a learning arrangement. Up to now, few prior publications on BL or FL have investigated closely the full design process (Halverson et al. 2014). The process for constructing and validating an FL design model reported in this study, and its implications for improvement course-level practice and educational innovation are thought to contribute to the growing literature on this meaning approach to learning.
Review of literature
Studies on FL began to appear around 2012, after FL practitioners had begun to share anecdotal reports with other (Goodwin and Miller 2013). Even if FL takes a fresh approach to learning by redefining class time and the role of technology, however, the pedagogy underlying FL is not entirely novel (Milman 2013). Numerous distance education and e-learning researchers have studied online lecture, just as those researching collaborative/cooperative learning, inquiry-oriented learning, or problem-based learning have studied F2F learner-centered learning. Team-based learning of Michaelsen, Knight, and Fink (2004) even proposed similar format to FL, but with more emphasis on the F2F part. Both lines of literature are relevant to FL because its success depends on defining the right blend of online and F2F sessions and arranging each ingredient in such a way as to build an integrated optimum learning experience (Halverson et al. 2014; Singh 2003).
The development of instructional design models specific to FL may provide invaluable assistance in the instructional design process. The steps in the instructional design process are based on knowledge of what creates a successful design product, knowledge that is primarily derived from related theoretical implications (Reigeluth 1992; Richey 1986). The BL literature and the burgeoning body of FL literature provide such insights, and can inform the development of a design model for FL. BL studies can inform decisions about the best blend of the two FL components, while FL studies can offer detailed and practical implications for design within each of the components.
Studies on blended learning
Although typical forms of BL include a mix of face-to-face instructor-led lectures and supplementary online activities (Garrison and Kanuka 2004; Ginns and Ellis 2007; Kember et al. 2010; Kerres and de Witt 2003), BL scholars and practitioners have long sought to identify the optimal blend of the two learning modes. According to a thematic synthesis of BL research performed by Halverson et al. (2014), publications on BL from 2000 through 2011 mostly questioned instructional design issues like models, strategies, and course structure.
The didactical 3C model developed by Kerres and de Witt (2003) involves a framework specifying three components of BL and corresponding learning tasks. The three components of BL were: (1) a content component that makes factual learning content available to learners, (2) a communication component that provides interpersonal interactions for more complex or arguable learning tasks between learners or learners and instructors, and (3) a constructive component that facilitates learners’ active engagement in most complex learning tasks. In this 3C model, the three ingredients are arranged according to the complexities of the learning tasks, and the weight of each ingredient depending on the learning objectives.
Several theories have addressed how these components of the 3C model can best be arranged using available media. One is media richness theory of Daft and Lengel (1984) and Rice (1992), which categorizes media into richer media (media with richer sensory input, immediate feedback or personal focus, for example, face to face meeting) and less rich media (media with less rich sensory input, delayed feedback or shared focus, for example, online discussion board), and learning tasks into equivocal tasks and unequivocal tasks. Media richness theory proposes that equivocal learning tasks with multiple or conflicting interpretations or solutions are better supported by richer media, while unequivocal tasks with low haziness (such as learning factual knowledge) may be allocated to less rich media.
Another theory that informs the allocation of BL components and tasks involves the cost of media/communication (Hollingshead, 1996; Kerres and de Witt 2003). Cost, then, is seen as related to the dependency of learning on time and space (asynchronous or synchronous), and the directionality of communication (uni-directional transmission or bi-/multi-directional interaction) in the learning process. In other words, the more dependent learning is on time and location, and the more directions in which interactions occur in the learning experience, the greater the cost of media or communication. Thus, face-to-face classroom learning would be a more expensive mode of communication, and would be recommended for intensively challenging, high-benefit learning activities.
Blending suggestions in related BL theories
Didactical 3C model (Kerres and Witt 2003)
Communication component that provides interpersonal interactions for more complex or arguable learning tasks
Constructive component that facilitates learners to actively engage in most complex learning tasks
Content component that makes factual learning content available to learners
Dependent on time and location
Intensive high benefit challenging learning activities
Equivocal learning tasks with multiple interpretations
Less dependent on time and location
Less intensive, less challenging learning activities
Unequivocal tasks with low haziness
Media synchronicity theory (Dennis and Valacich 1999)
Convergence process for learners’ shared focus and a common understanding at the same time
Conveyance process for the exchange of information
Studies on flipped learning
The FL research literature may be divided into studies with conceptual or empirical discussions. The conceptual studies concern definitions, comparisons with traditional approaches, educational rationales, and subject appropriateness of FL (e.g. Bergmann and Sams 2013; Bishop and Verleger 2013; Bush 2013; Mason et al. 2013; Milman 2013), while the empirical studies typically report on the effectiveness of actual cases of FL instruction and include implications for design that can guide potential practitioners (e.g. Covil et al. 2013; Herreid and Schiller 2013; Slomanson 2014; Strayer 2012; Talbert 2012).
The term FL derives from Baker’s (2000) phrase “the classroom flipped” (p. 9). More recently, Bergmann and Sams (2012) described FL in this way: in this way: what is traditionally done in class is done at home, and what is traditionally done as homework is done in class. Learners begin at home, learning with video lectures or screencasts by themselves prior to the class, and then engage in enriching activities that help them apply content at a deeper level (Bergmann and Sams 2013; Collins et al. 2001; Covil et al. 2013; Gannod et al. 2008; Lage et al. 2000; Strayer 2012). This way of blending maximizes the advantages of two modes of learning environment. A great strength of the online learning environment is that it can readily provide visual representations like space simulations or comparisons and allow for self-pacing (Bush 2013; Goodwin and Miller 2013). The F2F learning environment can be undoubtedly optimal media for group work, idea sharing with one-to-one assistance, immediate scaffolding, and individual inspiration. Thus, by flipping or inverting the allocation of time and locations for learning, FL combines two types of learning activities—active problem-solving learning activities and direct instruction/mastery learning activities (Bishop and Verleger 2013)—while making both approaches core components of a coherent and meaningful learning experience (Stannard 2012).
Most FL literature stresses that the most critical issue in making FL successful is structuring online and F2F learning experiences so that each component coherently supports the other at the macro-course level and micro-lesson level (Ginns and Ellis 2007). Most researchers commenting on the macro-course level emphasize the need for a much tighter link between the online and F2F modes of learning (Bush, 2013; Covil et al. 2013; Mason et al. 2013; Stannard 2012; Strayer 2012). In order to achieve a desirable macro-structure, the lecture part of learning should be kept to a minimum and the inquiry, discovery, and application parts of the learning process should be allocated maximum time (Stannard 2012). Technology should be used for what it can do better than humans, saving for instructors what only humans can do (Bush 2013). These recommendations largely correspond with recommendations in the BL literature. In addition, the FL literature specifically addresses the need for a consistent course structure with thorough descriptions of the course and design rationales in the syllabus, as well as an introductory orientation that addresses learners’ potential fear or resistance to a new method (Herreid and Schiller 2013; Mason et al. 2013; McLaughlin et al. 2014). Assessment of FL courses should include a variety of instruments designed to evaluate higher-order thinking skills like critical and creative thinking skills and problem-solving skills along with content mastery (McLaughlin et al. 2014). Assessment also should provide learners with multiple ways to demonstrate their understanding (Milman 2013).
Studies investigating the design of online sessions contain three important concerns: (1) physical features of online lectures, such as length, speed, and auditory quality (Bush 2013; Goodwin and Miller 2013; Hattie 2008; Khan, 2012; Mason et al. 2013; Smith and McDonald 2013); (2) content features of online lectures, such as proper allocation of online portions, interactivity, and clarity (Bush 2013; Goodwin and Miller 2013; Milman 2013; Shim, 2013; Smith and McDonald 2013); and (3) logistic features outside of the online lecture, such as formative evaluation, scheduling, and after-activity (Bush 2013; Mason et al. 2013; Slomanson 2014; Talbert 2012). In order to retain the learners’ attention the online lecture should be less than twenty minutes (Mason et al., 2013; Smith and McDonald 2013) and even less than ten minutes if it is a static screencast that does not show the instructor’s face (Khan 2012). Further, creating one clip per topic is better for further reorganizing and modularizing (Smith and McDonald 2013). The clip should have a function for controlling speed (Bush 2013; Goodwin and Miller 2013; Hattie 2008) and the audio quality should eliminate auditory distractions (Mason et al., 2013; Smith and McDonald 2013). In terms of content, online lectures should contain a basic overview or prerequisite content for an upcoming unit (Milman 2013), including a clear introduction and summary of main points (Smith and McDonald 2013).
The critical feature of a successful FL online lecture is its interactivity. Some researchers have suggested embedding interactive activities such as stop-think-answer, interactive graphs, or optional synchronous Q & A for this purpose (Bush 2013; Goodwin and Miller 2013; Shim 2013). In order to improve the quality of online lectures both instructionally and technically, formative evaluation of the resulting instructional product with potential learners and following revisions of the design are invaluable (Slomanson 2014). Among the most crucial advice emerging from FL practitioners is that it is important to design a speculative schedule for studying online content in order to help learners come to class prepared (Mason et al. 2013; Talbert 2012). Summarizing activities or self-checkup quiz also can help to ensure their understanding of online instruction (Bush 2013; Slomanson 2014).
Designing the F2F portion of FL can be much more demanding because it is the core element of FL. Relevant studies primarily concern: (1) initial activities such as verification quizzes and mini-lectures (Roehl et al. 2013; Strayer 2012; Talbert 2012); (2) main learning activities in terms of task features and the instructor’s role (Bergmann and Sams 2012; Goodwin and Miller 2013; Mason et al. 2013; Talbert 2012); and (3) external features such as after-activities, ground rules, and classroom culture (Covil et al. 2013; Hamdan et al. 2013; Smith and McDonald 2013; Talbert 2012). One suggestion for F2F sessions is starting with a quick verification quiz to check learners’ understanding of online content (Roehl et al. 2013; Talbert 2012), or conducting a 5–10 min Q and A session instead. These initial activities are a very important part of the FL design because successful engagement in those activities as entry tickets for the classroom activities can support the link between online and F2F session, and also promote learners’ motivation to come to class prepared. Furthermore, such initial activities enable the instructor to catch students’ lack of understanding or misconceptions before the class activity begins. Mini-lectures can then address the identified issues, and be geared to the level of the class. The relative ratio of lecture to class activity depends on the radicality of the instructor’s approach of the course: the more radical the approach, the higher the ratio of class activity to lecture. Though one researcher cautioned against entirely eliminating lectures or re-teaching (Talbert 2012), another suggested that mini-lectures should gradually be reduced as learners become more familiar with the FL structure (Strayer 2012).
Micro-level design elements/issues for designing FL from previous studies
FL online part
Physical feature design
Length; speed control; audio quality
Content feature design
Appropriateness; interactivity; clarity
External feature design
Formative evaluation of design; study Schedule; after activity
FL F2F part
Initial activity design
Verification quiz; mini-lecture
Main learning activity design
Task; instructor’s role
External feature design
After activity; ground rule; culture
Synthesizing Findings from the BL and FL Literature
An ID model is a systematic tool that assists designers in understanding related instructional variables and/or guides them through the design process (Lee and Jang 2014). Research on ID models may be classified into three different types: model development, model validation, and model use. This study concerned the first two of these, and largely followed the research methodology for model development and model validation explicated by Richey and Klein (2007), who noted that ID models may be developed through theoretical or practical means, or both. Theoretical approaches work by synthesizing related literature, while practical approaches utilize simulated design tasks or real-life design projects. The FL design model in this study was developed from a synthesis of relevant literature, simulated design tasks, and real-life design project data.
Further, a developed model may be validated. The validity of a model refers both to the appropriateness of the model components and the usefulness of the model with respect to purpose (Barlas 1994). ID model validation is a carefully planned process of collecting and analyzing empirical data to (1) provide support for each component of the model, or (2) prove its usefulness in practice (Richey 2005). ID model validation can be performed either internally or externally, or through the use of both methods. Internal model validation is a validation that addresses the integrity and its usefulness of a model (Richey and Klein 2007). The integrity of the model refers to how valid the components or processes of a model are; and the usefulness of the model points to how effectively the model “assists designers in understanding related instructional variables and/or guides them through the process of analyzing, designing, developing, implementing, and evaluating instructional products” (Lee and Jang 2014, p. 744). Prevalent methods of internal validation are expert reviews and model usability tests. External model validation deals with the effects of using the model: the quality of the ID products it creates and the benefits of these products for learners, clients, or organizations. Questions that might be asked in the course of an external validation include: “To what extent does the resulting instruction meet learner needs, motivate learners, or satisfy clients?” or “To what extent do changes occur in the learners’ or organization’s performance or learning?” (Richey 2005; Richey and Klein 2007). However, such outcome issues may be influenced by a variety of factors such as instructor variables, learner characteristics, or organizational priorities or policies. Typical methods used for external validation are field evaluations or controlled tests. The FL design model in this study underwent internal validation through expert reviews and model usability tests, and external validation through field evaluations.
Construction of the initial model
Participants for internal validation of the FL design model
Years of teaching experience
Master student/secondary school teacher
Master student/secondary school teacher
Instructor/FL instructional design
10 years (3 FL courses)
Instructor/FL instructional design
25 years (2 FL courses)
Instructor/FL instructional design
12 years (2 FL courses)
10 years (1 FL courses)
Instructor/FL instructional design
16 years (1 FL courses)
Internal validation I: model usability test
The refinement of the initial model led to the second model that the instructional design team used to design and implement a semester-long college calculus course for mathematics education majors at a Korean university. As the semester progressed, the team continued to develop both online learning content and F2F learning activities. We collected the team members’ reflections on the FL design process via individual interviews. The interview questions concerned conceptual and actual design tasks, such as, “Does this FL design model appropriately reflect actual design practice?”; Are there any step that are hard for designers to follow?; and “Can you elaborate on the component design process in a more detailed manner?” Insights gained from this first usability test were then used to create a revised second FL design model.
Internal validation II: model usability test
The second FL design model also underwent a the second model usability test involving members of the original design team and additionally recruited three experts who have experience in the design of FL (see Table 3 for information on these participants). These individuals completed a model usability questionnaire modified from Tracey’s (2001) model development study, and additional questions added for this study. The items on the model usability test concerned on the usability of overall FL design model, specific stages of the model, and resulting products. The responses to the questionnaire resulted in revisions to the second FL design model and, ultimately, the final FL design model for this study.
Internal validation III: expert review
The internal validity of the final model was tested using experts’ reviews. Total of five experts who have theoretical expertise and practical experience in the FL design were asked to answer questions on validity, as elaborated into explicability and comprehensibility (for addressing the integrity of the model components); and usability and generality (for addressing the usefulness of the model) of FL design model (See Table 1 for information on experts). Four of the experts were asked to review the model through individual F2F interviews, and one expert was asked to do it via email. The final content validity index (CVI) and inter-rater agreement (IRA) were calculated and reported. The CVI is a measure of how valid the model is, and the CVI of an item is calculated by dividing the number of experts with positive ratings (i.e. 3 or 4) by total number of experts. A CVI higher than 0.80 is recommended (Davis 1992). The IRA is a measure of the reliability of their ratings and overall agreement among experts (Lynn 1986). The IRA of a scale is calculated by dividing the number of items with an item IRA over 0.80 by the total number of items. The CVI and IRA represent the integrity of a model, that is, the validity of the components of a model.
The data used for external validation included (1) pre- and post-semester scores from a survey of changes in student views about mathematics (VAMS) (Carlson, 1999); (2) the reflection journal scores; (3) a class survey of learners’ satisfaction and follow-up interviews with selected students.
Views about mathematics (VAMS) test items
The instructor of the FL algebra course also asked students to record their reflections after each class. The students then submitted their reflection journals to the teaching assistants, who analyzed the content qualitatively according to the three types of reflective thinking identified by (Mezirow 1990a, b): content reflection, process reflection, and critical reflection. These types served as a priori categories for qualitative content. The reflections also were evaluated quantitatively for completeness and punctuality.
Learner satisfaction survey and follow-up interviews
Learners’ satisfaction data were collected through a class survey in the seventh week of class. In the fifteenth week of class, three students who had reported high satisfaction and two who reported low satisfaction in the seventh week were interviewed one-to-one using questions such as, “What do you find most helpful in the FL class?” “What do you find most challenging in the FL class?” and “What do you think is the difference between the FL class and the traditional lecture-centered class?”
The study involved three sets of participants whose feedback was sought at different times in the model construction and validations. The first set of participants was composed of four members of an instructional team (one university professor and three teaching assistants) who served as model users. The instructional design team took part in the model usability test and provided ratings on the general model usability, specific stages of the model, and the resulting products. The instructor was a professor with twenty-one years of experience in the field of mathematics education. The teaching assistants, one doctoral student and two masters students (who concurrently were employed as secondary school math teachers), had from three to eight years of teaching experience.
The second set of participants included five professors from US and South Korean universities who provided the expert review in the second test of internal validity. These five experts were recruited based on their theoretical expertise and experience in designing and implementing FL for at least one course. Their fields of study were physics, mathematics education, electronic engineering, and educational technology. Their experience in designing FL ranged from one to three classes, as indicated in Table 3.
The third set of participants in the study included 18 college students enrolled an algebra course developed from the study’s initial FL design model and taught by the first set of participants, the instructional team. The algebra course was offered in the department of mathematics education of a South Korean university in the fall 2013 semester. The group of students contained eight females (44%) and ten males (56%), and all were freshman mathematics education majors.
Initial FL design model
Second design model
The initial FL design model was first validated via two rounds of model usability tests. The results of the first model usability test led to revisions that were incorporated in the second FL design model. In the process of reflecting on the design and implementation process and sharing their insights, the instructional team helped identify two major ways the model might be improved. The first related to course level design and lesson level design; these were seen as needing to be divided since the instructional designers usually do different tasks for each level of design. At the course level, the analysis of goal, content, learner, and environment, and the content outline and instructional strategy design are performed at the macro-level. At the lesson level, however, the objectives of each lesson must be analyzed and online and F2F sessions must be designed and developed at the micro level. The second improvement inspired by the feedback of the instructional team was to include a formative evaluation step in the center of the model in order to reflect how the actual FL design process happens. An iterative revision process proceeds after a set of online and F2F sessions is designed and implemented. The observation and students’ feedback data collected during implementation of a lesson promotes the modification of the lesson or the next lessons.
Final FL design model
Finally, the FL design model can have three different usage scenarios at the lesson level. In the first usage scenario, a full set of online sessions is developed, and then a full set of F2F sessions is developed. After development, implementation begins. In the second scenario, a full set of online sessions is developed and each F2F session is developed for immediate implementation. In the third scenario, pairs of online and F2F sessions are developed for immediate implementation.
Final evaluation of final FL design model
Internal validation (expert review)
The panel for the expert review consisted of five professors from a variety of disciplines (educational technology, math education, electrical engineering, physics, and foreign language education), all with experience in the design of FL and theoretical expertise with FL. They were asked to evaluate the validity of final FL design model by providing ratings on the validity, explicability, usability, generality, and comprehensibility of the final model. Mean scores ranged from 2.8 to 3.8 on a scale of 4 to 1, with 4 indicating “strongly agree” and 1 indicating “strongly disagree”. The content validity index (CVI) and inter-rater agreement (IRA) was higher than 0.80 for all items, indicating that the validity of the model is acceptable (Davis 1992; Lynn 1986), and experts’ evaluations were mostly in agreement about the usefulness of the model. They were especially positive about the separated course and lesson level design in the higher education context, and the inclusion of a usability test step by which the FL design could be tailored to the particulars of a course. Additionally, they felt strongly that a detailed description of the model needed to be included for novice instructors who might otherwise find it difficult to effectively utilize the model. In addition, they believed that guidelines for allocating learning content to online or F2F sessions would be very helpful.
External validation (field evaluation)
A 15 week algebra course was designed according to the model. The topics of a traditional algebra class were assigned throughout the fifteen weeks, with a mid-term exam in the eighth week and a final exam in the fifteenth week. Each week, two clips with 20 min lectures recorded by the instructor were uploaded on the online course management system for students. In addition, students had a 75 min F2F class each week. Each F2F class started with a quiz on the lecture clips, followed by a 10–15 min mini-lecture. For approximately the next hour, students then engaged in group problem-solving activities with three to four problems provided by the instructors. The difficulty of these problems gradually increased over the class time. Students first discussed the problem in teams, and then participated in a whole class discussion. Students also were assigned to submit a reflection journal within the class day.
After implementing the FL course designed according to the final FL design model, the various data indicate how resulting instruction impacted students’ views of mathematics (VAMS), student reflections about the FL design of the class, and students’ satisfactions in regard to the course overall.
VAMS (views about mathematics survey)
Although the normality assumptions of Shapiro–Wilk statistics were met (p = 0.20, and p = 0.61, respectively), both parametric and non-parametric statistics were calculated due to small sample size (n = 17). Both Wilcoxon signed rank test and paired t test indicated that participants showed significantly higher scores in the post-VAMS (M = 3.59, SE = 0.085) than in the pre-VAMS (M = 3.28, SE = 0.061), (Z = 2.33, t(17) = 2.774, p < 0.05, r = 0.56).
Studies suggest that students with low VAMS scores usually have naïve views toward math and knowledge whereas students with high VAMS scores have advanced mathematical views and epistemological beliefs closer to those of experts (Hofer 2004). Students in the former group regard knowledge as a solid, absolute, structured, separate, and stable truth. To the contrary, students in the latter group regard knowledge as a variable and provisional agreement that is complex and constructed by interaction, and these students tend to have better mathematical problem-solving ability (Higgins 1997; Verschaffel et al. 1999) and greater ability to synthesize information than naïve viewers (Muis, 2008). Research also has indicated that the VAMS scores rarely increase much in a short period like a single semester (Carlson, 1999; Chris et al. 2006). In this study, then, the meaningful difference between students’ pre- and post-VAMS scores after just a single semester suggests that the FL design model had a very positive impact on the development and implementation of the FL algebra course.
In the seventh week of the algebra course, a short-answer mid-term survey was conducted to measure learners’ satisfaction. Students’ responses were somewhat mixed. Some students complained that their workload was too heavy due to the lecture clips, discussion participation, and reflection journals. Other complaints involved the limited time for problem solving during the F2F sessions and the inaccessibility of instant Q & A when learning with the lecture clips. Students also commented on the positive impacts of FL on their learning. Many valued being able to pause and repeat the clips when they did not understand a specific part or needed a while to reflect or think deeper. Students also responded very positively to the increased opportunities for discussion and active engagement in the overall learning process, and they reported enhanced motivation for learning itself.
I think group work was the best part here. When I asked other friends about what I couldn’t understand, I found at least one of us understood it well. Hearing this, I learned, and at the same time, I could practice explaining things that I understood well. That made me feel organized and I loved it. (Learner A)
In my group, there were some smart guys, and I felt more comfortable asking them questions than asking the teacher. Of course, they knew me [and my needs] better than the teacher and so they could explain it better. (Learner C)
Thanks to the group work, I think I studied harder this semester than last semester. Last semester, I rarely studied after the mid-term exam. However, this semester, in order to participate in discussions in class, I had to bring some basic knowledge to the classroom. I had to watch the video lectures and study the subject before the class. That was the main difference from the last semester. (Learner B)
I believe the key to success is students’ participation. I myself became engaged in the process of understanding during group discussions. Just watching lectures doesn’t guarantee understanding and they do not always give clear explanations about what I haven’t understood. Even when what I couldn’t understand was not a big part, [my lack of understanding] but was eliminated in group discussions. I loved the group work. (Learner E)
Last semester, the teacher couldn’t finish the class in time because of too much content to cover, and some students dozed off or gave up. However, when watching the video lectures we don’t doze off and we can take a nap if it is really needed and ask the teaching assistants questions in class. That was the good part. There was a synergistic effect when combining the video lecture and offline class. (Learner C)
First of all, the video lectures were helpful for reviewing the study materials when preparing for exams. They could be replayed anytime, and [helped me] to master calculus anyway because calculus requires incremental understandings. I felt I had to master theories by myself but it gave chances to think deeply about the subject. (Learner B)
It should be a four-credit course with this much homework. There were lots of small homework assignments and they counted for 5% of our total grade. We were evaluated multi-dimensionally. That was a really good part of it. It was possible because it was flipped classroom. So, it went past rote learning and went deep into the subject. More subjects were introduced, and students were more interested in them. (Learner D)
I didn’t study that much for the mid- and final exams. Actually, I didn’t need to…’cause I already had studied and mastered the content. Last semester, I usually stayed up nights for the exams because I had to start to review and recall the content from the beginning. (Learner E)
I would take it again because it was fun. I confess I am one who dozes off a lot in class, but I never did in this one. (Learner D)
If I could choose between a normal class and a flipped learning class, I would definitely choose a flipped one. (Learner E)
Discussion and conclusion
This study was a developmental investigation of a process for constructing and validating a flipped learning design model. Through an iterative process of review and revision, the FL design model was formatively improved and internally validated. The implementation of the final model was shown to result in meaningful increases in students’ maturity of mathematical views and epistemological beliefs, reflections, and satisfactions. Related issues that emerged during the process will be discussed below.
Distinctive features of the final FL design model
Two distinctive features of the final FL design model would worth further discussion: the macro and micro two-level approach and the integrative design of online and F2F sessions. First, in taking both a macro-view of the course level, and a micro-view of each lesson level, the FL design model creates a more systematic approach to FL design. At the macro-level, course goals are defined, which in turn guides the design of macro-level content and micro-level objectives and content. The macro-design corresponds to the syllabus design, and the micro-design corresponds to the lesson plan design, by which the model reflects the design context of higher education. This sort of two-level approach for designing FL has been suggested in the FL literature, though without any explicit recommendations. In fact, the review of FL literature revealed its clear division into studies concerning either macro-level or micro-level design elements. Macro-level studies tended to concern issues like content allocation and a consistent structure throughout the lessons of a course (Bush 2013; Covil et al. 2013; Mason et al. 2013; Stannard 2012; Strayer 2012), an introductory orientation to the overall design rationale (Herreid and Schiller 2013; Mason et al. 2013; McLaughlin et al. 2014), and course outcomes and assessment at course end (McLaughlin et al. 2014; Milman 2013). By contrast, micro-level studies concerned the length of lessons (Mason et al. 2013; Smith and McDonald 2013), speed control of lessons (Bush 2013; Goodwin and Miller 2013; Hattie, 2008), audio quality of lecture clips (Mason et al. 2013; Smith and McDonald 2013), strategies for interactivity (Bush 2013; Goodwin and Miller 2013; Shim 2013) the study schedule of online lectures (Mason et al. 2013; Talbert 2012), in-class verification quizzes (Roehl et al. 2013; Talbert 2012), mini-lectures (Strayer 2012; Talbert 2012); and in- and after-activity design of F2F sessions (Covil et al. 2013; Goodwin and Miller 2013; Mason et al. 2013). The apparent distinction between the two lines of FL studies indicates that design tasks at the macro- and micro-level are best modeled separately, as the final FL design model in this study proposes.
Secondly, the final model starts with a collective analysis step dealing with both online and F2F sessions, and proceeds to the formative design and development of the online lessons. The design and development of F2F lessons start with another analysis specifically intended for the design of F2F activities. Results from the common implementation and evaluation are to be reflected in the analysis step of the next lessons. These model changes can be interpreted as working towards the right blend, that is, the optimal integration of the two parts of FL. The common analysis, implementation, and evaluation steps will undoubtedly improve the amalgamation of the online and F2F pair by aligning the two sessions with a shared lesson objective and by allocating content into on and off, thus maximizing the advantages of each environment.
These strategies are consistent with recommendations derived from related FL literature. Most FL studies have stressed that online and F2F learning experiences should coherently support each other in order to create a successful FL experience (Bush 2013; Covil et al. 2013; Mason et al. 2013; Stannard 2012; Strayer 2012). This kind of coherent support means that the two FL components have a complementary rather than supplementary relationship. Put simply, without the online sessions, F2F activity should not be effective; and without the F2F sessions, learning with online lectures should not be complete or have enough depth. Conducting an integral evaluation of online and F2F learning, in particular, can increase the congruency of all the learning experiences in a FL course. This is because such an evaluation reveals the extent to which each portion of the design as well as the entire design optimally contributes to the achievement of course and lesson goals and objectives. Another analysis for designing F2F activities represents an effort to increase the tight integration of the two parts, and to design the F2F sessions more deliberately.
The final FL design model created in this study evolved gradually, and more making it possible to create a specified guide to the FL design process with model assumptions, usage scenarios, and step-by-step descriptions. Model specificity can be achieved both by specified model assumptions and by specified step components. The model assumptions and usage scenarios specify the target users, scope, and design context for which the final FL design model should be used. The step-by-step descriptions provide more precise guides that are useful when designers make decisions on the actual design.
However, in general, the more specific a model gets, the narrower the application of the model becomes. Downes (2003) referred to this dilemma when asserting that design requires specificity but specificity is incompatible with reusability and general application. The task of finding the best balance between a useful model with specific and practical guidelines and a wide-reaching model with general and flexible guidelines is crucial, but challenging. In the process of internal validation in this study, the experts closer to being practitioners preferred the former approach whereas the experts who were closer to being theorists prefer the latter. These preferences reflect the theoretical and practical roles of ID models, that is, to promote understandings of ID realities and to guide ID performance respectively (Branch and Kopcha 2014; Davies 1996; Gustafson and Branch 2002; Jung and Rha 1989; Lee and Jang 2014; Rubinstein 1975; Seels and Glasgow 1998). Since an ID model for designing FL is not yet available in the research literature, the final FL design model developed in this study was intended for general application within the higher education context, while clearly specifying the meanings of the component steps so that the steps can cognitively guide designers to make intelligent design decisions.
This study may be limited in that the FL design model was developed from a single case, suggesting the need for confirmation in more cases. Also, since the case was an algebra course, the model may include some features relevant to courses in natural sciences but not to disciplines such as liberal arts, social sciences, or art and music, which may require different design processes. Further, even though the FL design model in this study underwent internal validation from experts from diverse disciplines like engineering, physics, mathematics education, and educational technology, actual implementation of the model in courses in these fields might reveal different aspects of FL design.
Another potential limitation of this study that suggests future research topics regards the broad definitions used for outcomes of FL. Many FL studies have found meaningful improvements in students’ satisfaction or attitude (e.g. Bland 2006; Kellogg 2009; Talbert 2012). However, other studies have reported insignificant or marginal increases in academic performance when comparing FL to traditional methods (e.g. Kellogg 2009; Papadopoulos and Roman 2010). In those cases, the academic achievement of students in FL courses was measured using the same instruments as in traditional courses. By contrast, in this study the outcomes of the FL course were broadly defined as epistemology change, quality of reflection, and satisfaction. Observation throughout the semester revealed other areas in which FL has positive outcomes, including study skills, presentation skills, collaborative or communicative skills, and inquisitive attitudes toward learning, some of which are mentioned in previous studies (Bishop and Verleger 2013; Mason et al. 2013; Talbert 2012). Thus, balancing qualitative and quantitative assessment, and content knowledge and general competencies, research on more comprehensive approaches to evaluating the learning impacts of FL is recommended for the future.
The interview data from students taking the course pointed to changes in their epistemology and classroom culture. Thus, a potential impact of the course designed using the FL design model may be innovation in learning from a broader institutional context, which may be due to the transformative potential of design by envisioning different types of learning experiences (Collins et al. 2004). Design has been described as systematic planning for future innovation (Rowland 2008; Simon 1969; Smith and Boling 2009), an approach that diverges from the widely-accepted view of design as a solution to a problem. Increasingly, design is seen as providing “a big picture lens of the problem” in a larger context rather than “a direct path” (Daly et al. 2012, p. 200) or an immediate intervention to a problem at hand. Some scholars have even argued that design is a matter of “shaping the world into desired states” (Molenda and Boling 2008, p. 121) or “how things ought to be” (Simon 1969, p. 133), thus “making the world a better place” (Laurillard 2012, p. 225). Design models, then, can inform, guide, and lead to successful educational innovations (Smith and Boling 2009). Although proposed innovative potential of FL design models is somewhat speculative, they nevertheless may serve as a bridge that approaches FL from the individual course level as well as from the level of institutional support and policy. It is hoped that the FL design model developed in this study would contribute at both level.
We are grateful to the experts and students whose valuable comments enriched this research. We owe particular thanks to Professor Onam Kwon and Dr. Younggon Bae for their support in implementing the FL course under study.
- Allen, I. E., Seaman, J., & Garrett, R. (2007). Blending in: The extent and promise of blended education in the United States. Newburyport, MA: Sloan Consortium.Google Scholar
- Arum, R., Cho, E., Kim, J., & Roska, J. (2012). Documenting uncertain times: post- graduate transitions of the academically adrift cohort. New York: Social Science Research Council.Google Scholar
- Arum, R., & Roska, J. (2011). Academically adrift: Limited learning on college campuses. Chicago: University of Chicago Press.Google Scholar
- Baker, J. W. (2000). The classroom flipped: Using web course management tools to become the guide on the side. Paper presented at the 11th International Conference on College Teaching and Learning, Jacksonville, FL.Google Scholar
- Barlas, Y. (1994). Model validation in system dynamics. Paper presented at the International System Dynamics Conference.Google Scholar
- Bergmann, J., & Sams, A. (2012). Flip your classroom. Eugene, OR: International Society for Technology in EducationGoogle Scholar
- Bergmann, J., & Sams, A. (2013). Flipping for mastery. Educational Leadership, 71(4), 24–29.Google Scholar
- Bishop, J. L., & Verleger, M. A. (2013). The flipped classroom: A survey of the research. Paper presented at 120th ASEE Annual Conference & Exposition, Atlanta, GA.Google Scholar
- Bland, L. (2006). Applying flip/inverted classroom model in electrical engineering to establish life-long learning. Paper presented at 113th ASEE Annual Conference & Exposition, Chicago, IL.Google Scholar
- Bush, M. D. (2013). Educational technology points of inflection: what MOOCs, flipped classrooms, and OLPC teach us about individualization of learning. Educational Technology, 53(6), 60–63.Google Scholar
- Covil, D., Patel, B. A., & Gill, D. S. (2013). Flipping the classroom to support learning: an overview of flipped class from science, engineering and product design. School Science Review, 95(350), 73–80.Google Scholar
- Daft, R. L., & Lengel, R. H. (1984). Information richness: A new approach to managerial behavior and organization design. In B. M. Staw & L. L. Cummings (Eds.), Research in organizational behavior (Vol. 6, pp. 191–233). Greenwich: JAI Press.Google Scholar
- Davies, I. K. (1996). Educational technology: Archetypes, paradigms and models. In D. P. Ely & T. Plomp (Eds.), Classic writings on instructional technology (pp. 15–30). Englewood. CO: Libraries Unlimited Inc.Google Scholar
- Dennis, A., & Valacich, J. (1999). Rethinking media richness: Towards a theory of media synchronicity. Paper presented at the 32th Hawaii International Conference of Systems Sciences, Maui, HI.Google Scholar
- Dick, W. (1997). A model for the systematic design of instruction. In R. D. Tennyson, E. Schott, N. M. Seel, & S. Dijkstras (Eds.), Instructional design: international perspectives (pp. 361–369). Mahwah: Lawrence Erlbaum Associates.Google Scholar
- Downes, S. (2003). Design and reusability of learning objects in an academic context: a new economy of education? Journal of the United States Distance Learning Association, 17(1), 3–22.Google Scholar
- Gannod, G. C., Burge, J. E., & Helmick, M. T. (2008). Using the inverted classroom to teach software engineering. Paper presented at the 30th International Conference on Software Engineering, Leipzig, Germany.Google Scholar
- Garrison, D. R., & Vaughan, N. D. (2008). Blended learning in higher education: Framework, principles, and guidelines. San Francisco: John Wiley & Sons.Google Scholar
- Goodwin, B., & Miller, K. (2013). Evidence on flipped classrooms is still coming in. Educational Leadership, 70(6), 78–80.Google Scholar
- Graham, C. R. (2006). Blended learning systems: Definition, current trends, and future directions. In C. J. Bonk & C. R. Graham (Eds.), Handbook of blended learning: global perspectives, local designs (pp. 3–21). San Francisco: Pfeiffer Publishing.Google Scholar
- Gustafson, K., & Branch, R. M. (2002). Survey of instructional development models (4th ed.). Syracuse: Eric Clearinghouse on Information & Technology.Google Scholar
- Hamdan, N., McKnight, P., McKnight, K. & Arfstrom, K.M. (2013). A review of flipped learning. http://flippedlearning.org/review.Retrieved 18 Feb 2014
- Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.Google Scholar
- Herreid, C. F., & Schiller, N. A. (2013). Case studies and the flipped classroom. Journal of College Science Teaching, 42(5), 62–66.Google Scholar
- Hofmann, J. (2002). Blended learning case study. In A. Rossett (Ed.), The ASTD e-learning handbook: Best practices, strategies, and case studies for an emerging field (pp. 516–519). New York: McGraw-Hill.Google Scholar
- Jung, I., & Rha, I. (1989). Instructional design theories. Seoul: Educational Science.Google Scholar
- Kellogg, S. (2009). Developing online materials to facilitate and inverted classroom approach. Paper presented at the 39th ASEE/IEEE Frontiers in Education Conference, San Antonio, TX.Google Scholar
- Khan, S. (2012). The one world schoolhouse: Education re-imagined. London: Hodder & Stoughton Ltd.Google Scholar
- Laurillard, D. (2012). Teaching as a design science: building pedagogical patterns for learning and technology. New York: Routledge.Google Scholar
- Mezirow, J. (Ed.). (1990a). Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning. San Francisco: Jossey-Bass.Google Scholar
- Mezirow, J. (1990b). Fostering critical reflection in adulthood: a guide to transformative and emancipatory learning. San Francisco: Jossey-Bass.Google Scholar
- Michaelsen, L. K., Knight, A. B., & Fink, L. D. (Eds.). (2004). Team-based learning: a transformative use of small groups in college teaching. Sterling: Stylus Publishing.Google Scholar
- Milman, N. B. (2013). The flipped classroom strategy: what is it and how can it best be used? Distance Learning, 9(3), 85–87.Google Scholar
- Molenda, M., & Boling, E. (2008). Creating. Januszewski, A., & Molenda, M. (Eds.). Educational technology: A definition with commentary. New York, NY: Routledge.Google Scholar
- Overmyer, J. (2012). Flipped classrooms 101. Principal, 9–10, 46–47.Google Scholar
- Papadopoulos, C., & Roman, S.A. (2010). Implementing and inverted classroom model in engineering statics: Initial results. Paper presented at the 117th ASEE Annual Conference & Exposition, Louisville, KY.Google Scholar
- Richey, R. C. (1986). The theoretical and conceptual bases of instructional design. New York: Kogan Page.Google Scholar
- Richey, R. C. (2005). Validating instruction design and development models. In: Spector J. M. and Wiley D. A. (Eds.), Innovations in instructional technology: Essays in honor of M. David Merrill (pp. 171–185.) Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
- Richey, R. C., & Klein, J. D. (2007). Design and development research. Mahwah: Lawrence Erlbaum.Google Scholar
- Rubinstein, M. (1975). Patterns of problem solving. New York: Prentice Hall.Google Scholar
- Seels, B. A., & Glasgow, Z. (1998). Making instructional design decisions (2nd ed.). Upper Saddle River: Prentice Hall.Google Scholar
- Shim, H. (2013). A commentary on the flipped classroom instructional model. Paper presented at the 30th Symposium on Mathematics Education. Seoul, Korea.Google Scholar
- Simon, H. A. (1969). The sciences of the artificial. Cambridge: MIT Press.Google Scholar
- Singh, H. (2003). Building effective blended learning programs. Educational Technology, 43(6), 51–54.Google Scholar
- Slomanson, W. R. (2014). Blended learning: a flipped classroom experiment. Journal of Legal Education, 64, 1–10.Google Scholar
- Smith, K. M., & Boling, E. (2009). What do we make of design? Design as a concept in educational technology. Educational Technology, 49(4), 3–17.Google Scholar
- Stannard, R. (2012). The flipped classroom or the connected classroom? Looks at the benefits of putting more input online. Modern English Teacher, 21(1), 35–37.Google Scholar
- Talbert, R. (2012). Learning matlab in the inverted classroom. Paper presented at 119th ASEE Annual Conference and Exposition, San Antonio, TX.Google Scholar
- Tracey, M.W., (2001). The construction and validation of an instructional systems design model incorporating multiple intelligences. Unpublished doctoral dissertation. Detroit: Wayne State University.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.