Abstract
We discuss a mixed methods approach for assessing the flipped classroom, which we applied to a school-wide initiative starting in the fall of 2013. Assessment of a flipped classroom is, in many ways, no different than rigorous assessment of any good pedagogy. Assessment planning must first consider the objectives of the pedagogical initiative. The critical question we asked was “What educational gains or advantages should students experience as a result of course flipping?” We then focused on the selection of instruments and protocols for measurement. To study student learning and achievement, we analysed pre-flip versus flip exam and homework results and formally interviewed instructors. To investigate in-class engagement and active learning, we conducted classroom observation using a validated protocol. Using web analytics video access data, we investigated preparation with the flipped classroom and its relationship to achievement. Finally, to assess student perceptions, we used an evaluation survey tailored to the flipped classroom and a research-based classroom environment instrument. A comprehensive and thorough assessment plan provides the advantage of both formative and summative data for an initiative and can guide future directions with it.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Boulmetis, J., & Dutwin, P. (2011). The ABCs of evaluation: Timeless techniques for program and project managers (p. 140). San Francisco, CA: Wiley.
Brent, R. (2012). Process evaluation: The vital (and usually) missing piece in educational research. In Proceedings of the ASEE Annual Conference and Exposition, San Antonio, TX.
Clark, R., Besterfield-Sacre, M., Budny, D., Bursic, K., Clark, W., Norman, B., et al. (2016). Flipping engineering courses: A school wide initiative. Advances in Engineering Education, 5(3).
Clark, R., Budny, D., Bursic, K., & Besterfield-Sacre, M. (2014b). Preliminary experiences with “flipping” a freshman engineering programming course. In Proceedings of First Year Engineering Experience (FYEE) Conference, College Station, TX.
Clark, R., Clark, W., & Besterfield-Sacre, M. (2017). Experiences with “flipping” an introductory mechanical design course. In C. Reidsema, L. Kavanagh, R. Hadgraft, & N. Smith (Eds.), The flipped classroom: Practice and practices. Sydney: Springer.
Clark, R., Norman, B., & Besterfield-Sacre, M. (2014a). Preliminary experiences with ‘flipping’ a facility layout/material handling course. In Proceedings of the Industrial and Systems Engineering Research Conference, Montreal.
Coll, R., Taylor, N., & Fisher, D. (2002). An application of the Questionnaire on Teacher Interaction and College and University Classroom Environment Inventory in a multicultural tertiary context. Research in Science & Technological Education, 20(2), 165–183.
Creswell, J. (2013). Qualitative inquiry and research design: Choosing among five approaches (p. 251). Thousand Oaks, CA: SAGE Publications Inc.
Fitzpatrick, J., Sanders, J., & Worthen, B. (2011). Program evaluation: Alternative approaches and practical guidelines (pp. 385–386, 422, 425). Upper Saddle River, NJ: Pearson Education, Inc.
Fraser, B. (2012). Classroom learning environments: Retrospect, context and prospect. In B. Fraser, K. Tobin, & C. McRobbie (Eds.), Second international handbook of science education (pp. 1191–1239). Dordrecht: Springer.
Fraser, B. J. (Personal Communication, October 2014). Ph.D., Distinguished Professor, Associate Dean, and Director of the Science and Mathematics Education Center, Curtin University, Perth, Western Australia.
Fraser, B., & Treagust, D. (1986). Validity and use of an instrument for assessing classroom psychosocial environment in higher education. Higher Education, 15, 37–57.
Gray, P. (2010). Student learning assessment. In E. Crawley, J. Malmqvist, S. Ostlund, & D. Brodeur (Eds.), Rethinking engineering education: The CDIO approach (pp. 152–165). New York: Springer.
Hake, R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64–74.
Hora, M., & Ferrare, J. (2013). Instructional systems of practice: A multidimensional analysis of math and science undergraduate course planning and classroom teaching. Journal of the Learning Sciences, 22(2), 212–257.
Hora, M., & Ferrare, J. (2014). Teaching Dimensions Observation Protocol (TDOP) 2.1 users guide. Madison, WI: University of Wisconsin-Madison, Wisconsin Center for Education Research. Retrieved from http://tdop.wceruw.org/Document/TDOP-2.1-Users-Guide.pdf on 12/10/2014.
Hora, M., Ferrare, J., & Oleson, A. (2012). Findings from classroom observations of 58 math and science faculty. In Research report: Culture, cognition, and evaluation of STEM higher education reform NSF # DRL-0814724.
Hora, M., Oleson, A., & Ferrare, J. (2013). Teaching Dimensions Observation Protocol (TDOP) 1.0 users manual. Madison, WI: University of Wisconsin-Madison, Wisconsin Center for Education Research.
Hora, M. T. (Personal Communication, February 2014). Ph.D., Researcher, Wisconsin Center for Educational Research, University of Wisconsin-Madison, Madison, Wisconsin.
Kober, N. (2015). Reaching students: What research says about effective instruction in undergraduate science and engineering (p. 126). Washington, DC: The National Academies Press.
Lawson, A. (1983). Rank analysis of covariance: Alternative approaches. The Statistician, 32(3), 331–337.
Leicht, R., Zappe, S., Litzinger, T., & Messner, J. (2012). Employing the classroom flip to move ‘lecture’ out of the classroom. Journal of Applications and Practices in Engineering Education, 3(1), 19–31.
McCawley, P. (2001). The logic model for program planning and evaluation. Moscow, Idaho: University of Idaho Extension. Retrieved from http://www.cals.uidaho.edu/edcomm/detail.asp?IDnum=798 on December 15, 2014.
Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.
Norusis, M. (2005). SPSS 14.0 statistical procedures companion (p. 183). Upper Saddle River, NJ: Prentice Hall Inc.
Olds, B., & Miller, R. (2008). Using formative assessment for program improvement. In J. Spurlin, S. Rajala, & J. Lavelle (Eds.), Designing better engineering education through assessment (pp. 266–284). Sterling, VA: Stylus Publishing LLC.
Pasley, J., Weiss, I., Shimkus, E., & Smith, P. S. (2004). Looking inside the classroom: Science teaching in the United States. Science Educator, 13(1), 1–12.
Quade, D. (1967). Rank analysis of covariance. Journal of the American Statistical Association, 62(320), 1187–1200.
Salkind, N. (Ed.). (2010). Encyclopedia of research design (Vol. 1). Thousand Oaks, CA: Sage Publications.
Sawada, D., Piburn, M., Judson, E., Turley, J., Falconer, K., Benford, R., et al. (2002). Measuring reform practices in science and mathematics classrooms: The reformed teaching observation protocol. School Science and Mathematics, 102(6), 245–253.
Smith, M., Jones, F., Gilbert, S., & Wieman, C. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE-Life Sciences Education, 12(4), 618–627.
Spurlin, J. (2008). Assessment methods used in undergraduate program assessment. In J. Spurlin, S. Rajala, & J. Lavelle (Eds.), Designing better engineering education through assessment (pp. 59–116). Sterling, VA: Stylus Publishing LLC.
Stodolsky, S. (1990). Classroom observation. In J. Millman & L. Darling-Hammond (Eds.), The new handbook of teacher evaluation: Assessing elementary and secondary school teachers (pp. 175–190). Newbury Park, CA: Sage Publications.
Strayer, J. (2007). The effects of the classroom flip on the learning environment: A comparison of learning activity in a traditional classroom and a flip classroom that used an intelligent tutoring system. Doctoral dissertation. Retrieved from ProQuest Dissertations and Theses. Publication No. 3279789.
Strayer, J. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15(2), 171–193.
Streveler, R., Smith, K., & Pilotte, M. (2012). Aligning course content, assessment, and delivery: Creating a context for outcome-based education. In K. M. Yusof, N. A. Azli, A. M. Kosnin, S. K. S. Yusof, & Y. M. Yusof (Eds.), Outcome-based science, technology, engineering, and mathematics education: Innovative practices (pp. 1–32). Hershey, PA: IGI Global.
Suskie, L. (2008). Understanding the nature and purpose of assessment. In J. Spurlin, S. Rajala, & J. Lavelle (Eds.), Designing better engineering education through assessment (pp. 3–19). Sterling, VA: Stylus Publishing LLC.
Wiggins, G., & McTighe, J. (2005). Understanding by design (pp. 141, 146, 148, 339–340). Upper Saddle River, NJ: Pearson Education, Inc.
Zappe, S., Leicht, R., Messner, J., Litzinger, T., & Lee, H. (2009). ‘Flipping’ the classroom to explore active learning in a large undergraduate course. In Proceedings of the ASEE Annual Conference and Exposition, Austin, TX.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendix 1
Faculty/instructor interview or focus group questions
Interview/focus group questions | |
|---|---|
1 | You had an objective of “particular objective” with the flipped classroom. Did you reach this objective? If not, why do you think this was so? |
2 | With the flipped classroom, what benefits could you provide to the students? |
3 | With the flipped classroom, did students have time to do more active learning or practice during class and/or could you provide more individualised help to students during class? Please expound on |
4 | With the flipped classroom, did you notice that the students experienced fewer software execution problems or less frustration with the software? (for software-based courses) |
5 | With the flipped classroom, were there benefits to you as an instructor? |
6 | With the flipped classroom, what drawbacks existed for you or the students, including any drawbacks the students may have mentioned to you? |
7 | With the flipped classroom, did you notice improvements in problem solving ability, deep learning, quality of the students’ work, or student engagement? |
8 | With the flipped classroom, did you notice any other good outcomes or improvements compared to previous semesters? |
9 | Do you plan to flip additional courses or continue to flip this course? |
10 | What advice would you give to a faculty member who is contemplating flipping? |
Appendix 2
Course flipping evaluation survey
Evaluation survey question | Response options or type | |
|---|---|---|
1 | Do you prefer a “flipped” classroom over a traditional lecture class? | Yes No Not sure yet |
2 | What percentage of the videos did you watch? (approximate as needed, and use 0 or 100 as appropriate) | 0–100% |
3 | When did you primarily view the videos? | Before the class period for which they were assigned After the class period for which they were assigned |
4 | How often did you re-watch the videos or any portions of them? | Never Rarely Sometimes Often Almost always or Always |
5 | Why did you re-watch videos or portions of them? (select all that apply) | The topic was difficult or challenging to grasp The instructor’s explanation or lecture was not clear (please provide specifics) To reinforce my understanding as I was learning new material To review or study course material prior to an exam or homework problem Poor audio or visual quality of the video or other technical difficulty (please provide specifics) Other (please provide specifics) |
6 | Did you experience any technical difficulties with the videos? (e.g. clarity/volume of speech, text size, visual quality, availability, etc.) | Yes (please provide specifics) No |
7 | How did you primarily use the videos? | To learn new material To review or reinforce material after it was demonstrated or presented in class |
8 | With the “flipped” classroom, how would you rate the overall time required of you (both in and out of class), compared to a traditional lecture class? | Less than regular lecture About the same More than regular lecture |
9 | I prefer using class time for problem solving or active learning exercises (with the instructor or TAs present for assistance) rather than listening to a lecture | Strongly disagree Disagree Neutral Agree Strongly agree |
10 | I am NOT able to learn from a video | Strongly disagree Disagree Neutral Agree Strongly agree |
11 | More time needed to be spent at the beginning of class reviewing the video content | Strongly disagree Disagree Neutral Agree Strongly agree |
12 | I understand the reasons or rationale for the “flipped” classroom style in this course | Strongly disagree Disagree Neutral Agree Strongly agree |
13 | Was there any important course content that was missing from the videos? | Yes (please provide specifics) ______ No |
14 | Please describe the length of each of the videos | Too short Just right Too long |
15 | What did you like most about this “flipped” class, and what benefits did you perceive? | Open ended |
16 | What suggestions do you have for improving this “flipped” class, and what drawbacks did you perceive? | Open ended |
Appendix 3
Framework for coding of open-ended responses to benefits
Category | Benefit description/examples |
|---|---|
Video/online learning | Re-watch videos Work at one’s own pace; pause video Flexibility, convenience, own preferences Modularisation of topics |
Enhanced learning or learning process | Better understanding; less confusion Enhanced learning/effectiveness/depth/ability Subject matter retention Multiple sources/resources for understanding Reinforcement and review Multiple attempts |
Alternative use of class time | In-class active learning, problem solving, clickers In-class support and questions In-class group time for projects Student interactivity and peer support |
Specific to course or course’s videos | Videos concise or had a good pace Overall work time less Videos had relevant content (e.g. demo or examples) or were of high quality |
Preparation, engagement and professional behaviours | Engaged during class; paid attention; not bored Enjoyed class Arrived to class prepared Ability to learn on one’s own; independence Drove motivation and accountability |
No benefit or neutral result | No benefits perceived Did not like flipped instruction Videos not used Instructional differences not noticed |
Appendix 4
Framework for coding of open-ended responses to suggestions/drawbacks
Category | Suggestion and drawback description/examples |
|---|---|
Specific to course or course’s videos | Include more examples or problems in the videos Videos needed editing or bug/technical fixes Videos were too long Videos were not sufficiently described Videos were dry or boring Videos did not have an appropriate pace Videos repeated information Video material was too complex |
In-class time | Increase time for active learning or problem solving Increase effectiveness or relevancy of problems; grade them Provide appropriate amount of lecture or content review Have more instructor-types during class to assist Synchronize class activity and video content |
Prepare, Equip & incentivize students to flip | Prepare students for the flipped learning style Incentivize students, including video quizzes Clarify/emphasize expectations, including video watching Provide video “lecture” notes Ensure videos available in advance for students |
No drawbacks or neutral result | No drawbacks or suggestions |
Load, burden, stressors | Insufficient time to complete out-of-class activities Increased work load Increased time burden Concerns over grades or impacts to the grade Accountability quizzes (including surprise) |
Approach differently | Do not flip courses in general; use traditional teaching Do not flip this course in particular Provide students with a choice on flipping Flip only a portion of the class periods |
Video/online learning | Students unable to ask questions during a video Instructor unable to sense student understanding in a video Distractors to viewing videos in a non-classroom setting Less motivation to attend class |
Student learning | Lesser understanding or learning Difficulty learning from a video |
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Clark, R.M., Besterfield-Sacre, M. (2017). Assessing Flipped Classrooms. In: Reidsema, C., Kavanagh, L., Hadgraft, R., Smith, N. (eds) The Flipped Classroom. Springer, Singapore. https://doi.org/10.1007/978-981-10-3413-8_4
Download citation
DOI: https://doi.org/10.1007/978-981-10-3413-8_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-3411-4
Online ISBN: 978-981-10-3413-8
eBook Packages: EducationEducation (R0)