Our study addresses the need for new approaches to prepare novice elementary teachers to teach both science and engineering, and for new tools to measure how well those approaches are working. This in particular would inform the teacher educators of the extent to which novice teachers are developing expertise in facilitating their students’ engineering design work. One important dimension to measure is novice teachers’ abilities to notice the substance of student thinking and to respond in productive ways. This teacher noticing is particularly important in science and engineering education, where students’ initial, idiosyncratic ideas and practices influence the likelihood that particular instructional strategies will help them learn. This paper describes evidence of validity and reliability for the Video Case Diagnosis (VCD) task, a new instrument for measuring pre-service elementary teachers’ engineering teaching responsiveness. To complete the VCD, participants view a 6-min video episode of children solving an engineering design problem, describe in writing what they notice about the students’ science ideas and engineering practices, and propose how a teacher could productively respond to the students. The rubric for scoring VCD responses allowed two independent scorers to achieve inter-rater reliability. Content analysis of the video episode, systematic review of literature on science and engineering practices, and solicitation of external expert educator responses establish content validity for VCD. Field test results with three different participant groups who have different levels of engineering education experience offer evidence of construct validity.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Similar content being viewed by others
Abell, S. K., & Cennamo, K. S. (2004). Video cases in elementary science teacher preparation. In J. Brophy (Ed.), Using video in teacher education. (Advances in research on teaching, vol. 10) (pp. 103–129). New York: Elsevier Science.
Barlex, D. (2011) Nuffield primary design & technology. In: C Benson and J Lunt (Ed) International handbook of primary technology education. Rotterdam: Sense Publishers.
Bolger, M. S., Kobiela, M., Weinberg, P. J., & Lehrer, R. (2012). Children’s mechanistic reasoning. Cognition and Instruction, 30(2), 170–206.
Benenson, G. (2001). The unrealized potential of everyday technology as a context for learning. Journal of Research in Science Teaching, 38(7), 730–745.
Capobianco, B. M., Diefes-Dux, H. A., & Mena, I. B. (2011). Elementary school teachers’ attempts at integrating engineering design: transformation or assimilation? Proceedings from 118th American Society for Engineering Education Annual Conference and Exposition, Vancouver, British Columbia.
Carmines, E.G., & Zeller, R.A. (1979). Reliability and validity assessment. Quantitative applications in the social sciences, 17. Thousand Oaks, CA: Sage Publications, Inc.
Cejka, E., Rogers, C., & Portsmore, M. (2006). Kindergarten robotics: using robotics to motivate math, science, and engineering literacy in elementary school. International Journal of Engineering Education, 22(4), 711–722.
Crismond, D. (2001). Learning and using science ideas when doing investigate-and-redesign tasks: a study of naive, novice, and expert designers doing constrained and scaffolded design work. Journal of Research in Science Teaching, 38(7), 791–820.
Crismond, D., & Adams, R. (2012). The informed design teaching and learning matrix. Journal of Engineering Education, 101(4), 738–797.
Cunningham, C. M., & Lachapelle, C. P. (2014). Designing engineering experiences to engage all students. In S. Purzer, J. Strobel, & M. Cardella (Eds.), Engineering in pre-college settings: synthesizing research, policy, and practices. Lafayette, IN: Purdue University Press.
Engle, R. A., & Conant, F. C. (2002). Guiding principles for fostering productive disciplinary engagement: explaining an emergent argument in a community of learners classroom. Cognition and Instruction, 20(4), 399–483.
Fleer, M. (1999). The science of technology: young children working technologically. International Journal of Technology and Design Education, 9(3), 269–291.
Fleer, M. (2002). Technology and design education: are we developing professionals or technicians? In C. Yin Cheong, T. Kwok Tung, C. King Wai, & M. Magdalena Mo Ching (Eds.), Subject teaching and teacher education in the new century: research and innovation (pp. 33–46). Hong Kong: Kluwer Academic Publishers and Hong Kong Institute of Education.
Forbes, C. T. (2011). Preservice elementary teachers’ adaptation of science curriculum materials for inquiry-based elementary science. Science Education, 95(5), 1–29.
Frederiksen, J. R., Sipusic, M., Sherin, M., & Wofle, E. W. (1998). Video portfolio assessment: creating a framework for viewing functions of teaching. Educational Assessment, 11(2), 255–297.
Full Option Science System. (2013). FOSS third edition—framework and NGSS. https://www.fossweb.com/delegate/ssi-wdf-ucm-webContent?dDocName=D2691958
Hammer, D. (1997). Discovery learning and discovery teaching. Cognition and Instruction, 15(4), 485–529.
Hammer, D., Goldberg, F., & Fargason, S. (2012). Responsive teaching and the beginnings of energy in a third grade classroom. Review of Science, Mathematics and ICT Education, 6(1), 51–72.
Hammer, D. & van Zee, E. H. (2006). Seeing the science in children’s thinking: case studies of student inquiry in physical science. (Book and DVD) Portsmouth, NH: Heinemann
Hsu, M., Cardella, M., & Purzer, Ş. (2010). Elementary teachers’ perceptions of engineering and familiarity with design, engineering, and technology. Louisville KY: Proceedings from American Society for Engineering Education Conference.
Kendall, A. (2013). Teachers’ attention to student thinking during the engineering design process: a case study of three elementary classroom. San Antonio TX: Proceedings from American Society for Engineering Education Annual Conference and Exposition.
Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, B., Gray, J., Holbrook, J., & Ryan, M. (2003). Problem-based learning meets case-based reasoning in the middle-school science classroom: Putting Learning by DesignTM into practice. Journal of The Learning Sciences, 12(4), 495–547.
Levin, D. M., Hammer, D., & Coffey, J. E. (2009). Novice teachers’ attention to student thinking. Journal of Teacher Education, 60(2), 142–154.
Levin, D. M., Hammer, D., Elby, A., & Coffey, J. E. (2012). Becoming a responsive science teacher: focusing on student thinking in secondary science. Arlington, VA: NSTA Press.
McCormick, M., & Hynes, M. M. (2012). Engineering in a fictional world: early findings from integrating engineering and literacy. San Antonio TX: Proceedings from American Society for Engineering Education Annual Conference.
Maskiewicz, A., & Winters, V. (2012). Understanding the co-construction of inquiry practices: a case study of a responsive teaching environment. Journal of Research in Science Teaching, 49(4), 429–464.
Michaels, S., O’Connor, C., Hall, M., & Resnick, L. (2002). Accountable talk: classroom conversation that works (CD-ROM set). Pittsburgh: University of Pittsburgh.
Museum of Science Boston. (2015). Engineering is elementary: engineering and technology lessons for children. www.eie.org
Nadelson, L., Seifer, A. L., & Hettinger, J. K. (2012). Teaching by design: preparing K-12 teachers to use design across the curriculum. San Antonio, TX: Proceedings from 119th American Society for Engineering Education Annual Conference and Exposition.
National Academy of Engineering and National Research Council. (2014). STEM integration in K-12 education: status, prospects, and an agenda for research. Washington, DC: The National Academies Press. doi:10.17226/18612.
National Curriculum in England: design and technology programmes of study (September, 2013) https://www.gov.uk/government/publications/national-curriculum-in-england-design-and-technology-programmes-of-study/national-curriculum-in-england-design-and-technology-programmes-of-study
National Research Council. (2012). A framework for K-12 science education: practices, crosscutting concepts, and core ideas. Committee on Conceptual Framework for the New K-12 Science Education Standards. Board on Science Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
NGSS Lead States. (2013). Next generation science standards: for states, by states. Washington, DC: National Academies Press.
Norton, A., McCloskey, A., Hudson, Rick A. (2011). Using video-based predictions to assess prospective teachers’ knowledge of students’ mathematical thinking. Journal of Mathematics Teacher Education, 14(4), 305–325.
Olivero, J. L. (1965). The use of video recordings in teacher education. Stanford University. (ERIC Document Reproduction Service No. ED 011 074).
O’Neill, K., & Polman, J. L. (2004). Why educate little scientists? Examining the potential of practice-based scientific literacy. Journal of Research in Science Teaching, 41(3), 234–266.
Philipp, R. A., Ambrose, R., Lamb, L. L., Sowder, J. T., Schappelle, B. t., Sowder, L., Thanheiser, E., & Chauvot, J. (2007). Effects of early field experiences on the mathematical content knowledge and beliefs of prospective elementary school teachers: an experimental study. Journal for Research in mathematics Education, 38, 438–476.
Portsmore, M. (2013). Exploring first grade students’ drawing and artifact construction during an engineering design problem. In B. M. Brizuela & B. E. Gravel (Eds.), "Show me what you know" exploring representations across STEM disciplines. New York: Teachers’ College Press.
Prince, M., Vigeant, M., & Nottis, K. (2012). Development of the heat and energy concept inventory: preliminary results on the prevalence and persistence of engineering students’ misconceptions. Journal of Engineering Education, 101(3), 412–438.
Roth, K. J., Garnier, H. E., Chen, C., Lemmens, M., Schwille, K., & Wickler, N. I. Z. (2011). Videobased lesson analysis: effective PD for teacher and student learning. Journal of Research in Science Teaching, 48(2), 117–148.
Roth, W. M. (1995). Authentic school science: knowing and learning in open-inquiry science laboratories. Dordrecht, Netherlands: Kluwer Academic Publishers.
Roth, W. M. (2001a). Learning science through technological design. Journal of Research in Science Teaching, 38(7), 768–790.
Sadler, P., Coyle, H., & Schwartz, M. (2000). Engineering competitions in the middle school classroom: Key elements in developing effective design challenges. Journal of The Learning Sciences, 9(3): 299--327.
Sherin, M. G. (2001). Developing a professional vision of classroom events. In T. Wood, B. S. Nelson, & J. Warfield (Eds.), Beyond classical pedagogy: teaching elementary school mathematics (pp. 75–93). Hillsdale, NJ: Erlbaum.
Sherin, M. G. (2003). New Perspectives on the role of video in teacher education. Advances in Research on Teaching, 10, 1–27.
Sherin, M. G., & Han, S. (2004). Teacher learning in the context of a video club. Teaching and Teacher Education, 20, 163–183.
Sherin, M. G. (2007) The development of teachers’ professional vision in video clubs. In: R. Goldman, R. Pea, B. Barron, & S. Derry (eds) Video research in the learning sciences (pp. 383-395). Hillsdale, NJ: Lawrence Erlbaum.
Sherin, M. G., & van Es, E. (2002). Using video to support teachers’ ability to interpret classroom interactions. In: D. Willis et al. (Eds.), Proceedings from Society for Information Technology & Teacher Education International Conference, Chesapeake, VA: Association for the Advancement of Computing in Education (AACE)
Sherin, M. G., & van Es, E. A. (2005). Using video to support teachers’ ability to interpret classroom interactions. Journal of Technology and Teacher Education, 13, 475–491.
Sherin, M. G., & van Es., E. A. (2009). Effects of video club participation on teachers’ professional vision. Journal of Teacher Education, 60(1), 20–37.
Star, J. R., & Strickland, S. K. (2008). Learning to observe: using video to improve preservice mathematics teachers’ ability to notice. Journal of Mathematics Teacher Education, 11, 107–125.
Stockero, S. (2008). Using a video-based curriculum to develop a reflective stance in prospective mathematics teachers. Journal of Mathematics Teacher Education, 11, 373–394.
Streveler, R. T., Litzinger, T., Miller, R., & Steif, P. (2008). Learning conceptual knowledge in the engineering sciences: overview and future research directions. Journal of Engineering Education, 97(3), 279–294.
Swenson, J. (2013). Dynamics of 5th grade students engineering service learning projects. Proceedings from American Society for Engineering Education Annual Conference and Exposition, San Antonio, TX
Tafur, M., Douglas, K. A., & Diefes-Dux, H. A. (2014). Changes in elementary students’ engineering knowledge over two years of integrated science instruction (research to practice). Proceedings from American Society for Engineering Education Annual Conference and Exposition, Indianapolis, IN
Tochon, F. V. (1999). Video study groups for education, professional development and change. Madison, WI: Atwood.
van Es, E. A., & Sherin, M. G. (2002). Learning to notice: scaffolding new teachers’ interpretations of classroom interactions. Journal of Technology and Teacher Education, 10, 571–596.
Watkins, J., Spencer, K., & Hammer, D. (2014). Examining young students’ problem scoping in engineering design. Journal of Pre-College Engineering Education, 4(1), 43–53.
Weilan, I., Rogers, M. P., Akerson, V., & Pongsanon, K. (2010). Proposing a video-based measure of preservice teachers’ abilities to predict elementary students’ scientific reasoning. Paper presented at the annual conference of the association for science teacher education.
Welch. M. & Lim, H. S. (1999). From stick figure to design proposal: teaching novice designers to “think on paper”. N C. Benson & W. Till (Eds.). Second International Primary Design and Technology Conference 1999 (136–141). Birmingham, UK: CRIPT at University of Central England
Welch, M., Barlex, D., Christie, C., Mueller, A., Munby H., Chin, P. and Taylor, J. (2000) Teaching elementary science and technology in Ontario. In: P. H. Roberts and E. W. L. Norman (Eds.) IDATER2000: Thirteenth International Conference on Design and Technology Educational Research and Curriculum Development: 180–185, Loughborough, UK: Loughborough University
Wendell, K. B. (2011). Science through engineering in elementary school: comparing three enactments of an engineering-design-based curriculum on the science of sound. (Doctoral dissertation) Retrieved from http://gradworks.umi.com/34/45/3445103.html. Accessed 1 Dec 2015.
Wendell, K. B., & Rogers, C. B. (2013). Engineering design-based science, science content performance, and science attitudes in elementary school. Journal of Engineering Education, 102(4), 513–540.
Wendell, K. B., & Kolodner, J. (2014). Learning disciplinary concepts and practices through engineering design. In B. Olds and A. Johri (Eds.), Cambridge handbook of engineering education research. Cambridge University Press.
Vygotsky, L. S. (1962). Thought and language. Cambridge MA: MIT Press.
This research was supported by the National Science Foundation under grant DRL-1253344. Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Video Case Diagnosis No. 1: the “Peach Lifter” Case
We will view a 6-min video clip (in two parts) of two fourth-grade students attempting to design a “peach lifter” for the characters in the book James and the Giant Peach by Roald Dahl.
□ Before viewing the video, read over the transcript.
□ Feel free to take notes on the transcript before or while viewing the video.
□ After viewing the video, please respond to the following four questions.
1. What science ideas, or ideas or thoughts about science phenomena, did the students express?
2. What practices, processes, or skills of science did the students exhibit, and what evidence do you have that they were exhibited?
Evidence they were exhibited
3. What practices, processes, or skills of engineering did the students exhibit, and what evidence do you have that they were exhibited?
Evidence they were exhibited
4. What are three possible ways the teacher might respond to help these students further develop their ideas and practices? (In other words, what are three possible next steps for the teacher?)
About this article
Cite this article
Dalvi, T., Wendell, K. Using Student Video Cases to Assess Pre-service Elementary Teachers’ Engineering Teaching Responsiveness. Res Sci Educ 47, 1101–1125 (2017). https://doi.org/10.1007/s11165-016-9547-5