Research in Science Education

, Volume 47, Issue 5, pp 1101–1125 | Cite as

Using Student Video Cases to Assess Pre-service Elementary Teachers’ Engineering Teaching Responsiveness

  • Tejaswini DalviEmail author
  • Kristen Wendell


Our study addresses the need for new approaches to prepare novice elementary teachers to teach both science and engineering, and for new tools to measure how well those approaches are working. This in particular would inform the teacher educators of the extent to which novice teachers are developing expertise in facilitating their students’ engineering design work. One important dimension to measure is novice teachers’ abilities to notice the substance of student thinking and to respond in productive ways. This teacher noticing is particularly important in science and engineering education, where students’ initial, idiosyncratic ideas and practices influence the likelihood that particular instructional strategies will help them learn. This paper describes evidence of validity and reliability for the Video Case Diagnosis (VCD) task, a new instrument for measuring pre-service elementary teachers’ engineering teaching responsiveness. To complete the VCD, participants view a 6-min video episode of children solving an engineering design problem, describe in writing what they notice about the students’ science ideas and engineering practices, and propose how a teacher could productively respond to the students. The rubric for scoring VCD responses allowed two independent scorers to achieve inter-rater reliability. Content analysis of the video episode, systematic review of literature on science and engineering practices, and solicitation of external expert educator responses establish content validity for VCD. Field test results with three different participant groups who have different levels of engineering education experience offer evidence of construct validity.


Video cases Elementary engineering education Responsive teaching 



This research was supported by the National Science Foundation under grant DRL-1253344. Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.


  1. Abell, S. K., & Cennamo, K. S. (2004). Video cases in elementary science teacher preparation. In J. Brophy (Ed.), Using video in teacher education. (Advances in research on teaching, vol. 10) (pp. 103–129). New York: Elsevier Science.Google Scholar
  2. Barlex, D. (2011) Nuffield primary design & technology. In: C Benson and J Lunt (Ed) International handbook of primary technology education. Rotterdam: Sense Publishers.Google Scholar
  3. Bolger, M. S., Kobiela, M., Weinberg, P. J., & Lehrer, R. (2012). Children’s mechanistic reasoning. Cognition and Instruction, 30(2), 170–206.CrossRefGoogle Scholar
  4. Benenson, G. (2001). The unrealized potential of everyday technology as a context for learning. Journal of Research in Science Teaching, 38(7), 730–745.CrossRefGoogle Scholar
  5. Capobianco, B. M., Diefes-Dux, H. A., & Mena, I. B. (2011). Elementary school teachers’ attempts at integrating engineering design: transformation or assimilation? Proceedings from 118th American Society for Engineering Education Annual Conference and Exposition, Vancouver, British Columbia.Google Scholar
  6. Carmines, E.G., & Zeller, R.A. (1979). Reliability and validity assessment. Quantitative applications in the social sciences, 17. Thousand Oaks, CA: Sage Publications, Inc.Google Scholar
  7. Cejka, E., Rogers, C., & Portsmore, M. (2006). Kindergarten robotics: using robotics to motivate math, science, and engineering literacy in elementary school. International Journal of Engineering Education, 22(4), 711–722.Google Scholar
  8. Crismond, D. (2001). Learning and using science ideas when doing investigate-and-redesign tasks: a study of naive, novice, and expert designers doing constrained and scaffolded design work. Journal of Research in Science Teaching, 38(7), 791–820.CrossRefGoogle Scholar
  9. Crismond, D., & Adams, R. (2012). The informed design teaching and learning matrix. Journal of Engineering Education, 101(4), 738–797.CrossRefGoogle Scholar
  10. Cunningham, C. M., & Lachapelle, C. P. (2014). Designing engineering experiences to engage all students. In S. Purzer, J. Strobel, & M. Cardella (Eds.), Engineering in pre-college settings: synthesizing research, policy, and practices. Lafayette, IN: Purdue University Press.Google Scholar
  11. Engle, R. A., & Conant, F. C. (2002). Guiding principles for fostering productive disciplinary engagement: explaining an emergent argument in a community of learners classroom. Cognition and Instruction, 20(4), 399–483.CrossRefGoogle Scholar
  12. Fleer, M. (1999). The science of technology: young children working technologically. International Journal of Technology and Design Education, 9(3), 269–291.CrossRefGoogle Scholar
  13. Fleer, M. (2002). Technology and design education: are we developing professionals or technicians? In C. Yin Cheong, T. Kwok Tung, C. King Wai, & M. Magdalena Mo Ching (Eds.), Subject teaching and teacher education in the new century: research and innovation (pp. 33–46). Hong Kong: Kluwer Academic Publishers and Hong Kong Institute of Education.Google Scholar
  14. Forbes, C. T. (2011). Preservice elementary teachers’ adaptation of science curriculum materials for inquiry-based elementary science. Science Education, 95(5), 1–29.CrossRefGoogle Scholar
  15. Frederiksen, J. R., Sipusic, M., Sherin, M., & Wofle, E. W. (1998). Video portfolio assessment: creating a framework for viewing functions of teaching. Educational Assessment, 11(2), 255–297.Google Scholar
  16. Full Option Science System. (2013). FOSS third edition—framework and NGSS.
  17. Hammer, D. (1997). Discovery learning and discovery teaching. Cognition and Instruction, 15(4), 485–529.CrossRefGoogle Scholar
  18. Hammer, D., Goldberg, F., & Fargason, S. (2012). Responsive teaching and the beginnings of energy in a third grade classroom. Review of Science, Mathematics and ICT Education, 6(1), 51–72.Google Scholar
  19. Hammer, D. & van Zee, E. H. (2006). Seeing the science in children’s thinking: case studies of student inquiry in physical science. (Book and DVD) Portsmouth, NH: HeinemannGoogle Scholar
  20. Hsu, M., Cardella, M., & Purzer, Ş. (2010). Elementary teachers’ perceptions of engineering and familiarity with design, engineering, and technology. Louisville KY: Proceedings from American Society for Engineering Education Conference.Google Scholar
  21. Kendall, A. (2013). Teachers’ attention to student thinking during the engineering design process: a case study of three elementary classroom. San Antonio TX: Proceedings from American Society for Engineering Education Annual Conference and Exposition.Google Scholar
  22. Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, B., Gray, J., Holbrook, J., & Ryan, M. (2003). Problem-based learning meets case-based reasoning in the middle-school science classroom: Putting Learning by DesignTM into practice. Journal of The Learning Sciences, 12(4), 495–547.CrossRefGoogle Scholar
  23. Levin, D. M., Hammer, D., & Coffey, J. E. (2009). Novice teachers’ attention to student thinking. Journal of Teacher Education, 60(2), 142–154.CrossRefGoogle Scholar
  24. Levin, D. M., Hammer, D., Elby, A., & Coffey, J. E. (2012). Becoming a responsive science teacher: focusing on student thinking in secondary science. Arlington, VA: NSTA Press.Google Scholar
  25. McCormick, M., & Hynes, M. M. (2012). Engineering in a fictional world: early findings from integrating engineering and literacy. San Antonio TX: Proceedings from American Society for Engineering Education Annual Conference.Google Scholar
  26. Maskiewicz, A., & Winters, V. (2012). Understanding the co-construction of inquiry practices: a case study of a responsive teaching environment. Journal of Research in Science Teaching, 49(4), 429–464.CrossRefGoogle Scholar
  27. Michaels, S., O’Connor, C., Hall, M., & Resnick, L. (2002). Accountable talk: classroom conversation that works (CD-ROM set). Pittsburgh: University of Pittsburgh.Google Scholar
  28. Museum of Science Boston. (2015). Engineering is elementary: engineering and technology lessons for children.
  29. Nadelson, L., Seifer, A. L., & Hettinger, J. K. (2012). Teaching by design: preparing K-12 teachers to use design across the curriculum. San Antonio, TX: Proceedings from 119th American Society for Engineering Education Annual Conference and Exposition.Google Scholar
  30. National Academy of Engineering and National Research Council. (2014). STEM integration in K-12 education: status, prospects, and an agenda for research. Washington, DC: The National Academies Press. doi: 10.17226/18612.Google Scholar
  31. National Research Council. (2012). A framework for K-12 science education: practices, crosscutting concepts, and core ideas. Committee on Conceptual Framework for the New K-12 Science Education Standards. Board on Science Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.Google Scholar
  32. NGSS Lead States. (2013). Next generation science standards: for states, by states. Washington, DC: National Academies Press.Google Scholar
  33. Norton, A., McCloskey, A., Hudson, Rick A. (2011). Using video-based predictions to assess prospective teachers’ knowledge of students’ mathematical thinking. Journal of Mathematics Teacher Education, 14(4), 305–325.Google Scholar
  34. Olivero, J. L. (1965). The use of video recordings in teacher education. Stanford University. (ERIC Document Reproduction Service No. ED 011 074).Google Scholar
  35. O’Neill, K., & Polman, J. L. (2004). Why educate little scientists? Examining the potential of practice-based scientific literacy. Journal of Research in Science Teaching, 41(3), 234–266.CrossRefGoogle Scholar
  36. Philipp, R. A., Ambrose, R., Lamb, L. L., Sowder, J. T., Schappelle, B. t., Sowder, L., Thanheiser, E., & Chauvot, J. (2007). Effects of early field experiences on the mathematical content knowledge and beliefs of prospective elementary school teachers: an experimental study. Journal for Research in mathematics Education, 38, 438–476.Google Scholar
  37. Portsmore, M. (2013). Exploring first grade students’ drawing and artifact construction during an engineering design problem. In B. M. Brizuela & B. E. Gravel (Eds.), "Show me what you know" exploring representations across STEM disciplines. New York: Teachers’ College Press.Google Scholar
  38. Prince, M., Vigeant, M., & Nottis, K. (2012). Development of the heat and energy concept inventory: preliminary results on the prevalence and persistence of engineering students’ misconceptions. Journal of Engineering Education, 101(3), 412–438.CrossRefGoogle Scholar
  39. Roth, K. J., Garnier, H. E., Chen, C., Lemmens, M., Schwille, K., & Wickler, N. I. Z. (2011). Videobased lesson analysis: effective PD for teacher and student learning. Journal of Research in Science Teaching, 48(2), 117–148.CrossRefGoogle Scholar
  40. Roth, W. M. (1995). Authentic school science: knowing and learning in open-inquiry science laboratories. Dordrecht, Netherlands: Kluwer Academic Publishers.CrossRefGoogle Scholar
  41. Roth, W. M. (2001a). Learning science through technological design. Journal of Research in Science Teaching, 38(7), 768–790.CrossRefGoogle Scholar
  42. Sadler, P., Coyle, H., & Schwartz, M. (2000). Engineering competitions in the middle school classroom: Key elements in developing effective design challenges. Journal of The Learning Sciences, 9(3): 299--327.Google Scholar
  43. Sherin, M. G. (2001). Developing a professional vision of classroom events. In T. Wood, B. S. Nelson, & J. Warfield (Eds.), Beyond classical pedagogy: teaching elementary school mathematics (pp. 75–93). Hillsdale, NJ: Erlbaum.Google Scholar
  44. Sherin, M. G. (2003). New Perspectives on the role of video in teacher education. Advances in Research on Teaching, 10, 1–27.CrossRefGoogle Scholar
  45. Sherin, M. G., & Han, S. (2004). Teacher learning in the context of a video club. Teaching and Teacher Education, 20, 163–183.CrossRefGoogle Scholar
  46. Sherin, M. G. (2007) The development of teachers’ professional vision in video clubs. In: R. Goldman, R. Pea, B. Barron, & S. Derry (eds) Video research in the learning sciences (pp. 383-395). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  47. Sherin, M. G., & van Es, E. (2002). Using video to support teachers’ ability to interpret classroom interactions. In: D. Willis et al. (Eds.), Proceedings from Society for Information Technology & Teacher Education International Conference, Chesapeake, VA: Association for the Advancement of Computing in Education (AACE)Google Scholar
  48. Sherin, M. G., & van Es, E. A. (2005). Using video to support teachers’ ability to interpret classroom interactions. Journal of Technology and Teacher Education, 13, 475–491.Google Scholar
  49. Sherin, M. G., & van Es., E. A. (2009). Effects of video club participation on teachers’ professional vision. Journal of Teacher Education, 60(1), 20–37.CrossRefGoogle Scholar
  50. Star, J. R., & Strickland, S. K. (2008). Learning to observe: using video to improve preservice mathematics teachers’ ability to notice. Journal of Mathematics Teacher Education, 11, 107–125.CrossRefGoogle Scholar
  51. Stockero, S. (2008). Using a video-based curriculum to develop a reflective stance in prospective mathematics teachers. Journal of Mathematics Teacher Education, 11, 373–394.CrossRefGoogle Scholar
  52. Streveler, R. T., Litzinger, T., Miller, R., & Steif, P. (2008). Learning conceptual knowledge in the engineering sciences: overview and future research directions. Journal of Engineering Education, 97(3), 279–294.CrossRefGoogle Scholar
  53. Swenson, J. (2013). Dynamics of 5th grade students engineering service learning projects. Proceedings from American Society for Engineering Education Annual Conference and Exposition, San Antonio, TXGoogle Scholar
  54. Tafur, M., Douglas, K. A., & Diefes-Dux, H. A. (2014). Changes in elementary students’ engineering knowledge over two years of integrated science instruction (research to practice). Proceedings from American Society for Engineering Education Annual Conference and Exposition, Indianapolis, INGoogle Scholar
  55. Tochon, F. V. (1999). Video study groups for education, professional development and change. Madison, WI: Atwood.Google Scholar
  56. van Es, E. A., & Sherin, M. G. (2002). Learning to notice: scaffolding new teachers’ interpretations of classroom interactions. Journal of Technology and Teacher Education, 10, 571–596.Google Scholar
  57. Watkins, J., Spencer, K., & Hammer, D. (2014). Examining young students’ problem scoping in engineering design. Journal of Pre-College Engineering Education, 4(1), 43–53.Google Scholar
  58. Weilan, I., Rogers, M. P., Akerson, V., & Pongsanon, K. (2010). Proposing a video-based measure of preservice teachers’ abilities to predict elementary students’ scientific reasoning. Paper presented at the annual conference of the association for science teacher education.Google Scholar
  59. Welch. M. & Lim, H. S. (1999). From stick figure to design proposal: teaching novice designers to “think on paper”. N C. Benson & W. Till (Eds.). Second International Primary Design and Technology Conference 1999 (136–141). Birmingham, UK: CRIPT at University of Central EnglandGoogle Scholar
  60. Welch, M., Barlex, D., Christie, C., Mueller, A., Munby H., Chin, P. and Taylor, J. (2000) Teaching elementary science and technology in Ontario. In: P. H. Roberts and E. W. L. Norman (Eds.) IDATER2000: Thirteenth International Conference on Design and Technology Educational Research and Curriculum Development: 180–185, Loughborough, UK: Loughborough UniversityGoogle Scholar
  61. Wendell, K. B. (2011). Science through engineering in elementary school: comparing three enactments of an engineering-design-based curriculum on the science of sound. (Doctoral dissertation) Retrieved from Accessed 1 Dec 2015.
  62. Wendell, K. B., & Rogers, C. B. (2013). Engineering design-based science, science content performance, and science attitudes in elementary school. Journal of Engineering Education, 102(4), 513–540.CrossRefGoogle Scholar
  63. Wendell, K. B., & Kolodner, J. (2014). Learning disciplinary concepts and practices through engineering design. In B. Olds and A. Johri (Eds.), Cambridge handbook of engineering education research. Cambridge University Press.Google Scholar
  64. Vygotsky, L. S. (1962). Thought and language. Cambridge MA: MIT Press.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.Center of Science and Mathematics in ContextUniversity of MassachusettsBostonUSA
  2. 2.Department of Mechanical Engineering and the Center for Engineering Education and OutreachTufts UniversityMedfordUSA

Personalised recommendations