Educational Technology Research and Development

, Volume 68, Issue 1, pp 137–162 | Cite as

Project-based learning for middle school students monitoring standby power: replication of impact on stem knowledge and dispositions

  • Gerald KnezekEmail author
  • Rhonda Christensen
Research Article


Middle school students participating in energy-monitoring activities guided by their teachers during 2009–2011 gained (p < .05) in content knowledge and became more positive in their dispositions toward STEM (science, technology, engineering, and mathematics). No comparison group data were gathered for this initial study. Activities were replicated with a new group of treatment students during 2013–2014, adding a comparison group not receiving the treatment. Matched pre-post data from 2013 to 2014 confirmed gains in knowledge of environmental science and vampire power (p < .0001, effect size = .86). Aggregate dispositions toward science, mathematics, engineering and technology became more positive for treatment versus comparison group students (p = .023). Gains in STEM dispositions for girls were more positive (effect size = .37) than for boys. Implications of these findings are that hands-on, inquiry-based science activities may help increase the STEM career pipeline in ways that can lead to broader participation in STEM careers in the future.


STEM dispositions STEM content knowledge Energy conservation Middle school students Replication study 



The authors would like to acknowledge the encouragement and guidance received for many years and dedicate this article in the memory of Julio Lopez-Ferrao, the NSF Program Officer for the MSOSW project.


This research was funded in part by the National Science Foundation (NSF) Innovative Technology Experiences for Students and Teachers (ITEST) Grants #0833706 and #1312168.

Compliance with Ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Alexakos, K., & Antoine, W. (2003). The gender gap in science education: strategies to encourage female participation in science. Science Teaching,70(3), 30–33.Google Scholar
  2. Baker, D. (2013). What works: using curriculum and pedagogy to increase girls’ interest and participation in science. Theory into Practice,52(1), 14–20. Scholar
  3. Barton, A., Tan, E., & Rivet, A. (2008). Creating hybrid spaces for engaging school science among urban middle school girls. American Educational Research Journal,45, 68–103.CrossRefGoogle Scholar
  4. Benson, L., & Borrego, M. (2015). The role of replication in engineering education research. Journal of Engineering Education,104(4), 388–392.CrossRefGoogle Scholar
  5. Bialo, E., & Sivin-Kachala, J. (1996). The effectiveness of technology in schools: A summary of recent research. School Library Media Quarterly,25(1), 51–57.Google Scholar
  6. Blumenfeld, P. C., Soloway, E., Marx, R. W., Krajcik, J. S., Guzdial, M., & Palincsar, A. (1991). Motivating project-based learning: Sustaining the doing, supporting the learning. Educational Psychologist,26(3&4), 369–398.CrossRefGoogle Scholar
  7. Bouvier, S. (2011). Increasing student interest in science, technology, engineering, and math (STEM): Massachusetts STEM pipeline fund programs using promising practices. Report prepared for the Massachusetts Department of Higher Education by the University of Massachusetts Donahue Institute. Retrieved from,Technology,Engineering,andMath(STEM).pdf
  8. Bybee, R. W. (2010). Advancing STEM education: A 2020 vision. Technology and Engineering Teacher,70(6), 30–35.Google Scholar
  9. Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin Company.Google Scholar
  10. Cavallo, A. M. L., & Laubach, T. A. (2001). Students’ science perceptions and enrollment decisions in different learning cycle classrooms. Journal of Research on Science Teaching,41(4), 392–414.Google Scholar
  11. Ceci, S., Williams, W., & Barnett, S. (2009). Women’s underrepresentation in science: sociocultural and biological considerations. Psychological Bulletin,135(2), 218–261. Scholar
  12. Choi, N., & Chang, M. (2009). Performance of middle school students: Comparing U.S and Japanese inquiry-based science practices in middle schools. Middle Grades Research Journal,6(1), 15–26.Google Scholar
  13. Christensen, R., & Knezek, G. (2013). Contrasts in student perceptions of STEM content and careers. In R. McBride & M. Searson (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2013 (pp. 2048–2053). Chesapeake, VA: AACE. Retrieved Jan 2, 2014 from
  14. Christensen, R., & Knezek, G. (2015). Active learning approaches to integrating technology into middle school science classrooms: Reconceptualizing a middle school science curriculum based on 21st century skills. In X. Ge, D. Ifenthaler, & J. M. Spector (Eds.), Full steam ahead: Emerging technologies for STEAM (pp. 17–37). New York: Springer Academic.CrossRefGoogle Scholar
  15. Christensen, R., & Knezek, G. (2017). Relationship of middle school student STEM interest to career intent. Journal of Education in Science, Environment and Health,3(1), 1–13.CrossRefGoogle Scholar
  16. Christensen, R., Knezek, G., & Tyler-Wood, T. (2015). Gender differences in high school dispositions toward science, technology, engineering and mathematics careers. Journal of Computers in Mathematics and Science Teaching,34(4), 395–408.Google Scholar
  17. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.Google Scholar
  18. DeVellis, R. F. (1991). Scale development. Newbury Park, NJ: Sage.Google Scholar
  19. Earp, B. D. (2015). How to fix psychology’s replication crisis. The Chronicle of Higher Education,62, 1–9.Google Scholar
  20. George, R. (2006). A cross-domain analysis of change in students’ attitudes toward science and attitudes about utility of science. International Journal of Science Education,28(6), 571–589. Scholar
  21. Gokhale, A. A., Rabe-Hemp, C., Woeste, L., & Machina, K. (2015). Gender differences in attitudes toward science and technology among majors. Journal of Science Education and Technology,24(4), 509–516. Scholar
  22. Hammouri, H. (2004). Attitudinal and motivational variables related to mathematics achievement in Jordan: Findings from the Third International Mathematics and Science Study (TIMSS). Educational Research,46(3), 241–257. Scholar
  23. Hattie, J. (2009). Visible learning. London: Routledge.Google Scholar
  24. Heard, P. F., Divall, S. A., & Johnson, S. D. (2000). Can ‘ears’on’ help hands-on science learning for girls and boys? International Journal of Science Education,22, 1133–1146.CrossRefGoogle Scholar
  25. Heilbronner, N. N. (2011). Stepping onto the STEM pathway. Journal for the Education of the Gifted,34(6), 876–899.CrossRefGoogle Scholar
  26. Hirsch, L., Carpinelli, J., Kimmel, H., Rockland, R., & Bloom, J. (2007). The differential effects of pre-engineering curricula on middle school students’ attitudes to and knowledge of engineering careers. Presented at the 37th ASEE/IEEE Frontiers in Education Conference. Retrieved from
  27. Hodges, C. B. (2015). Replication studies in educational technology. TechTrends,59(4), 3–4.CrossRefGoogle Scholar
  28. Huziak-Clark, T., Sondergeld, T., van Staaden, M., Knagga, C., & Bullerjahn, A. (2015). Assessing the impact of a research-based STEM program on STEM majors’ attitudes and beliefs. Journal of School Science and Mathematics,115(5), 226–236.CrossRefGoogle Scholar
  29. Kim, H. (2016). Inquiry-based science and technology enrichment program for middle school-aged female students. Journal of Science Education and Technology,25, 174–186. Scholar
  30. Knezek, G., & Christensen, R. (1998). Internal consistency reliability for the Teachers Attitudes toward Information Technology Survey. In S. McNeil, J. Price, S. Boger-Mehall, B. Robin, & J. Willis (Eds.), Proceedings of SITE 1998Society for Information Technology & Teacher Education International Conference (pp. 843–844). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE).Google Scholar
  31. Knezek, G., Christensen, R., & Tyler-Wood, T. (2011). Contrasting perceptions of STEM content and careers. Contemporary Issues in Technology and Teacher Education,11(1), 92–117.Google Scholar
  32. Knezek, G., Christensen, R., Tyler-Wood, T., & Periathiruvadi, S. (2013). Impact of environmental power monitoring activities on middle school student perceptions of STEM. Science Education International,24(1), 98–123.Google Scholar
  33. Lenhard, W. & Lenhard, A. (2016). Calculation of effect sizes. Retrieved from: Dettelbach: Psychometrica.
  34. Liu, F. (2008). Impact of online discussion on elementary teacher candidates’ anxiety towards teaching mathematics. Education,128(4), 614–629.Google Scholar
  35. Makel, M.C., & Plucker, J.A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Research, 20(10), 1–13. Retrieved from
  36. Maltese, A.V., & Tai, R.H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among U.S. students. Science Education, 95(5), 877-907.Google Scholar
  37. McCoy, J. (2006). Improving middle school students’ attitudes towards science. Journal of Teacher Initiated Research, 3(10), 129–134. Retrieved from
  38. Meier, A. (2009). National Geographic Vampire Power Test. Retrieved from
  39. Miettinen, R. (2000). The concept of experiential learning and John Dewey’s theory of reflective thought and action. International Journal of Lifelong Education,19(1), 54–72.CrossRefGoogle Scholar
  40. Milman, N., Christensen, R., Spector, J.M., Branch, R., Schmidt-Crawford, D., Hodges, C., et al. (2016). A revised replication study typology and a call for participation: Replication studies involving technology and teacher education. In Proceedings of Society for Information Technology & Teacher Education International Conference 2016 (pp. 1054–1058). Chesapeake, VA.Google Scholar
  41. Misiti, F., Shrigley, R., & Hanson, L. (1991). Science attitude scale for middle school students. Science Education,75(5), 525–540. Scholar
  42. National Center for Education Statistics. (2012a). The Nation’s Report Card: Science, 2009 and 2011. Retrieved from
  43. National Center for Education Statistics. (2012b). The Nation’s Report Card: Science in Action: Hands-On and Interactive Computer Tasks From the 2009 Science Assessment (NCES 2012-468). Institute of Education Sciences: Department of Education, Washington, D.C.Google Scholar
  44. National Science Foundation. (2014). Women, minorities, and persons with disabilities in science and engineering: Data tables. Retrieved from
  45. Neuhauser, A. (2014). Space Cadets: STEM Program Gives Students Control of Satellites. U.S. News & World Report. Retrieved from
  46. Nolte, P. (2014). Middle Schoolers Out to Save the World (MSOSW) External Evaluation Report. Unpublished document. Institute for the Integration of Technology into Teaching & Learning: Denton, TX.Google Scholar
  47. Osborne, J., Simon, S., & Collins, S. (2003). Attitudes toward science: A review of the literature and its implications. International Journal of Science Education,25(9), 1049–1079.CrossRefGoogle Scholar
  48. Palmer, D. G. (2009). Student interest generated during an inquiry skills lesson. Journal of Research on Science Teaching,46(2), 147–165.CrossRefGoogle Scholar
  49. Peterman, K., Kermish-Allen, R., Knezek, G., & Christensen, R. (2016). Measuring student career interest within the context of technology-enhanced STEM projects: A cross-project comparison study based on the career interest questionnaire. Journal of Science Education and Technology,25(4), 833–845. Scholar
  50. Pine, J., Aschbacher, P., Roth, E., Joes, M., McPhee, C., Martin, C., et al. (2006). Fifth graders’ science inquiry abilities: A comparative study of students in hands-on and textbook curricula. Journal of Research on Science Teaching,43(5), 467–484.CrossRefGoogle Scholar
  51. Spector, J. M., Johnson, T. E., & Young, P. A. (2015). An editorial on replication studies and scaling up efforts. Educational Technology Research and Development,63(1), 1–4. Scholar
  52. Super, D. E. (1969). Vocational development theory. The Counseling Psychologist,1(1), 2–30.CrossRefGoogle Scholar
  53. Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. (2006). Planning early for careers in science. Science,312, 1143–1144.CrossRefGoogle Scholar
  54. Tyler-Wood, T., Knezek, G., & Christensen, R. (2010). Instruments for assessing interest in STEM content and careers. Journal of Technology and Teacher Education,18(2), 341–363.Google Scholar
  55. Tyler-Wood, T. L., Ellison, A., Lim, O., & Periathiruvadi, S. (2011). Bringing up girls in science (BUGS): The effectiveness of an afterschool environmental science program for increasing female student’s interest in science careers. Journal of Science Education Technology.,21, 46–55. Scholar
  56. U.S. Department of Energy. (2011). When to turn off personal computers. Retrieved from
  57. Zaichkowsky, J. L. (1985). Measuring the involvement construct. Journal of Consumer Research,12(3), 341–352.CrossRefGoogle Scholar

Copyright information

© Association for Educational Communications and Technology 2019

Authors and Affiliations

  1. 1.Department of Learning TechnologiesUniversity of North TexasDentonUSA
  2. 2.Institute for the Integration of Technology into Teaching and LearningUniversity of North TexasDentonUSA

Personalised recommendations