Skip to main content

There is an Evidence Crisis in Science Educational Policy

Educational Psychology Review volume 34, pages 1157–1176 (2022)Cite this article

Abstract

There is a considerable gap between many of the findings from educational psychology research and educational practice. This gap is especially notable in the field of science education. In this article, the implications of three categories of research and their findings for science educational policy in the USA and other jurisdictions were reviewed. We indicate that a particular category of research that we call “Program-Based Studies,” has dominated the formulation of educational standards while a large number of critical findings from randomized, controlled studies and correlational studies that overwhelmingly show minimal support for the suggested policy have been marked as irrelevant and excluded. The current blanket-emphasis on program-based studies at the expense of the other types of research is misplaced. Educational standards should represent a balanced view of the available data including findings from controlled and correlational studies. Finally, we indicate how these different forms of research might inform each other and provide coherent and consistent implications for educational procedures.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Abrahams, I., & Millar, R. (2008). Does practical work really work? A study of the effectiveness of practical work as a teaching and learning method in school science. International Journal of Science Education, 30(14).

  • Aditomo, A., & Klieme, E. (2020). Forms of inquiry-based science instruction and their relations with learning outcomes: Evidence from high and low-performing education systems. International Journal of Science Education. https://doi.org/10.1080/09500693.2020.1716093

  • Areepattamannil, S. (2012). Effects of inquiry-based science instruction on science achievement and interest in science: Evidence from Qatar. The Journal of Educational Research, 105(2), 134–146.

    Article  Google Scholar 

  • Ashman, G., Kalyuga, S., & Sweller, J. (2020). Problem-solving or explicit instruction: Which should go first when element interactivity is high? Educational Psychology Review, 32, 229–247. https://doi.org/10.1007/s10648-019-09500-5

    Article  Google Scholar 

  • Australian Curriculum Assessment and Reporting Authority. (2018). Year 10 Australian Curriculum Retrieved from www.australiancurriculum.edu.au/f-10-curriculum/science/aims/

  • Barton, A. C., & Tan, E. (2010). We be burnin’! agency, identity, and science learning. Journal of the Learning Sciences, 19(2), 187–229.

    Article  Google Scholar 

  • British Columbia Ministry of Education. (2015). B.C.'s Curriculum: Science. Retrieved from https://curriculum.gov.bc.ca/curriculum/science

  • Cairns, D., & Areepattamannil, S. (2019). Exploring the relations of inquiry-based teaching to science achievement and dispositions in 54 countries. Research in Science Education, 49(1), 1–23.

    Article  Google Scholar 

  • Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Houghton Mifflin Company.

    Google Scholar 

  • Carlson, R. A., Lundy, D. H., & Schneider, W. (1992). Strategy guidance and memory aiding in learning a problem-solving skill. Human Factors, 34(2), 129–145.

    Article  Google Scholar 

  • Chen, O., Kalyuga, S., & Sweller, J. (2015). The worked example effect, the generation effect, and element interactivity. Journal of Educational Psychology, 107, 689–704.

    Article  Google Scholar 

  • Chen, O., Kalyuga, S., & Sweller, J. (2016). Relations between the worked example and generation effects on immediate and delayed tests. Learning and Instruction, 45, 20–30.

    Article  Google Scholar 

  • Coördinatiegroep curriculum.nu. (2019). Samen bouwen aan het primair en voortgezet onderwijs van morgen Deel 1: Adviezen van de Coördinatiegroep Curriculum.nu. [Building tomorrow's primary and secondary education together Part 1: Advice from the Curriculum.nu Coordination Group]. Den Haag, the Netherlands: Coördinatiegroep curriculum.nu. Retrieved from https://www.curriculum.nu/download/advies-coordinatiegroep/

  • DeCaro, M. S., & Rittle-Johnson, B. (2012). Exploring mathematics problems prepares children to learn from instruction. Journal of Experimental Child Psychology, 113(4), 552–568.

    Article  Google Scholar 

  • Edelson, D. C., & Reiser, B. J. (2006). Making authentic practices accessible to learners: Design challenges and strategies. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 335–354). Cambridge University Press.

    Google Scholar 

  • Forbes, C. T., Neumann, K., & Schiepe-Tiska, A. (2020). Patterns of inquiry-based science instruction and student science achievement in PISA 2015. International Journal of Science Education. https://doi.org/10.1080/09500693.2020.1730017

    Article  Google Scholar 

  • Furtak, E. M. (2006). The problem with answers: An exploration of guided scientific inquiry teaching. Science Education, 90(3), 453–467.

    Article  Google Scholar 

  • Fyfe, E. R., DeCaro, M. S., & Rittle-Johnson, B. (2014). An alternative time for telling: When conceptual instruction prior to problem solving improves mathematical knowledge. British Journal of Educational Psychology, 84(3), 502–519.

    Article  Google Scholar 

  • Gao, S., Wang, J., & Zhong, Z. (2017). Influence of science instruction reform on academic performance of eighth grade students in Chinese inner-Mongolia autonomous region. Compare: A Journal of Comparative and International Education. https://doi.org/10.1080/03057925.2017.1365285

    Article  Google Scholar 

  • García-Carmona, A. (2020). From inquiry-based science education to the approach based on scientific practices. Science & Education. https://doi.org/10.1007/s11191-020-00108-8

    Article  Google Scholar 

  • Geier, R., Blumenfeld, P. C., Marx, R. W., Krajcik, J. S., Fishman, B., Soloway, E., & Clay-Chambers, J. (2008). Standardized test outcomes for students engaged in inquiry-based science curricula in the context of urban reform. Journal of Research in Science Teaching, 45(8), 922–939. Retrieved from - https://doi.org/10.1002/tea.20248

  • Glogger-Frey, I., Fleischer, C., Grüny, L., Kappich, J., & Renkl, A. (2015). Inventing a solution and studying a worked solution prepare differently for learning from direct instruction. Learning and Instruction, 39(72–87).

  • Hedges, L. V., & Schauer, J. (2018). Randomised trials in education in the USA. Educational Research. https://doi.org/10.1080/00131881.2018.1493350

    Article  Google Scholar 

  • Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107.

    Article  Google Scholar 

  • Hodson, D. (1996). Practical work in school science: Exploring some directions for change. International Journal of Science Education, 18(7), 755–760.

    Article  Google Scholar 

  • Hofstein, A., & Lunetta, V. N. (1982). The role of the laboratory in science teaching: Neglected aspects of research. Review of Educational Research, 52(2), 201–217.

    Article  Google Scholar 

  • Hofstein, A., & Lunetta, V. N. (2004). The laboratory in science education: Foundations for the twenty-first century. Science Education, 88(1), 28–54. https://doi.org/10.1002/sce.10106

    Article  Google Scholar 

  • Hsu, C.-Y., Kalyuga, S., & Sweller, J. (2015). When should guidance be presented during physics instruction? Archives of Scientific Psychology, 3(1), 37–53. https://doi.org/10.3886/ICPSR35626.v1

    Article  Google Scholar 

  • Jacobson, M. J., Markauskaite, L., Portolese, A., Kapur, M., Lai, P. K., & Roberts, G. (2017). Designs for learning about climate change as a complex system. Learning and Instruction, 1–14.

  • Jerrim, J., Oliver, M., & Sims, S. (2019). The relationship between inquiry-based teaching and students’ achievement. New evidence from a longitudinal PISA study in England. Learning and Instruction, 61, 35–44.

    Article  Google Scholar 

  • Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38, 23–31.

    Article  Google Scholar 

  • Kapur, M. (2008). Productive failure. Cognition and Instruction, 26, 379–424.

    Article  Google Scholar 

  • Kapur, M. (2010). Productive failure in mathematical problem solving. Instructional Science, 38, 523–550.

    Article  Google Scholar 

  • Kapur, M., & Bielaczyc, K. (2012). Designing for productive failure. Journal of the Learning Sciences, 21(1), 45–83.

    Article  Google Scholar 

  • Kaya, S., & Rice, D. C. (2010). Multilevel effects of student and classroom factors on elementary science achievement in five countries. International Journal of Science Education, 32(10), 1337–1363.

    Article  Google Scholar 

  • Kirschner, P. A. (1992). Epistemology, practical work and Academic skills in science education. Science & Education, 1(3), 273–299.

    Article  Google Scholar 

  • Kirschner, P. A. (2000). The inevitable duality of education: Cooperative higher education. Inaugural Address. Retrieved from Maarsticht, The Netherlands:

  • Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86.

    Article  Google Scholar 

  • Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science instruction effects of direct instruction and discovery learning. Psychological Science, 15(10), 661–667.

    Article  Google Scholar 

  • Kuhn, D. (2007). Is direct instruction an answer to the right question? Educational Psychologist, 42(2), 109–113.

    Article  Google Scholar 

  • Kyun, S., Kalyuga, S., & Sweller, J. (2013). The effect of worked examples when learning to write essays in English literature. The Journal of Experimental Education, 81(3), 385–408.

    Article  Google Scholar 

  • Lavonen, J., & Laaksonen, S. (2009). Context of teaching and learning school science in Finland: Reflections on PISA 2006 results. Journal of Research in Science Teaching, 46(8), 922–944.

    Article  Google Scholar 

  • Lazonder, A. W., & Harmsen, R. (2016). Meta-analysis of inquiry-based learning: Effects of guidance. Review of Educational Research, 86(3), 681–718.

    Article  Google Scholar 

  • Lee, H. S., & Anderson, J. R. (2013). Student learning: What has instruction got to do with it? Annual Review of Psychology, 64, 445–469.

    Article  Google Scholar 

  • Levin, J. R., & O’Donnell, A. M. (1999). What to do about educational research’s credibility gaps? Issues in Education, 5(2), 177–229.

    Article  Google Scholar 

  • Liou, P.-Y. (2020). Students' attitudes toward science and science achievement: An analysis of the differential effects of science instructional practices. Journal of Research in Science Teaching, Online First, 1–25.

  • Loibl, K., Roll, I., & Rummel, N. (2017). Towards a theory of when and how problem solving followed by instruction supports learning. Educational Psychology Review, 29, 693–715.

    Article  Google Scholar 

  • Loibl, K., & Rummel, N. (2014). The impact of guidance during problem-solving prior to instruction on students’ inventions and learning outcomes. Instructional Science, 42, 305–326.

    Article  Google Scholar 

  • Martin, A. J., Ginns, P., Burns, E. C., Kennett, R., & Pearson, J. (2020). Load reduction instruction in science and students’ science engagement and science achievement. Journal of Educational Psychology, Advance online publication. https://doi.org/10.1037/edu0000552

  • Matlen, B. J., & Klahr, D. (2013). Sequential effects of high and low instructional guidance on children’s acquisition of experimentation skills: Is it all in the timing? Instructional Science, 41(3), 621–634.

    Article  Google Scholar 

  • Mayer, R. E. (2003). Learning environments: The case for evidence-based practice and issue-driven research. Educational Psychology Review, 15(4), 359–366.

    Article  Google Scholar 

  • Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? American Psychologist, 59(1), 14–19. https://doi.org/10.1037/0003-066x.59.1.14

    Article  Google Scholar 

  • McConney, A., Oliver, M. C., Woods-McConney, A., Schibeci, R., & Maor, D. (2014). Inquiry, engagement, and literacy in science: A retrospective, cross-national analysis using PISA 2006. Science Education, 98(6), 963–980.

    Article  Google Scholar 

  • Minner, D. D., Levy, A. J., & Century, J. (2010). Inquiry-based science instruction—What is it and does it matter? Results from a research synthesis years 1984 to 2002. Journal of Research in Science Teaching, 47(4), 474–496.

    Article  Google Scholar 

  • Mistler-Jackson, M., & Songer, N. (2000). Student motivation and Internet technology: Are students empowered to learn science? Journal of Research in Science Teaching, 37(5), 459–479.

    Article  Google Scholar 

  • Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science, 32, 99–113.

    Article  Google Scholar 

  • National Academies of Sciences, Engineering, and Medicine. (2021). Call to action for science education: Building opportunity for the future. Washington, DC: The National Academies Press Retrieved from https://doi.org/10.17226/26152

  • National Research Council. (1996). National science education standards. National Academy Press.

    Google Scholar 

  • National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. The National Academies Press.

    Google Scholar 

  • NGSS Lead States. (2013). Next generation science standards: For states, by states. The National Academies Press.

    Google Scholar 

  • OECD. (2016). PISA 2015 results (Volume II). Policies and Practices for Successful Schools. Paris: OECD Publishing.

  • Oliver, M., McConney, A., & Woods-McConney, A. (2019). The efficacy of inquiry-based instruction in science: A comparative analysis of six countries using PISA 2015. Research in Science Education. https://doi.org/10.1007/s11165-019-09901-0

    Article  Google Scholar 

  • Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4.

    Article  Google Scholar 

  • Platform Onderwijs2032. (2016). Ons Onderwijs2032: Eindadvies [Our Education2032: Final advice]. The Hague, The Netherlands: Platform Onderwijs2032 Retrieved from https://curriculum.nu/wp-content/uploads/2018/05/Ons-Onderwijs2032-Eindadvies-januari-2016.pdf

  • Renken, M. D., & Nunez, N. (2010). Evidence for improved conclusion accuracy after reading about rather than conducting a belief-inconsistent simple physics experiment. Applied Cognitive Psychology, 24(6), 792–811.

    Article  Google Scholar 

  • Renkl, A. (2013). Why practice recommendations are important in use-inspired basic research and why too much caution is dysfunctional. Educational Psychology Review, 25(3), 317–324. https://doi.org/10.1007/s10648-013-9236-0

    Article  Google Scholar 

  • Renkl, A. (2015). Different roads lead to Rome: the case of principle-based cognitive skills. Learning: Research and Practice 1(1), 79 - 90.

  • Rittle-Johnson, B. (2006). Promoting transfer: Effects of self-explanation and direct instruction. Child Development, 7(1), 1–15.

    Article  Google Scholar 

  • Robinson, D. H., Levin, J. R., Schraw, G., Patall, E. A., & Hunt, E. B. (2013). On going (way) beyond one’s data: A proposal to restrict recommendations for practice in primary educational research journals. Educational Psychology Review, 25(2), 291–302.

    Article  Google Scholar 

  • Roussel, S., Joulia, D., Tricot, A., & e., & Sweller, J. (2017). Learning subject content through a foreign language should not ignore human cognitive architecture: A cognitive load theory approach. Learning and Instruction, 52, 69–79.

    Article  Google Scholar 

  • Schmidt, H. G., Loyens, S. M. M., Gog, T., & v., & Paas, F. (2007). Problem-based learning is compatible with human cognitive architecture: Commentary on Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 91–97.

    Article  Google Scholar 

  • Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.

    Article  Google Scholar 

  • Schwichow, M., Croker, S., Zimmerman, C., Höffler, T., & Härtig, H. (2016). Teaching the control-of-variables strategy: A meta-analysis. Developmental Review, 39, 37–63.

    Article  Google Scholar 

  • Shaffer, D. W. (2004). Pedagogical praxis: The professions as models for postindustrial education. Teachers College Record, 106(7).

  • Shavelson, R. J., & Towne, L. (2002). Scientific Research in Education. National Academy Press.

    Google Scholar 

  • Slavin, R. E. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15–21.

    Article  Google Scholar 

  • Songer, N. B., Lee, H.-S., & Kam, R. (2002). Technology-rich inquiry science in urban classrooms: What are the barriers to inquiry pedagogy? Journal of Research in Science Teaching, 39(2), 128–150.

    Article  Google Scholar 

  • Songer, N. B., Lee, H.-S., & McDonald, S. (2003). Research towards an expanded understanding of inquiry science beyond one idealized standard. Science Education, 87(4), 490–516.

    Article  Google Scholar 

  • Stull, A. T., & Mayer, R. E. (2007). Learning by doing versus learning by viewing: Three experimental comparisons of learner-generated versus author-provided graphic organizers. Journal of Educational Psychology, 99(4), 808–820.

    Article  Google Scholar 

  • Sweller, J. (2009). What human cognitive architecture tells us about constructivism. In S. Tobias & T. M. Duffy (Eds.), Constructivist instruction: Success or failure? (pp. 127–143). Routledge.

    Google Scholar 

  • Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer.

    Book  Google Scholar 

  • Sweller, J., Kirschner, P. A., & Clark, R. E. (2007). Why minimally guided teaching techniques do not work: A reply to commentaries. Educational Psychologist, 42(2), 115–121.

    Article  Google Scholar 

  • Sweller, J., van Merriënboer, J., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31, 261–292.

    Article  Google Scholar 

  • Taylor, J., Furtak, E., Kowalski, S., Martinez, A., Slavin, R., Stuhlsatz, M., & Wilson, C. (2016). Emergent themes fromrecent research syntheses in science education and their implications for research design, replication, and reporting practices. Journal of Research in Science Teaching, 53(8), 1216–1231.

    Article  Google Scholar 

  • Teig, N., Scherer, R., & Nilsen, T. (2018). More isn’t always better: The curvilinear relationship between inquiry-based teaching and student achievement in science. Learning and Instruction, 56, 20–29.

    Article  Google Scholar 

  • Tobias, S., & Duffy, T. M. (Eds.). (2009). The success or failure of constructivist instruction. Routledge.

    Google Scholar 

  • Tuovinen, J. E., & Sweller, J. (1999). A comparison of cognitive load associated with discovery learning and worked examples. Journal of Educational Psychology, 91(2), 334–341.

    Article  Google Scholar 

  • van Gog, T., Kester, L., & Paas, F. (2011). Effects of worked examples, example-problem, and problem-example pairs on novices’ learning. Contemporary Educational Psychology, 36(3), 212–218.

    Article  Google Scholar 

  • Weaver, J. P., Chastain, R. J., DeCaro, D. A., & DeCaro, M. S. (2018). Reverse the routine: Problem solving before instruction improves conceptual knowledge in undergraduate physics. Contemporary Educational Psychology, 52, 36–47.

    Article  Google Scholar 

  • Wecker, C. (2013). How to support prescriptive statements by empirical research: Some missing parts. Educational Psychology Review, 25(1), 1–18. https://doi.org/10.1007/s10648-012-9208-9

    Article  Google Scholar 

  • Whitehurst, G. J. (2002). Evidence-based education. Paper presented at the Student Achievement and School Accountability Conference.

  • Whitehurst, G. J. (2003). The Institute of Education Sciences: New wine, new bottles. Paper presented at the 2003 American Educational Research Association Annual Meeting Presidential Invited Session.

  • Williams, M., & Linn, M. C. (2002). WISE inquiry in fifth grade biology. Research in Science Education, 32, 415–436.

    Article  Google Scholar 

  • Zhang, L. (2016). Is inquiry-based science teaching worth the effort? Some thoughts worth considering. Science & Education, 25(7), 897–915.

    Article  Google Scholar 

  • Zhang, L. (2018). Withholding answers during hands-on scientific investigations? Comparing effects on developing students’ scientific knowledge, reasoning, and application. International Journal of Science Education, 40(4), 459–469. https://doi.org/10.1080/09500693.2018.1429692

    Article  Google Scholar 

  • Zhang, L. (2019). “Hands-on” plus “inquiry”? Effects of withholding answers coupled with physical manipulations on students’ learning of energy-related science concepts. Learning and Instruction, 60, 199–205. https://doi.org/10.1016/j.learninstruc.2018.01.001

    Article  Google Scholar 

  • Zhang, L., & Li, Z. (2019). How does inquiry-based scientific investigation relate to the development of students’ science knowledge, knowing, applying, and reasoning? An examination of TIMSS data. Canadian Journal of Science, Mathematics and Technology Education, 19(3), 334–345. https://doi.org/10.1007/s42330-019-00055-9

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Elementary & Special Education, 324 Harkins Hall, Providence College, 1 Cunningham Square, Providence, RI, 02918, USA

    Lin Zhang

  2. Open University of the Netherlands, Postbus 2960, 64901 DL, Heerlen, The Netherlands

    Paul A. Kirschner

  3. Thomas More University of Applied Sciences, Sint-Andriesstraat 2, 2000, Antwerp, Belgium

    Paul A. Kirschner

  4. The Mallinson Institute for Science Education, Western Michigan University, Kalamazoo, MI, USA

    William W. Cobern

  5. School of Education, University of New South Wales, Sydney, NSW, 2052, Australia

    John Sweller

Authors
  1. Lin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul A. Kirschner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. William W. Cobern
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John Sweller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lin Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Kirschner, P.A., Cobern, W.W. et al. There is an Evidence Crisis in Science Educational Policy. Educ Psychol Rev 34, 1157–1176 (2022). https://doi.org/10.1007/s10648-021-09646-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10648-021-09646-1

Keywords

  • Methodology
  • Educational practice
  • Educational policy
  • Controlled studies
  • Correlational studies
  • Program-based studies
  • Science education