This paper describes a randomized and controlled efficacy study conducted in high school biology classrooms across the USA. In this study, teachers implemented the use of Data Nuggets, activities designed to bring real research and data into the classroom. These materials can be embedded within the existing instructional modality of any given curriculum, thus infusing these curricula with science stories and associated datasets. Our design had teachers incorporate Data Nuggets into one of their class sections, while teaching a second class section in a business-as-usual manner. Although students in both conditions improved similarly in quantitative reasoning over the course of the study semester, we saw several key differences for students taught using the intervention as compared to those taught using only standard instruction. Students in classrooms that utilized Data Nuggets spent more time engaged in the practices of science and had greater improvement in their ability to construct scientific explanations. In addition, students using the intervention activities showed increases in both their self-efficacy in data-related tasks and their interest in STEM careers. Finally, the effects of teacher implementation on student outcomes when using Data Nuggets were assessed.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Ahlfeldt, S., Mehta, S., & Sellnow, T. (2005). Measurement and analysis of student engagement in university classes where varying levels of PBL methods of instruction are in use. Higher Education Research & Development, 24(1), 5–20. https://doi.org/10.1080/0729436052000318541
Aikens, M. L., & Dolan, E. L. (2014). Teaching quantitative biology: Goals, assessments, and resources. Molecular Biology of the Cell, 25(22), 3478–3481. https://doi.org/10.1091/mbc.e14-06-1045
American Association for the Advancement of Science [AAAS]. (1993). Benchmarks for science literacy. Oxford University Press.
Bandura, A. (1997). Self-efficacy: The exercise of control. Freeman.
Bandura, A., & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88(1), 87–99. https://doi.org/10.1037/0021-9010.88.1.87
Bargagliotti, A., Franklin, C., Arnold, P., Gould, R., Johnson, S., Perez, L., & Spangler, D. (2020). Guidelines for Assessment and Instruction in Statistics Education (GAISE) Pre-K–12 report II. Retrieved January 15, 2021 from https://www.amstat.org/asa/education/Guidelines-for-Assessment-and-Instruction-in-Statistics-Education-Reports.aspx
Becker, M. L., & Nilsson, M. R. (2021). College chemistry textbooks fail on gender representation. Journal of Chemical Education, 98(4), 1146–1151. https://doi.org/10.1021/acs.jchemed.0c01037
Bond, T., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human Sciences (2nd ed.). Routledge.
Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multiple-choice tests. Science Education, 90(2), 253–269. https://doi.org/10.1002/sce.20106
Bourdeau, V. D., & Arnold, M. E. (2009). The science process skills inventory. 4-H Youth Development Education, Oregon State University.
Britner, S. L., & Pajares, F. (2006). Sources of science self-efficacy beliefs of middle school students. Journal of Research in Science Teaching, 43(5), 485–499. https://doi.org/10.1002/tea.20131
Britton, B. K. (1983). What makes stories interesting. Behavioral and Brain Sciences, 6(4), 596–597.
Capraro, M. M., Caparo, R. M., & Jones, M. (2014). Numeracy and algebra: A path to full participation in community and society? Reading Psychology, 35(5), 422–436. https://doi.org/10.1080/02702711.2012.739263
Center for the Advancement of Informal Science Education. (2018). Broadening participation task force: February 2018 update. Retrieved January 15, 2021 from http://www.informalscience.org/news-views/broadening-participation-task-force-february-2018-update
Chemers, M. M., Zurbriggen, E. L., Syed, M., Goza, B. K., & Bearman, S. (2011). The role of efficacy and identity in science career commitment among underrepresented minority students. Journal of Social Issues, 67(3), 469–491. https://doi.org/10.1111/j.1540-4560.2011.01710.x
Collins, S. N. (2021). The importance of storytelling in chemical education. Nature Chemistry, 13(1), 1–2. https://doi.org/10.1038/s41557-020-00617-7
Crawford, B. A. (2000). Embracing the essence of inquiry: New roles for science teachers. Journal of Research in Science Teaching, 37(9), 916–937. https://doi.org/10.1002/1098-2736(200011)37:9%3c916::AID-TEA4%3e3.0.CO;2-2
Damschen, E. I., Rosenfeld, K. M., Wyer, M., Murphy-Medely, D., Wentworth, T. R., & Haddad, N. M. (2005). Visibility matters: Increasing knowledge of women’s contributions to ecology. Frontiers in Ecology and the Environment, 3(4), 212–219. https://doi.org/10.1890/1540-9295(2005)003[0212:VMIKOW]2.0.CO;2
Driver, R., Asoko, H., Leach, J., Scott, P., & Mortimer, E. (1994). Constructing scientific knowledge in the classroom. Educational Researcher, 23(7), 5–12. https://doi.org/10.3102/0013189X023007005
Estrada, M., Woodcock, A., Hernandez, P. R., & Schultz, P. W. (2011). Toward a model of social influence that explains minority student integration into the scientific community. Journal of Educational Psychology, 103(1), 206–222. https://doi.org/10.1037/a0020743
Franz-Odendaal, T. A., Blotnicky, K., French, F., & Joy, P. (2016). Experiences and perceptions of STEM subjects, careers, and engagement in STEM activities among middle school students in the Maritime Provinces. Canadian Journal of Science, Mathematics and Technology Education, 16, 153–168. https://doi.org/10.1080/14926156.2016.1166291
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111
Gentner, D. R. (1976). The structure and recall of narrative prose. Journal of Verbal Learning and Verbal Behavior, 15, 411–418.
Giamellaro, M., O’Connell, K., & Knapp, M. (2020). Teachers as participant-narrators in authentic data stories. International Journal of Science Education, 42(3), 406–425. https://doi.org/10.1080/09500693.2020.1714093
Gibson, D. E. (2004). Role models in career development: New directions for theory and research. Journal of Vocational Behavior, 65(1), 134–156. https://doi.org/10.1016/S0001-8791(03)00051-4
Gladstone, J. R., & Cimpian, A. (2021). Which role models are effective for which students? A systematic review and four recommendations for maximizing the effectiveness of role models in STEM. International Journal of STEM Education, 8(1), 59. https://doi.org/10.1186/s40594-021-00315-x
Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science motivation questionnaire II: Validation with science majors and nonscience majors. Journal of Research in Science Teaching, 48(10), 1159–1176. https://doi.org/10.1002/tea.20442
Gould, R., Sunbury, S., & Dussault, M. (2014). In praise of messy data: Lessons from the search for alien worlds. The Science Teacher, 81(8), 31–36.
Graesser, A. C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101(3), 371–395. https://doi.org/10.1037/0033-295X.101.3.371
Hammett, A., & Dorsey, C. (2020). Messy data, real science: Exploring harmful algal blooms with real-world data. The Science Teacher, 87(8), 40–49.
Hiebert, J., & Stigler, J. W. (2000). A proposal for improving classroom teaching: Lessons from the TIMSS video study. The Elementary School Journal, 101(1), 3–20. https://doi.org/10.1086/499656
Hoffmann, R. (2014). The tensions of scientific storytelling. American Scientist, 102(4), 250–253. https://doi.org/10.1511/2014.109.250
Holmes, N. G., Wieman, C. E., & Bonn, D. A. (2015). Teaching critical thinking. Proceedings of the National Academy of Sciences, 112(36), 11199–11204. https://doi.org/10.1073/pnas.1505329112
Homish, G. G., Edwards, E. P., Eiden, R. D., & Leonard, K. E. (2010). Analyzing family data: A GEE approach for substance use researchers. Addictive Behaviors, 35(6), 558–563. https://doi.org/10.1016/j.addbeh.2010.01.002
Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. Science, 326(5958), 1410–1412. https://doi.org/10.1126/science.1177067
Jin, H., Yan, D., Mehl, C. E., Llort, K., & Cui, W. (2020). An empirically grounded framework that evaluates argument quality in scientific and social contexts. International Journal of Science and Mathematics Education, 19(4), 681–700. https://doi.org/10.1007/s10763-020-10075-9
Karaali, G., Hernandez, E. H. V., & Taylor, J. A. (2016). What's in a name? A critical review of definitions of quantitative literacy, numeracy, and quantitative reasoning. Numeracy: Advancing Education in Quantitative Literacy, 9(1), Article 2. https://doi.org/10.5038/1936-46188.8.131.52
Kjelvik, M. K., Schultheis, E. H., & Gardner, S. (2019). Getting Messy with Authentic Data: Exploring the Potential of Using Data from Scientific Research to Support Student Data Literacy. CBE—Life Sciences Education, 18(2), es2. https://doi.org/10.1187/cbe.18-02-0023
Lawrenz, F., Huffman, D., & Gravely, A. (2007). Impact of the collaboratives for excellence in teacher preparation program. Journal of Research in Science Teaching, 44(9), 1348–1369. https://doi.org/10.1002/tea.20207
Lent, R. W., Sheu, H. B., Miller, M. J., Cusick, M. E., Penn, L. T., & Truong, N. N. (2018). Predictors of science, technology, engineering, and mathematics choice options: A meta-analytic path analysis of the social-cognitive choice model by gender and race/ethnicity. Journal of Counseling Psychology, 65(1), 17–35. https://doi.org/10.1037/cou0000243
Linacre, J. M. (2021). Winsteps® Rasch measurement computer program (version 5.1.1). Winsteps.com.
Linnenbrink-Garcia, L., Durik, A. M., Conley, A. M., Barron, K. E., Tauer, J. M., Karabenick, S. A., & Harackiewicz, J. M. (2010). Measuring situational interest in academic domains. Educational and Psychological Measurement, 70(4), 647–671. https://doi.org/10.1177/0013164409355699
Marx, R. W., Blumenfeld, P. C., Krajcik, J. S., & Soloway, E. (1997). Enacting project-based science. The Elementary School Journal, 97(4), 341–358. https://doi.org/10.1086/461781
Mayes, R., Forrester, J., Schuttlefield Christus, J., Peterson, F., & Walker, R. (2014). Quantitative reasoning learning progression: The matrix. Numeracy: Advancing Education in Quantitative Literacy, 7(2), Article 5. https://doi.org/10.5038/1936-46184.108.40.206
McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://doi.org/10.11613/BM.2012.031
McNeill, K. L., & Krajcik, J. S. (2008a). Scientific explanations: Characterizing and evaluating the effects of teachers’ instructional practices on student learning. Journal of Research in Science Teaching, 45(1), 53–78. https://doi.org/10.1002/tea.20201
McNeill, K. L., & Krajcik, J. S. (2008b). Assessing middle school students’ content knowledge and reasoning through written scientific explanations. In J. Coffey, R. Douglas, & C. Stearns (Eds.), Assessing science learning: Perspectives from research and practice (pp. 101–116). NSTA Press.
McNeill, K. L., Lizotte, D. J., Krajcik, J. S., & Marx, R. W. (2006). Supporting students’ construction of scientific explanations by fading scaffolds in instructional materials. The Journal of the Learning Sciences, 15(2), 153–191. https://doi.org/10.1207/s15327809jls1502_1
Murcia, K., Pepper, C., & Williams, J. (2020). Youth STEM career choices: What’s influencing secondary students’ decision making. Issues in Educational Research, 30(2), 593–611.
National Research Council [NRC]. (1996). National science education standards. National Academies Press. https://doi.org/10.17226/4962
National Research Council [NRC]. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press. https://doi.org/10.17226/13165
National Research Council [NRC]. (2013). Next Generation Science Standards: For states, by states. National Academies Press. https://doi.org/10.17226/18290
National Research Council [NRC]. (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research. National Academies Press. https://doi.org/10.17226/18612
Nelson, M. C., Cordray, D. S., Hulleman, C. S., Darrow, C. L., & Sommer, E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. The Journal of Behavioral Health Services & Research, 39(4), 374–396. https://doi.org/10.1007/s11414-012-9295-x
Neumann, M. M., Hood, M., Ford, R. M., & Neumann, D. L. (2013). Letter and numeral identification: Their relationship with early literacy and numeracy skills. European Early Childhood Education Research Journal, 21(4), 489–501. https://doi.org/10.1080/1350293X.2013.845438
Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41(10), 994–1020. https://doi.org/10.1002/tea.20035
Osborne, J. F., Henderson, B., MacPherson, A., Szu, E., Wild, A., & Shi-Ying, Y. (2016). The development and validation of a learning progression for argumentation in science. Journal of Research in Science Teaching, 53(6), 821–846. https://doi.org/10.1002/tea.21316
Rabe-Hesketh, S., & Skrondal, A. (2008). Multilevel and longitudinal modeling using Stata. STATA press.
Roth, W.-M., & Bowen, G. (2001). Professionals read graphs: A semiotic analysis. Journal for Research in Mathematics Education, 32(2), 159–194. https://doi.org/10.2307/749672
Schinske, J. N., Perkins, H., Snyder, A., & Wyer, M. (2016). Scientist spotlight homework assignments shift students’ stereotypes of scientists and enhance science identity in a diverse introductory science class. CBE—Life Sciences Education, 15(3), ar47. https://doi.org/10.1187/cbe.16-01-0002
Schochet, P. Z. (2008). Technical Methods Report: Guidelines for Multiple Testing in Impact Evaluations (NCEE 2008–4018). National Center for Education Evaluation and Regional Assistance.
Schultheis, E. H., & Kjelvik, M. K. (2015). Data Nuggets. The American Biology Teacher, 77(1), 19–29. https://doi.org/10.1525/abt.2015.77.1.4
Schultheis, E. H., & Kjelvik, M. K. (2020). Using Messy, Authentic Data to Promote Data Literacy & Reveal the Nature of Science. The American Biology Teacher, 82(7), 439–446. https://doi.org/10.1525/abt.2020.82.7.439
Settles, B. (2009). Active learning literature survey. Department of Computer Sciences, University of Wisconsin-Madison.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton.
Shernoff, D., Knauth, S., & Makris, E. (2000). The quality of classroom experiences. In M. Csikszentmihalyi & B. Schneider (Eds.), Becoming adult: How teenagers prepare for the world of work (pp. 141–164). Basic Books.
Simpson, A., & Bouhafa, Y. (2020). Youths’ and adults’ identity in STEM: A systematic literature review. Journal for STEM Education Research, 3(2), 1–28.
Šorgo, A., Verčkovnik, T., & Kocijančič, S. (2010). Information and communication technologies (ICT) in biology teaching in Slovenian secondary schools. Eurasia Journal of Mathematics, Science and Technology Education, 6(1), 37–46. https://doi.org/10.12973/ejmste/75225
Stains, M., & Vickrey, T. (2017). Fidelity of implementation: An overlooked yet critical construct to establish effectiveness of evidence-based instructional practices. CBE—Life Sciences Education, 16(1), rm1. https://doi.org/10.1187/cbe.16-03-0113
StataCorp. (2017). Stata Statistical Software: Release 15. StataCorp LLC.
Stuhlsatz, M., Snowden, J., & Donovan, B. (2020). Quantitative reasoning in high school biology assessment [Unpublished manuscript]. Colorado, Springs: BSCS Science Learning
Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., Chambwe, N., Cintrón, D. L., Cooper, J. D., Dunster, G., Grummer, J. A., Hennessey, K., Hsiao, J., Iranon, N., Jones, L., II, Jordt, H., Keller, M., Lacey, M. E., Littlefield, C. E., ... Freeman, S. (2020). Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences, 117(12), 6476–6483. https://doi.org/10.1073/pnas.1916903117
Toulmin, S. E. (1958). The uses of argument. Cambridge University Press.
Weinburgh, M. H., & Steele, D. (2000). The modified attitudes toward science inventory: Developing an instrument to be used with fifth grade urban students. Journal of Women and Minorities in Science and Engineering, 6(1), 87–94. https://doi.org/10.1615/JWomenMinorScienEng.v6.i1.50
Wilkerson, M. H., Lanouette, K., & Shareff, R. L. (2021). Exploring variability during data preparation: A way to connect data, chance, and context when working with complex public datasets. Mathematical Thinking and Learning, 1–19. https://doi.org/10.1080/10986065.2021.1922838
Willingham, D. T. (2004). Ask the cognitive scientist: The privileged status of story. American Educator. Retrieved January 15, 2021 from https://www.aft.org/periodical/american-educator/summer-2004/ask-cognitive-scientist
Wilson, E. O. (2002). The power of story. American Educator, 26(1), 8–11.
Wise, A. F. (2020). Educating data scientists and data literate citizens for a new generation of data. Journal of the Learning Sciences, 29(1), 165–181. https://doi.org/10.1080/10508406.2019.1705678
Yair, G. (2000). Educational battlefields in America: The tug-of-war over students’ engagement with instruction. Sociology of Education, 73(4), 247–269. https://doi.org/10.2307/2673233
Zeger, S. L., Liang, K. Y., & Albert, P. S. (1988). Models for longitudinal data: A generalized estimating equation approach. Biometrics, 44(4), 1049–1060. https://doi.org/10.2307/2531734
Thanks to Brian Donovan, Kjelvik and Schultheis, Christopher Wilson, May Lee, Alexa Warwick, Monica Weindling, Alex Duncan, Paul Strode, Audrey Mohan, Zoë Buck Bracey, Kristin Bass, and the participating teachers for their contributions to the research study. Thank you to the anonymous reviewers for their thoughtful comments. This material is based upon work supported by the National Science Foundation under DRK-12 grant numbers 1503211 and 1503005. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. Additional funding from Kellogg Biological Station (KBS) Long-Term Ecological Research program (NSF DEB 1832042) and NSF IUSE 2012014. This is KBS Contribution #2294.
This research was completed with funding from NSF DRK-12 1503211 and 1503005.
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Schultheis, E.H., Kjelvik, M.K., Snowden, J. et al. Effects of Data Nuggets on Student Interest in STEM Careers, Self-efficacy in Data Tasks, and Ability to Construct Scientific Explanations. Int J of Sci and Math Educ 21, 1339–1362 (2023). https://doi.org/10.1007/s10763-022-10295-1
- Authentic data
- Career motivation
- Data literacy
- Quantitative reasoning
- Science storytelling