Advertisement

Assessing Science Inquiry and Reasoning Using Dynamic Visualizations and Interactive Simulations

  • Jodi L. Davenport
  • Edys S. Quellmalz
Chapter

Abstract

How can we leverage dynamic visualizations and interactive simulations to assess complex science learning? Science systems involve many dynamic processes, and visualizations of these processes are central components of scientific thinking and communication. Although multiple-choice items may effectively measure declarative knowledge such as scientific facts or definitions, they fail to capture evidence of science inquiry practices such as making observations, collecting data, or modeling dynamic systems. As computers become more widely accessible, interactive, simulation-based assessments have the promise of capturing information about students’ ability to apply these more complex science practice skills. Tasks using dynamic visualizations and simulations are likely to provide more valid measures of the scientific proficiencies called for internationally by science educators, such as those reflected in the United States’ National Assessment of Educational Progress and the Next Generation Science Standards. In this chapter, we describe the range of science content knowledge and practice skills we seek to measure, outline research-based assessment design principles, and review a test creation process that mitigates the potential limitations of dynamic assessments. Next, we provide examples of how the stimulus and response affordances of dynamic visualizations allow us to assess scientific inquiry and reasoning skills. Finally, we provide findings from an efficacy study that support the claim that dynamic and interactive assessments provide better measures of student proficiency with complex scientific inquiry and reasoning. Throughout the chapter, we give concrete examples of the principles and processes in the context of two science assessment platforms, SimScientists and ChemVLab+.

Keywords

Science System Science Practice Processing Demand Interactive Format Dynamic Visualization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This material chapter is based upon work supported by the National Science Foundation under Grant No. DRL-0814776 awarded to WestEd, Edys Quellmalz, Principal Investigator and work supported by the Institute of Education Sciences, through Grant R305A100069 to WestEd, Jodi Davenport, Principal Investigator. Any opinions, findings and conclusions or recommendations expressed in this material chapter are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or the U.S. Department of Education.

References

  1. Adams, W. K., Reid, S., Lemaster, R., McKagan, S. B., Perkins, K. K., Dubson, M., et al. (2008). A study of educational simulations part 1: Engagement and learning. Journal of Interactive Learning Research, 19, 397–419.Google Scholar
  2. Ainsworth, S. (2008). The educational value of multiple-representations when learning complex scientific concepts. In J. K. Gilbert, M. Reiner, & M. Nakhleh (Eds.), Visualization: Theory and practice in science education (pp. 191–208). New York: Springer.CrossRefGoogle Scholar
  3. Bétrancourt, M. (2005). The animation and interactivity principles in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 287–296). New York: Cambridge University Press.CrossRefGoogle Scholar
  4. Boucheix, J.-M. (2008). Young learners’ control of technical animations. In R. K. Lowe & W. Schnotz (Eds.), Learning with animation: Research implications for design (pp. 208–234). New York: Cambridge University Press.Google Scholar
  5. Boucheix, J.-M., & Lowe, R. K. (2010). An eye tracking comparison of external pointing cues and internal continuous cues in learning with complex animations. Learning and Instruction, 20, 123–135.CrossRefGoogle Scholar
  6. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.Google Scholar
  7. Brennan, R. (2001). mGENOVA. Retrieved from http://www.education.uiowa.edu/centers/casma/computer-programs.aspx.
  8. Buckley, B. C., Gobert, J., Horwitz, P., & O’Dwyer, L. (2010). Looking inside the black box: Assessing model-based learning and inquiry in BioLogica. International Journal of Learning Technologies, 5, 166–190.CrossRefGoogle Scholar
  9. Buckley, B. C., & Quellmalz, E. S. (2013). Supporting and assessing complex biology learning with computer-based simulations and representations. In D. Treagust & C. Y. Tsui (Eds.), Multiple representations in biological education (pp. 247–267). New York: Springer.CrossRefGoogle Scholar
  10. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105.CrossRefGoogle Scholar
  11. Clark, R. C., & Mayer, R. E. (2011). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: Pfeiffer.CrossRefGoogle Scholar
  12. Cuadros, J., Leinhardt, G., & Yaron, D. (2007). One firm spot: The role of homework as lever in acquiring conceptual and performance competence in college chemistry. Journal of Chemical Education, 84, 1047–1052.CrossRefGoogle Scholar
  13. Davenport, J. L., Rafferty, A., Timms, M. J., Yaron, D., & Karabinos, M. (2012a). ChemVLab+: Evaluating a virtual lab tutor for high school chemistry. In J. van Aalst, K. Thompson, M. J. Jacobson, & P. Reimann (Eds.), Proceedings of the 10th international conference of the learning sciences (pp. 381–385). Sydney, Australia: International Society of the Learning Sciences.Google Scholar
  14. Davenport, J. L., Timms, M. J., Yaron, D., & Karabinos, M. (2012b) ChemVlab+: Supporting science learning using a virtual chemistry lab with embedded assessments. Paper presented at the 2012 Annual Meeting of the American Educational Research Association in Vancouver, BC.Google Scholar
  15. Davenport, J.L., Powers, J., Rafferty, A., Timms, M., Karabinos, M., & Yaron, D. (2014a, April). Classroom factors and students learning with online virtual chemistry lab activities. Paper presented at the 2014 Annual Meeting of the American Educational Research Association in Philadelphia, PA.Google Scholar
  16. Davenport, J. L., Rafferty, A., Yaron, D, Karabinos, M., & Timms, M. (2014b, April). ChemVLab+: Simulation-based lab activities to support chemistry learning. Paper presented at the 2014 Annual Meeting of the American Educational Research Association in Philadelphia, PA.Google Scholar
  17. De Jong, T. (2006). Technological advances in inquiry learning. Science, 312(5773), 532–533.CrossRefGoogle Scholar
  18. De Koning, B. B., & Jarodzka, H. (2017). Attention guidance strategies for supporting learning from dynamic visualizations. In R. Lowe & R. Ploetzner (Eds.), Learning from dynamic visualization – Innovations in research and application. Berlin: Springer (this volume).Google Scholar
  19. DeBoer, G. E., Quellmalz, E. S., Davenport, J. L., Timms, M. J., Herrmann-Abell, C. F., Buckley, B. C., et al. (2014). Comparing three online testing modalities: Using static, active, and interactive online testing modalities to assess middle school students’ understanding of fundamental ideas and use of inquiry skills related to ecosystems. Journal of Research in Science Teaching, 51, 523–554.CrossRefGoogle Scholar
  20. Gallegos, L., Jerezano, M., & Flores, F. (1994). Preconceptions and relations used by children in the construction of food chains. Journal of Research in Science Teaching, 31, 259–272.CrossRefGoogle Scholar
  21. Goldman, S. R. (2003). Learning in complex domains: When and why do multiple representations help? Learning and Instruction, 13, 239–244.CrossRefGoogle Scholar
  22. Hegarty, M. (2004). Dynamic visualizations and learning: Getting to the difficult questions. Learning and Instruction, 14, 343–351.CrossRefGoogle Scholar
  23. Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742.CrossRefGoogle Scholar
  24. Hmelo-Silver, C. E., Jordan, R., Liu, L., Gray, S., Demeter, M., Rugaber, S., et al. (2008). Focusing on function: Thinking below the surface of complex science systems. Science Scope, 31(9), 27–35.Google Scholar
  25. Horwitz, P., Gobert, J. D., Buckley, B. C., & O’Dwyer, L. M. (2010). Learning genetics with dragons: From computer-based manipulatives to hypermodels. In M. J. Jacobson & P. Reimann (Eds.), Designs for learning environments of the future: International perspectives from the learning sciences (pp. 61–87). New York: Springer.CrossRefGoogle Scholar
  26. Ioannidou, A., Repenning, A., Webb, D., Keyser, D., Luhn, L., & Daetwyler, C. (2010). Mr. Vetro: A collective simulation for teaching health science. International Journal of Computer-Supported Collaborative Learning, 5, 141–166.CrossRefGoogle Scholar
  27. Krajcik, J., Marx, R., Blumenfeld, P., Soloway, E., & Fishman, B. (2000). Inquiry-based science supported by technology: Achievement and motivation among urban middle school students. Paper presented at the American Educational Research Association, New Orleans, LA.Google Scholar
  28. Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learning from animations. International Journal of Human-Computer Studies, 65, 911–930.CrossRefGoogle Scholar
  29. Kühl, T., Scheiter, K., Gerjets, P., & Edelmann, J. (2011). The influence of text modality on learning with static and dynamic visualizations. Computers in Human Behavior, 27, 29–35.CrossRefGoogle Scholar
  30. Larkin, J., & Simon, H. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–100.CrossRefGoogle Scholar
  31. Lee, H., Plass, J. L., & Homer, B. D. (2006). Optimizing cognitive load for learning from computer-based science simulations. Journal of Educational Psychology, 98, 902–913.CrossRefGoogle Scholar
  32. Lehrer, R., Schauble, L., Strom, D., & Pligge, M. (2001). Similarity of form and substance: Modeling material kind. In D. K. S. Carver (Ed.), Cognition and instruction: 25 years of progress (pp. 39–74). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  33. Loehlin, J. C. (1998). Latent variable models: An introduction to factor, path, and structural analysis. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  34. Lowe, R. K. (2008). Learning from animation: Where to look, when to look. In R. K. Lowe & W. Schnotz (Eds.), Learning with animation: Research implications for design (pp. 49–70). New York: Cambridge University Press.Google Scholar
  35. Lowe, R., & Boucheix, J.-M. (2017). A composition approach to design of educational animations. In R. Lowe & R. Ploetzner (Eds.), Learning from dynamic visualization – Innovations in research and application. Berlin: Springer (this volume).CrossRefGoogle Scholar
  36. Lowe, R., Boucheix, J.-M., & Fillisch, B. (2017). Demonstration tasks for assessment. In R. Lowe & R. Ploetzner (Eds.), Learning from dynamic visualization – Innovations in research and application. Berlin: Springer (this volume).CrossRefGoogle Scholar
  37. Lowe, R., & Schnotz, W. (Eds.). (2008). Learning with animation: Research implications for design. New York: Cambridge University Press.Google Scholar
  38. Lowe, R. K., & Schnotz, W. (2014). Animation principles in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 279–315). New York: Cambridge University Press.Google Scholar
  39. Mayer, R. E. (Ed.). (2005). The Cambridge handbook of multimedia learning. New York: Cambridge University Press.Google Scholar
  40. Mayer, R. E. (Ed.). (2014). The Cambridge handbook of multimedia learning (2nd ed.). New York: Cambridge University Press.Google Scholar
  41. Mayer, R. E., & Fiorella, L. (2014). Principals for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 279–315). New York: Cambridge University Press.CrossRefGoogle Scholar
  42. Mayer, R. E., & Johnson, C. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100, 380–386.CrossRefGoogle Scholar
  43. Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design (Research Report No. RR-03-16). Princeton, NJ: Educational Testing Service.Google Scholar
  44. Mullis, I. V. S., & Martin, M. O. (Eds.). (2013). TIMSS 2015 assessment frameworks. Chestnut Hill, MA: TIMSS & PIRLS International Study Center at Boston College.Google Scholar
  45. National Assessment Governing Board (NAGB). (2010). Science framework for the 2011 national assessment of educational progress. Washington, DC: National Assessment Governing Board.Google Scholar
  46. National Center for Education Statistics (NCES). (2012). The nation’s report card: science in action: Hands-on and interactive computer tasks from the 2009 science assessment (NCES 2012–468). Washington, DC: Institute of Education Sciences, U.S. Department of Education.Google Scholar
  47. NGSS Lead States (NGSS). (2013). Next generation science standards: For states, by states. Washington, DC: The National Academies Press.Google Scholar
  48. Organization for Economic Cooperation and Development (OECD). (2009). PISA 2009 assessment framework: Key competencies in reading, mathematics and science. Paris, France: OECD.Google Scholar
  49. Papaevripidou, M., Constantinou, C. P., & Zacharia, Z. C. (2007). Modeling complex marine ecosystems: An investigation of two teaching approaches with fifth graders. Journal of Computer Assisted Learning, 23, 145–157.CrossRefGoogle Scholar
  50. Pedone, R., Hummel, J. E., & Holyoak, K. J. (2001). The use of diagrams in analogical problem solving. Memory and Cognition, 29, 214–221.CrossRefGoogle Scholar
  51. Pellegrino, J. W. (2013). Proficiency in science: Assessment challenges and opportunities. Science, 340(6130), 320–323.CrossRefGoogle Scholar
  52. Petre, M., & Green, T. R. G. (1993). Learning to read graphics: Some evidence that “seeing” an information display is an acquired skill. Journal of Visual Languages & Computing, 4, 55–70.CrossRefGoogle Scholar
  53. Plass, J. L., Homer, B. D., & Hayward, E. O. (2009). Design factors for educationally effective animations and simulations. Journal of Computing in Higher Education, 21(1), 31–61.CrossRefGoogle Scholar
  54. Quellmalz, E. S., Davenport, J. L., Timms, M. J., DeBoer, G. E., Jordan, K. A., Huang, C., et al. (2013). Next-generation environments for assessing and promoting complex science learning. Journal of Educational Psychology, 5, 1100.CrossRefGoogle Scholar
  55. Quellmalz, E. S., & Pellegrino, J. W. (2009). Technology and testing. Science, 323(5910), 75–79.CrossRefGoogle Scholar
  56. Quellmalz, E. S., Timms, M. J., & Buckley, B. (2010). The promise of simulation-based science assessment: The Calipers project. International Journal of Learning Technology, 5, 243–263.CrossRefGoogle Scholar
  57. Quellmalz, E. S., Timms, M. J., Silberglitt, M. D., & Buckley, B. C. (2011). Science assessments for all: Integrating science simulations into balanced state science assessment systems. Journal of Research in Science Teaching, 49, 363–393.CrossRefGoogle Scholar
  58. Reiner, M., & Eilam, B. (2001). Conceptual classroom environment: A system view of learning. International Journal of Science Education, 23, 551–568.CrossRefGoogle Scholar
  59. Rieber, L. P., Tzeng, S., & Tribble, K. (2004). Discovery learning, representation, and explanation within a computer-based simulation. Computers and Education, 27, 45–58.CrossRefGoogle Scholar
  60. Schwartz, D. L., & Black, T. (1999). Inferences through imagined actions: Knowing by simulated doing. Journal of Experimental Psychology: Learning, Memory & Cognition, 25, 116–136.Google Scholar
  61. Schwartz, D. L., & Heiser, J. (2006). Spatial representations and imagery in learning. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 283–298). New York: Cambridge University Press.Google Scholar
  62. Schweingruber, H., Keller, T., & Quinn, H. (Eds.). (2012). A framework for K-12 Science Education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.Google Scholar
  63. Slotta, J. D., & Chi, M. T. H. (2006). Helping students understand challenging topics in science through ontology training. Cognition and Instruction, 24, 261–289.CrossRefGoogle Scholar
  64. Stewart, J., Cartier, J. L., & Passmore, C. M. (2005). Developing understanding through model-based inquiry. In M. S. Donovan & J. D. Bransford (Eds.), How students learn (pp. 515–565). Washington, DC: The National Academies Press.Google Scholar
  65. Tversky, B., Heiser, J., Lozano, S., MacKenzie, R., & Morrison, J. (2008). Enriching animations. In R. K. Lowe & W. Schnotz (Eds.), Learning with animation: Research implications for design (pp. 263–285). New York: Cambridge University Press.Google Scholar
  66. Van Merriënboer, J. J. G., & Kester, L. (2005). The four-component instructional design model: Multimedia principles in environments for complex learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 71–93). New York: Cambridge University Press.CrossRefGoogle Scholar
  67. Vattam, S. S., Goel, A. K., Rugaber, S., Hmelo-Silver, C. E., Jordan, R., Gray, S., et al. (2011). Understanding complex natural systems by articulating structure-behavior- function models. Journal of Educational Technology & Society, 14, 66–81.Google Scholar
  68. Webb, N. M., & Shavelson, R. J. (2005). Generalizability theory: Overview. In B. Everitt & D. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 2, pp. 717–719). Chichester, UK: Wiley.Google Scholar
  69. Zacharia, Z. C. (2007). Comparing and combining real and virtual experimentation: An effort to enhance students’ conceptual understanding of electric circuits. Journal of Computer Assisted Learning, 23, 120–132.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.WestEdOaklandUSA
  2. 2.WestEdRedwood CityUSA

Personalised recommendations