Technology, Knowledge and Learning

, Volume 19, Issue 1–2, pp 183–204 | Cite as

Improving Learners’ Ability to Recognize Emergence with Embedded Assessment in a Virtual Watershed

  • Benjamin E. ErlandsonEmail author
Original research


Measures of participants’ water cycle knowledge and ability to recognize emergence were taken at various points throughout a 2-h experience with the Cloverdale virtual watershed socioecological simulation. Multilevel growth models were estimated for analysis of hypothesized predictive relationships between measured variables. Significant growth was found for ability to recognize emergence in participants that completed the Cloverdale virtual scenario. Preliminary water cycle knowledge was found to predict initial ability to recognize emergence. Application of results to unobtrusive assessment of learning in virtual worlds is discussed.


Virtual worlds Complex systems Environmental education Growth modeling Unobtrusive assessment 


  1. Almond, R., Steinberg, L., & Mislevy, R. (2002). Enhancing the design and delivery of assessment systems: A four-process architecture. The Journal of Technology, Learning and Assessment, 1(5), 3–63.Google Scholar
  2. Barab, S. A., Thomas, M., Dodge, T., Carteaux, R., & Tuzan, H. (2005). Making learning fun: Quest Atlantis, a game without guns. Educational Technology Research and Development, 53(1), 86–108.Google Scholar
  3. Bar-Yam, Y. (1997). Dynamics of complex systems. Boulder, CO: Westview Press.Google Scholar
  4. Behrens, J. T., DiCerbo, K. E., & Ferrara, S. (2012). Intended and unintended deceptions in the use of simulations. K-12 Center at ETS invitational research symposium on technology enhanced assessments. Retrieved from
  5. Berkes, F., Colding, J., & Folke, C. (Eds.). (2003). Navigating social-ecological systems: Building resilience for complexity and change. Cambridge: Cambridge University Press.Google Scholar
  6. Berkowitz, A. E., Ford, M. E., & Brewer, C. A. (2005). A framework for integrating ecological literacy, civics literacy, and environmental citizenship in environmental education. In E. A. Johnson & M. J. Mappin (Eds.), Environmental education or advocacy: Perspectives of ecology in environmental education. New York: Cambridge University Press.Google Scholar
  7. Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy (structure of the observed learning outcome). New York: Academic Press.Google Scholar
  8. Capra, F. (2000). Ecoliteracy: A systems approach to education. In Center for Ecoliteracy. Ecoliteracy: Mapping the terrain. Berkeley, CA.Google Scholar
  9. Capra, F. (2005). Speaking nature’s language: Principles for sustainability. In M. K. Stone & Z. Barlow (Eds.), Ecological literacy: Educating our children for a sustainable world (pp. 18–29). San Francisco: Sierra Club Books.Google Scholar
  10. Cilliers, P. (1998). Complexity and postmodernism: Understanding complex systems. London: Routledge.Google Scholar
  11. Clark, D. B., Martinez-Garza, M., Nelson, B. C., D’Angelo, C. M., & Slack, K. (2010). SURGE: Intended and unintended science learning in games. In K. Gomez, L. Lyons, & J. Radinsky (Eds.), ICLS '10 proceedings of the 9th international conference of the learning sciences (p. 247).Google Scholar
  12. Erlandson, B., Nelson, B., & Savenye, W. (2010). Collaboration modality, cognitive load, and science inquiry learning in situated inquiry environments. Educational Technology Research and Development, 58(6), 693–710.Google Scholar
  13. Eseryel, D., Ge, X., Ifenthaler, D., & Law, V. (2011). Dynamic modeling as a cognitive regulation scaffold for developing complex problem-solving skills in an educational massively multiplayer online game environment. Journal of Educational Computing Research, 45(3), 265–286.Google Scholar
  14. Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave Macmillan.Google Scholar
  15. Hmelo-Silver, C. E., & Azevedo, R. (2006). Understanding complex systems: Some core challenges. Journal of the Learning Sciences, 15(1), 53–61.Google Scholar
  16. Ifenthaler, D., & Eseryel, D. (2013). Facilitating complex learning by mobile augmented reality learning environments. In R. Huang & J. M. Spector (Eds.), Reshaping learning: Frontiers of learning technology in a global context (pp. 415–438). Berlin: Springer.CrossRefGoogle Scholar
  17. Jacobson, M. J., & Wilenksy, U. (2006). Complex systems in education: Scientific and educational importance and implications for the learning sciences. Journal of the Learning Sciences, 15(1), 11–34.Google Scholar
  18. Jen, E. (Ed.). (1990). 1989 Lectures in complex systems. Redwood City, CA: Addison-Wesley.Google Scholar
  19. Kafai, Y. B., Feldon, D., Fields, D., Giang, M., & Quintero, M. (2007). Life in the times of WhyPox: A virtual epidemic as a community event. In Pentland Steinfield & Contractor Ackerman (Eds.), Communities and technologies 2007: Proceedings of the third communities and tech conference, Michigan State University (pp. 171–190). London: Springer.CrossRefGoogle Scholar
  20. Ketelhut, D. J., & Nelson, B. C. (2010). Designing for real-world scientific inquiry in virtual environments. Educational Research, 52(2), 151–167.CrossRefGoogle Scholar
  21. Ketelhut, D. J., Nelson, B. C., Clarke, J., & Dede, C. (2010). A multi-user virtual environment for building and assessing higher order inquiry skills in science. British Journal of Educational Technology, 41(1), 56–68.CrossRefGoogle Scholar
  22. Lesh, R. (2006). Modeling students' modeling abilities: The teaching and learning of complex systems in education. Journal of the Learning Sciences, 15(1), 45–52.Google Scholar
  23. Meadows, D. (2005). Dancing with systems. In M. K. Stone & Z. Barlow (Eds.), Ecological literacy: Educating our children for a sustainable world (pp. 193–205). San Francisco: Sierra Club Books.Google Scholar
  24. Meadows, D. (2008). In D. Wright (Ed.), Thinking in systems. Chelsea Green Publishing.Google Scholar
  25. Metcalf, S., Kamarainen, A., Tutwiler, M. S., Grotzer, T., & Dede, C. (2011). Ecosystem science learning via multi-user virtual environments. International Journal of Gaming and Computer-Mediated Simulations, 3(1), 86.CrossRefGoogle Scholar
  26. Nelson, B. C. (2007). Exploring the use of individualized, reflective guidance in an educational multi-user environment. Journal of Science Education and Technology, 16(1), 83–97.CrossRefGoogle Scholar
  27. Nelson, B., & Erlandson, B. (2008). Managing cognitive load in educational multi-user virtual environments: Reflection on design practice. Educational Technology Research and Development, 56(5–6), 619–641.CrossRefGoogle Scholar
  28. Nelson, B., & Erlandson, B. (2012). Design for learning in virtual worlds. New York: Routledge.Google Scholar
  29. Nelson, B., Erlandson, B., & Denham, A. (2011). Global channels of evidence for learning and assessment in complex game environments. British Journal of Educational Technology, 42(1), 88–100.CrossRefGoogle Scholar
  30. Nicolis, G., & Prigogine, I. (1989). Exploring complexity. New York: Freeman & Co.Google Scholar
  31. Penner, D. E. (2000). Explaining systems: Investigating middle school students’ understanding of emergent phenomena. Journal of Research in Science Teaching, 37, 784–806.CrossRefGoogle Scholar
  32. Penner, D. E. (2001). Complexity, emergence, and synthetic models in science education. In K. Crowley, C. D. Schunn, & T. Okada (Eds.), Designing for science (pp. 177–208). Mahwah, NJ: Lawrence Earlbaum Associates, Inc.Google Scholar
  33. Serra, R., & Zanarini, G. (1990). Complex systems and cognitive processes. Berlin: Springer.CrossRefGoogle Scholar
  34. Shaffer, D. W. (1997). Learning mathematics through design: The anatomy of Escher’s World. Journal of Mathematical Behavior, 16(2), 95–112.CrossRefGoogle Scholar
  35. Shute, V., & Ventura, M. (2013). Stealth assessment: Measuring and supporting learning in video games. Cambridge: MIT Press.Google Scholar
  36. Shute, V. J., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning. In U. Ritterfeld, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects, 295–321.Google Scholar
  37. Stables, A. (1996). Reading the environment as text: Literary theory and environmental education. Environmental Education Research, 2(2), 189–195.Google Scholar
  38. Stables, A. (1998). Environmental literacy: Functional, cultural, critical. The case of the SCAA guidelines. Environmental Education Research, 4(2), 155–164.Google Scholar
  39. Steinkuehler, C. A. (2004). Learning in massively multiplayer online games. In Y. B. Kafai, W. A. Sandoval, N. Enyedy, A. S. Nixon, & F. Herrera (Eds.), Proceedings of the 6th international conference on learning sciences (pp. 521–528). Mahwah, NJ: Lawrence Earlbaum Associates.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  1. 1.Arizona State UniversityTempeUSA
  2. 2.ElkinUSA

Personalised recommendations