Press Play! How Immersive Environments Support Problem-Solving Skills and Productive Failure

  • Benjamin EmihovichEmail author
  • Logan Arrington
  • Xinhao Xu
Part of the Advances in Game-Based Learning book series (AGBL)


Well-designed video games provide exciting prospects for teaching, training, learning, and research. Moreover, immersive virtual reality (VR) environments offer flexibility to design learning and training scenarios that are authentic. Gameplay in immersive environments often requires players to test and refine new strategies when confronted with progressively more challenging scenarios where learning from failure is a function of game design. Over time, players hone their skills through internal game mechanics and interactions with the environment, such as feedback and pedagogical agents to support long-term learning. This is known as productive failure. However, there are still challenges in assessing learners’ targeted competencies, for example, problem-solving skills, during video gameplay. In this chapter, we examine how student interactions during gameplay can be assessed in immersive environments without disrupting the flow of gameplay. This type of assessment is known as stealth assessment. We also describe the challenges with assessing productive failure in game-based learning and implications for future research on assessment of learning in immersive environments.


Gameplay Productive failure Virtual reality Problem-solving skills Stealth assessment 


  1. Abrahamson, D., & Kapur, M. (2018). Reinventing discovery learning: A field-wide research program. Instructional Science, 46(1), 1–10.CrossRefGoogle Scholar
  2. Adams Becker, S., Freeman, A., Giesinger Hall, C., Cummins, M., & Yuhnke, B. (2016). Horizon report: 2016 K-12 edition. Austin, TX: New Media Consortium.Google Scholar
  3. Andersen, S. A. W., Konge, L., & Sørensen, M. S. (2018). The effect of distributed virtual reality simulation training on cognitive load during subsequent dissection training. Medical Teacher, 40, 684–689.CrossRefGoogle Scholar
  4. Anderson, C. G., Dalsen, J., Kumar, V., Berland, M., & Steinkuehler, C. (2018). Failing up: How failure in a game environment promotes learning through discourse. Thinking Skills and Creativity, 30, 135–144.CrossRefGoogle Scholar
  5. Bellini, H., Chen, W., Sugiyama, M., Shin, M., Alam, S., & Takayama, D. (2016). Profiles in innovation: Virtual & augmented reality. New York, NY: Goldman Sachs Group, Inc.Google Scholar
  6. Brown, A., & Green, T. (2016). Virtual reality: Low-cost tools and resources for the classroom. TechTrends, 60(5), 517–519. Scholar
  7. Chang, T. P., & Weiner, D. (2016). Screen-based simulation and virtual reality for pediatric emergency medicine. Clinical Pediatric Emergency Medicine, 17(3), 224–230.CrossRefGoogle Scholar
  8. Cho, J. S., Hahn, K. Y., Kwak, J. M., Kim, J., Baek, S. J., Shin, J. W., & Kim, S. H. (2013). Virtual reality training improves da Vinci performance: A prospective trial. Journal of Laparoendoscopic & Advanced Surgical Techniques, 23(12), 992–998.CrossRefGoogle Scholar
  9. Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research on Technology in Education, 42(3), 309–328.CrossRefGoogle Scholar
  10. Dede, C. (2005). Planning for neomillennial learning styles. Educause Quarterly, 28(1), 7–12.Google Scholar
  11. Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323(66), 66–69.CrossRefGoogle Scholar
  12. DiCerbo, K., Shute, V. J., & Kim, Y. J. (2017). The future of assessment in technology rich environments: Psychometric considerations. In J. M. Spector, B. Lockee, & M. Childress (Eds.), Learning, design, and technology: An international compendium of theory, research, practice, and policy (pp. 1–21). New York, NY: Springer.Google Scholar
  13. Eseryel, D., Law, V., Ifenthaler, D., Ge, X., & Miller, R. (2014). An investigation of the interrelationships between motivation, engagement, and complex problem solving in game-based learning. Educational Technology & Society, 17(1), 42–53.Google Scholar
  14. Fong, K., Jenson, J., & Hebert, C. (2018). Challenges with measuring learning through digital gameplay in K-12 classrooms. Media and Communication, 6(2), 112–125.CrossRefGoogle Scholar
  15. Gauthier, A., & Jenkinson, J. (2018). Designing productively negative experiences with serious game mechanics: Qualitative analysis of gameplay and game design in a randomized trial. Computers & Education, 127, 66–89.CrossRefGoogle Scholar
  16. Gee, J. P. (2005). Learning by design: Good video games as learning machines. E-Learning and Digital Media, 2(1), 5–16.CrossRefGoogle Scholar
  17. Gee, J. P. (2008). Learning and games. In K. Salen (Ed.), The ecology of games: Connecting youth, games, and learning (pp. 21–40). Cambridge, MA: MIT.Google Scholar
  18. Glogger-Frey, I., Fleischer, C., Grüny, L., Kappich, J., & Renkl, A. (2015). Inventing a solution and studying a worked solution prepare differently for learning from direct instruction. Learning and Instruction, 39, 72–87. Scholar
  19. Hart Research Associates. (2018). Fulfilling the American dream: Liberal education and the future of work. Washington, DC: Association of American Colleges and Universities.Google Scholar
  20. Hew, K. F., & Cheung, W. S. (2010). Use of three-dimensional (3-D) immersive virtual worlds in K-12 and higher education settings: A review of the research. British Journal of Educational Technology, 41(1), 33–55.CrossRefGoogle Scholar
  21. Jagust, T., Boticki, I., & So, H. (2018). Examining competitive, collaborative and adaptive gamification in young learners’ math learning. Computers & Education, 125, 444–457.CrossRefGoogle Scholar
  22. Jensen, L., & Konradsen, F. (2018). A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies, 23(4), 1515–1529.CrossRefGoogle Scholar
  23. Johnson-Glenberg, M. C., & Megowan-Romanowicz, C. (2017). Embodied science and mixed reality: How gesture and motion capture affect physics education. Cognitive Research: Principles and Implications, 2(1), 24.Google Scholar
  24. Jonassen, D. H. (1997). Instructional design models for well-structured and III-structured problem-solving learning outcomes. Educational Technology Research and Development, 45(1), 65–94.CrossRefGoogle Scholar
  25. Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48(4), 63–85.CrossRefGoogle Scholar
  26. Kapur, M. (2008). Productive failure. Cognition and Instruction, 26(3), 379–424. Scholar
  27. Kapur, M. (2009). Productive failure in mathematical problem solving. Instructional Science, 38(6), 523–550. Scholar
  28. Kapur, M. (2010). A further study of productive failure in mathematical problem solving: Unpacking the design components. Instructional Science, 39(4), 561–579. Scholar
  29. Kapur, M. (2012). Productive failure in learning the concept of variance. Instructional Science, 40(4), 651–672. Scholar
  30. Kapur, M. (2013). Comparing learning from productive failure and vicarious failure. Journal of the Learning Sciences, 23(4), 651–677. Scholar
  31. Kapur, M. (2014). Productive failure in learning math. Cognitive Science, 38(5), 1008–1022. Scholar
  32. Kapur, M. (2015). The preparatory effects of problem solving versus problem posing on learning from instruction. Learning and Instruction, 39, 23–31. Scholar
  33. Kapur, M. (2016). Examining productive failure, productive success, unproductive failure, and unproductive success in learning. Educational Psychologist, 51(2), 289–299. Scholar
  34. Kapur, M., & Bielaczyc, K. (2012). Designing for productive failure. Journal of the Learning Sciences, 21(1), 45–83. Scholar
  35. Kapur, M., & Kinzer, C. K. (2009). Productive failure in CSCL groups. International Journal of Computer-Supported Collaborative Learning, 4(1), 21–46. Scholar
  36. Kapur, M., & Rummel, N. (2012). Productive failure in learning from generation and invention activities. Instructional Science, 40(4), 645–650. Scholar
  37. Ke, F., Lee, S., & Xu, X. (2016). Teaching training in a mixed-reality integrated learning environment. Computers in Human Behavior, 62, 212–220.CrossRefGoogle Scholar
  38. Kirk, D., & MacPhail, A. (2002). Teaching games for understanding and situated learning: Rethinking the Bunker-Thorpe model. Journal of Teaching in Physical Education, 21, 117–192.Google Scholar
  39. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
  40. Leder, J., Horlitz, T., Puschmann, P., Wittstock, V., & Schütz, A. (2019). Comparing immersive virtual reality and powerpoint as methods for delivering safety training: Impacts on risk perception, learning, and decision making. Safety Science, 111, 271–286.CrossRefGoogle Scholar
  41. Loibl, K., & Rummel, N. (2014a). The impact of guidance during problem-solving prior to instruction on students’ inventions and learning outcomes. Instructional Science, 42(3), 305–326. Scholar
  42. Loibl, K., & Rummel, N. (2014b). Knowing what you don’t know makes failure productive. Learning and Instruction, 34, 74–85. Scholar
  43. Macedonia, M., & von Kriegstein, K. (2012). Gestures enhance foreign language learning. Biolinguistics, 6(3–4), 393–416.Google Scholar
  44. Makransky, G., Terkildsen, T. S., & Mayer, R. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learning and Instruction, 60, 225–236.CrossRefGoogle Scholar
  45. Mayer, R., & Wittrock, M. (2006). Problem solving. In P. Alexander & P. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 287–303). Mahwah, NJ: Erlbaum Publishers.Google Scholar
  46. Mazziotti, C., Loibl, K., & Rummel, N. (2015). Collaborative or individual learning within productive failure. Does the social form of learning make a difference? In O. Lindwall, P. Häkkinen, T. Koschman, P. Tchounikine, & S. Ludvigsen (Eds.), Exploring the material conditions of learning: The computer supported collaborative learning (CSCL) conference 2015 (Vol. 2, pp. 570–575). Gothenburg, Sweden: ISLS.Google Scholar
  47. McGrath, J. L., Taekman, J. M., Dev, P., Danforth, D. R., Mohan, D., Kman, N., … Talbot, T. B. (2018). Using virtual reality simulation environments to assess competence for emergency medicine learners. Academic Emergency Medicine, 25(2), 186–195.CrossRefGoogle Scholar
  48. McGregor, C., Bonnis, B., Stanfield, B., & Stanfield, M. (2017). Integrating Big Data analytics, virtual reality, and ARAIG to support resilience assessment and development in tactical training. In 2017 IEEE 5th International Conference on Serious Games and Applications for Health (pp. 1–7). IEEE.Google Scholar
  49. Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3–62.Google Scholar
  50. Nagendran, M., Gurusamy, K. S., Aggarwal, R., Loizidou, M., & Davidson, B. R. (2013). Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database of Systematic Reviews, (8), CD006575.Google Scholar
  51. Nathan, M. J., & Walkington, C. (2017). Grounded and embodied mathematical cognition: Promoting mathematical insight and proof using action and language. Cognitive Research: Principles and Implications, 2(1), 9.Google Scholar
  52. Passig, D., Tzuriel, D., & Eshel-Kedmi, G. (2016). Improving children’s cognitive modifiability by dynamic assessment in 3D Immersive Virtual Reality environments. Computers & Education, 95, 296–308.CrossRefGoogle Scholar
  53. Richards, D., & Taylor, M. (2015). A Comparison of learning gains when using a 2D simulation tool versus a 3D virtual world: An experiment to find the right representation involving the Marginal Value Theorem. Computers & Education, 86, 157–171.CrossRefGoogle Scholar
  54. Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503–524). Charlotte, NC: Information Age Publishers.Google Scholar
  55. Shute, V. J., & Emihovich, B. (2018). Assessing problem-solving skills in immersive environments. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), International handbook of information technology in primary and secondary education. Cham, Switzerland: Springer.Google Scholar
  56. Shute, V. J., Hansen, E. G., & Almond, R. G. (2008). You can’t fatten a hog by weighing it—Or can you? Evaluating an assessment for learning system called ACED. International Journal of Artificial Intelligence in Education, 18(4), 289–316.Google Scholar
  57. Shute, V. J., Ke, F., & Wang, L. (2017). Assessment and adaptation in games. In P. Wouters & H. van Oostendorp (Eds.), Instructional techniques to facilitate learning and motivation of serious games (pp. 59–78). New York, NY: Springer.CrossRefGoogle Scholar
  58. Shute, V. J., Rahimi, S., & Emihovich, B. (2018). Assessment for learning in immersive environments. In D. Lui, C. Dede, R. Huang, & J. Richards (Eds.), Virtual, augmented, and mixed realities in education (pp. 71–89). Heidelberg, Germany: Springer.Google Scholar
  59. Shute, V. J., Ventura, M., & Ke, F. (2015). The power of play: The effects of Portal 2 and Lumosity on cognitive and noncognitive skills. Computers & Education, 80, 58–67.CrossRefGoogle Scholar
  60. Shute, V. J., Wang, L., Greiff, S., Zhao, W., & Moore, G. R. (2016). Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior, 63, 106–117.CrossRefGoogle Scholar
  61. Smith, P. C., & Hamilton, B. K. (2015). The effects of virtual reality simulation as a teaching strategy for skills preparation in nursing students. Clinical Simulation in Nursing, 11(1), 52–58.CrossRefGoogle Scholar
  62. Stanica, I., Dascalu, M. I., Bodea, C. N., & Moldoveanu, A. D. B. (2018). VR job interview simulator: Where virtual reality meets artificial intelligence for education. In 2018 Zooming Innovation in Consumer Technologies Conference (ZINC) (pp. 9–12). IEEE.Google Scholar
  63. Sugden, C., Aggarwal, R., Banerjee, A., Haycock, A., Thomas-Gibson, S., Williams, C. B., & Darzi, A. (2012). The development of a virtual reality training curriculum for colonoscopy. Annals of Surgery, 256(1), 188–192.CrossRefGoogle Scholar
  64. Taub, M., Azevedo, R., Bradbury, A. E., Millar, G. C., & Lester, J. (2018). Using sequence mining to reveal the efficiency in scientific reasoning during STEM learning with a game-based learning environment. Learning and Instruction, 54, 93–103.CrossRefGoogle Scholar
  65. Tiffany, J. M., & Hoglund, B. A. (2016). Using virtual simulation to teach inclusivity: A case study. Clinical Simulation in Nursing, 12(4), 115–122.CrossRefGoogle Scholar
  66. Toh, P. L. L., & Kapur, M. (2017). Is having more prerequisite knowledge better for learning from productive failure? Instructional Science, 45(3), 377–394. Scholar
  67. Van Eck, R. N., Hung, W., Bowman, R., & Love, S. (2009). 21st century game design: A model and prototype for promoting scientific problem solving. In Proceedings of the Twelfth IASTED International Conference on Computers and Advanced Technology in Education: Globalization of Education Through Advanced Technology. Calgary, Canada. ACTA Press.Google Scholar
  68. Van Eck, R. N., Shute, V. J., & Rieber, L. P. (2017). Leveling up: Game design research and practice for instructional designers. In R. Reiser & J. Dempsey (Eds.), Trends and issues in instructional design and technology (4th ed., pp. 227–285). New York, NY: Pearson.Google Scholar
  69. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.Google Scholar
  70. Westermann, K., & Rummel, N. (2012). Delaying instruction: Evidence from a study in a university relearning setting. Instructional Science, 40(4), 673–689. Scholar
  71. Xu, X., & Ke, F. (2014). From psychomotor to ‘motorpsycho’: Learning through gestures with body sensory technologies. Educational Technology Research and Development, 62(6), 711–741.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Benjamin Emihovich
    • 1
    Email author
  • Logan Arrington
    • 2
  • Xinhao Xu
    • 3
  1. 1.University of Michigan-FlintFlintUSA
  2. 2.University of West GeorgiaCarrolltonUSA
  3. 3.University of MissouriColumbiaUSA

Personalised recommendations