Skip to main content

Interaction and Communication in an Immersive Learning Game: The Challenges of Modelling Real-Time Collaboration in a Virtual Operating Room

  • 4489 Accesses

Abstract

In this chapter, we describe the methodology we have engineered during the design process of the collaborative and immersive learning game 3D Virtual Operating Room. The game targets an audience of practitioners involved in the operating room and the training consists in virtually re-enacting typical perioperative activities so as to learn or improve skills related to patient safety. The challenges faced in this project include multiplayer collaboration in a shared, interactive and dynamically evolving virtual environment, and modelling educational scenarios on the basis of actual observations inside the operating room. The model we detail is grounded on a semantic definition of the environment which allowed for three innovative features. A game-mediated communication system where information pertaining to the game is exchanged in real time by the players. AI-controlled characters replacing missing players as fully equal partners. And, the ability for the game to provide feedback in real time or during a debriefing on the team’s performance against predefined pedagogical objectives.

Keywords

  • Immersive learning game
  • Design methodology
  • Multiplayer collaboration
  • In-game communication
  • Business process modelling and notation

Serious Game Research Network, http://sgrn.univ-jfc.fr

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-51645-5_7
  • Chapter length: 40 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   219.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-51645-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   279.99
Price excludes VAT (USA)
Hardcover Book
USD   279.99
Price excludes VAT (USA)
Fig. 7.1
Fig. 7.2
Fig. 7.3
Fig. 7.4
Fig. 7.5
Fig. 7.6
Fig. 7.7
Fig. 7.8
Fig. 7.9
Fig. 7.10
Fig. 7.11
Fig. 7.12

References

  • Allweyer, T.: BPMN 2.0: Introduction to the Standard for Business Process Modeling. BoD–Books on Demand, Norderstedt (2010)

    Google Scholar 

  • Bellotti, F., Berta, R., De Gloria, A., Primavera, L.: Supporting authors in the development of task-based learning in serious virtual worlds. Br. J. Educ. Technol. 41 (1), 86–107 (2010)

    Google Scholar 

  • Carayon, P., Schultz, K., Hundt, A.S.: Righting wrong site surgery. Jt. Comm. J. Qual. Patient Saf. 30 (7), 405–410 (2004)

    Google Scholar 

  • Cassell, J.: Embodied Conversational Agents. MIT Press, Cambridge (2000)

    Google Scholar 

  • Cook, D.A., Triola, M.M.: Virtual patients: a critical literature review and proposed next steps. Med. Educ. 43 (4), 303–311 (2009)

    CrossRef  Google Scholar 

  • Coulter, R., Saland, L., Caudell, T., Goldsmith, T.E., Alverson, D.: The effect of degree of immersion upon learning performance in virtual reality simulations for medical education. In: Medicine Meets Virtual Reality 15: In Vivo, in Vitro, in Silico: Designing the Next in Medicine, vol. 125, pp. 155. IOS Press, Amsterdam/Washington, DC (2007)

    Google Scholar 

  • Currie, L.: Fall and injury prevention. In: Hughes, R.G. (ed.) Patient safety and quality: an evidence-based handbook for nurses. Agency for Healthcare Research and Quality (US), Rockville, MD (2008). Chapter 10

    Google Scholar 

  • De Freitas, S.: Learning in Immersive Worlds. Joint Information Systems Committee, London (2006)

    Google Scholar 

  • De Freitas, S., Rebolledo-Mendez, G., Liarokapis, F., Magoulas, G., Poulovassilis, A.: Learning as immersive experiences: using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world. Br. J. Educ. Technol. 41 (1), 69–85 (2010)

    Google Scholar 

  • Dede, C.: Immersive interfaces for engagement and learning. Science 323 (5910), 66–69 (2009)

    CrossRef  Google Scholar 

  • Djaouti, D., Alvarez, J., Jessel, J.P., Rampnoux, O.: Origins of serious games. In: Serious Games and Edutainment Applications, pp. 25–43. Springer, London/New York (2011)

    Google Scholar 

  • Duval, Y., Panzoli, D., Reymonet, A., Plantec, J., Thomas, J., Jessel, J.: Serious games scenario modeling for non-experts. In: CSEDU 2015 – Proceedings of the 7th International Conference on Computer Supported Education, Lisbon, vol. 1, pp. 474–479, 23–25 May 2015

    Google Scholar 

  • Fischler, I.S., Kaschub, C.E., Lizdas, D.E., Lampotang, S.: Understanding of anesthesia machine function is enhanced with a transparent reality simulation. Simul. Healthc. 3 (1), 26–32 (2008)

    CrossRef  Google Scholar 

  • Fried, M.P., Satava, R., Weghorst, S., Gallagher, A., Sasaki, C., Ross, D., Sinanan, M., Cuellar, H., Uribe, J.I., Zeltsan, M., et al.: The use of surgical simulators to reduce errors. Adv. Patient Saf. 4 (2005)

    Google Scholar 

  • Gaba, D.M.: The future vision of simulation in health care. Qual. Saf. Health Care 13 (suppl 1), i2–i10 (2004)

    CrossRef  Google Scholar 

  • Gallagher, A.G., Ritter, E.M., Champion, H., Higgins, G., Fried, M.P., Moses, G., Smith, C.D., Satava, R.M.: Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann. Surg. 241 (2), 364–372 (2005)

    CrossRef  Google Scholar 

  • Gibson, J.J.: The Ecological Approach to Visual Perception. Houghton, Mifflin and Company, Boston (1979)

    Google Scholar 

  • Göbel, S., Mehm, F., Radke, S., Steinmetz, R.: 80days: adaptive digital storytelling for digital educational games. In: Proceedings of the 2nd International Workshop on Story-Telling and Educational Games (STEG’09), Aachen, vol. 498 (2009)

    Google Scholar 

  • Graafland, M., Schraagen, J., Schijven, M.P.: Systematic review of serious games for medical education and surgical skills training. Br. J. Surg. 99 (10), 1322–1330 (2012)

    CrossRef  Google Scholar 

  • Hennigan, B.: Making the case for NLP in dialogue systems for serious games. In: 8th International Conference on Natural Language Processing (JapTAL), 1st Workshop on Games and NLP, Kanazawa (2012)

    Google Scholar 

  • Hughes, C.E., Moshell, J.M.: Shared virtual worlds for education: the ExploreNet experiment. Multimed. Syst. 5 (2), 145–154 (1997)

    CrossRef  Google Scholar 

  • Johnson, M.T., Clary, M.: A second life virtual clinic for medical student training. In: Second Life Education Community Conference (SLEDcc’08), Tampa (2008)

    Google Scholar 

  • Kallmann, M., Thalmann, D.: Modeling behaviors of interactive objects for real-time virtual environments. J. Vis. Lang. Comput. 13 (2), 177–195 (2002)

    CrossRef  Google Scholar 

  • Kohn, L., Corrigan, J., Donaldson, M.: To Err Is Human: Building a Safer Health System. National Academy Press, Washington, DC (2000)

    Google Scholar 

  • Kwaan, M.R., Studdert, D.M., Zinner, M.J., Gawande, A.A.: Incidence, patterns, and prevention of wrong-site surgery. Arch. Surg. 141 (4), 353–358 (2006)

    CrossRef  Google Scholar 

  • Lagarrigue, P., Lubrano, V., Minville, V., Pons Lelardeux, C.: The 3dvor project. http://3dvor.univ-jfc.fr/, Online (2013)

  • Lave, J., Wenger, E.: Situated Learning: Legitimate Peripheral Participation. Cambridge University Press, Cambridge/New York (1991)

    CrossRef  Google Scholar 

  • Lee, A., Berge, Z.L.: Second life in healthcare education: virtual environment’s potential to improve patient safety. Knowl. Manag. E-Learning: Int. J. (KM&EL) 3 (1), 17–23 (2011)

    Google Scholar 

  • Lee, C., Liu, A., Del Castillo, S., Bowyer, M., Alverson, D., Muniz, G., Caudell, T.: Towards an immersive virtual environment for medical team training. Stud. Health Technol. Inform. 125, 274–279 (2006)

    Google Scholar 

  • Lingard, L., Espin, S., Whyte, S., Regehr, G., Baker, G., Reznick, R., Bohnen, J., Orser, B., Doran, D., Grober, E.: Communication failures in the operating room: an observational classification of recurrent types and effects. Qual. Saf. Health Care 13 (5), 330–334 (2004)

    CrossRef  Google Scholar 

  • Mathieu, P., Panzoli, D., Picault, S.: Virtual customers in an agent world. In: 10th International Conference on Practical Application of Agent and Multi-Agent Systems (PAAMS’12), Salamanca, vol. 155, pp. 147–152. University of Salamanca, Salamanca, 28–30 Mar 2012

    Google Scholar 

  • Mateas, M., Stern, A.: Natural language understanding in façade: surfacetext processing. In: Proceedings of the Conference on Technologies for Interactive Digital Storytelling and Entertainment (TIDSE), Darmstadt (2004)

    Google Scholar 

  • Michael, D.R., Chen, S.L.: Serious Games: Games that Educate, Train, and Inform. Muska & Lipman/Premier-Trade (2005)

    Google Scholar 

  • Mori, D., Berta, R., De Gloria, A., Fiore, V., Magnani, L.: An easy to author dialogue management system for serious games. J. Comput. Cult. Herit. (JOCCH) 6 (2), 10 (2013)

    Google Scholar 

  • Morningstar, C., Farmer, R.F.: The lessons of Lucasfilm’s Habitat. In: Benedikt, M. (ed.) The First International Conference on Cyberspace, Austin, May 1990

    Google Scholar 

  • Murphy, J., Cremonini, F., Kane, G., Dunn, W.: Is simulation based medicine training the future of clinical medicine? Eur. Rev. Med. Pharmacol. Sci. 11 (1), 1 (2007)

    Google Scholar 

  • Orkin, J., Roy, D.: The restaurant game: learning social behavior and language from thousands of players online. J. Game Dev. 3 (1), 39–60 (2007)

    Google Scholar 

  • Parvati, D., Heinrichs, W.L., Patricia, Y.: CliniSpaceTM: a multiperson 3d online immersive training environment accessible through a browser. Med. Meets Virtual Real. 18: NextMed 163, 173 (2011)

    Google Scholar 

  • Pons Lelardeux, C., Panzoli, D., Lubrano, V., Minville, V., Jean-Pierre, J., Lagarrigue, P.: Communication system and team situation awareness in a multiplayer real-time learning environment: application to a virtual operating room. Vis. Comput. (2016, to appear)

    Google Scholar 

  • Porteous, J., Cavazza, M., Charles, F.: Applying planning to interactive storytelling: narrative control using state constraints. ACM Trans. Intell. Syst. Technol. 1 (2), 10:1–10:21 (2010)

    Google Scholar 

  • Prensky, M., Prensky, M.: Digital Game-Based Learning, vol. 1. Paragon House, St. Paul (2007)

    Google Scholar 

  • Rattner, G.N.: The king’s midwife. A history and mystery of madame du Coudray. University of California Press, Berkeley (1998)

    Google Scholar 

  • Reason, J.T., Reason, J.T.: Managing the risks of organizational accidents, vol. 6. Ashgate Aldershot, Brookfield (1997)

    Google Scholar 

  • Riedl, M.O., Li, B., Ai, H., Ram, A.: Robust and authorable multiplayer storytelling experiences. In: Proceedings of the 7th Annual Conference on Artificial Intelligence and Interactive Digital Entertainment, Palo Alto (2011)

    Google Scholar 

  • Rosen, K.R.: The history of medical simulation. J. Crit. Care 23 (2), 157–166 (2008)

    CrossRef  Google Scholar 

  • Rus, V., D’Mello, S., Hu, X., Graesser, A.: Recent advances in conversational intelligent tutoring systems. AI Mag. 34 (3), 42–54 (2013)

    Google Scholar 

  • Sanchez, S., Balet, O., Luga, H., Duthen, Y.: Autonomous virtual actors. In: 2nd International Conference on Technologies for Interactive Digital Storytelling and Entertainment, Darmstadt, 24/06/2004-26/06/2004. LNCS, pp. 68–78. Springer (juin 2004)

    Google Scholar 

  • Sanselone, M., Sanchez, S., Sanza, C., Panzoli, D., Duthen, Y.: Control of non-playing characters in a medical learning game with Monte Carlo Tree Search (regular paper). In: IEEE Conference on Computational Intelligence and Games, Dortmund, 26/08/2014-29/08/2014, pp. 208–215. http://www.computer.org. IEEE Computer Society (ao. 2014)

  • Satava, R.M.: Medical virtual reality. the current status of the future. Stud. Health Technol. Inform. 29, 100 (1996)

    Google Scholar 

  • Sharma, M., Ontañón, S., Mehta, M., Ram, A.: Drama management and player modeling for interactive fiction games. Comput. Intell. 26 (2), 183–211 (2010)

    MathSciNet  CrossRef  Google Scholar 

  • Taekman, J.M., Segall, N., Hobbs, E., Wright, M.: 3diteams–healthcare team training in a virtual environment. Anesthesiology 107 (A2145), A2145 (2007)

    Google Scholar 

  • Thomas, D., Vlacic, L.: Collaborative decision making amongst human and artificial beings. In: Intelligent Decision Making: An AI-Based Approach, pp. 97–133. Springer, Berlin (2008)

    Google Scholar 

  • von Websky, M.W., Vitz, M., Raptis, D.A., Rosenthal, R., Clavien, P., Hahnloser, D.: Basic laparoscopic training using the simbionix lap mentor: setting the standards in the novice group. J. Surg. Educ. 69 (4), 459–467 (2012)

    CrossRef  Google Scholar 

  • Warburton, S.: Second life in higher education: Assessing the potential for and the barriers to deploying virtual worlds in learning and teaching. Br. J. Educ. Technol. 40 (3), 414–426 (2009)

    CrossRef  Google Scholar 

  • Waters, A., Bassendowski, S., Petrucka, P.: Serious games for students in healthcare: engaging a technically inclined generation. Can. J. Nurs. Inf. 3 (4), 16–27 (2008)

    Google Scholar 

  • Ziv, A., Small, S.D., Wolpe, P.W.: Patient safety and simulation-based medical education. Med. Teach. 22 (5), 489–495 (2000)

    CrossRef  Google Scholar 

Download references

Acknowledgements

3DVOR is supported by the 12th innovation cluster French funding scheme “Fonds Unique Interministériel” (FUI). 3DVOR is a collaborative research project between KTM Advance, Novamotion, the University of Toulouse, Toulouse Hospital and the University JF Champollion. The steering committee of 3DVOR is composed of Pr. Pierre Lagarrigue, M.D. Ph.D. Vincent Lubrano, M.D. Ph.D. Vincent Minville and Catherine Pons-Lelardeux. The scenarios used in the game were designed by Thomas Rodsphon and Vincent Lubrano from Toulouse University Hospital. The experiment described in this chapter has been conducted under the supervision of Christiane Paban, Hoang-Minh Truong and Amélie Lafitte. The authors are also grateful to the designers and programmers who have contributed to the game: Cyrielle Guimbal, Jules de Guglielmi and Romain Régis.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to David Panzoli .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Panzoli, D., Lelardeux, C.P., Galaup, M., Lagarrigue, P., Minville, V., Lubrano, V. (2017). Interaction and Communication in an Immersive Learning Game: The Challenges of Modelling Real-Time Collaboration in a Virtual Operating Room. In: Ma, M., Oikonomou, A. (eds) Serious Games and Edutainment Applications . Springer, Cham. https://doi.org/10.1007/978-3-319-51645-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-51645-5_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-51643-1

  • Online ISBN: 978-3-319-51645-5

  • eBook Packages: Computer ScienceComputer Science (R0)