Skip to main content

Advertisement

SpringerLink
Log in

Communication system and team situation awareness in a multiplayer real-time learning environment: application to a virtual operating room

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Digital multi-player learning games are believed to represent an important step forward in risk management training, especially related to human factors, where they are trusted to improve the performance of a team of learners in reducing serious adverse events, near-misses and crashes in complex socio-technical systems. Team situation awareness is one of the critical factors that can lead the team to consider the situation with an erroneous mental representation. Then, inadequate decisions are likely to be made regarding the actual situation. This paper describes an innovative communication system designed to be used in digital learning games. The system aims at enabling the learners to share information and build a common representation of the situation to help them take appropriate actions, anticipate failures, identify, reduce or correct errors. This innovative system is neither based on voice-chat nor branching dialogues, but on the idea that pieces of information can be manipulated as tangible objects in a virtual environment. To that end, it provides a handful of graphic interactions allowing users to collect, memorize, exchange, listen and broadcast information, ask and answer questions, debate and vote. The communication system was experimented on a healthcare training context with students and their teacher. The training scenario is set in a virtual operating room and features latent critical events (wrong-patient or wrong-side surgery). Teams have to manage such a critical situation, detect anomalies hidden in the environment and share them to make the most suitable decision. Analyzing the results demonstrated the efficacy of the communication system as per the ability for the players to actually exchange information, build a common representation of the situation and make collaborative decisions accordingly. The communication system was considered user-friendly by the users and successfully exposed lifelike behaviors such as debate, conflict or irritation. More importantly, every matter or implicit disagreement was raised while playing the game and led to an argued discussion, although eventually the right decision was not always taken by the team. So, improving the gameplay should help theplayers to manage a conflict and to make them agree on the most suitable decision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Reason, J.: A Life in Error. Ashgate Publishing Ltd, Farnham (2013)

    Google Scholar 

  2. Reason, J.: Human error: models and management. Br. Med. J. 320(7237), 768–770 (2000)

    Article  Google Scholar 

  3. Endsley, M.R., Robertson, M.M.: Situation awareness in aircraft maintenance teams. Int. J. Ind. Ergon. 26(2), 301–325 (2000)

    Article  Google Scholar 

  4. Fracker, M.L.: Attention gradients in situation awareness. In: Situational awareness in aerospace operations, pp. 6/1–6/10. Nato-Agard, Neuilly Sur Seine (1990)

  5. Sarter, N.B., Woods, D.D.: Situation awareness: a critical but ill-defined phenomenon. Int. J. Aviat. Psychol. 1(1), 45–47 (1991)

    Article  Google Scholar 

  6. Hartel, C., Smith, K., Prince, C.: Defining aircrew coordination: searching mishaps for meaning. In: Fifth international symposium on aviation psychology. UQ Business School Publications, Ohio State University, Columbus (1989)

  7. Halverson, A.L., Casey, J.T., Andersson, J., Anderson, K., Park, C., Rademaker, A., Moorman, D.: Communication failure in the operating room. Surgery 149(3), 305–310 (2011)

    Article  Google Scholar 

  8. Kohn, L.T., Corrigan, J.M., Donaldson, M.S. (eds.): To Err is Human: Building a Safer Health System. National Academies Press, Washington (2000)

    Google Scholar 

  9. Lingard, L., Espin, S., Whyte, S., Regehr, G., Baker, G., Reznick, R.: Communication failure in the operating room: observational classification of reccurent types and effects. Qual. Saf. Heathc. 13(5), 330–334 (2004)

    Article  Google Scholar 

  10. Authority, P.P.S: Pennsylvania patient safety authority 2007. Annual report, Pennsylvania (2007)

  11. Joint Commission: Improving Americas hospitals: the joint commission annual report on quality and safety. Retreived February, vol. 25 (2008)

  12. Vicente, K.: Cognitive Work Analysis. Lawrence Erlbaum Associates Inc., Mahwah (1999)

    Google Scholar 

  13. Effken, J.A.: Different lenses, improved outcomes: a new approach to the analysis and design of healthcare information systems. Int. J. Med. Inform. 65(1), 59–74 (2002)

    Article  Google Scholar 

  14. Plasters, C.L., Seagull, F.J., Xiao, Y.: Coordination challenges in operating-room management: an in-depth field study. In: Proceedings of AMIA annual symposium (2003)

  15. Carayon, P.: Human factors of complex sociotechnical systems. Appl. Ergon. 37(4), 525–535 (2006)

    Article  Google Scholar 

  16. Plsek, P.E., Greenhalgh, T.: The challenge of complexity in health care. Br. Med. J. 323(7313), 625 (2001)

    Article  Google Scholar 

  17. Keyton, J., Beck, S.J., Asbury, M.B.: Macrocognition a communication perspective. Theor. Issues Ergon. Sci. 11, 272–286 (2010)

    Article  Google Scholar 

  18. Keyton, J., Beck, S.J.: Perspective: examining communication as macrocognition in STS. Hum. Factors J. Hum. Factors Ergon. Soc. 52(2), 335–339 (2010)

    Article  Google Scholar 

  19. Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Hum. Factors 37(1), 32–64 (1995)

    Article  Google Scholar 

  20. Kaber, D.B., Endsley, M.R.: Team situation awareness for process control safety and performance. Process Saf. Progr. 17(1), 43–48 (1998)

    Article  Google Scholar 

  21. Mathieu, J.E., Heffner, T.S., Goodwin, G.F., Salas, E., Cannon-Bowers, J.A.: The influence of shared mental models on team process and performance. J. Appl. Psychol. 85(2), 273 (2000)

    Article  Google Scholar 

  22. Devreux, G., Faure, G., Chevalier, A., Cegarra, J., Lubrano, V., Rodsphon, T.: The role of expertise and team situation awareness in a dynamic system: the case of the operating room. In: 28th international congress of applied psychology, Paris (2014)

  23. Kolb, D.A.: Experiential Learning: Experience as the Source of Learning and Development. Pearson Education, Englewood Cliffs (1984)

    Google Scholar 

  24. Capin, T.K., Noser, H., Thalmann, D., Pandzic, I.S., Thalmann, N.M.: Virtual human representation and communication in VLNet. IEEE Comput. Graph. Appl. 2, 42–53 (1997)

    Article  Google Scholar 

  25. Egges, A., Papagiannakis, G., Magnenat-Thalmann, N.: Presence and interaction in mixed reality environments. Vis. Comput. 23(5), 317–333 (2007)

    Article  Google Scholar 

  26. Kopp, S., Wachsmuth, I.: Synthesizing multimodal utterances for conversational agents. Comput. Animat. Virtual worlds 15(1), 39–52 (2004)

    Article  Google Scholar 

  27. Ma, J., Cole, R.: Animating visible speech and facial expressions. Vis. Comput. 20(2–3), 86–105 (2004). doi:10.1007/s00371-003-0234-y. http://link.springer.com/article/10.1007/s00371-003-0234-y

  28. Grice, H.P., Cole, P., Morgan, J.L.: Syntax and semantics. Log. Conversat. 3, 41–58 (1975)

    Google Scholar 

  29. Duranti, A.: Linguistic Anthropology, 0th edn. Cambridge University Press, Cambridge (1997)

    Book  Google Scholar 

  30. Sacks, H., Schegloff, E.A., Jefferson, G.: A simplest systematics for the organization of turn-taking for conversation. Language 50(4), 696–735 (1974)

    Article  Google Scholar 

  31. Duncan Jr., S.: Toward a grammar for dyadic conversation. Semiotica 9(1), 29–46 (1973)

    Article  MathSciNet  Google Scholar 

  32. Goodwin, C.: Conversational Organization: Interaction Between Speakers and Hearers. Academic Press, New York (1981)

    Google Scholar 

  33. Herring, S.: Interactional coherence in CMC. J. Comput. Mediat. Commun. 4(4) (1999)

  34. Fraser, N.: Assessment of Interactive Systems. Handbook on Standards and Resources for Spoken Language SystemsAssessment of Interactive Systems Handbook on Standards and Resources for Spoken Language Systems, pp. 564–615. Mouton de Gruyter, Berlin (1997)

    Google Scholar 

  35. Tear, M.F.M.: Spoken dialogue technology: enabling the conversationnal user interface. In: ACM Computing Surveys, vol. 34 (2002)

  36. Fraser, N.M., Gilbert, G.N.: Simulating speech systems. Comput. Speech Lang. 5(1), 81–99 (1991)

    Article  Google Scholar 

  37. Navarathna, R., Lucey, P., Dean, D., Fookes, C., Sridharan, S.: Lip detection for audio-visual speech recognition in-car environment. In: Proceedings of the 10th international conference on information science, signal processing and their applications, pp. 598–601. IEEE (2010)

  38. Lipovic, I.: Speech and Language Technologies. InTech Open Access Publisher (2011)

  39. Mirzaei, M.R., Ghorshi, S., Mortazavi, M.: Audio-visual speech recognition techniques in augmented reality environments. Vis. Comput. 30(3), 245–257 (2013). doi:10.1007/s00371-013-0841-1

    Article  Google Scholar 

  40. Lelardeux, C.: Healthcare Games and the Metaphoric Approach. In: Serious games for healthcare: applications and implications: applications and implications, pp. 24–49. IGI Global (2012)

  41. Michael, D.R., Chen, S.L.: Serious Games: Games that Educate, Train, and Inform. Muska & Lipman/Premier-Trade, Boston (2005)

    Google Scholar 

  42. Lelardeux, C.P., Galaup, M., Segonds, F., Lagarrigue, P.: Didactic study of a learning game to teach mechanical engineering. Proc. Eng. 132, 242–250 (2015)

    Article  Google Scholar 

  43. Zichermann, G., Cunningham, C.: Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps. O’Reilly Media, Inc., Sebastopol (2011)

    Google Scholar 

  44. Potier, V., Lelardeux, C.P., Lalanne, M., Lagarrigue, P.: Making complexity fun—machining procedures in mechanical engineering. Gamification Eng. Educ. 1, 1–10 (2016)

    Google Scholar 

  45. Daesign: Renault Academy (2009)

  46. Disney Stars: The virtual sell (2012)

  47. Rilling, S., Wechselberger, U.: A framework to meet didactical requirements for serious game design. Vis. Comput. 27(4), 287–297 (2011). doi:10.1007/s00371-011-0550-6

    Article  Google Scholar 

  48. Csikszentmihalyi, M., Abuhamdeh, S., Nakamura, J.: Flow. In: Elliot, A.J., Dweck, C.S. (eds.) Handbook of competence and motivation, pp. 598–609. Guilford Press, New York (2005)

    Google Scholar 

  49. Sanselone, M., Sanchez, S., Sanza, C., Panzoli, D., Duthen, Y.: Control of non-playing characters in a medical learning game with Monte Carlo Tree Search (regular paper). In: IEEE conference on computational intelligence and games, pp. 208–215. IEEE Computer Society, Dortmund (2014)

  50. Thomas, D., Vlacic, L.: Collaborative decision making amongst human and artificial beings. In: Intelligent decision making: an AI-based approach, pp. 97–133. Springer (2008)

  51. Lepper, M.R., Malone, T.W.: Intrinsic motivation and instructional effectiveness in computer-based education. Aptit. Learn. Instruct. 3, 255–286 (1987)

    Google Scholar 

  52. Malone, T.W.: What makes things fun to learn? Heuristics for designing instructional computer games. In: Proceedings of the 3rd ACM SIGSMALL symposium and the first SIGPC symposium on small systems, pp. 162–169. ACM, New York (1980)

  53. Malone, T.W., Lepper, M.R.: Making learning fun: a taxonomy of intrinsic motivations for learning. Aptit. Learn. Instruct. 3(1987), 223–253 (1987)

    Google Scholar 

  54. Godden, D.R., Baddeley, A.D.: Context-dependent memory in two natural environments: on land and underwater. Br. J. Psychol. 66(3), 325–331 (1975)

    Article  Google Scholar 

  55. Cowan, B., Rojas, D., Kapralos, B., Moussa, F., Dubrowski, A.: Effects of sound on visual realism perception and task performance. Vis. Comput. 31(9), 1207–1216 (2014). doi:10.1007/s00371-014-1006-6

    Article  Google Scholar 

  56. Poling, N.D.: Collaboration, teamwork, and team cohesion in a starcraft 2 digital game-based course. Ph.D. thesis, University of Florida (2013)

  57. Lagarrigue, P., Lubrano, V., Minville, V., Pons-Lelardeux, C.: The 3dvor project (2012). http://3dvor.univ-jfc.fr/

  58. Chin, T.J., You, Y., Coutrix, C., Lim, J.H., Chevallet, J.P., Nigay, L.: Mobile phone-based mixed reality. Snap2play Game 25(1), 25–37 (2009)

    Google Scholar 

  59. Liarokapis, F., Macan, L., Malone, G., Rebolledo-Mendez, G., Freitas, S.D.: Multimodal augmented reality tangible gaming. Vis. Comput. 25(12), 1109–1120 (2009). doi:10.1007/s00371-009-0388-3. http://link.springer.com/article/10.1007/s00371-009-0388-3

  60. Pandzic, I.S., Ostermann, J., Millen, D.: User evaluation: synthetic talking faces for interactive services. Vis. Comput. 15(7), 330–340 (1999)

    Article  Google Scholar 

  61. Parvati, D., Heinrichs, W.L.: CliniSpace: A multiperson 3d online immersive training environment accessible through a browser. Medicine Meets Virtual Reality 18: NextMed 163, 173 (2011)

  62. Taekman, J.M., Segall, N., Hobbs, E., Wright, M.: 3diteamsHealthcare team training in a virtual environment. Anesthesiology 107(A2145), A2145 (2007)

    Google Scholar 

  63. Mateas, M., Stern, A.: Natural language understanding in faade: surface text processing. In: Proceedings of the conference on technologies for interactive digital storytelling and entertainment (TIDSE) (2004)

  64. Cassell, J.: Embodied conversational agents. MIT press, Cambridge (2000)

    Google Scholar 

  65. Rus, V., D’mello, S., Hu, X., Graesser, A.: Recent advances in conversational intelligent tutoring systems. AI Mag. 34(3), 42–54 (2013)

    Google Scholar 

  66. Hennigan, B.: Making the case for NLP in dialogue systems for serious games. In: 8th international conference on natural language processing (JapTAL), 1st workshop on games and NLP (2012)

  67. Morningstar, C., Farmer, R.F.: The Lessons of Lucasfilm’s Habitat. In: Benedikt, M. (ed.) The first international conference on cyberspace. University of Texas, Austin (1990)

    Google Scholar 

  68. Johnson, M.T., Clary, M.: A second life virtual clinic for medical student training. Second Life Education Community Conference (SLEDcc’08) (2008)

  69. Salas, E., Prince, C., David, Baker, Shrestha, L.: Situation awareness in team performance. Implic. Meas. Train. 37(1), 123–136 (1995)

    Google Scholar 

  70. Leckie, G.J., Pettigrew, K.E., Sylvain, C.: Modeling the information seeking of professionals: a general model derived from research on engineers, health care professionals, and lawyers. The Library Quarterly, pp. 161–193 (1996)

  71. Reddy, M., Jansen, B., Spence, P.: collaborative information behavior: exploring collaboration and coordination during information seeking and retrieval activities. In: Collaborative information behavior: user engagement and communication sharing. IGI Global, Hershey (2010)

  72. Reddy, M., Bernard, J.J.: A model for understanding collaborative information behavior in context: a study of two healthcare teams (2006)

  73. Panzoli, D., Sanselone, M., Sanchez, S., Sanza, C., Lelardeux, C., Lagarrigue, P., Duthen, Y.: Introducing a design methodology for multi-character collaboration in immersive learning games (regular paper). In: Proceedings of the sixth international conference on virtual worlds and games for serious applications (VS-Games14), p. (electronic medium). IEEExplore digital library, University of Malta (2014)

  74. WHO: Surgical safety checklist (2009)

  75. Authority, P.P.S.: Pennsylvania Patient Safety Authority 2012. Annual report 2012, Pennsylvania (2012)

  76. Seiden, S.C., Barach, P.: Wrong-side/wrong-site, wrong-procedure, and wrong-patient adverse events: are they preventable? Arch. Surg. 141(9), 931–939 (2006)

    Article  Google Scholar 

  77. Busemann, A., Heidecke, C.D.: Safety checklists in the operating room. Dtsch. rztebl. Int. 109(42), 693–694 (2012). doi:10.3238/arztebl.2012.0693. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3489073/

Download references

Acknowledgements

The steering committee of 3DVOR is composed of Pr. Pierre Lagarrigue, M.D. Ph.D. Vincent Lubrano, M.D. Ph.D. Vincent Minville and Catherine Pons-Lelardeux. The following authors are also grateful to contributors to the project 3D operating room: Thomas Rodsphon, Cyrielle Guimbal, Michel Galaup and Jules de Guglielmi. The experiment described in Sect. 6.3 has been conducted under the supervision of Christiane Paban (teacher) and two students of the anesthetist nurse school of Toulouse: Hoang and Amelie. These works are part of a global national innovative IT program whose partners are KTM Advance company, Novamotion company, Serious Game Research Network and University Hospital of Toulouse (France). This R&D project is supported by French National Funding : Bpifrance Financement.

Author information

Authors and Affiliations

  1. IRIT, University of Toulouse, INU Champollion, Serious Game Research Network, Toulouse, France

    Catherine Pons Lelardeux & David Panzoli

  2. ToNIC, University of Toulouse, UPS, University Hospital, Inserm, Serious Game Research Network, Toulouse, France

    Vincent Lubrano

  3. MATN, University of Toulouse, UPS, University Hospital, Serious Game Research Network, Toulouse, France

    Vincent Minville

  4. ICA, University of Toulouse, INU Champollion, Serious Game Research Network, Toulouse, France

    Pierre Lagarrigue

  5. IRIT, University of Toulouse, UPS, Serious Game Research Network, Toulouse, France

    Jean-Pierre Jessel

Authors
  1. Catherine Pons Lelardeux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Panzoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vincent Lubrano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vincent Minville
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pierre Lagarrigue
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jean-Pierre Jessel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Catherine Pons Lelardeux.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pons Lelardeux, C., Panzoli, D., Lubrano, V. et al. Communication system and team situation awareness in a multiplayer real-time learning environment: application to a virtual operating room. Vis Comput 33, 489–515 (2017). https://doi.org/10.1007/s00371-016-1280-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-016-1280-6

Keywords

Search

Navigation