Skip to main content
SpringerLink
Log in

Developing a scenario-based video game generation framework for computer and virtual reality environments: a comparative usability study

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

Serious games—games that have additional purposes rather than only entertainment—aim to educate people, solve, and plan several real-life tasks and circumstances in an interactive, efficient, and user-friendly way. Emergency training and planning provide structured curricula, rule-based action items, and interdisciplinary collaborative entities to imitate and teach real-life tasks. This rule-based structure enables the curricula to be transferred into other systematic learning platforms. Although emergency training includes these highly structured and repetitive action responses, a general framework to map the training scenarios’ actions, roles, and collaborative structures to serious games’ game mechanics and game dialogues, is still not available. To address this issue, in this study, a scenario-based game generator, which maps domain-oriented tasks to game rules and game mechanics, was developed. Also, two serious games (i.e., Hospital game and BioGarden game) addressing the training mechanisms of Chemical, Biological, Radiological, Nuclear, and Explosives (CBRNe) domain, were developed by both the game developers and the scenario-based game generator for comparative analysis. Finally, the outcomes of these games were mapped to the virtual reality environment to provide a thorough training program. To test the usability, immersion, presence, and technology acceptance aspects of the proposed game generator’s outcomes, 15 game developer participants tested a complete set of games and answered the questionnaires of the corresponding phenomenon. The results show that although the game generator has higher CPU time and memory usage, it highly outperforms the game development pipeline performance of the game developers and provides usable and immersive games. Thus, this study provides a promising game generator which bridges the CBRNe practitioners and game developers to transform real-life training scenarios into video games efficiently and quickly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Santos R, Magalhaes C, Capretz LF, Neto JC, da Silva FQB, Saher A (2018) Computer games are serious business and so is their quality: particularities of software testing in game development from the perspective of practitioners. arXiv preprint arXiv:1812.05164

  2. Michael DR, Chen SL (2005) Serious games: games that educate, train, and inform. Muska & Lipman/Premier-Trade, Dubai

    Google Scholar 

  3. Pirovano M, Surer E, Mainetti R, Lanzi PL, Borghese NA (2016) Exergaming and rehabilitation: a methodology for the design of effective and safe therapeutic exergames. Entertain. Comput. 14:55–65

    Article  Google Scholar 

  4. Susi T, Johannesson M, Backlund P (2007) Serious games: an overview. University of Skövde, Skovde

    Google Scholar 

  5. Crookall D (2010) Serious games, debriefing, and simulation/gaming as a discipline. Simulat. Gaming 41(6):898–920

    Article  Google Scholar 

  6. Tinati R, Luczak-Roesch M, Hall W (2017) An investigation of player motivations in Eyewire, a gamified citizen science project. Comput Hum Behav 73:527–540

    Article  Google Scholar 

  7. Curtis V (2015) Motivation to participate in an online citizen science game: a study of Foldit. Sci Commun 37(6):723–746

    Article  Google Scholar 

  8. Dorval M, Pepin M (1986) Effect of playing a video game on a measure of spatial visualization. Perceptual Motor Skills 62(1):159–162

    Article  Google Scholar 

  9. Nelson R, Strachan I (2009) Action and puzzle video games prime different speed/accuracy tradeoffs. Perception 38(11):1678–1687

    Article  Google Scholar 

  10. Tobler-Ammann BC, Surer E, de Bruin ED, Rabuffetti M, Borghese NA, Mainetti R, Pirovano M, Wittwer L, Knols RH (2017) Exergames encouraging exploration of hemineglected space in stroke patients with visuospatial neglect: a feasibility study. JMIR Serious Games 5(3):17

    Article  Google Scholar 

  11. Tobler-Ammann BC, Surer E, Knols RH, Borghese NA, de Bruin ED (2017) User perspectives on exergames designed to explore the hemineglected space for stroke patients with visuospatial neglect: usability study. JMIR Serious Games 5(3):e18

    Article  Google Scholar 

  12. Milgram P, Takemura H, Utsumi A, Kishino F (1995) Augmented reality: a class of displays on the reality-virtuality continuum, Telemanipulator and Telepresence Technologies

  13. XVR Simulation | Incident command training tool for safety and security (2019). Retrieved from https://www.xvrsim.com/en-cn/

  14. Djaouti D, Alvarez J, Jessel J, Rampnoux O (2011) Origins of Serious Games. In: Ma M, Oikonomou A, Jain LC (eds) Serious games and edutainment applications. Springer, London, pp 25–43

    Chapter  Google Scholar 

  15. Abt C (1970) Serious games. University Press of Americak, Lanham

    Google Scholar 

  16. Laamarti F, Eid M, El Saddik A (2014) An overview of serious games. Int J Comput Games Technol 2014:1–15

    Article  Google Scholar 

  17. Michael D, Chen S (2006) Serious games: games that educate, train, and inform. Thomson Course Technology PTR, Boston

    Google Scholar 

  18. Heldal I (2016) Simulation and serious games in emergency management: experiences from two case studies, 22nd International Conference on Virtual System & Multimedia (VSMM), pp. 1-9. IEEE

  19. Lukosch H, van Ruijven T, Verbraeck A (2012) The participatory design of a simulation training game. In: Proceedings of the winter simulation conference, winter simulation conference

  20. Ingrassia PL, Ragazzoni L, Carenzo L, Barra FL, Colombo D, Gugliotta G, Corte FD (2012) Virtual reality and live scenario simulation: options for training medical students in mass casualty incident triage. Crit Care 16(1):P479

    Article  Google Scholar 

  21. Ragazzoni L, Ingrassia PL, Echeverri L, Maccapani F, Berryman L, Burkle FM, Corte FD (2015) Virtual reality simulation training for Ebola deployment. Disaster Med Public Health Preparedness 9(5):543–546

    Article  Google Scholar 

  22. Ferdinandus GR (2012) Automated scenario generation, coupling planning techniques with smart objects, Master’s thesis

  23. Niehaus J, Riedl MO (2009) Scenario adaptation: an approach to customizing computer-based training games and simulations. In: Proceedings of the AIED 2009 workshop on intelligent educational games, vol 3, pp 89–98

  24. Unity Asset Store - Storyteller (2020) Retrieved from https://assetstore.unity.com/packages/tools/utilities/storyteller-97715

  25. Hjelseth S, Morrison A, Nordby K (2015) Design and computer simulated user scenarios: exploring real-time 3D game engines and simulation in the maritime sector

  26. User Expectations Questionnaire for eNotice Serious Game (2019) Retrieved from http://ec.europa.eu/eusurvey/runner/eNoticeSeriousGamePreDevelopment

  27. European Network Of CBRN TraIning CEnters—Official Website (2019) Retrieved from https://www.h2020-enotice.eu

  28. Surer E, Atalay TB, Demirkan DÇ, Düzgün HŞ (2019) Serious gaming in CBRNe domain: a survey on user expectations, concerns and suggestions. In: 3rd international conference CBRNE—research & innovation, Nantes, France, pp 43

  29. Garrett J (2011) The elements of user experience. New Riders, Berkeley

    Google Scholar 

  30. Witmer B, Singer M (1998) Measuring presence in virtual environments: a presence questionnaire. Presence Teleoper Virt Environ 7(3):225–240

    Article  Google Scholar 

  31. Davis F (1986) A technology acceptance model for empirically testing new end-user information systems: theory and results, Doctoral dissertation, MIT Sloan School of Management, Cambridge, MA

  32. Venkatesh V, Davis F (1996) A model of the antecedents of perceived ease of use: development and test. Decis Sci 27(3):451–481

    Article  Google Scholar 

  33. Brooke J (1996) SUS-A quick and dirty usability scale, usability evaluation in industry. CRC Press, Boca Raton

    Google Scholar 

  34. Surer E, Erkayaoglu M, Oztürk ZN, Yücel F, Bıyık EA, Altan B, Senderin B, Oguz Z, Gürer S, Düzgün HS (2019) Developing a scenario-based video game generation framework: preliminary results. In: 15th international summer workshop on multimodal interfaces, pp 19

  35. Valve Corporation (2020) Retrieved from https://www.valvesoftware.com/en/

  36. Bjork S, Holopainen J (2005) Patterns in game design. Charles River Media, Hingham

    Google Scholar 

  37. Unity Asset Store: Hospital LowPoly (2019) Retrieved from https://assetstore.unity.com/packages/3d/environments/hospital-lowpoly-82552

  38. Unity Asset Store: low poly laboratory pack (2019) Retrieved from https://assetstore.unity.com/packages/3d/environments/low-poly-laboratory-pack-47677

  39. Hollan J, Hutchins E, Kirsh D (2000) Distributed cognition: toward a new foundation for human-computer interaction research. ACM Trans Comput Hum Interact 7(2):174–196

    Article  Google Scholar 

  40. Hamilton P, Wigdor DJ (2014) Conductor: enabling and understanding cross-device interaction. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp 2773–2782

  41. Villaroman N, Rowe D, Swan B (2011) Teaching natural user interaction using OpenNI and the Microsoft Kinect sensor. In: Proceedings of the 2011 conference on Information technology education, pp 227–232

  42. Markus F, Kritzler M, Michahelles F (2017) HoloLens is more than air Tap: natural and intuitive interaction with holograms. In: Proceedings of the seventh international conference on the internet of things, pp 1–2

  43. Ergun O, Akin S, Dino IG, Surer E (2019) Architectural design in virtual reality and mixed reality environments: a comparative analysis. In: 2019 IEEE conference on virtual reality and 3D user interfaces (VR). IEEE, pp 914–915

  44. Furht B (ed) (2008) Encyclopedia of multimedia. Springer, Berlin

    Google Scholar 

  45. Rendering Profiler: Unity Documentation (2019) Retrieved from https://docs.unity3d.com/560/Documentation/Manual/ProfilerRendering.html

  46. Surer E, Erkayaoğlu M, Öztürk ZN, Yücel F, Bıyık EA, Altan B, Şenderin B, Oğuz Z, Gürer S, Düzgün HŞ (2019) Developing a Scenario-Based Video Game Generation Framework: Preliminary Results, arXiv preprint arXiv:1911.07380

  47. Bangor A, Kortum PT, Miller JT (2008) An empirical evaluation of the system usability scale. Intl J Hum Comput Interact 24(6):574–94

    Article  Google Scholar 

  48. Bangor A, Kortum PT, Miller JT (2009) Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 4(3):114–23

    Google Scholar 

  49. Brooke J (2013) SUS: a retrospective. J Usability Stud 8(2):29–40

    Google Scholar 

  50. Fishbein M, Ajzen I (1980) Understanding attitudes and predicting social behavior. Prentice-Hall, Upper Saddle River

    Google Scholar 

  51. Tavakol M, Dennick R (2011) Making sense of Cronbach’s alpha. Int J Med Edu 2:53

    Article  Google Scholar 

  52. Dyer R, Matthews JJ, Stulac JF, Wright CE, Yudowitch K (1976) Questionnaire construction manual, annex literature survey and bibliography. Operations Research Associates, Palo Alto

    Book  Google Scholar 

Download references

Acknowledgements

This study was developed during the 15th Summer Workshop on Multimodal Interfaces (eNTERFACE’19) which was held at Bilkent University, Ankara, Turkey between the dates of July 8 and August 2, 2019. This framework is fully supported by European Network Of CBRN TraIning Centers (eNOTICE) project funded under EU H2020 (Project ID: 740521). The authors would like to thank Dr. Olga Vybornova (Center for Applied Molecular Technologies, Université Catholique de Louvain, Brussels, Belgium) and Prof. Gilles Dusserre (IMT Mines Alès, Alès, Languedoc Roussillon, France) for their help in scenarios of BioGarden and Hospital games, respectively. The authors would also like to thank Oğuzcan Ergün (Multimedia Informatics, Middle East Technical University, Ankara, Turkey) for his help and feedback on VR instrumentation.

Author information

Authors and Affiliations

  1. Department of Modeling and Simulation, Graduate School of Informatics, Middle East Technical University, 06800, Ankara, Turkey

    Elif Surer, Furkan Yücel, Emin Alp Bıyık, Burak Altan & Büşra Şenderin

  2. Department of Mining Engineering, Middle East Technical University, 06800, Ankara, Turkey

    Mustafa Erkayaoğlu & Servet Gürer

  3. Department of Computer Engineering, Bilkent University, 06800, Ankara, Turkey

    Zeynep Nur Öztürk

  4. Department of Psychology, Bilkent University, 06800, Ankara, Turkey

    Zeliha Oğuz

  5. Colorado School of Mines, 1500 Illinois Street, 80401, Golden, Colorado, USA

    H. Şebnem Düzgün

Authors
  1. Elif Surer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mustafa Erkayaoğlu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zeynep Nur Öztürk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Furkan Yücel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Emin Alp Bıyık
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Burak Altan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Büşra Şenderin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Zeliha Oğuz
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Servet Gürer
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. H. Şebnem Düzgün
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elif Surer.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This framework is fully supported by European Network Of CBRN TraIning Centers (eNOTICE) project funded under EU H2020 (Project ID: 740521)

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 80 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Surer, E., Erkayaoğlu, M., Öztürk, Z.N. et al. Developing a scenario-based video game generation framework for computer and virtual reality environments: a comparative usability study. J Multimodal User Interfaces 15, 393–411 (2021). https://doi.org/10.1007/s12193-020-00348-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-020-00348-6

Keywords

Search

Navigation