Skip to main content
SpringerLink
Log in

Immersive visual scripting based on VR software design patterns for experiential training

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Virtual reality (VR) has re-emerged as a low-cost, highly accessible consumer product, and training on simulators is rapidly becoming standard in many industrial sectors. However, the available systems are either focusing on gaming context, featuring limited capabilities or they support only content creation of virtual environments without any rapid prototyping and modification. In this project, we propose a code-free, visual scripting platform to replicate gamified training scenarios through rapid prototyping and VR software design patterns. We implemented and compared two authoring tools: a) visual scripting and b) VR editor for the rapid reconstruction of VR training scenarios. Our visual scripting module is capable of generating training applications utilizing a node-based scripting system, whereas the VR editor gives user/developer the ability to customize and populate new VR training scenarios directly from the virtual environment. We also introduce action prototypes, a new software design pattern suitable to replicate behavioral tasks for VR experiences. In addition, we present the training scenegraph architecture as the main model to represent training scenarios on a modular, dynamic and highly adaptive acyclic graph based on a structured educational curriculum. Finally, a user-based evaluation of the proposed solution indicated that users—regardless of their programming expertise—can effectively use the tools to create and modify training scenarios in VR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. MARUI 3 plugin for Autodesk Maya. https://www.marui-plugin.com/marui3/. Accessed 30 July 2020

  2. Our clinical trial (citation not provided for the reviewing process)

  3. Our VR training platform (citation not provided for the reviewing process)

  4. Unity 2020 roadmap. https://unity3d.com/unity/roadmap

  5. Alsumait, A., Almusawi, Z.S.: Creative and innovative e-learning using interactive storytelling. Int. J. Pervasive Comput. Commun. 9(3), 209–226 (2013). https://doi.org/10.1108/IJPCC-07-2013-0016

    Article  Google Scholar 

  6. Andersen, S.A.W., Konge, L., Cayé-Thomasen, P., Sørensen, M.S.: Retention of mastoidectomy skills after virtual reality simulation training. JAMA Otolaryngol. Head Neck Surg. 142(7), 635–640 (2016). https://doi.org/10.1001/jamaoto.2016.0454

    Article  Google Scholar 

  7. Bouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, E., Forget, H., Loranger, C., Xavier Roucaut, F.: Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: a three-arm randomised controlled trial. Br psychiatr J Ment sci 210, (2016). https://doi.org/10.1192/bjp.bp.116.184234

  8. Ebrahimi, A., West, T., Schoen, M., Urquidi, D.: Unity: Editorvr. In: ACM SIGGRAPH 2017 Real Time Live!, SIGGRAPH ’17, pp. 27–27. ACM, New York, NY, USA (2017). https://doi.org/10.1145/3098333.3098918

  9. Ellis, T.O., Heafner, J.F., Sibley, W.L.: The grail project: an experiment in man-machine communications (RM-5999-ARPA). RAND Corporation, Santa Monica (1969)

    Google Scholar 

  10. de Faria, J.W.V., Teixeira, M.J., de Moura Sousa Júnior, L., Otoch, J.P., Figueiredo, E.G, : Virtual and stereoscopic anatomy: when virtual reality meets medical education. J. Neurosurg. JNS 125(5), 1105–1111 (2016)

  11. Ganier, F., Hoareau, C., Tisseau, J.: Evaluation of procedural learning transfer from a virtual environment to a real situation: a case study on tank maintenance training. Ergonomics 10(1080/00140139), 899628 (2014)

  12. Greenleaf, W.: How vr technology will transform healthcare. In: ACM SIGGRAPH 2016 VR Village, pp. 1–2 (2016). https://doi.org/10.1145/2929490.2956569

  13. Greenwald, S., Kulik, A., Kunert, A., Beck, S., Froehlich, B., Cobb, S., Parsons, S., Newbutt, N., Gouveia, C., Cook, C., Snyder, A., Payne, S., Holland, J., Buessing, S., Fields, G., Corning, W., Lee, V., Xia, L., Maes, P.: Technology and applications for collaborative learning in virtual reality. In: CSCL (2017)

  14. Göbel, S., Salvatore, L., Konrad, R.: Storytec: A digital storytelling platform for the authoring and experiencing of interactive and non-linear stories. In: 2008 International Conference on Automated Solutions for Cross Media Content and Multi-Channel Distribution, pp. 103–110 (2008). https://doi.org/10.1109/AXMEDIS.2008.45

  15. Kotis, K.I.: Artist–a real-time low-effort multi-entity interaction system for creating reusable and optimized MR experiences. Res. Ideas Outcomes 5, e36464 (2019). https://doi.org/10.3897/rio.5.e36464

    Article  Google Scholar 

  16. Monahan, T., McArdle, G., Bertolotto, M.: Virtual reality for collaborative e-learning. Comp. Educ. 50(4), 1339–1353 (2008). https://doi.org/10.1016/j.compedu.2006.12.008

    Article  Google Scholar 

  17. Murcia-López, M., Steed, A.: A comparison of virtual and physical training transfer of bimanual assembly tasks. IEEE Trans. Vis. Comp. Gr. 24(4), 1574–1583 (2018). https://doi.org/10.1109/TVCG.2018.2793638

    Article  Google Scholar 

  18. Nielsen, J.: Usability testing. In: Nielsen, J. (ed.) Usability Engineering, pp. 165–206. Morgan Kaufmann, San Diego (1993)

    Chapter  Google Scholar 

  19. Nystrom, R.: Game Programming Patterns, 3rd edn. Genever Benning (2014)

  20. Pan, X., Hamilton, A.: Why and how to use virtual reality to study human social interaction: the challenges of exploring a new research landscape. Br. J. Psychol. (2018). https://doi.org/10.1111/bjop.12290

    Article  Google Scholar 

  21. Papagiannakis, P., Trahanias, G., Kenanidis, E., Tsiridis, E.: Psychomotor Surgical Training in Virtual Reality.Master Case Series and Techniques, pp. 827–830. Adult Hip, Springer, Cham (2017)

    Google Scholar 

  22. Pasternak, E., Fenicheland, R., Marshall, A.N.: Tips for creating a block language with blockly. In: 2017 IEEE Blocks and Beyond Workshop (B B), pp. 21–24 (2017). https://doi.org/10.1109/BLOCKS.2017.8120404

  23. Pfeiffer-Leßmann, N., Pfeiffer, T.: Exprotovar: a lightweight tool for experience-focused prototyping of augmented reality applications using virtual reality. In: Stephanidis, C. (ed.) HCI International 2018—Posters’ Extended Abstracts, pp. 311–318. Springer International Publishing, Cham (2018)

    Chapter  Google Scholar 

  24. Roque, R.: Openblocks : an extendable framework for graphical block programming systems (2008)

  25. Slater, M.: Implicit Learning Through Embodiment in Immersive Virtual Reality, pp. 19–33. Springer, Singapore (2017)

    Google Scholar 

  26. Stefanidi, E., Arampatzis, D., Leonidis, A., Papagiannakis, G.: BricklAyeR: A Platform for Building Rules for AmI Environments in AR, pp. 417–423 (2019)

  27. Tullis, T., Albert, W.: Measuring the User Experience, Second Edition: Collecting, Analyzing, and Presenting Usability Metrics, 2nd edn. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (2013)

    Google Scholar 

  28. Villela, R.: Working with the CodeDOM, pp. 155–177. Apress, Berkeley, CA (2019)

    Google Scholar 

Download references

Acknowledgements

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871793 (ACCORDION) and No727585 (STARS-PCP) and supported by Greek national funds (projects VRADA and vipGPU).

Author information

Authors and Affiliations

  1. ORamaVR, Heraklion, Greece

    Paul Zikas, Nick Lydatakis & Steve Kateros

  2. ORamaVR, Institute of Computer Science, Foundation for Research and Technology-Hellas, Department of Computer Science, University of Crete, Heraklion, Greece

    George Papagiannakis

  3. Institute of Computer Science Foundation for Research and Technology-Hellas, Heraklion, Greece

    Stavroula Ntoa & Ilia Adami

  4. Institute of Computer Science, Foundation for Research and Technology-Hellas, Department of Computer Science, University of Crete, Heraklion, Greece

    Constantine Stephanidis

Authors
  1. Paul Zikas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George Papagiannakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nick Lydatakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Steve Kateros
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stavroula Ntoa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ilia Adami
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Constantine Stephanidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Zikas.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zikas, P., Papagiannakis, G., Lydatakis, N. et al. Immersive visual scripting based on VR software design patterns for experiential training. Vis Comput 36, 1965–1977 (2020). https://doi.org/10.1007/s00371-020-01919-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01919-0

Keywords

Search

Navigation