Skip to main content
SpringerLink
Log in

Cost–benefit analysis of virtual reality-based training for emergency rescue workers: a socio-technical systems approach

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Virtual reality (VR) is widely recognised as a promising technology for training emergency first responders and other safety–critical workers. It is uniquely able to immerse trainees in extreme situations that are too risky or dangerous to be examined in traditional real-world safety training. Most organisations seeking to implement VR safety training often limit their decisions to financial and technological factors. However, in this paper, we argue that a socio-technical systems approach is required to better appreciate the social costs and benefits of VR training, which are important for a successful implementation. The paper also reports our own research on a real-world implementation of VR safety training for the Mine Rescue Brigades in New South Wales, Australia. The training—conducted in both fully immersive (360 VR) and non-immersive (Desktop VR) virtual reality—involved a search and rescue operation which was necessitated by an underground fire at the bottom of the transport drift in a coal mine. Following this training, the 368 trainees not only completed a post-training questionnaire, but also were interviewed, to assess their training experiences in the VR environment. The findings provide a comprehensive account of the social costs and benefits of adopting VR as a safety training tool. Overall, the trainees perceived the benefits to far outweigh the costs, with an overall high inclination to recommend the VR training to other colleagues. Desktop VR was found to be as fit for delivering successful training as the more immersive 360 VR. However, this Desktop VR generated considerably less motion sickness in trainees. These findings should help organisations and training providers decide on: (1) whether or not to invest in VR safety training solutions; (2) which type technology/method of delivery to use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Appelbaum SH (1997) Socio-technical systems theory: an intervention strategy for organizational development. Manag Decis 35:452–463

    Article  Google Scholar 

  • Ausburn LJ, Ausburn FB (2004) Desktop virtual reality: A powerful new technology for teaching and research in industrial teacher education. J Ind Teacher Educ 41(4):1–16

    Google Scholar 

  • Backlund P, Engstrom H, Hammar C, Johannesson M, Lebram M (2007) Sidh–a game based firefighter training simulation. 2007 11th International Conference Information Visualization (IV'07). IEEE, pp. 899–907

  • Bahaei SS, Gallina B (2019) Extending safe concert for modelling augmented reality-equipped socio-technical systems. In 2019 4th International Conference on System Reliability and Safety (ICSRS). IEEE, pp. 275–282

  • Bhoir S, Esmaeili B (2015) State-of-the-art review of virtual reality environment applications in construction safety, AEI

  • Blumenfeld PC (1992) Classroom learning and motivation: clarifying and expanding goal theory. J Educ Psychol 84:272–281

    Article  Google Scholar 

  • Bostrom RP, Heinen JS (1977) MIS problems and failures: a socio-technical perspective. The causes. MIS quarterly, Part I, pp 17–32

    Google Scholar 

  • Bowen Z, Jun Y, Jia G, Junbo W, Yuxin Z, Ming Y (2018) Using virtual reality for the training of physical protection system in nuclear power plant. 2018 International Conference on Power System Technology (POWERCON). IEEE, pp. 4806–4810

  • Caserman P, Cornel M, Dieter M, Göbel S (2018) A concept of a training environment for police using VR game technology. Joint International Conference on Serious Games. Springer, pp. 175–181

  • Chakraborty PR, Bise CJ (2000) A virtual-reality-based model for task-training of equipment operators in the mining industry. Miner Resours Eng 9:437–449

    Article  Google Scholar 

  • Checa D, Bustillo A (2020) Advantages and limits of virtual reality in learning processes: Briviesca in the fifteenth century. Virtual Real 24(1):151–161

    Article  Google Scholar 

  • Chen CJ, Toh SC, Fauzy WM (2004) The theoretical framework for designing desktop virtual reality-based learning environments. J Interact Learn Res 15:147

    Google Scholar 

  • Chittaro L, Corbett CL, Mclean G, Zangrando N (2018) Safety knowledge transfer through mobile virtual reality: a study of aviation life preserver donning. Saf Sci 102:159–168

    Article  Google Scholar 

  • Cordeiro C, Paludo J, Tanaka E, Domingues L, Gadbem E, Euflausino A (2015) Development of immersive virtual reality environment to train electricians to work on substations. 2015 XVII Symposium on Virtual and Augmented Reality. IEEE, pp. 142–146

  • Da Dalto L, Benus F, Balet O (2010) The use and benefits of Virtual Reality tools for the welding training. 63rd Annual Assembly & International Conference of the International Institute of Welding, Istanbul, Turkey

  • Dalgarno B, Lee MJ (2010) What are the learning affordances of 3-D virtual environments? Br J Educ Technol 41:10–32

  • Dishaw MT, Strong DM (1999) Extending the technology acceptance model with task–technology fit constructs. Inf Manag 36:9–21

  • Düking P, Holmberg H-C, Sperlich B (2018) The potential usefulness of virtual reality systems for athletes: a short SWOT analysis. Front Physiol 9:128

  • Engelbrecht H, Lindeman R, Hoermann S (2019) A SWOT Analysis of the field of virtual reality for firefighter training. Front Robot AI 6:101. https://doi.org/10.3389/frobt

    Article  Google Scholar 

  • Fernández-alvarez J, Colombo D, Botella C, García-Palacios A, Riva G (2019) Virtual eality for anxiety and stress-related disorders: a SWOT analysis. International Symposium on Pervasive Computing Paradigms for Mental Health. Springer, pp. 43–54

  • Filigenzi MT, Orr TJ, Ruff TM (1999) Mining publication: Computerized accident reconstruction and training for metal/non-metal mines. Am J Ind Med 36(S1):116–118

  • Filigenzi MT, Orr TJ, Ruff TM (2000) Virtual reality for mine safety training. Appl Occup Environ Hyg 15:465–469

    Article  Google Scholar 

  • Fox J, Arena D, Bailenson JN (2009) Virtual reality. J Media Psychol: Theories, Methods, Appl 21:95–113

    Article  Google Scholar 

  • Gagliati A, Donvito G, Gasco S, Munteanu D (2011) Immersive training in oil & gas industries. VTT Symp. 20–26

  • Gazit E, Yair Y, Chen D (2006) The gain and pain in taking the pilot seat: learning dynamics in a non immersive virtual solar system. Virtual Real 10:271–282

    Article  Google Scholar 

  • Graham CR (2006) Blended learning systems: Definition, current trends, and future directions. In: Bonk CJ, Graham CR (eds) Handbook of blended learning: Global perspectives, local designs. Pfeiffer Publishing, San Francisco, CA, pp. 1–32

    Google Scholar 

  • Gregoriades A, Pampaka M (2016) Validating sociotechnical systems requirements through immersion. In ICEIS (1) pp. 456–463

  • Hanson K, Shelton BE (2008) Design and development of virtual reality: analysis of challenges faced by educators. J Educ Technol Soc 11(1):118–131

    Google Scholar 

  • Hao Z, Hao W, Yang G (2012) Virtual reality and its application in the coal mine. Springer, Communications and Information Processing

    Book  Google Scholar 

  • Haque S, Srinivasan S (2006) A meta-analysis of the training effectiveness of virtual reality surgical simulators. IEEE Trans Inf Technol Biomed 10:51–58

    Article  Google Scholar 

  • Hatsushika D, Nagata K, Hashimoto Y (2018) Underwater VR experience system for scuba training using underwater wired HMD. OCEANS 2018 MTS/IEEE Charleston. IEEE, pp. 1–7

  • Heaton L (1998) Talking heads vs. Virtual workspaces: a comparison of design across cultures. J Inform Technol 13(4):259–272

    Article  Google Scholar 

  • Henderson JV (2005) The virtual terrorism response academy: training for high-risk, low-frequency threats. Stud Health Technol Inform 111:185–190

    Google Scholar 

  • Hoang RV, Sgambati MR, Brown TJ, Coming DS, Harris JR, F. C. (2010) VFire: Immersive wildfire simulation and visualization. Comput Graph 34:655–664

    Article  Google Scholar 

  • Hsu EB, LI Y, Bayram JD, Levinson D, Yang S, Monahan C (2013) State of virtual reality based disaster preparedness and response training. PLoS curr 5

  • Juricic I, Malaguti L, Poggi M (2015) Virtual reality of a typical ENI platform to anticipate and train for start-up, maintenance and emercency operations. Offshore Mediterranean Conference and Exhibition

  • Kavanagh S, Luxton-Reilly A, Wuensche B, Plimmer B (2017) A systematic review of virtual reality in education. Themes Sci Technol Educ 10(2):85–119

    Google Scholar 

  • Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG (1993) Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 3:203–220

    Article  Google Scholar 

  • Kinateder M, Ronchi E, Nilsson D, Kobes M, Müller M, Pauli P, Mühlberger A (2014) Virtual reality for fire evacuation research. 2014 Federated Conference on Computer Science and Information Systems. IEEE, pp. 313–321

  • Kowalski-Trakofler KM, Barrett EA (2003) The concept of degraded images applied to hazard recognition training in mining for reduction of lost-time injuries. J Safety Res 34:515–525

  • Krug S (2000) Don't make me think!: a common sense approach to Web usability. Pearson Education India

  • Le Callet P, Möller S, Perkis A (2012) Qualinet white paper on definitions of quality of experience. European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003). Ver.1.2. 2013

  • Lee KM (2004) Presence, explicated. Commun Theory 14:27–50. https://doi.org/10.1111/j.1468-2885.2004.tb00302.x

    Article  Google Scholar 

  • Li M, Sun Z, Jiang Z, Tan Z, Chen J (2020) A virtual reality platform for safety training in coal mines with AI and cloud computing. Discrete Dyn Nat Soc 2020

  • Li L, Zhang M, Xu F, Liu S (2005) ERT-VR: an immersive virtual reality system for emergency rescue training. Virtual Real 8(3):194–197

    Article  Google Scholar 

  • Lloréns R, Noé E, Colomer C, Alcañiz M (2015) Effectiveness, usability, and cost-benefit of a virtual reality–based telerehabilitation program for balance recovery after stroke: a randomized controlled trial. Arch Phys Med Rehabil 96(418–425):e2

    Google Scholar 

  • Longo F, Nicoletti L, Padovano A (2019) Emergency preparedness in industrial plants: a forward-looking solution based on industry 4.0 enabling technologies. Comput Ind 105:99–122

    Article  Google Scholar 

  • Lucas J, Thabet W (2008) Implementation and evaluation of a VR task-based training tool for conveyor belt safety training. ITcon

  • Lucas HJ, Lucas HC, Ginzberg MJ, Ginzberg MJ, Schultz RL, Schultz RL (1990) Information systems implementation: testing a structural model, Intellect Books

  • Macy BA, Izumi H (1993) Organizational change, design, and work innovation: a meta-analysis of 131 North American field studies—1961–1991. Res Org Change Develop 7:235–313

    Google Scholar 

  • Makransky G, Lilleholt L (2018) A structural equation modeling investigation of the emotional value of immersive virtual reality in education. Educ Technol Res Develop 66(5):1141–1164

    Article  Google Scholar 

  • Markus ML, Benjamin RI (1996) Change agentry-the next IS frontier. Mis Quarterly 385–407

  • Mccomas J, Mackay M, Pivik J (2002) Effectiveness of virtual reality for teaching pedestrian safety. Cyberpsychol Behav 5:185–190

    Article  Google Scholar 

  • Mendes JB, Caponetto GH, Lopes GP, Ramos ACB (2010) Low cost helicopter training simulator a case study from the brazilian Military Police. 2010 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems. IEEE, pp. 18–22

  • Merchant Z, Goetz ET, Cifuentes L, Keeney-Kennicutt W, Davis TJ (2014) Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: a meta-analysis. Comput Educ 70:29–40

    Article  Google Scholar 

  • Mikropoulos TA, Natsis A (2011) Educational virtual environments: a ten-year review of empirical research (1999–2009). Comput Educ 56:769–780

    Article  Google Scholar 

  • Morel M, Bideau B, Lardy J, Kulpa R (2015) Advantages and limitations of virtual reality for balance assessment and rehabilitation. Neurophysiol Clin/Clin Neurophysiol 45:315–326

    Article  Google Scholar 

  • Paquette DL, Gouveia LM (2000) Virtual training for live fire naval gunfire support. OCEANS 2000 MTS/IEEE Conference and Exhibition. Conference Proceedings (Cat. No. 00CH37158). IEEE, pp. 733–737

  • Pasmore W, Francis C, Haldeman J, Shani A (1982) Sociotechnical systems: a North American reflection on empirical studies of the seventies. Human Relat 35:1179–1204

    Article  Google Scholar 

  • Patle DS, Manca D, Nazir S, Sharma S (2019) Operator training simulators in virtual reality environment for process operators: a review. Virtual Real 23(3):293–311

    Article  Google Scholar 

  • Patton MQ (1997) Toward distinguishing empowerment evaluation and placing it in a larger context. Eval Pract 18:147–163

    Article  Google Scholar 

  • Pedram S, Perez P, Palmisano S, Farrelly M (2018) The factors affecting the quality of learning process and outcome in virtual reality environment for safety training in the context of mining industry. International Conference on Applied Human Factors and Ergonomics. Germany: Springer, pp. 404–411

  • Pedram S, Palmisano S, Skarbez R, Perez P, Farrelly M (2020) Investigating the process of mine rescuers’ safety training with immersive virtual reality: a structural equation modelling approach. Comput Educ 153:103891

    Article  Google Scholar 

  • Pedram S, Palmisano S, Skarbez R, Perez P, Farrelly M (2020) Investigating the process of mine rescuers' safety training with immersive virtual reality: a structural equation modelling approach. Comput Educ 103891

  • Rizzo A, Kim GJ (2005) A swot analysis of the field of virtual reality rehabilitation and therapy. Presence: Teleoperators Virtual Environ 14:119–146

  • Ryan SD, Harrison DA (2000) Considering social subsystem costs and benefits in information technology investment decisions: a view from the field on anticipated payoffs. J Manag Inform Syst 16:11–40

    Article  Google Scholar 

  • Ryan SD, Harrison DA, Schkade LL (2002) Information-technology investment decisions: when do costs and benefits in the social subsystem matter? J Manag Inform Syst 19:85–127

    Article  Google Scholar 

  • Salzman MC, Dede C, Loftin RB, Chen J (1999) A model for understanding how virtual reality Aids complex conceptual Learning. Presence: Teleoperators Virtual Environ 8:293–316

  • Schultze U (2010) Embodiment and presence in virtual worlds: a review. J Inform Technol 25(4):434–449

    Article  Google Scholar 

  • Schwebel DC, Combs T, Rodriguez D, Severson J, Sisiopiku V (2016) Community-based pedestrian safety training in virtual reality: a pragmatic trial. Accid Anal Prev 86:9–15

    Article  Google Scholar 

  • Shu Y, Huang YZ, Chang SH, Chen MY (2019) Do virtual reality head-mounted displays make a difference? A comparison of presence and self-efficacy between head-mounted displays and desktop computer-facilitated virtual environments. Virtual Real 23(4):437–446

    Article  Google Scholar 

  • Simon SC, Greitemeyer T (2019) The impact of immersion on the perception of pornography: a virtual reality study. Comput Hum Behav 93:141–148

    Article  Google Scholar 

  • Squelch A (2001) Virtual reality for mine safety training in South Africa. J-South Afri Inst Mining Metall 101:209–216

    Google Scholar 

  • Stansfield SA, Shawver D (1997) Using virtual reality to train and plan response actions to acts of terrorism. Terrorism and counter-terrorism methods and technologies. Int Socr Opt Photo 2933:93–100

    Google Scholar 

  • Sutherland I (1965) The ultimate display. Proceedings of the IFIPS Congress 65(2):506–508. New York: IFIP

  • Tanaka EH, Paludo JA, Cordeiro CS, Domingues LR, Gadbem EV, Euflausino A (2015) Using immersive virtual reality for electrical substation training. International Association for Development of the Information Society

  • Thomson JA, Tolmie AK, Foot HC, Whelan KM, Sarvary P, Morrison S (2005) Influence of virtual reality training on the roadside crossing judgments of child pedestrians. J Exp Psychol Appl 11:175

    Article  Google Scholar 

  • Tichon J, Burgess-Limerick R (2011) A review of virtual reality as a medium for safety related training in mining. J Health Safety Res Pract 3:33–40

    Google Scholar 

  • Van Wyk E, de Villiers R (2009) Virtual reality training applications for the mining industry. Proceedings of the 6th international conference on computer graphics, virtual reality, visualisation and interaction in Africa. ACM, pp. 53–63

  • Vincent DS, Sherstyuk A, Burgess L, Connolly KK (2008) Teaching mass casualty triage skills using immersive three-dimensional virtual reality. Acad Emerg Med 15:1160–1165

    Article  Google Scholar 

  • Walton RE (1989) Up and running: integrating information technology and the organization. Harvard Business School Press

    Google Scholar 

  • Xiuwen L, Fangbing X, Yicheng J (2009) A prototype of marine search and rescue simulator. 2009 International Conference on Information Technology and Computer Science. IEEE, pp. 343–346

  • Zhao D, Mccoy AP, Kleiner BM, Smith-Jackson TL (2015) Control measures of electrical hazards: an analysis of construction industry. Saf Sci 77:143–151

    Article  Google Scholar 

  • Zhou B-J, Jia Q-L (2011) Application of virtual reality simulation technology to earthquake emergency drill. J Nat Diss 5:008

    Google Scholar 

Download references

Acknowledgements

This project was funded by Coal Services Health and Safety Trust. The authors would like to thank the Mines Rescue Pty Ltd for agreeing to participate in this study and for granting access to its training facilities. Special appreciation goes to the personnel of the Mines Rescue Pty Ltd for their helpfulness and support, particularly the Heads of Stations, the trainers and the VR program designers.

Funding

This project was funded by Coal Services Health and Safety Trust.

Author information

Authors and Affiliations

  1. Smart Infrastructure Facility, Faculty of Engineering & Information Sciences, University of Wollongong, Wollongong, Australia

    Shiva Pedram, Robert Ogie & Pascal Perez

  2. School of Psychology, Faculty of Social Sciences, University of Wollongong, Wollongong, Australia

    Stephen Palmisano

  3. Mines Rescue, Coal Services Pty. Ltd, Sydney, Australia

    Matthew Farrelly

Authors
  1. Shiva Pedram
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert Ogie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephen Palmisano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matthew Farrelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pascal Perez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiva Pedram.

Ethics declarations

Conflict of interest

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Availability of data and material

All the qualitative and quantitative data are stored on computer at university of Wollongong and are available upon request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pedram, S., Ogie, R., Palmisano, S. et al. Cost–benefit analysis of virtual reality-based training for emergency rescue workers: a socio-technical systems approach. Virtual Reality 25, 1071–1086 (2021). https://doi.org/10.1007/s10055-021-00514-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-021-00514-5

Search

Navigation