Evaluating Training Efficacy and Return on Investment for Augmented Reality: A Theoretical Framework
Augmented Reality (AR) has been in existence since the 1960s, however there is a lack of empirical research of areas in which using AR technology have the most benefit – and which areas do not see positive impacts. Being able to identify tradeoffs involved in acquiring and deploying an AR-based training system with respect to utility, efficacy, and return on investment (ROI) are valuable to any training program and acquisitions manager. In order to address this innovation gap, we present the Media Efficacy Return on Investment Tool (MERIT). MERIT is a theoretical framework methodology for quantitatively evaluating AR and other training solutions for a given training environment based on four factors: Task Factors, Training Media Factors, Return on Investment (ROI) Factors, and Training Efficacy Factors. The MERIT framework, based upon the “House of Quality” matrix, helps compare, visualize, and assess user and technical requirements as well as research gaps.
KeywordsAugmented Reality Training Training efficacy Return on investment Human systems
The work described in this paper was funded by the Naval Air Warfare Center Training Systems Division (NAWCTSD) via a Small Business Innovative Research (SBIR) Phase I contract (N68335-17-C-0490), and does not express the views of the funding agency.
- 1.Bayer, M.M., Rash, C.E., Brindle, J.H.: Introduction to helmet-mounted displays. In: Rash, C.E., Russo, M.B., Letowski, T.R., Schmeisser, E.T. (eds.) Helmet mounted displays: sensation, perception, and cognitive issues, pp. 47–109. US Army Aeromedical Research Laboratory, Fort Rucker (2009)Google Scholar
- 2.Goldiez, B. Liarokapis, F.: Trends and perspectives in augmented reality training. In: Cohn, J., Nicholson, D., Schmorrow, D. (eds.), The PSI Handbook of Virtual Environments for Training and Education: Developments for the Military and Beyond, pp 278–289. Praeger Security International (2008)Google Scholar
- 3.Baldwin, R.: Ferrari’s AR showroom app delves inside your next supercar. Endgaget 15 April 2015. https://www.engadget.com/2015/04/15/ferrari-ar-app/. Accessed 21 Jan 2017
- 4.Allison, P.R.: Augmented reality business applications start to get real. Computer Weekly (2015). http://www.computerweekly.com/feature/Augmented-reality-business-applications-start-to-get-real. Accessed 21 Jan 2017
- 5.Dill, K., Schaffer, R., Cullen, S., Meas, P.: Mixed and augmented reality for marine corps training. In: Human Computer Interaction International (2013)Google Scholar
- 6.Newman, D.: Hyper-training and the future augmented reality workplace. Forbes, 20 September 2016. http://www.forbes.com/sites/danielnewman//09/20/hyper-training-and-the-future-augmented-reality-workplace/#78a8264435f3. Accessed 21 Jan 2017
- 7.Hauser, J.R., Clausing, D.P.: The house of quality. Harvard Bus. Rev. 66(3), 63–73 (1988)Google Scholar
- 10.Bransford, J.D., Brown, A.L., Cocking, R.R.: How people learn: Brain, mind, experience, and school. National Academy of Sciences, Washington D.C (2004)Google Scholar
- 14.Sherwood, S., Neville, K., Sonnenfeld, N., Mooney, J., Walwanis, M., Bolton, A.: Fidelity requirements for effective live-virtual-constructive training of Navy F/A-18 pilots: an exploratory survey study. In: Proceedings of the 2015 Annual Meeting of the Human Factors and Ergonomics Society. Sage Publishing, Thousand Oaks (2015)CrossRefGoogle Scholar