Evaluating Training Efficacy and Return on Investment for Augmented Reality: A Theoretical Framework

  • Amanda Bond
  • Kelly Neville
  • Joseph Mercado
  • Lauren Massey
  • Adam Wearne
  • Sherry Ogreten
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 785)


Augmented Reality (AR) has been in existence since the 1960s, however there is a lack of empirical research of areas in which using AR technology have the most benefit – and which areas do not see positive impacts. Being able to identify tradeoffs involved in acquiring and deploying an AR-based training system with respect to utility, efficacy, and return on investment (ROI) are valuable to any training program and acquisitions manager. In order to address this innovation gap, we present the Media Efficacy Return on Investment Tool (MERIT). MERIT is a theoretical framework methodology for quantitatively evaluating AR and other training solutions for a given training environment based on four factors: Task Factors, Training Media Factors, Return on Investment (ROI) Factors, and Training Efficacy Factors. The MERIT framework, based upon the “House of Quality” matrix, helps compare, visualize, and assess user and technical requirements as well as research gaps.


Augmented Reality Training Training efficacy Return on investment Human systems 



The work described in this paper was funded by the Naval Air Warfare Center Training Systems Division (NAWCTSD) via a Small Business Innovative Research (SBIR) Phase I contract (N68335-17-C-0490), and does not express the views of the funding agency.


  1. 1.
    Bayer, M.M., Rash, C.E., Brindle, J.H.: Introduction to helmet-mounted displays. In: Rash, C.E., Russo, M.B., Letowski, T.R., Schmeisser, E.T. (eds.) Helmet mounted displays: sensation, perception, and cognitive issues, pp. 47–109. US Army Aeromedical Research Laboratory, Fort Rucker (2009)Google Scholar
  2. 2.
    Goldiez, B. Liarokapis, F.: Trends and perspectives in augmented reality training. In: Cohn, J., Nicholson, D., Schmorrow, D. (eds.), The PSI Handbook of Virtual Environments for Training and Education: Developments for the Military and Beyond, pp 278–289. Praeger Security International (2008)Google Scholar
  3. 3.
    Baldwin, R.: Ferrari’s AR showroom app delves inside your next supercar. Endgaget 15 April 2015. Accessed 21 Jan 2017
  4. 4.
    Allison, P.R.: Augmented reality business applications start to get real. Computer Weekly (2015). Accessed 21 Jan 2017
  5. 5.
    Dill, K., Schaffer, R., Cullen, S., Meas, P.: Mixed and augmented reality for marine corps training. In: Human Computer Interaction International (2013)Google Scholar
  6. 6.
    Newman, D.: Hyper-training and the future augmented reality workplace. Forbes, 20 September 2016. Accessed 21 Jan 2017
  7. 7.
    Hauser, J.R., Clausing, D.P.: The house of quality. Harvard Bus. Rev. 66(3), 63–73 (1988)Google Scholar
  8. 8.
    Santos, M.E.C., Taketomi, T., Yamamoto, G., Rodrigo, M.M.T., Sandor, C., Kato, H.: Augmented reality as multimedia: the case for situated vocabulary learning. Res. Pract. Technol. Enhanc. Learn. 11(1), 4 (2016)CrossRefGoogle Scholar
  9. 9.
    Wu, H.K., Lee, S.W.Y., Chang, H.Y., Liang, J.C.: Current status, opportunities and challenges of augmented reality in education. Comput. Educ. 62, 41–49 (2013)CrossRefGoogle Scholar
  10. 10.
    Bransford, J.D., Brown, A.L., Cocking, R.R.: How people learn: Brain, mind, experience, and school. National Academy of Sciences, Washington D.C (2004)Google Scholar
  11. 11.
    Bransford, J.D., Franks, J.J., Vye, N.J., Sherwood, R.D.: New approaches to instruction: because wisdom can’t be told. In: Vosniadou, S., Ortony, A. (eds.) Similarity and Analogical Reasoning, pp. 470–497. Cambridge University Press, New York (1989)CrossRefGoogle Scholar
  12. 12.
    Craik, F.I., Lockhart, R.S.: Levels of processing: a framework for memory research. J. Verbal Learn. Verbal Behav. 11(6), 671–684 (1972)CrossRefGoogle Scholar
  13. 13.
    Renkl, A., Mandl, H., Gruber, H.: Inert knowledge: analyses and remedies. Educ. Psychol. 31(2), 115–121 (1996)CrossRefGoogle Scholar
  14. 14.
    Sherwood, S., Neville, K., Sonnenfeld, N., Mooney, J., Walwanis, M., Bolton, A.: Fidelity requirements for effective live-virtual-constructive training of Navy F/A-18 pilots: an exploratory survey study. In: Proceedings of the 2015 Annual Meeting of the Human Factors and Ergonomics Society. Sage Publishing, Thousand Oaks (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  • Amanda Bond
    • 1
  • Kelly Neville
    • 1
  • Joseph Mercado
    • 2
  • Lauren Massey
    • 1
  • Adam Wearne
    • 1
  • Sherry Ogreten
    • 1
  1. 1.Soar Technology, Inc.OrlandoUSA
  2. 2.Naval Air Warfare Center Training Systems DivisionOrlandoUSA

Personalised recommendations