The Use of Immersive Virtual Reality for the Test and Evaluation of Interactions with Simulated Agents

  • Gabrielle VasquezEmail author
  • Rhyse BendellEmail author
  • Andrew TaloneEmail author
  • Blake NguyenEmail author
  • Florian JentschEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 780)


We aim to better inform the scientific community regarding test and evaluation techniques for validating devices that will potentially be used by individuals interfacing with autonomous robotic teammates (particularly, members of the U.S. Military). Testing within immersive virtual environments (IVRs) similar to those experienced in military operations will be discussed with focus on the use of a commercial gaming engine for task development. Highlights of using commercial gaming engines will be illustrated throughout the paper to emphasize their utility for evaluating future technologies with attention given to testing efficiency and ecological validity. The study of interactions with simulated agents and future communication devices will be described in the context of the Robotics Collaborative Technology Alliance (RCTA) research program.


Human-robot interaction Immersive virtual reality Simulation 



The research reported in this document was performed in connection with Contract Number W911NF-10- 2-0016 with the U.S. Army Research Laboratory. The views and conclusions contained in this document are those of the authors and should not be interpreted as presenting the official policies or position, either expressed or implied, of the U.S. Army Research Laboratory, or the U.S. Government unless so designated by other authorized documents. Citation of manufacturers or trade names does not constitute an official endorsement or approval of the use thereof. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon. Support for this endeavor was additionally provided by the University of Central Florida’s Office of Research and Commercialization in the form of financial assistance via Mr. Bendell’s ORC fellowship.


  1. 1.
    Yanco, H.A., Drury, J.L.: Rescuing interfaces: a multi-year study of human-robot interaction at the AAAI robot rescue competition. Auton. Robots 22(4), 333–352 (2007)CrossRefGoogle Scholar
  2. 2.
    Chen, J.Y.C., Haas, E.C., Barnes, M.J.: Human performance issues and user interface design for teleoperated robots. IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 37(6), 1231–1245 (2007)CrossRefGoogle Scholar
  3. 3.
    Perzanowski, D., Schultz, A.C., Adams, W., Marsh, E., Bugajska, M.: Building a multimodal human-robot interface. IEEE Intell. Syst. 16(1), 16–21 (2001)CrossRefGoogle Scholar
  4. 4.
    Evans III, A.W., Hoeft, R.M., Rehfeld, S.A., Feldman, M., Curtis, M., Fincannon, T., Ottlinger, J., Jentsch, F.: Advancing robotics research through the use of a scale MOUT facility. Proc. Hum. Factors Ergon. Soc. Ann. Meet. 49(6), 742–746 (2005)CrossRefGoogle Scholar
  5. 5.
    Rizzo, A., Buckwalter, J.: Virtual reality and cognitive assessment and rehabilitation: the state of the art. Stud. Health Technol. Inf. 44, 123–145 (1997)Google Scholar
  6. 6.
    Deterding, S.: Gamification: designing for motivation. Interactions 19(4), 14–17 (2012)CrossRefGoogle Scholar
  7. 7.
    Denny, P.: The effect of virtual achievements on student engagement. In: Proceedings of CHI 2013: Changing Perspectives, pp. 763–772 (2013)Google Scholar
  8. 8.
    Landers, R.N., Armstrong, M.B.: Enhancing instructional outcomes with gamification: an empirical test of the technology-enhanced training effectiveness model. Comput. Hum. Behav. 71, 499–507 (2017)CrossRefGoogle Scholar
  9. 9.
    Michael, D.R., Chen, S.: Serious games: games that educate, train, and inform. Muska & Lipman/Premier-Trade. Thomson Course Technology, Boston (2006)Google Scholar
  10. 10.
    Abt, C.C.: Serious Games. University Press of America, Boston (1987)Google Scholar
  11. 11.
    Landers, R.N.: Developing a theory of gamified learning: linking serious games and gamification of learning. Simul. Gaming 45(6), 752–768 (2014)CrossRefGoogle Scholar
  12. 12.
    Roth, S.: Serious gamification: on the redesign of a popular paradox. Games Cult. 12(1), 100–111 (2017)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T., Boyle, J.M.: A systematic literature review of empirical evidence on computer games and serious games. Comput. Educ. 59(2), 661–686 (2012)CrossRefGoogle Scholar
  14. 14.
    Boyle, E., Connolly, T.M., Hainey, T.: The role of psychology in understanding the impact of computer games. Entertain. Comput. 2(2), 69–74 (2011)CrossRefGoogle Scholar
  15. 15.
    Heeter, C.: Being there: the subjective experience of presence. Presence: Teleoper. Virtual Environ. 1(2), 262–271 (1992)CrossRefGoogle Scholar
  16. 16.
    Snow, M.P., Williges, R.C.: Empirical modeling of perceived presence in virtual environments using sequential experimentation techniques. In: Proceedings of the Human Factors and Ergonomics Society, 41st Annual Meeting, pp. 1224–1228 (1997)Google Scholar
  17. 17.
    Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments (Tech. Rep. No. 1014). U.S. Army Research Institute, Washington, DC (1994)Google Scholar
  18. 18.
    Barfield, W., Sheridan, T., Zeltzer, D., Slater, M.: Presence and performance within virtual environments. In: Barfield, W., Furness, T.A. (eds.) Virtual Environments and Advanced Interface Design, pp. 473–513 (1995)Google Scholar
  19. 19.
    Barfield, W., Weghorst, S.: The sense of presence within virtual environments: a conceptual framework. In: Salvendy, G., Smith, M. (eds.) Human–Computer Interaction: Software and Hardware Interfaces, pp. 699–704 (1993)Google Scholar
  20. 20.
    Steuer, J.: Defining virtual reality: dimensions determining telepresence. J. Commun. 42, 73–93 (1992)CrossRefGoogle Scholar
  21. 21.
    Bowman, D., McMahan, R.: Virtual reality: how much immersion is enough? Computer 40, 36–43 (2007)CrossRefGoogle Scholar
  22. 22.
    McMahan, R.P., Bowman, D.A., Zielinski, D.J., Brady, R.B.: Evaluating display fidelity and interaction fidelity in a virtual reality game. IEEE Trans. Vis. Comput. Graph. 18, 626–633 (2012)CrossRefGoogle Scholar
  23. 23.
    Slater, M.: Place illusion and plausibility can lead to realistics behaviour in immersive virtual environments. Philos. Trans. R. Soc. B 364, 3549–3557 (2009)CrossRefGoogle Scholar
  24. 24.
    Endsley, M.R.: Design and evaluation for situation awareness enhancement. Proc. Hum. Factors Soc. Ann. Meet. 32, 97–101 (1988)CrossRefGoogle Scholar
  25. 25.
    Moray, N.: Mental Workload: Its Theory and Measurement. Plenum, New York (1979)CrossRefGoogle Scholar
  26. 26.
    Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50, 449–455 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  1. 1.University of Central FloridaOrlandoUSA

Personalised recommendations