From CAVE2TM to Mobile: Adaptation of Hearts and Minds Virtual Reality Project Interaction

  • Arthur Nishimoto
  • Daria TsoupikovaEmail author
  • Scott Rettberg
  • Roderick Coover
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9732)


Hearts and Minds: The Interrogations Project is an interactive performance made for the CAVE2TM [1] large-scale 320-degree panoramic virtual reality environment that describes veterans’ testimonies about military interrogations in Iraq during the American counter-insurgency campaign. The project is based on interviews of American soldiers and on their actual testimonies [2]. The project was achieved through technical innovation, cross-disciplinary and international collaboration. It was developed using a novel method for direct output of the Unity-based virtual reality projects into the CAVE2 environment. Other portable versions of the work were developed to reach new audiences across educational, arts and public arenas which include (1) personal computer version navigable using Xbox 360 controller; (2) web-based version available for free download; (3) Oculus Rift immersive virtual reality HMD version; (4) mobile version of the project for Apple iPad (in progress). This paper describes the development and compares the interaction experiences across platforms.


Virtual reality Navigation Interaction Immersive environments 3D user interface CAVE2TM 

List of Abbreviations




Application programming interface


CAVE automatic virtual environment


Graphical user interface


Head mounted display


Virtual reality


Virtual reality environment



The authors wish to thank Electronic Visualization Lab (EVL) at the University of Illinois at Chicago, the UIC School of Design, Temple University, the Electronic Literature Organization, and the Norwegian Research Council. Thanks to Lance Long for research engineering support, to Mark Partridge for sound design, voice artists Richard Garella, Jeffrey Cousar, Laurel Katz, Darin Dunston, production assistant Mark Baratta and performance artist Mark Jeffrey. Dr. Jeffrey Murer of St. Andrews University, Scotland also contributed as a consultant on the project. The project is based on interviews of American soldiers conducted by political scientist, Dr. John Tsukayama. Special thanks to all visitors who participated in the exhibitions and discussions.


  1. 1.
    Febretti, A., Nishimoto, A., Thigpen, T., Talandis, J., Long, L., Pirtle, J.D., Peterka, T., Verlo, A., Brown, M., Plepys, D., Sandin, D., Renambot, L., Johnson, A., Leigh, J.: CAVE2: a hybrid reality environment for immersive simulation and information analysis. In: Proceedings of SPIE 8649, The Engineering Reality of Virtual Reality 2013, p. 864903 (2013)Google Scholar
  2. 2.
    Tsukayama, J.: By any means necessary: an interpretive phenomenological analysis study of post 9/11 American abusive violence in Iraq. Ph.D. dissertation, University of St Andrews, St Andrews, UK (2014)Google Scholar
  3. 3.
    Grau, O.: Virtual Art, From Illusion to Immersion. Leonardo Book Series. MIT Press, Cambridge (2004)Google Scholar
  4. 4.
    Preddy, S.M. Nance, R.E.: Key requirements for CAVE simulations. In: Proceedings of the 34th Conference on Winter Simulation: Exploring New Frontiers, San Diego, pp. 127–135 (2002)Google Scholar
  5. 5.
    Tsoupikova, D., Coover, R., Rettberg, S., Nishimoto, A.: Hearts and minds: the interrogations project. In: Proceedings of the IEEE VIS Arts Program (VISAP), Paris, France (2014)Google Scholar
  6. 6.
    Tsoupikova, D., Rettberg, S., Coover, R., Nishimoto, A.: The battle for hearts and minds: interrogation and torture in the age of war. In: Proceedings of SIGGRAPH 2015 ACM SIGGRAPH 2015 Posters, article no. 12. ACM, New York (2015)Google Scholar
  7. 7.
    Omicron. Accessed 8 Feb 2016
  8. 8.
    Mechdyne Corporation, getReal3D. Accessed 8 Feb 2016
  9. 9.
    Sutcliffe, A., Gault, B., Fernando, T., Tan, K.: Investigating interaction in CAVE virtual environments. ACM Trans. Comput. Hum. Interact. 13(2), 235–267 (2006)CrossRefGoogle Scholar
  10. 10.
    Muhanna, M.: Virtual reality and the CAVE: taxonomy, interaction challenges and research directions. J. King Saud Univ. Comput. Inf. Sci. 27(3), 344–361 (2015)Google Scholar
  11. 11.
    Sutclıffe, A.G., Kaur, K.D.: Evaluating the usability of virtual reality user interfaces. Behav. Inf. Technol. 19(6), 415–426 (2000)CrossRefGoogle Scholar
  12. 12.
    Bowman, D.A., Johnson, D.B., Hodges, L.F.: Testbed evaluation of virtual environments interaction techniques. In: Proceedings of Virtual Reality Software Technology (VRST) 1999, pp. 26–33. ACM Press, New York (1999)Google Scholar
  13. 13.
    Poupyrev, I., Ichıkawa, T.: Manipulating objects in virtual worlds: categorization and empirical evaluation of interaction techniques. J. Vis. Lang. Comput. 10(1), 19–35 (1999)CrossRefGoogle Scholar
  14. 14.
    Darken, R.P., Sibert, J.L.: Wayfinding strategies and behaviors in large virtual worlds. In: Tauber, M., Bellotti, V., Jeffries, R., MaCkinlay, J.D., Nielsen, J. (eds.) Human Factors in Computing Systems CHI 1996 Conference Proceedings (Vancouver BC), pp. 142–149. ACM Press, New York (1996)Google Scholar
  15. 15.
    Vinson, N.G.: Design guidelines for landmarks to support navigation in virtual environments. In: Human Factors in Computing Systems CHI 1999 Conference Proceedings, Pittsburgh PA (1999)Google Scholar
  16. 16.
    Usoh M., Arthur K., Whitton MC., Bastos R., Steed A., Slater M., et al. Walking > walking-in-place > flying, in virtual environments. In: Proceedings of the 26th Annual Conference on Computer-Graphics and Interactive Techniques, pp. 359–364. ACM Press, Addison-Wesley Publishing Co. (1999)Google Scholar
  17. 17.
    Hauber, J., Billinghurst, M., Regenbrecht, H.: Tangible teleconferencing. In: Masoodian, M., Jones, S., Rogers, B. (eds.) APCHI 2004. LNCS, vol. 3101, pp. 143–152. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Bourdot, P., Touraine D.: Polyvalent display framework to control virtual navigations by 6DOF tracking. In: Proceedings of the IEEE Virtual Reality 2002 (VR 2002), pp. 277–278, 24–28 Mar 2002Google Scholar
  19. 19.
    Schneiderman, B.: Direct manipulation: a step beyond programming languages. IEEE Comput. 16(8), 57–69 (1983)CrossRefGoogle Scholar
  20. 20.
    Schneiderman, B., Plaisant, C., Cohen, M., Jacobs, S.: Designing the User Interface: Strategies for Effective Human-Computer Interaction, 5th edn. Addison Wesley, Boston (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Arthur Nishimoto
    • 1
  • Daria Tsoupikova
    • 2
    Email author
  • Scott Rettberg
    • 3
  • Roderick Coover
    • 4
  1. 1.Computer ScienceUniversity of Illinois at ChicagoChicagoUSA
  2. 2.School of DesignUniversity of Illinois at ChicagoChicagoUSA
  3. 3.University of BergenBergenNorway
  4. 4.Temple UniversityPhiladelphiaUSA

Personalised recommendations