Gesture-Based Interaction for Virtual Reality Environments Through User-Defined Commands

  • David Céspedes-HernándezEmail author
  • Juan Manuel González-Calleros
  • Josefina Guerrero-García
  • Liliana Rodríguez-Vizzuett
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 847)


Natural user interaction has been recently identified as a field of interest of computer sciences, mainly due to the development of hardware devices that enable fast identification and recognition of gestures within a diversity of contexts. In this paper, the investigation of body gestures that users naturally perform to navigate virtual reality environments is reported. For this purpose, users were asked to show the commands they would perform when realizing basic navigation tasks within a virtual world, and they were observed following the Wizard of Oz approach. The performance gesture-based interaction is evaluated in terms of usability, contrasted with traditional desktop interaction, and analyzed. Finally, as result of this work, a set of gestures (language) to navigate virtual reality environments is defined, along with insights regarding this interaction modality, and comments on the future direction of natural user interfaces using full-body movement as input.


Virtual reality environment Gesture-based interaction Natural user interfaces Body gestures Usability evaluation 



This work has been funded by PRODEP, CONACYT and BUAP-VIEP projects. We truly appreciate the work of students of the computer science and architecture schools of the Autonomous University of Puebla (BUAP) for their tremendous contribution on the creation of the VR environment. Particularly to the students: Cortés García Raymundo, Rodríguez Hernández Fredy, Tlapaya Tepech Ismael, Roberto Pérez Hernández, Daniel Romero Dominguez, and Manuel Juarez.


  1. 1.
    Boudoin, P., Otmane, S., Mallem, M.: Design of a 3d navigation technique supporting vr interaction. In: Arioui, H., Merzouki, R., Abbassi, H.A. (eds.) AIP Conference Proceedings, vol. 1019, no. 1, pp. 149–153 (2008).
  2. 2.
    Constantine, L.L.: Canonical abstract prototypes for abstract visual and interaction design. In: Jorge, J.A., Jardim Nunes, N., Falcão e Cunha, J. (eds.) DSV-IS 2003. LNCS, vol. 2844, pp. 1–15. Springer, Heidelberg (2003). Scholar
  3. 3.
    Wüthrich, C.A.: An analysis and a model of 3d interaction methods and devices for virtual reality. In: Duke, D., Puerta, A. (eds.) Design, Specification and Verification of Interactive Systems 1999, pp. 18–29. Springer, Vienna (1999). Scholar
  4. 4.
    Bowman, D.A., Koller, D., Hodges, L.F.: Travel in immersive virtual environments: an evaluation of viewpoint motion control techniques. In: Virtual Reality Annual International Symposium 1997, pp. 45–52. IEEE (1997).
  5. 5.
    Vanderdonckt, J.: A MDA-compliant environment for developing user interfaces of information systems. In: Pastor, O., Falcão e Cunha, J. (eds.) CAiSE 2005. LNCS, vol. 3520, pp. 16–31. Springer, Heidelberg (2005). Scholar
  6. 6.
    Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., Vanderdonckt, J.: A unifying reference framework for multi-target user interfaces. Interact. Comput. 15(3), 289–308 (2003). Scholar
  7. 7.
    Paterno, F., Mancini, C., Meniconi, S.: ConcurTaskTrees: a diagrammatic notation for specifying task models. In: Howard, S., Hammond, J., Lindgaard, G. (eds.) Human-Computer Interaction INTERACT 1997. ITIFIP, pp. 362–369. Springer, Boston, MA (1997). Scholar
  8. 8.
    Fonseca, J.M.C., et al.: Model-based ui xg final report. W3C Incubator Group Report, May, p. 32 (2010)Google Scholar
  9. 9.
    Kelley, J.F.: An iterative design methodology for user-friendly natural language office information applications. ACM Trans. Inf. Syst. (TOIS) 2(1), 26–41 (1984). Scholar
  10. 10.
    Dow, S., Lee, J., Oezbek, C., MacIntyre, B., Bolter, J.D., Gandy, M.: Wizard of Oz interfaces for mixed reality applications. In: CHI 2005. ACM, April 2005 (2005).
  11. 11.
    Höysniemi, J., Hämäläinen, P., Turkki, L.: Wizard of Oz prototyping of computer vision-based action games for children. In: Proceedings of the 2004 Conference on Interaction Design and Children: Building a Community, pp. 27–34. ACM (2004).
  12. 12.
    Maike, V.R.M.L., de Sousa Britto Neto, L., Baranauskas, M.C.C., Goldenstein, S.K.: Seeing through the kinect: a survey on heuristics for building natural user interfaces environments. In: Stephanidis, C., Antona, M. (eds.) UAHCI 2014. LNCS, vol. 8513, pp. 407–418. Springer, Cham (2014). Scholar
  13. 13.
    Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7(1), 57–78 (1995). Scholar
  14. 14.
    Aliakseyeu, D., Subramanian, S., Martens, J.B., Rauterberg, M.: Interaction techniques for navigation through and manipulation of 2 D and 3 D data. In: ACM International Conference Proceeding Series, vol. 23, pp. 179–188 (2002)Google Scholar
  15. 15.
    González-Calleros, J.M., Vanderdonckt, J., Muñoz-Arteaga, J.: A structured methodology for developing 3D web applications. In: Integrating Usability Engineering for Designing the Web Experience: Methodologies and Principles, pp. 15–43 (2010).
  16. 16.
    Kaur, K.: Designing virtual environments for usability. In: Howard, S., Hammond, J., Lindgaard, G. (eds.) Human-Computer Interaction INTERACT 1997. ITIFIP, pp. 636–639. Springer, Boston, MA (1997). Scholar
  17. 17.
    Tan, D.S., Robertson, G.G., Czerwinski, M.: Exploring 3D navigation: combining speed-coupled flying with orbiting. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 418–425. ACM (2001).
  18. 18.
    Kray, C., Nesbitt, D., Dawson, J., Rohs, M.: User-defined gestures for connecting mobile phones, public displays, and tabletops. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 239–248. ACM (2010).
  19. 19.
    Ruiz, J., Li, Y., Lank, E.: User-defined motion gestures for mobile interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 197–206. ACM (2011).
  20. 20.
    Kurdyukova, E., Redlin, M., André, E.: Studying user-defined iPad gestures for interaction in multi-display environment. In: Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces, pp. 93–96. ACM (2012).
  21. 21.
    Piumsomboon, T., Clark, A., Billinghurst, M., Cockburn, A.: User-defined gestures for augmented reality. In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds.) INTERACT 2013. LNCS, vol. 8118, pp. 282–299. Springer, Heidelberg (2013). Scholar
  22. 22.
    Valdes, C., et al.: Exploring the design space of gestural interaction with active tokens through user-defined gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 4107–4116. ACM (2014).
  23. 23.
    Vatavu, R.D., Wobbrock, J.O.: Between-subjects elicitation studies: formalization and tool support. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 3390–3402. ACM (2016).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Facultad de Ciencias de la ComputaciónBenemérita Universidad Autónoma de PueblaPueblaMexico
  2. 2.Facultad de Ciencias de la ElectrónicaBenemérita Universidad Autónoma de PueblaPueblaMexico

Personalised recommendations