Virtual Reality

, Volume 18, Issue 3, pp 203–217 | Cite as

Evaluation of direct manipulation using finger tracking for complex tasks in an immersive cube

  • Emmanuelle ChapoulieEmail author
  • Maud Marchal
  • Evanthia Dimara
  • Maria Roussou
  • Jean-Christophe Lombardo
  • George Drettakis
Original Article


A solution for interaction using finger tracking in a cubic immersive virtual reality system (or immersive cube) is presented. Rather than using a traditional wand device, users can manipulate objects with fingers of both hands in a close-to-natural manner for moderately complex, general purpose tasks. Our solution couples finger tracking with a real-time physics engine, combined with a heuristic approach for hand manipulation, which is robust to tracker noise and simulation instabilities. A first study has been performed to evaluate our interface, with tasks involving complex manipulations, such as balancing objects while walking in the cube. The user’s finger-tracked manipulation was compared to manipulation with a 6 degree-of-freedom wand (or flystick), as well as with carrying out the same task in the real world. Users were also asked to perform a free task, allowing us to observe their perceived level of presence in the scene. Our results show that our approach provides a feasible interface for immersive cube environments and is perceived by users as being closer to the real experience compared to the wand. However, the wand outperforms direct manipulation in terms of speed and precision. We conclude with a discussion of the results and implications for further research.


Virtual reality Direct manipulation Immersive cube Finger tracking 



The authors wish to thank Rachid Guerchouche for his help in many aspects of the project, also Theophanis Tsandilas, Martin Hachet, Anatole Lecuyer, Peter Vangorp and Sylvain Duchêne for their help and their useful comments. We also thank all the participants of the study. This work was supported in part by the Regional Council of Provence Alpes-Côte d’Azur and the EU project VERVE ( The textures in the VEs come from the following websites:,,,,,,, and

Supplementary material

MP4 (47,620 KB)

10055_2014_246_MOESM2_ESM.odt (12 kb)
ODT (699 KB)


  1. Agarawala A, Balakrishnan R (2006) Keepin’ it real: pushing the desktop metaphor with physics, piles and the pen. CHI ’06, pp 1283–1292Google Scholar
  2. Aleotti J, Caselli S (2011) Physics-based virtual reality for task learning and intelligent disassembly planning. Virtual Reality 15:41–54CrossRefGoogle Scholar
  3. Bolt RA (1980) “put-that-there”: Voice and gesture at the graphics interface. In: Proceedings of the 7th annual conference on computer graphics and interactive techniques, ACM, New York, NY, USA, SIGGRAPH ’80, pp 262–270. doi: 10.1145/800250.807503
  4. Borst C, Indugula A (2005) Realistic virtual grasping. In: IEEE VR 2005, pp 91–98Google Scholar
  5. Bowman D, Coquillart S, Froehlich B, Hirose M, Kitamura Y, Kiyokawa K, Stuerzlinger W (2008) 3D user interfaces: new directions and perspectives. IEEE Comput Graph Appl 28(6):20–36CrossRefGoogle Scholar
  6. Bowman D, McMahan R, Ragan E (2012) Questioning naturalism in 3d user interfaces. Commun ACM 55(9):78–88CrossRefGoogle Scholar
  7. Buchmann V, Violich S, Billinghurst M, Cockburn A (2004) Fingartips: gesture based direct manipulation in augmented reality. In: Proceedings GRAPHITE ’04, pp 212–221Google Scholar
  8. Cabral MC, Morimoto CH, Zuffo MK (2005) On the usability of gesture interfaces in virtual reality environments. In: Proceedings CLIHC ’05, pp 100–108Google Scholar
  9. Dipietro L, Sabatini AM, Dario P (2008) A survey of glove-based systems and their applications. IEEE Trans Syst Man Cybern Part C 38(4):461–482CrossRefGoogle Scholar
  10. Fröhlich B, Tramberend H, Beers A, Agrawala M, Baraff D (2000) Physically-based manipulation on the responsive workbench. In: IEEE VR 2000Google Scholar
  11. Heumer G, Amor H, Weber M, Jung B (2007) Grasp recognition with uncalibrated data gloves—a comparison of classification methods. In: Virtual reality conference, 2007. VR ’07. IEEE, pp 19 –26. doi: 10.1109/VR.2007.352459
  12. Hilliges O, Kim D, Izadi S, Weiss M, Wilson A (2012) HoloDesk: direct 3D interactions with a situated see-through display. In: CHI ’12, pp 2421–2430Google Scholar
  13. Hirota K, Hirose M (2003) Dexterous object manipulation based on collision response. In: IEEE VR ’03, IEEE Comput Soc, vol 2003, pp 232–239Google Scholar
  14. Holz D, Ullrich S, Wolter M, Kuhlen T (2008) Multi-contact grasp interaction for virtual environments. J Virtual Real Broadcast 5(7):1860–2037Google Scholar
  15. Jacobs J, Froehlich B (2011) A soft hand model for physically-based manipulation of virtual objects. In: IEEE VR 2011, IEEEGoogle Scholar
  16. Jacobs J, Stengel M, Froehlich B (2012) A generalized god-object method for plausible finger-based interactions in virtual environments. In: 3DUI’2012, IEEE, pp 43–51Google Scholar
  17. Koons DB, Sparrell CJ (1994) Iconic: speech and depictive gestures at the human-machine interface. In: Conference companion on human factors in computing systems, ACM, New York, NY, USA, CHI ’94, pp 453–454. doi: 10.1145/259963.260487
  18. Latoschik M, Frohlich M, Jung B, Wachsmuth I (1998) Utilize speech and gestures to realize natural interaction in a virtual environment. In: Industrial electronics society, 1998. IECON ’98. Proceedings of the 24th annual conference of the IEEE, vol 4, pp 2028 –2033. doi: 10.1109/IECON.1998.724030
  19. Latoschik ME (2001) A gesture processing framework for multimodal interaction in virtual reality. In: Proceedings of the 1st international conference on computer graphics, virtual reality and visualisation, ACM, New York, NY, USA, AFRIGRAPH ’01, pp 95–100. doi: 10.1145/513867.513888
  20. McMahan R, Alon A, Lazem S, Beaton R, Machaj D, Schaefer M, Silva M, Leal A, Hagan R, Bowman D (2010) Evaluating natural interaction techniques in video games. In: 3DUI, IEEE, pp 11–14Google Scholar
  21. Moehring M, Froehlich B (2011) Natural interaction metaphors for functional validations of virtual car models. IEEE TVCG 17(9):1195–1208Google Scholar
  22. O’Hagan R, Zelinsky A, Rougeaux S (2002) Visual gesture interfaces for virtual environments. Interact Comput 14(3):231–250CrossRefGoogle Scholar
  23. Ortega M, Redon S, Coquillart S (2007) A six degree-of-freedom god-object method for haptic display of rigid bodies with surface properties. IEEE TVCG 13(3):458–469Google Scholar
  24. Prachyabrued M, Borst C (2012) Visual interpenetration tradeoffs in whole-hand virtual grasping. In: 3DUI, IEEE, pp 39–42Google Scholar
  25. Slater M (2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos Trans R Soc B Biol Sci 364(1535):3549–3557CrossRefGoogle Scholar
  26. Sturman DJ, Zeltzer D (1994) A survey of glove-based input. IEEE Comput Graph Appl 14(1):30–39. doi: 10.1109/38.250916 CrossRefGoogle Scholar
  27. Sturman DJ, Zeltzer D, Pieper S (1989) Hands-on interaction with virtual environments. UIST ’89, pp 19–24Google Scholar
  28. Ullmann T, Sauer J (2000) Intuitive virtual grasping for non haptic environments. In: Pacific Graphics ’00, pp 373–381Google Scholar
  29. Wang RY, Popović J (2009) Real-time hand-tracking with a color glove. In: ACM SIGGRAPH 2009 Papers, ACM, New York, NY, USA, SIGGRAPH ’09, pp 63:1–63:8. doi: 10.1145/1576246.1531369
  30. Wexelblat A (1995) An approach to natural gesture in virtual environments. ACM Trans Comput Hum Interact 2(3):179–200CrossRefGoogle Scholar
  31. Wilson AD, Izadi S, Hilliges O, Garcia-Mendoza A, Kirk D (2008) Bringing physics to the surface. In: ACM UIST ’08, pp 67–76Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Emmanuelle Chapoulie
    • 1
    Email author
  • Maud Marchal
    • 2
    • 3
  • Evanthia Dimara
    • 4
  • Maria Roussou
    • 4
    • 5
  • Jean-Christophe Lombardo
    • 6
  • George Drettakis
    • 1
  1. 1.InriaREVES, Sophia AntipolisFrance
  2. 2.InriaHybrid, RennesFrance
  3. 3.IRISA-INSARennesFrance
  4. 4.University of AthensAthensGreece
  5. 5.Makebelieve Design and ConsultingAthensGreece
  6. 6.InriaSophia AntipolisFrance

Personalised recommendations