Full Body Locomotion with Video Game Motion Controllers

  • Brian Williamson
  • Chadwick Wingrave
  • Joseph J. LaViolaJr.


Sensing technologies of increasing fidelity are dropping in costs to the point that full body sensing hardware is commonplace in people’s homes. This presents an opportunity for users to interact and move through environments with their body. Not just walking or running, but jumping, dodging, looking, dancing and exploring. Three current generation videogame devices, the Nintendo Wii Remote, Playstation Move and Microsoft Kinect, are discussed in terms of their sensors and data, in order to explore two questions. First, how do you deal with the data from the devices including error, uncertainty and volume? Second, how do you use the devices to create an interface that allows the user to interact as they wish? While these devices will change in time, understanding the sensing methods and approach to interface design will act as a basis for further improvements to full body locomotion.


Video game motion controllers Full body interaction Locomotion Virtual environments Games Human–computer interaction 


  1. 1.
    Azuma R, Bishop G (1994) Improving static and dynamic registration in an optical see-through HMD. In: SIGGRAPH ’94: Proceedings of the 21st annual conference on computer graphics and interactive techniques. ACM, New York, pp 197–204Google Scholar
  2. 2.
    Bowman D, Kruijff E, LaViola J, Poupyrev I (2005) 3D user interfaces: theory and practice. Addison-Wesley, BostonGoogle Scholar
  3. 3.
    Buxton B (2007) Sketching user experiences: getting the design right and the right design. Morgan Kaufmann, San FranciscoGoogle Scholar
  4. 4.
    Cohn G, Morris D, Patel S, Tan D (2012) Humantenna: using the body as an antenna for real-time whole-body interaction. In: Proceedings of the 2012 ACM annual conference on human factors in computing systems (CHI ’12), pp 1901–1910Google Scholar
  5. 5.
    Crassidis JL, Markley FL (2003) Unscented filtering for spacecraft attitude estimation. J Guid Control Dyn 26(4):536–542CrossRefGoogle Scholar
  6. 6.
    Hoffman M, Varcholik P, LaViola J (2010) Breaking the status quo: improving 3D gesture recognition with spatially convenient input devices. IEEE VR 2010Google Scholar
  7. 7.
    Julier S, Uhlmann J (1997) A new extension of the Kalman filter to nonlinear systems. In: International symposium: aerospace/defense sensing, simulation and controls.Google Scholar
  8. 8.
    Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82 (Series D) 35–45Google Scholar
  9. 9.
    LaViola J (2008) Bringing VR and spatial 3D interaction to the masses through video games. IEEE Comput Graph Appl 28(5):10–15Google Scholar
  10. 10.
    Luinge H, Veltink P, Baten C (1999) Estimating orientation with gyroscopes and accelerometers. Technol Health Care 7(6):455–459CrossRefGoogle Scholar
  11. 11.
    Mankoff J, Hudson S, Abowd GD (2000) Interaction techniques for ambiguity resolution in recognition-based interfaces. In: Proceedings of the 13th annual ACM symposium UIST, pp 11–20Google Scholar
  12. 12.
    Mueller F, Edge D, Vetere F, Gibbs M, Agamanolis S, Bongers B, Sheridan J (2011) Designing sports: a framework for exertion games. In: Proceedings of the 2011 annual conference on human factors in computing systems (CHI ’11). ACM, New York, pp 2651–2660Google Scholar
  13. 13.
    Norton J, Wingrave C, LaViola J (2011) Exploring strategies and guidelines for developing full body video game interfaces. In: Foundations of digital games 2011Google Scholar
  14. 14.
    Oviatt S (1999) Mutual disambiguation of recognition errors in a multimodel architecture. In: Proceedings of the SIGCHI conference on human factors in computing systems: the CHI is the limit (CHI ’99). ACM, New York, pp 576–583Google Scholar
  15. 15.
    Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. In: Proceedings of the IEEE, 1989Google Scholar
  16. 16.
    Rubine D (1991) Specifying gestures by example. Comput Graph 25(4):329–337CrossRefGoogle Scholar
  17. 17.
    Sato M, Poupyrev I, Harrison C (2012) Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects. In: Proceedings of the 2012 ACM annual conference on human factors in computing systems (CHI ’12), pp 483–492Google Scholar
  18. 18.
    Sherman B, Craig A (2002) Understanding virtual reality: interface, application, and design. Morgan Kauffman, San FranciscoGoogle Scholar
  19. 19.
    Shirai A, Geslin E, Richir S (2007) Wiimedia: motion analysis methods and applications using a consumer video game controller. In: Sandbox ’07: Proceedings of the (2007) ACM SIGGRAPH symposium on video games. ACM, New York, pp 133–140Google Scholar
  20. 20.
    Shiratori T, Hodgins JK (2008) Accelerometer-based user interfaces for the control of a physically simulated character. ACM Trans. Graph. 27(5):1–9CrossRefGoogle Scholar
  21. 21.
    Shivaram G, Seetharaman G (1998) A new technique for finding the optical center of cameras. In: ICIP 98: Proceedings of 1998 international conference on image processing, vol 2, pp 167–171Google Scholar
  22. 22.
    Shotton J, Fizgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human pose recognition in parts from single depth images. In: IEEE computer vision and pattern recognition (CVPR) 2011. IEEEGoogle Scholar
  23. 23.
    Wan EA, van der Merwe R (2002) The unscented Kalman filter for nonlinear estimation, pp 153–158Google Scholar
  24. 24.
    Welch G, Foxlin E (2002) Motion tracking: no silver bullet, but a respectable arsenal. IEEE Comput Graph Appl 22(6):24–38CrossRefGoogle Scholar
  25. 25.
    Whitton M, Cohn J, Feasel J, Zimmons P, Razzaque S, Poulton S et al (2005) Comparing VE locomotion interfaces. In: IEEE virtual reality, pp 123–130Google Scholar
  26. 26.
    Williamson R, Andrews B (2001) Detecting absolute human knee angle and angular velocity using accelerometers and rate gyroscopes. Med Biol Eng Comput 39(3):294–302CrossRefGoogle Scholar
  27. 27.
    Williamson B, Wingrave C, LaViola J (2010) Realnav: exploring natural user interfaces for locomotion in video games. In: Proceedings of IEEE symposium on 3D user interfaces 2010. IEEE Computer Society, pp 3–10Google Scholar
  28. 28.
    Williamson B, Wingrave C, LaViola J, Roberts T, Garrity P (2011) Natural full body interaction for navigation in dismounted soldier training. In: Proceedings of the interservice/industry training, simulation, and education conference (I/ITSEC) 2011, pp 2103–2110, Dec 2011Google Scholar
  29. 29.
    Wingrave C, Williamson B, Varcholik P, Rose J, Miller A, Charbonneau E, Bott J, LaViola J (2010) Wii remote and beyond: using spatially convenient devices for 3DUIs. IEEE Comput Graph Appl 30(2):71–85Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Brian Williamson
    • 1
  • Chadwick Wingrave
    • 1
  • Joseph J. LaViolaJr.
    • 1
  1. 1.Department of Electrical Engineering and Computer ScienceUniversity of Central FloridaOrlandoUSA

Personalised recommendations