Machine Vision and Applications

, Volume 24, Issue 1, pp 187–204 | Cite as

Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models

  • Javier MolinaEmail author
  • Marcos Escudero-Viñolo
  • Alessandro Signoriello
  • Montse Pardàs
  • Christian Ferrán
  • Jesús Bescós
  • Ferran Marqués
  • José M. Martínez
Original Paper


The use of hand gestures offers an alternative to the commonly used human computer interfaces, providing a more intuitive way of navigating among menus and multimedia applications. This paper presents a system for hand gesture recognition devoted to control windows applications. Starting from the images captured by a time-of-flight camera (a camera that produces images with an intensity level inversely proportional to the depth of the objects observed) the system performs hand segmentation as well as a low-level extraction of potentially relevant features which are related to the morphological representation of the hand silhouette. Classification based on these features discriminates between a set of possible static hand postures which results, combined with the estimated motion pattern of the hand, in the recognition of dynamic hand gestures. The whole system works in real-time, allowing practical interaction between user and application.


Computer vision Human–computer interaction Hand gesture cognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alon J., Athitsos V., Yuan Q., Sclaroff S.: A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 31(9), 1685–1699 (2009)CrossRefGoogle Scholar
  2. 2.
    Athitsos, V., Sclaroff, S.: Estimating 3d hand pose from a cluttered image. Computer Vision and Pattern Recognition. In: IEEE Computer Society Conference, vol. 2, pp. 432 (2003)Google Scholar
  3. 3.
    Breuer, P., Eckes, C., Muller, S.: Hand gesture recognition with a novel ir time-of-flight range camera: a pilot study. In: Computer Vision/Computer Graphics Collaboration Techniques Third International Conference, MIRAGE, pp. 247–260 (2007)Google Scholar
  4. 4.
    Castilla, D., Miralles, I., Jorquera, M., Botella, C., Banos, R., Montesa, J., Ferran, C.: Analysis and testing of metaphors for the definition of a gestual language based on real users interaction: vision project. In: 13th International Conference on Human–Computer Interaction, San Diego (2009)Google Scholar
  5. 5.
    Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines (2001)Google Scholar
  6. 6.
    Chen, Y.T., Tseng, K.T.: Developing a multiple-angle hand gesture recognition system for human machine interactions. In: Industrial Electronics Society, 33rd Annual Conference of the IEEE, pp. 489–492 (2007)Google Scholar
  7. 7.
    Goutte, C., Gaussier, E.: A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. In: Losada, D.E., Fernández-Luna, J.M. (ed.) Advances in Information Retrieval, Lecture Notes in Computer Science, vol. 3408, pp. 345–359. Springer, Berlin (2005)Google Scholar
  8. 8.
    Grzeszczuk, R., Bradski, G., Chu, M., Bouguet, J.: Stereo based gesture recognition invariant to 3d pose and lighting. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. I: 826–833 (2000)Google Scholar
  9. 9.
    Guomundsson S., Pardás M., Larsen R., Aanaes H., Casas J.R.: TOF imaging in smart room environments towards improved people tracking. Comput. Vis. Image Underst. 114(12), 1376–1384 (2010)CrossRefGoogle Scholar
  10. 10.
    Hernandez P.C.C., Czyz J., Marqués F., Umeda T., Marichal X., Macq B.M.: Bayesian approach for morphology-based 2-d human motion capture. IEEE Trans. Multimedia 9(4), 754–765 (2007)CrossRefGoogle Scholar
  11. 11.
    Holden E.J., Lee G., Owens R.: Australian sign language recognition. Mach. Vis. Appl. 16, 312–320 (2005)CrossRefGoogle Scholar
  12. 12.
    Hong, P., Turk, M., Huang, T.S.: Constructing finite state machines for fast gesture recognition. In: Pattern Recognition, 2000. Proceedings of 15th International Conference, vol. 3, pp. 691–694 (2000)Google Scholar
  13. 13.
    Hu M.K.: Visual pattern recognition by moment invariants. IRE Trans. Inf. Theory 8(2), 179–187 (1962)zbMATHCrossRefGoogle Scholar
  14. 14.
    Kelly D., McDonald J., Markham C.: A person independent system for recognition of hand postures used in sign language. Pattern Recogn. Lett. 31(11), 1359–1368 (2010)CrossRefGoogle Scholar
  15. 15.
    Keskin, C., Akarun, L.: Stars: sign tracking and recognition system using input-output hmms. Pattern Recogn. Lett. 30(12), 1086–1095 (2009). Image/video-based Pattern Analysis and HCI ApplicationsGoogle Scholar
  16. 16.
    Kollorz E., Penne J., Hornegger J., Barke A.: Gesture recognition with a time-of-flight camera. Int. J. Intell. Syst. Technol. Appl. 5(3/4), 334–343 (2008)Google Scholar
  17. 17.
    Kuhl F., Giardina C.: Elliptic fourier features of a closed contour. Comput. Graphics Image Process. 18, 236–258 (1982)CrossRefGoogle Scholar
  18. 18.
    Lantuejoul C., Maisonneuve F.: Geodesic methods in quantitative image analysis. Pattern Recogn. 17(2), 177–187 (1984)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Laviola J.J.: Bringing vr and spatial 3d interaction to the masses through video games. IEEE Comput. Graphics Appl. 28(5), 10–15 (2008)CrossRefGoogle Scholar
  20. 20.
    Letessier, J., Bérard, F.: Visual tracking of bare fingers for interactive surfaces. In: UIST ’04: Proceedings of the 17th annual ACM symposium on User interface software and technology, pp. 119–122. ACM, New York (2004)Google Scholar
  21. 21.
    Liu, X., Fujimura, K.: Hand gesture recognition using depth data. In: Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on, pp. 529 – 534 (2004)Google Scholar
  22. 22.
    Malassiotis S., Strintzis M.: Real-time hand posture recognition using range data. Image Vis. Comput. 26(7), 1027–1037 (2008)CrossRefGoogle Scholar
  23. 23.
    Mitra S., Acharya T.: Gesture recognition: a survey, systems, man, and cybernetics, part C: applications and reviews. IEEE Trans. 37(3), 311–324 (2007)Google Scholar
  24. 24.
    Nickel K., Stiefelhagen R.: Visual recognition of pointing gestures for human–robot interaction. Image Vis. Comput. 25(12), 1875–1884 (2007)CrossRefGoogle Scholar
  25. 25.
    Premaratne P., Nguyen Q.: Consumer electronics control system based on hand gesture moment invariants. Iet Comput. Vis. 1(1), 35–41 (2007)CrossRefGoogle Scholar
  26. 26.
    Soille P.: Morphological Image Analysis: Principles and Applications. Springer, New York (2003)zbMATHGoogle Scholar
  27. 27.
    Soutschek, S., Penne, J., Hornegger, J., Kornhuber, J.: 3-d gesture-based scene navigation in medical imaging applications using time-of-flight cameras. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–6 (2008)Google Scholar
  28. 28.
    Stenger B., Thayananthan A., Torr P.H.S., Cipolla R.: Model-based hand tracking using a hierarchical bayesian filter. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 372–1384 (2006)CrossRefGoogle Scholar
  29. 29.
    Teh C.H., Chin R.T.: On image analysis by the methods of moments. IEEE Trans. Pattern Anal. Mach. Intell. 10(4), 496–513 (1988)zbMATHCrossRefGoogle Scholar
  30. 30.
    Teng X., Wu B., Yu W., Liu C.: A hand gesture recognition system based on local linear embedding. J. Vis. Lang. Comput. 16, 442–454 (2005)CrossRefGoogle Scholar
  31. 31.
    Usabiaga J., Erol A., Bebis G., Boyle R., Twombly X.: Global hand pose estimation by multiple camera ellipse tracking. Mach. Vis. Appl. 21, 1–15 (2009)CrossRefGoogle Scholar
  32. 32.
    Van den Bergh, M., Van Gool, L.: Combining rgb and tof cameras for real-time 3d hand gesture interaction. 66–72 (2011)Google Scholar
  33. 33.
    Zaki M.M., Shaheen S.I.: Sign language recognition using a combination of new vision based features. Pattern Recogn. Lett. 32(4), 572–577 (2011)CrossRefGoogle Scholar
  34. 34.
    Zheng G., Wang C.J., Boult T.E.: Application of projective invariants in hand geometry biometrics. IEEE Trans. Inf. Forensics Security 2(4), 758–768 (2007)CrossRefGoogle Scholar
  35. 35.
    Zhu, X., Yang, J., Waibel, A.: Segmenting hands of arbitrary color. In: FG ’00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, p. 446. IEEE Computer Society, Washington (2000)Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Javier Molina
    • 1
    Email author
  • Marcos Escudero-Viñolo
    • 1
  • Alessandro Signoriello
    • 2
  • Montse Pardàs
    • 2
  • Christian Ferrán
    • 3
  • Jesús Bescós
    • 1
  • Ferran Marqués
    • 2
  • José M. Martínez
    • 1
  1. 1.Video Processing and Understanding Lab Laboratorio C-111 Escuela Politécnica SuperiorUniversidad Autónoma de MadridMadridSpain
  2. 2.Image and Video Processing GroupPolytechnic University of CataloniaBarcelonaSpain
  3. 3.Telefónica I+DBarcelonaSpain

Personalised recommendations