Journal of Signal Processing Systems

, Volume 86, Issue 1, pp 17–25 | Cite as

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

  • Javier MolinaEmail author
  • José Antonio Pajuelo
  • José M. Martínez


This paper presents an innovative solution based on Time-Of-Flight (TOF) video technology to motion patterns detection for real-time dynamic hand gesture recognition. The resulting system is able to detect motion-based hand gestures getting as input depth images. The recognizable motion patterns are modeled on the basis of the human arm anatomy and its degrees of freedom, generating a collection of synthetic motion patterns that is compared with the captured input patterns in order to finally classify the input gesture. For the evaluation of our system a significant collection of gestures has been compiled, getting results for 3D pattern classification as well as a comparison with the results using only 2D information.


Computer vision Human-computer interaction Hand gesture recognition 


  1. 1.
    Athitsos, V., & Sclaroff, S. (2003). Estimating 3d hand pose from a cluttered image. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, 432.Google Scholar
  2. 2.
    Breuer, P., Eckes, C., & Muller, S. (2007). Hand gesture recognition with a novel ir time-of-flight range camera: a pilot study. In Computer vision/computer graphics collaboration techniques 3rd international conference, MIRAGE (pp. 247–260).Google Scholar
  3. 3.
    Castilla, D., Miralles, I., Jorquera, M., Botella, C., Baños, R., Montesa, J., & Ferran, C. (2009). Analysis and testing of metaphors for the definition of a gestual language based on real users interaction: vision project. In 13th international conference on human-computer interaction. San Diego.Google Scholar
  4. 4.
    Chen, Y.T., & Tseng, K.T. (2007). Developing a multiple-angle hand gesture recognition system for human machine interactions. In 33rd annual conference of the IEEE industrial electronics society, 2007. IECON 2007 (pp. 489–492).Google Scholar
  5. 5.
    Cheng, J., Xie, C., Bian, W., & Tao, D. (2012). Feature fusion for 3d hand gesture recognition by learning a shared hidden space. Pattern Recognition Letters, 33(4), 476–484. Intelligent Multimedia Interactivity.CrossRefGoogle Scholar
  6. 6.
    Fothergill, S., Mentis, H.M., Kohli, P., & Nowozin, S. (2012). Instructing people for training gestural interactive systems. In J. A. Konstan, E. H. Chi, & K. Höök (Eds.), CHI. ACM (pp. 1737–1746).Google Scholar
  7. 7.
    Grzeszczuk, R., Bradski, G., Chu, M., & Bouguet, J. (2000). Stereo based gesture recognition invariant to 3d pose and lighting. In IEEE conference on computer vision and pattern recognition (pp. I: 826–833).Google Scholar
  8. 8.
    Guomundsson, S., Pardás, M., Larsen, R., Aanaes H., & Casas, J.R. (2010). TOF imaging in smart room environments towards improved people tracking. Computer Vision and Image Understanding, 114(12), 1376–1384.CrossRefGoogle Scholar
  9. 9.
    Holden, E.J., Lee, G., & Owens, R. (2005). Australian sign language recognition. Machine Vision and Applications, 16, 312–320.CrossRefGoogle Scholar
  10. 10.
    Hu, J., Brown, M.K., & Turin, W. (1996). Hmm based on-line handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell., 18, 1039–1045.CrossRefGoogle Scholar
  11. 11.
    ISO9241-11. (1998). Ergonomic requirements for office work with visual display terminals (vdts) - part 11: Guidance on usability.Google Scholar
  12. 12.
    Jang, I.Y., & Lee, K. (2010). Depth video based human model reconstruction resolving self-occlusion. IEEE Transactions on Consumer Electronics, 56(3), 1933–1941.CrossRefGoogle Scholar
  13. 13.
    Kelly, D., McDonald, J., & Markham, C. (2010). A person independent system for recognition of hand postures used in sign language. Pattern Recognition Letters, 31(11), 1359–1368.CrossRefGoogle Scholar
  14. 14.
    Keskin, C., & Akarun, L. (2009). Stars: sign tracking and recognition system using input-output hmms. Pattern Recognition Letters, 30(12), 1086–1095. Image/video-based Pattern Analysis and HCI Applications.CrossRefGoogle Scholar
  15. 15.
    Kim, H.J., Lee, J., & Park, J.H. (2008). Dynamic hand gesture recognition using a cnn model with 3d receptive fields. In 2008 international conference on neural networks and signal processing (pp. 14–19).Google Scholar
  16. 16.
    Kim, S.Y., Cho, J.H., Koschan, A., & Abidi, M.A. (2010). Spatial and temporal enhancement of depth images captured by a time-of-flight depth sensor, (pp. 2358–2361).Google Scholar
  17. 17.
    Kollorz, E., Penne, J., Hornegger, J., & Barke, A. (2008). Gesture recognition with a time-of-flight camera. International Journal of Intelligent Systems Technologies and Applications, 5(3/4), 334–343.CrossRefGoogle Scholar
  18. 18.
    Kong, W., & Ranganath, S. (2010). Sign language phoneme transcription with rule-based hand trajectory segmentation. Journal of Signal Processing Systems, 59(2), 211–222.CrossRefGoogle Scholar
  19. 19.
    Kurakin, A., Zhang, Z., & Liu, Z. (2012). A real time system for dynamic hand gesture recognition with a depth sensor. In Proceedings of the 20th european signal processing conference, EUSIPCO 2012, Bucharest, Romania (pp. 1975–1979).Google Scholar
  20. 20.
    Lantuejoul, C., & Maisonneuve, F. (1984). Geodesic methods in quantitative image analysis. Pattern Recognition, 17(2), 177–187.MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Laviola, J.J. (2008). Bringing vr and spatial 3d interaction to the masses through video games. IEEE Computer Graphics and Applications, 28(5), 10–15.CrossRefGoogle Scholar
  22. 22.
    Liu, X., & Fujimura, K. (2004). Hand gesture recognition using depth data. In Proceedings of the 6th IEEE international conference on automatic face and gesture recognition, 2004 (pp. 529–534).Google Scholar
  23. 23.
    Malassiotis, S., & Strintzis, M. (2008). Real-time hand posture recognition using range data. Image and Vision Computing, 26(7), 1027–1037.CrossRefGoogle Scholar
  24. 24.
    Mitra, S., & Acharya, T. (2007). Gesture recognition: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37(3), 311–324.CrossRefGoogle Scholar
  25. 25.
    Molina, J., Escudero-Viñolo, M., Signoriello, A., Pardás, M., Ferrán, C., Bescós, J., Marqués, F., & Martínez, J. (2013). Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models. Machine Vision and Applications, 24(1), 187–204.CrossRefGoogle Scholar
  26. 26.
    Molina, J., & Martínez, J.M. (2014). A synthetic training framework for providing gesture scalability to 2.5d pose-based hand gesture recognition systems. Machine Vision And Applications, 25(5), 1309–1315.CrossRefGoogle Scholar
  27. 27.
    Molina, J., Pajuelo, J.A., Escudero-Viñolo, M., Bescós, J., & Martínez, J.M. (2014). A natural and synthetic corpus for benchmarking of hand gesture recognition systems. Machine Vision and Applications, 25 (4), 943–954.CrossRefGoogle Scholar
  28. 28.
    Nickel, K., & Stiefelhagen, R. (2007). Visual recognition of pointing gestures for human-robot interaction. Image and Vision Computing, 25(12), 1875–1884.CrossRefGoogle Scholar
  29. 29.
    Qin, S., Zhu, X., Yang, Y., & Jiang, Y. (2014). Real-time hand gesture recognition from depth images using convex shape decomposition method. Journal of Signal Processing Systems, 74(1), 47–58.CrossRefGoogle Scholar
  30. 30.
    Sakoe, H., & Chiba, S. (1978). Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics. Speech and Signal Processing, 26(1), 43–49.CrossRefzbMATHGoogle Scholar
  31. 31.
    Soutschek, S., Penne, J., Hornegger, J., & Kornhuber, J. (2008). 3-d gesture-based scene navigation in medical imaging applications using time-of-flight cameras. In IEEE computer society conference on computer vision and pattern recognition workshops (pp. 1–6).Google Scholar
  32. 32.
    Stenger, B., Thayananthan, A., Torr, P.H.S., & Cipolla, R. (2006). Model-based hand tracking using a hierarchical bayesian filter. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9), 1372–1384.CrossRefzbMATHGoogle Scholar
  33. 33.
    Teng, X., Wu, B., Yu, W., & Liu, C. (2005). A hand gesture recognition system based on local linear embedding. Journal of Visual Languages and Computing, 16, 442–454.CrossRefGoogle Scholar
  34. 34.
    Usabiaga, J., Erol, A., Bebis, G., Boyle, R., & Twombly, X. (2009). Global hand pose estimation by multiple camera ellipse tracking. Machine Vision and Applications, 21, 1–15.CrossRefGoogle Scholar
  35. 35.
    Wang, J., Liu, Z., Chorowski, J., Chen, Z., & Wu, Y. (2012). Robust 3d action recognition with random occupancy patterns. In Proceedings of the 12th european conference on computer vision - volume part II, ECCV’12 (pp. 872–885). Berlin: Springer.Google Scholar
  36. 36.
    Wenjun, T., Chengdong, W., Shuying, Z., & Li, J. (2010). Dynamic hand gesture recognition using motion trajectories and key frames. In 2010 2nd international conference on advanced computer control (ICACC) (vol. 3 pp. 163–167).Google Scholar
  37. 37.
    Yoon, H.S., Soh, J., Bae, Y.J., & Yang, H.S. (2001). Hand gesture recognition using combined features of location, angle and velocity. Pattern Recognition, 34(7), 1491–1501.CrossRefzbMATHGoogle Scholar
  38. 38.
    Zheng, G., Wang, C.J., & Boult, T.E. (2007). Application of projective invariants in hand geometry biometrics. IEEE Transactions on Information Forensics and Security, 2(4), 758–768.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Javier Molina
    • 1
    Email author
  • José Antonio Pajuelo
    • 1
  • José M. Martínez
    • 1
  1. 1.Video Processing and Understanding Laboratory Escuela Politécnica SuperiorUniversidad Autónoma de Madrid Avda. Francisco Tomás y Valiente, 11 Ciudad Universitaria de CantoblancoMadridSpain

Personalised recommendations