Skip to main content
SpringerLink
Log in

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

This paper presents an innovative solution based on Time-Of-Flight (TOF) video technology to motion patterns detection for real-time dynamic hand gesture recognition. The resulting system is able to detect motion-based hand gestures getting as input depth images. The recognizable motion patterns are modeled on the basis of the human arm anatomy and its degrees of freedom, generating a collection of synthetic motion patterns that is compared with the captured input patterns in order to finally classify the input gesture. For the evaluation of our system a significant collection of gestures has been compiled, getting results for 3D pattern classification as well as a comparison with the results using only 2D information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4

Similar content being viewed by others

Notes

  1. http://wii.com

  2. http://www.xbox.com/kinect/

  3. http://playstation.com/psmove/

  4. Atlas Gloves: A DIY Hand Gesture Interface for Google Earth, http://atlasgloves.org/about

  5. http://research.microsoft.com/en-us/um/people/zliu/actionrecorsrc/

  6. http://www-vpu.eps.uam.es/publications/papermotion/indexpaper.html, (user: vision, password: visionpaper)

  7. http://www.mesa-imaging.ch/

  8. http://www.obliquity.com/computer/speedtest.html

References

  1. Athitsos, V., & Sclaroff, S. (2003). Estimating 3d hand pose from a cluttered image. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, 432.

    Google Scholar 

  2. Breuer, P., Eckes, C., & Muller, S. (2007). Hand gesture recognition with a novel ir time-of-flight range camera: a pilot study. In Computer vision/computer graphics collaboration techniques 3rd international conference, MIRAGE (pp. 247–260).

  3. Castilla, D., Miralles, I., Jorquera, M., Botella, C., Baños, R., Montesa, J., & Ferran, C. (2009). Analysis and testing of metaphors for the definition of a gestual language based on real users interaction: vision project. In 13th international conference on human-computer interaction. San Diego.

  4. Chen, Y.T., & Tseng, K.T. (2007). Developing a multiple-angle hand gesture recognition system for human machine interactions. In 33rd annual conference of the IEEE industrial electronics society, 2007. IECON 2007 (pp. 489–492).

  5. Cheng, J., Xie, C., Bian, W., & Tao, D. (2012). Feature fusion for 3d hand gesture recognition by learning a shared hidden space. Pattern Recognition Letters, 33(4), 476–484. Intelligent Multimedia Interactivity.

    Article  Google Scholar 

  6. Fothergill, S., Mentis, H.M., Kohli, P., & Nowozin, S. (2012). Instructing people for training gestural interactive systems. In J. A. Konstan, E. H. Chi, & K. Höök (Eds.), CHI. ACM (pp. 1737–1746).

  7. Grzeszczuk, R., Bradski, G., Chu, M., & Bouguet, J. (2000). Stereo based gesture recognition invariant to 3d pose and lighting. In IEEE conference on computer vision and pattern recognition (pp. I: 826–833).

  8. Guomundsson, S., Pardás, M., Larsen, R., Aanaes H., & Casas, J.R. (2010). TOF imaging in smart room environments towards improved people tracking. Computer Vision and Image Understanding, 114(12), 1376–1384.

    Article  Google Scholar 

  9. Holden, E.J., Lee, G., & Owens, R. (2005). Australian sign language recognition. Machine Vision and Applications, 16, 312–320.

    Article  Google Scholar 

  10. Hu, J., Brown, M.K., & Turin, W. (1996). Hmm based on-line handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell., 18, 1039–1045.

    Article  Google Scholar 

  11. ISO9241-11. (1998). Ergonomic requirements for office work with visual display terminals (vdts) - part 11: Guidance on usability.

  12. Jang, I.Y., & Lee, K. (2010). Depth video based human model reconstruction resolving self-occlusion. IEEE Transactions on Consumer Electronics, 56(3), 1933–1941.

    Article  Google Scholar 

  13. Kelly, D., McDonald, J., & Markham, C. (2010). A person independent system for recognition of hand postures used in sign language. Pattern Recognition Letters, 31(11), 1359–1368.

    Article  Google Scholar 

  14. Keskin, C., & Akarun, L. (2009). Stars: sign tracking and recognition system using input-output hmms. Pattern Recognition Letters, 30(12), 1086–1095. Image/video-based Pattern Analysis and HCI Applications.

    Article  Google Scholar 

  15. Kim, H.J., Lee, J., & Park, J.H. (2008). Dynamic hand gesture recognition using a cnn model with 3d receptive fields. In 2008 international conference on neural networks and signal processing (pp. 14–19).

  16. Kim, S.Y., Cho, J.H., Koschan, A., & Abidi, M.A. (2010). Spatial and temporal enhancement of depth images captured by a time-of-flight depth sensor, (pp. 2358–2361).

  17. Kollorz, E., Penne, J., Hornegger, J., & Barke, A. (2008). Gesture recognition with a time-of-flight camera. International Journal of Intelligent Systems Technologies and Applications, 5(3/4), 334–343.

    Article  Google Scholar 

  18. Kong, W., & Ranganath, S. (2010). Sign language phoneme transcription with rule-based hand trajectory segmentation. Journal of Signal Processing Systems, 59(2), 211–222.

    Article  Google Scholar 

  19. Kurakin, A., Zhang, Z., & Liu, Z. (2012). A real time system for dynamic hand gesture recognition with a depth sensor. In Proceedings of the 20th european signal processing conference, EUSIPCO 2012, Bucharest, Romania (pp. 1975–1979).

  20. Lantuejoul, C., & Maisonneuve, F. (1984). Geodesic methods in quantitative image analysis. Pattern Recognition, 17(2), 177–187.

    Article  MathSciNet  MATH  Google Scholar 

  21. Laviola, J.J. (2008). Bringing vr and spatial 3d interaction to the masses through video games. IEEE Computer Graphics and Applications, 28(5), 10–15.

    Article  Google Scholar 

  22. Liu, X., & Fujimura, K. (2004). Hand gesture recognition using depth data. In Proceedings of the 6th IEEE international conference on automatic face and gesture recognition, 2004 (pp. 529–534).

  23. Malassiotis, S., & Strintzis, M. (2008). Real-time hand posture recognition using range data. Image and Vision Computing, 26(7), 1027–1037.

    Article  Google Scholar 

  24. Mitra, S., & Acharya, T. (2007). Gesture recognition: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37(3), 311–324.

    Article  Google Scholar 

  25. Molina, J., Escudero-Viñolo, M., Signoriello, A., Pardás, M., Ferrán, C., Bescós, J., Marqués, F., & Martínez, J. (2013). Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models. Machine Vision and Applications, 24(1), 187–204.

    Article  Google Scholar 

  26. Molina, J., & Martínez, J.M. (2014). A synthetic training framework for providing gesture scalability to 2.5d pose-based hand gesture recognition systems. Machine Vision And Applications, 25(5), 1309–1315.

    Article  Google Scholar 

  27. Molina, J., Pajuelo, J.A., Escudero-Viñolo, M., Bescós, J., & Martínez, J.M. (2014). A natural and synthetic corpus for benchmarking of hand gesture recognition systems. Machine Vision and Applications, 25 (4), 943–954.

    Article  Google Scholar 

  28. Nickel, K., & Stiefelhagen, R. (2007). Visual recognition of pointing gestures for human-robot interaction. Image and Vision Computing, 25(12), 1875–1884.

    Article  Google Scholar 

  29. Qin, S., Zhu, X., Yang, Y., & Jiang, Y. (2014). Real-time hand gesture recognition from depth images using convex shape decomposition method. Journal of Signal Processing Systems, 74(1), 47–58.

    Article  Google Scholar 

  30. Sakoe, H., & Chiba, S. (1978). Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics. Speech and Signal Processing, 26(1), 43–49.

    Article  MATH  Google Scholar 

  31. Soutschek, S., Penne, J., Hornegger, J., & Kornhuber, J. (2008). 3-d gesture-based scene navigation in medical imaging applications using time-of-flight cameras. In IEEE computer society conference on computer vision and pattern recognition workshops (pp. 1–6).

  32. Stenger, B., Thayananthan, A., Torr, P.H.S., & Cipolla, R. (2006). Model-based hand tracking using a hierarchical bayesian filter. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9), 1372–1384.

    Article  MATH  Google Scholar 

  33. Teng, X., Wu, B., Yu, W., & Liu, C. (2005). A hand gesture recognition system based on local linear embedding. Journal of Visual Languages and Computing, 16, 442–454.

    Article  Google Scholar 

  34. Usabiaga, J., Erol, A., Bebis, G., Boyle, R., & Twombly, X. (2009). Global hand pose estimation by multiple camera ellipse tracking. Machine Vision and Applications, 21, 1–15.

    Article  Google Scholar 

  35. Wang, J., Liu, Z., Chorowski, J., Chen, Z., & Wu, Y. (2012). Robust 3d action recognition with random occupancy patterns. In Proceedings of the 12th european conference on computer vision - volume part II, ECCV’12 (pp. 872–885). Berlin: Springer.

  36. Wenjun, T., Chengdong, W., Shuying, Z., & Li, J. (2010). Dynamic hand gesture recognition using motion trajectories and key frames. In 2010 2nd international conference on advanced computer control (ICACC) (vol. 3 pp. 163–167).

  37. Yoon, H.S., Soh, J., Bae, Y.J., & Yang, H.S. (2001). Hand gesture recognition using combined features of location, angle and velocity. Pattern Recognition, 34(7), 1491–1501.

    Article  MATH  Google Scholar 

  38. Zheng, G., Wang, C.J., & Boult, T.E. (2007). Application of projective invariants in hand geometry biometrics. IEEE Transactions on Information Forensics and Security, 2(4), 758–768.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Video Processing and Understanding Laboratory Escuela Politécnica Superior, Universidad Autónoma de Madrid Avda. Francisco Tomás y Valiente, 11 Ciudad Universitaria de Cantoblanco, Ctra. de Colmenar Viejo, km 15, E-28049, Madrid, Spain

    Javier Molina, José Antonio Pajuelo & José M. Martínez

Authors
  1. Javier Molina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José Antonio Pajuelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José M. Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Javier Molina.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Molina, J., Pajuelo, J.A. & Martínez, J.M. Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video. J Sign Process Syst 86, 17–25 (2017). https://doi.org/10.1007/s11265-015-1090-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-015-1090-5

Keywords

Search

Navigation