Skip to main content

Feature Descriptors for Depth-Based Hand Gesture Recognition

Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)

Abstract

Depth data acquired by consumer depth cameras provide a very informative description of the hand pose that can be exploited for accurate gesture recognition. A typical hand gesture recognition pipeline requires to identify the hand, extract some relevant features and exploit a suitable machine learning technique to recognize the performed gesture. This chapter deals with the recognition of static poses. It starts by describing how the hand can be extracted from the scene exploiting depth and color data. Then several different features that can be extracted from the depth data are presented. Finally, a multi-class support vector machines (SVM) classifier is applied to the presented features in order to evaluate the performance of the various descriptors.

Keywords

  • Support Vector Machine
  • Convex Hull
  • Gesture Recognition
  • Depth Data
  • Hand Shape

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-08651-4_11
  • Chapter length: 23 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   99.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-08651-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   129.00
Price excludes VAT (USA)
Hardcover Book
USD   129.00
Price excludes VAT (USA)
Fig. 11.1
Fig. 11.2
Fig. 11.3
Fig. 11.4
Fig. 11.5
Fig. 11.6
Fig. 11.7
Fig. 11.8
Fig. 11.9
Fig. 11.10

Notes

  1. 1.

    In Eqs. (11.3) and (11.4) \(L\) is considered as a periodic function with period \(2\pi \).

References

  1. Ballan L, Taneja A, Gall J, Van Gool L, Pollefeys M (2012) Motion capture of hands in action using discriminative salient points. In: Proceedings of the European conference on computer vision (ECCV), Firenze, October 2012

    Google Scholar 

  2. Biswas K, Basu S (2011) Gesture recognition using microsoft kinect. In: 5th international conference on automation, robotics and applications (ICARA), December 2011, pp 100–103

    Google Scholar 

  3. Doliotis P, Stefan A, McMurrough C, Eckhard D, Athitsos V (2011) Comparing gesture recognition accuracy using color and depth information. In: Proceedings of the 4th international conference on pervasive technologies related to assistive environments ( PETRA’11), pp 20:1–20:7

    Google Scholar 

  4. Dominio F, Donadeo M, Marin G, Zanuttigh P, Cortelazzo GM (2013) Hand gesture recognition with depth data. In: Proceedings of the 4th ACM/IEEE international workshop on analysis and retrieval of tracked events and motion in imagery stream, ACM, pp 9–16

    Google Scholar 

  5. Dominio F, Donadeo M, Zanuttigh P (2013) Combining multiple depth-based descriptors for hand gesture recognition. Pattern Recognition Lett

    Google Scholar 

  6. Garg P, Aggarwal N, Sofat S (2009) Vision based hand gesture recognition. World Acad Sci Eng Technol 49(1):972–977

    Google Scholar 

  7. Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334

    Google Scholar 

  8. Herrera Daniel, Kannala Juho, Heikkilä Janne (2012) Joint depth and color camera calibration with distortion correction. IEEE Trans Pattern Anal Mach Intell 34(10):2058–2064

    CrossRef  Google Scholar 

  9. Keskin G, Kirac G, Kara YE, Akarun L (2011) Real time hand pose estimation using depth sensors. In: ICCV Workshops, November 2011, pp 1228–1234

    Google Scholar 

  10. Keskin C, Furkan Kıraç, Kara YE, Akarun L (2012) Hand pose estimation and hand shape classification using multi-layered randomized decision forests. In: Proceedings of the European conference on computer vision (ECCV), pp 852–863

    Google Scholar 

  11. Eva K, Jochen P, Joachim H, Alexander B (2008) Gesture recognition with a time-of-flight camera. Int J Intell Syst Technol Appl 5(3/4):334–343

    Google Scholar 

  12. Kumar N, Belhumeur PN, Biswas A, Jacobs DW, Kress WJ, Lopez I, Soares JVB (2012) Leafsnap: a computer vision system for automatic plant species identification. In Proceedings of the European conference on computer vision (ECCV), October 2012

    Google Scholar 

  13. Kurakin A, Zhang Z, Liu Z (2012) A real-time system for dynamic hand gesture recognition with a depth sensor. In: Proceedings of EUSIPCO

    Google Scholar 

  14. Li Y (2012) Hand gesture recognition using kinect. In: IEEE 3rd international conference on software engineering and service science (ICSESS), June 2012, pp 196–199

    Google Scholar 

  15. Liu X, Fujimura K (2004) Hand gesture recognition using depth data. In: Proceedings sixth IEEE international conference on automatic face and gesture recognition, May 2004, pp 529–534

    Google Scholar 

  16. Manay S, Cremers D, Hong B-w, Yezzi AJ, Soatto S (2006) Integral invariants for shape matching. IEEE Trans Pattern Anal Mach Intell 28(10):1602–1618

    Google Scholar 

  17. Marin G, Fraccaro M, Donadeo M, Dominio F, Zanuttigh P (2013) Palm area detection for reliable hand gesture recognition. In: Proceedings of MMSP

    Google Scholar 

  18. Mo Z, Neumann U (2006) Real-time hand pose recognition using low-resolution depth images. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, vol 2, pp 1499–1505

    Google Scholar 

  19. Nanni L, Lumini A, Dominio F, Donadeo M, Zanuttigh P (2014) Ensemble to improve gesture recognition. Int J Autom Ident Technology (to appear)

    Google Scholar 

  20. Oikonomidis I, Kyriazis N, Argyros AA (2011) Efficient model-based 3d tracking of hand articulations using kinect. In: Proceedings of the 22nd British machine vision conference (BMVC 2011)

    Google Scholar 

  21. Pedersoli F, Adami N, Benini S, Leonardi R (2012) Xkin—extendable hand pose and gesture recognition library for kinect. In: Proceedings of ACM conference on multimedia 2012—open source competition, Nara, Japan, October 2012

    Google Scholar 

  22. Pedersoli F, Benini S, Adami N, Leonardi R (2014) Xkin: an open source framework for hand pose and gesture recognition using kinect. Vis Comput 1–16

    Google Scholar 

  23. Pugeault N, Bowden R (2011) Spelling it out: real-time asl fingerspelling recognition. In: Proceedings of the 1st IEEE workshop on consumer depth cameras for computer vision, pp 1114–1119

    Google Scholar 

  24. Ren Z, Meng J, Yuan J (2011) Depth camera based hand gesture recognition and its applications in human–computer-interaction. In: Proceedings of International conference on information, communications and signal processing (ICICS), December 2011, pp 1–5

    Google Scholar 

  25. Ren Z, Yuan J, Zhang Z (2011) Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In Proceedings of the 19th ACM international conference on multimedia, MM’11, ACM, NY, USA, 2011, pp 1093–1096

    Google Scholar 

  26. Sun C, Zhang T, Bao BK, Xu C, Mei T (2013) Discriminative exemplar coding for sign language recognition with kinect. IEEE Trans Cybern 43(5):1418–1428

    Google Scholar 

  27. Suryanarayan P, Subramanian A, Mandalapu D (2010) Dynamic hand pose recognition using depth data. In: Proceedings of international conference on pattern recognition (ICPR), August 2010, pp 3105–3108

    Google Scholar 

  28. Van den Bergh M, Van Gool L (2011) Combining rgb and tof cameras for real-time 3d hand gesture interaction. In: IEEE Workshop on applications of computer vision (WACV), January 2011, pp 66–72

    Google Scholar 

  29. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, CVPR 2001, vol 1, IEEE, pp I–511

    Google Scholar 

  30. Wachs JP, Kölsch M, Stern H, Edan Y (2011) Vision-based hand-gesture applications. Commun ACM 54(2):60–71

    Google Scholar 

  31. Wan T, Wang Y, Li J (2012) Hand gesture recognition system using depth data. In: Proceedings of 2nd international conference on consumer electronics, communications and networks (CECNet), April 2012, pp 1063–1066

    Google Scholar 

  32. Wang J, Liu Z, Chorowski J, Chen Z, Wu Y (2012) Robust 3d action recognition with random occupancy patterns. In: Proceedings of the European conference on computer vision (ECCV)

    Google Scholar 

  33. Wen Y, Hu C, Yu G, Wang C (2012) A robust method of detecting hand gestures using depth sensors. In: Proceedings of haptic audio visual environments and games (HAVE), 2012, pp 72–77

    Google Scholar 

  34. Zabulis X, Baltzakis H, Argyros A (2009) Vision-based hand gesture recognition for human computer interaction. In: The universal access handbook, human factors and ergonomics, Chap. 34, Lawrence Erlbaum Associates Inc. (LEA), June 2009, pp 34.1–34.30

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fabio Dominio .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Dominio, F., Marin, G., Piazza, M., Zanuttigh, P. (2014). Feature Descriptors for Depth-Based Hand Gesture Recognition. In: Shao, L., Han, J., Kohli, P., Zhang, Z. (eds) Computer Vision and Machine Learning with RGB-D Sensors. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-08651-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-08651-4_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-08650-7

  • Online ISBN: 978-3-319-08651-4

  • eBook Packages: Computer ScienceComputer Science (R0)