Feature Descriptors for Depth-Based Hand Gesture Recognition

  • Fabio Dominio
  • Giulio Marin
  • Mauro Piazza
  • Pietro Zanuttigh
Chapter

Abstract

Depth data acquired by consumer depth cameras provide a very informative description of the hand pose that can be exploited for accurate gesture recognition. A typical hand gesture recognition pipeline requires to identify the hand, extract some relevant features and exploit a suitable machine learning technique to recognize the performed gesture. This chapter deals with the recognition of static poses. It starts by describing how the hand can be extracted from the scene exploiting depth and color data. Then several different features that can be extracted from the depth data are presented. Finally, a multi-class support vector machines (SVM) classifier is applied to the presented features in order to evaluate the performance of the various descriptors.

References

  1. 1.
    Ballan L, Taneja A, Gall J, Van Gool L, Pollefeys M (2012) Motion capture of hands in action using discriminative salient points. In: Proceedings of the European conference on computer vision (ECCV), Firenze, October 2012Google Scholar
  2. 2.
    Biswas K, Basu S (2011) Gesture recognition using microsoft kinect. In: 5th international conference on automation, robotics and applications (ICARA), December 2011, pp 100–103Google Scholar
  3. 3.
    Doliotis P, Stefan A, McMurrough C, Eckhard D, Athitsos V (2011) Comparing gesture recognition accuracy using color and depth information. In: Proceedings of the 4th international conference on pervasive technologies related to assistive environments ( PETRA’11), pp 20:1–20:7Google Scholar
  4. 4.
    Dominio F, Donadeo M, Marin G, Zanuttigh P, Cortelazzo GM (2013) Hand gesture recognition with depth data. In: Proceedings of the 4th ACM/IEEE international workshop on analysis and retrieval of tracked events and motion in imagery stream, ACM, pp 9–16Google Scholar
  5. 5.
    Dominio F, Donadeo M, Zanuttigh P (2013) Combining multiple depth-based descriptors for hand gesture recognition. Pattern Recognition LettGoogle Scholar
  6. 6.
    Garg P, Aggarwal N, Sofat S (2009) Vision based hand gesture recognition. World Acad Sci Eng Technol 49(1):972–977Google Scholar
  7. 7.
    Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334Google Scholar
  8. 8.
    Herrera Daniel, Kannala Juho, Heikkilä Janne (2012) Joint depth and color camera calibration with distortion correction. IEEE Trans Pattern Anal Mach Intell 34(10):2058–2064CrossRefGoogle Scholar
  9. 9.
    Keskin G, Kirac G, Kara YE, Akarun L (2011) Real time hand pose estimation using depth sensors. In: ICCV Workshops, November 2011, pp 1228–1234Google Scholar
  10. 10.
    Keskin C, Furkan Kıraç, Kara YE, Akarun L (2012) Hand pose estimation and hand shape classification using multi-layered randomized decision forests. In: Proceedings of the European conference on computer vision (ECCV), pp 852–863Google Scholar
  11. 11.
    Eva K, Jochen P, Joachim H, Alexander B (2008) Gesture recognition with a time-of-flight camera. Int J Intell Syst Technol Appl 5(3/4):334–343Google Scholar
  12. 12.
    Kumar N, Belhumeur PN, Biswas A, Jacobs DW, Kress WJ, Lopez I, Soares JVB (2012) Leafsnap: a computer vision system for automatic plant species identification. In Proceedings of the European conference on computer vision (ECCV), October 2012Google Scholar
  13. 13.
    Kurakin A, Zhang Z, Liu Z (2012) A real-time system for dynamic hand gesture recognition with a depth sensor. In: Proceedings of EUSIPCOGoogle Scholar
  14. 14.
    Li Y (2012) Hand gesture recognition using kinect. In: IEEE 3rd international conference on software engineering and service science (ICSESS), June 2012, pp 196–199Google Scholar
  15. 15.
    Liu X, Fujimura K (2004) Hand gesture recognition using depth data. In: Proceedings sixth IEEE international conference on automatic face and gesture recognition, May 2004, pp 529–534Google Scholar
  16. 16.
    Manay S, Cremers D, Hong B-w, Yezzi AJ, Soatto S (2006) Integral invariants for shape matching. IEEE Trans Pattern Anal Mach Intell 28(10):1602–1618Google Scholar
  17. 17.
    Marin G, Fraccaro M, Donadeo M, Dominio F, Zanuttigh P (2013) Palm area detection for reliable hand gesture recognition. In: Proceedings of MMSPGoogle Scholar
  18. 18.
    Mo Z, Neumann U (2006) Real-time hand pose recognition using low-resolution depth images. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, vol 2, pp 1499–1505Google Scholar
  19. 19.
    Nanni L, Lumini A, Dominio F, Donadeo M, Zanuttigh P (2014) Ensemble to improve gesture recognition. Int J Autom Ident Technology (to appear)Google Scholar
  20. 20.
    Oikonomidis I, Kyriazis N, Argyros AA (2011) Efficient model-based 3d tracking of hand articulations using kinect. In: Proceedings of the 22nd British machine vision conference (BMVC 2011)Google Scholar
  21. 21.
    Pedersoli F, Adami N, Benini S, Leonardi R (2012) Xkin—extendable hand pose and gesture recognition library for kinect. In: Proceedings of ACM conference on multimedia 2012—open source competition, Nara, Japan, October 2012Google Scholar
  22. 22.
    Pedersoli F, Benini S, Adami N, Leonardi R (2014) Xkin: an open source framework for hand pose and gesture recognition using kinect. Vis Comput 1–16Google Scholar
  23. 23.
    Pugeault N, Bowden R (2011) Spelling it out: real-time asl fingerspelling recognition. In: Proceedings of the 1st IEEE workshop on consumer depth cameras for computer vision, pp 1114–1119Google Scholar
  24. 24.
    Ren Z, Meng J, Yuan J (2011) Depth camera based hand gesture recognition and its applications in human–computer-interaction. In: Proceedings of International conference on information, communications and signal processing (ICICS), December 2011, pp 1–5Google Scholar
  25. 25.
    Ren Z, Yuan J, Zhang Z (2011) Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In Proceedings of the 19th ACM international conference on multimedia, MM’11, ACM, NY, USA, 2011, pp 1093–1096Google Scholar
  26. 26.
    Sun C, Zhang T, Bao BK, Xu C, Mei T (2013) Discriminative exemplar coding for sign language recognition with kinect. IEEE Trans Cybern 43(5):1418–1428Google Scholar
  27. 27.
    Suryanarayan P, Subramanian A, Mandalapu D (2010) Dynamic hand pose recognition using depth data. In: Proceedings of international conference on pattern recognition (ICPR), August 2010, pp 3105–3108Google Scholar
  28. 28.
    Van den Bergh M, Van Gool L (2011) Combining rgb and tof cameras for real-time 3d hand gesture interaction. In: IEEE Workshop on applications of computer vision (WACV), January 2011, pp 66–72Google Scholar
  29. 29.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, CVPR 2001, vol 1, IEEE, pp I–511Google Scholar
  30. 30.
    Wachs JP, Kölsch M, Stern H, Edan Y (2011) Vision-based hand-gesture applications. Commun ACM 54(2):60–71Google Scholar
  31. 31.
    Wan T, Wang Y, Li J (2012) Hand gesture recognition system using depth data. In: Proceedings of 2nd international conference on consumer electronics, communications and networks (CECNet), April 2012, pp 1063–1066Google Scholar
  32. 32.
    Wang J, Liu Z, Chorowski J, Chen Z, Wu Y (2012) Robust 3d action recognition with random occupancy patterns. In: Proceedings of the European conference on computer vision (ECCV)Google Scholar
  33. 33.
    Wen Y, Hu C, Yu G, Wang C (2012) A robust method of detecting hand gestures using depth sensors. In: Proceedings of haptic audio visual environments and games (HAVE), 2012, pp 72–77Google Scholar
  34. 34.
    Zabulis X, Baltzakis H, Argyros A (2009) Vision-based hand gesture recognition for human computer interaction. In: The universal access handbook, human factors and ergonomics, Chap. 34, Lawrence Erlbaum Associates Inc. (LEA), June 2009, pp 34.1–34.30Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Fabio Dominio
    • 1
  • Giulio Marin
    • 1
  • Mauro Piazza
    • 1
  • Pietro Zanuttigh
    • 1
  1. 1.Department of Information EngineeringPadovaItaly

Personalised recommendations