Multimedia Tools and Applications

, Volume 75, Issue 19, pp 11929–11943 | Cite as

A novel approach to extract hand gesture feature in depth images

  • Zhaojie Ju
  • Dongxu Gao
  • Jiangtao Cao
  • Honghai Liu


This paper proposes a novel approach to extract human hand gesture features in real-time from RGB-D images based on the earth mover’s distance and Lasso algorithms. Firstly, hand gestures with hand edge contour are segmented using a contour length information based de-noise method. A modified finger earth mover’s distance algorithm is then applied applied to locate the palm image and extract fingertip features. Lastly and more importantly, a Lasso algorithm is proposed to effectively and efficiently extract the fingertip feature from a hand contour curve. Experimental results are discussed to demonstrate the effectiveness of the proposed approach.


Hand gestures Feature extraction Kinect sensor EMD 



The authors would like to acknowledge support from DREAM project of EU FP7-ICT 611391 and Research Project of State Key Laboratory of Mechanical System and Vibration China MSV201508.


  1. 1.
    Alexiadis DS et al. (2011) Evaluating a dancer’s performance using kinect-based skeleton tracking. Proceedings of the 19th ACM international conference on Multimedia. ACMGoogle Scholar
  2. 2.
    Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. Computer Vision–ECCV 2006. Springer Berlin Heidelberg. 404–417Google Scholar
  3. 3.
    Dollár P, Rabaud V, Cottrell G, Belongie S (2005) Behavior recognition via sparse spatio-temporal features. In: Visual surveillance and performance evaluation of tracking and surveillance, 2005. 2nd Joint IEEE International Workshop on, IEEE, 65–72Google Scholar
  4. 4.
    Farid H, Simoncelli EP (1997) Optimally rotation-equivariant directional deriva- tive kernels. In: Computer analysis of images and patterns. Springer, 207–214Google Scholar
  5. 5.
    Han J et al (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334CrossRefGoogle Scholar
  6. 6.
    Ju Z, Liu H (2011) A unified fuzzy framework for human hand motion recognition. IEEE Trans Fuzzy Syst 19(5):901–913CrossRefGoogle Scholar
  7. 7.
    Ju Z, Liu H (2012) Fuzzy Gaussian mixture models. Pattern Recogn 45(3):1146–1158CrossRefzbMATHGoogle Scholar
  8. 8.
    Ju Z, Wang Y, Chen SY, Liu H (2013) Depth and RGB image alignment for hand gesture segmentation using Kinect, Proc. International Conference on Machine Learning and Cybernetics, 1–8, Tianjing, ChinaGoogle Scholar
  9. 9.
    Keskin C et al. (2013) Real time hand pose estimation using depth sensors. Consumer Depth Cameras for Computer Vision. Springer London. 119–137Google Scholar
  10. 10.
    Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2):1437–1454CrossRefGoogle Scholar
  11. 11.
    Li C, Ma H, Yang C, Fu M (2014) Teleoperation of a virtual iCub robot under framework of parallel system via hand gesture recognition, Fuzzy Systems (FUZZ-IEEE). IEEE International Conference on, 1469–1474Google Scholar
  12. 12.
    Liang H, Yuan J, Thalmann D (2012) 3D fingertip and palm tracking in depth image sequences. Proceedings of the 20th ACM international conference on Multimedia. ACMGoogle Scholar
  13. 13.
    Matyunin S et al. (2011) Temporal filtering for depth maps generated by kinect depth camera. 3DTV Conference: the true vision-capture, Transmission and display of 3D video (3DTV-CON), 2011. IEEEGoogle Scholar
  14. 14.
    Reddivari H, Yang* C, Ju Z, Liang P, Li Z, Xu B (2014) Teleoperation Control of Baxter Robot using Body Motion Tracking, the 2014 I.E. International Conference on Multisensor Fusion and Information Integration, Beijing, China, September 28–30, pp. 1–6Google Scholar
  15. 15.
    Tang M (2011) Recognizing hand gestures with microsoft’s kinect. Palo Alto: Department of Electrical Engineering of Stanford University:[sn]Google Scholar
  16. 16.
    Tara R, Santosa P, Adji T (2012) Hand segmentation from depth image using anthropometric approach in natural interface development. Int J Sci Eng Res 3(5):1–4Google Scholar
  17. 17.
    Wang B, Yang C, Xie Q (2012) Human-machine Interfaces based on Electromyography and Kinect applied to Teleoperation of a Mobile Humanoid Robot, the 10th World Congress on Intelligent Control and Automation (WCICA), pp. 3903–3908, Beijing, ChinaGoogle Scholar
  18. 18.
    Zhou R, Yuan J, Meng J, Zhang Z (2013) Robust part-based hand gesture recognition using Kinect sensor. IEEE Trans. On Multimedia 15(5)Google Scholar
  19. 19.
    Zhou R et al. (2011) Minimum near-convex decomposition for robust shape representation. Computer Vision (ICCV), 2011 I.E. International Conference on. IEEEGoogle Scholar
  20. 20.
    Zhou R et al. (2011) Robust hand gesture recognition with kinect sensor. Proceedings of the 19th ACM international conference on Multimedia. ACMGoogle Scholar
  21. 21.
    Zhu H-M, Pun C-M (2011) Hand gesture recognition with motion tracking on spatial-temporal filtering. In: Proceedings of the 10th International Confer- ence on Virtual Reality Continuum and Its Applications in Industry, ACM, 273–278Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Zhaojie Ju
    • 1
    • 2
  • Dongxu Gao
    • 3
  • Jiangtao Cao
    • 4
  • Honghai Liu
    • 5
  1. 1.University of PortsmouthPortsmouthUK
  2. 2.State Key Laboratory of Mechanical System and VibrationShanghaiChina
  3. 3.University of PortsmouthPortsmouthUK
  4. 4.Liaoning Shihua UniversityFushunChina
  5. 5.University of PortsmouthPortsmouthUK

Personalised recommendations