Skip to main content
SpringerLink
Log in
Book cover

Human Motion Sensing and Recognition pp 193–205Cite as

A Novel Approach to Extract Hand Gesture Feature in Depth Images

  • Chapter
  • First Online:

Part of the book series: Studies in Computational Intelligence ((SCI,volume 675))

Abstract

This chapter proposes a novel approach to extract human hand gesture features in real-time from RGB-D images based on the earth mover’s distance and Lasso algorithms. Firstly, hand gestures with hand edge contour are segmented using a contour length information based de-noise method. A modified finger earth mover’s distance algorithm is then applied to locate the palm image and extract fingertip features. Lastly and more importantly, a Lasso algorithm is proposed to effectively and efficiently extract the fingertip feature from a hand contour curve. Experimental results are discussed to demonstrate the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. K. Khoshelham and S.O. Elberink. Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors, 12(2):1437–1454, 2012.

    Article  Google Scholar 

  2. J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, M. Cook, and R. Moore. Real-time human pose recognition in parts from single depth images. Communications of the ACM, 56(1):116–124, 2013.

    Article  Google Scholar 

  3. C. Herrera, J. Kannala, et al. Joint depth and color camera calibration with distortion correction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(10):2058–2064, 2012.

    Article  Google Scholar 

  4. R.A. Newcombe, A.J. Davison, S. Izadi, P. Kohli, O. Hilliges, J. Shotton, D. Molyneaux, S. Hodges, D. Kim, and A. Fitzgibbon. Kinectfusion: Real-time dense surface mapping and tracking. In IEEE International Symposium on Mixed and Augmented Reality, pages 127–136. IEEE, 2011.

    Google Scholar 

  5. S. Izadi, R. A Newcombe, D. Kim, O. Hilliges, D. Molyneaux, S. Hodges, P. Kohli, J. Shotton, A.J. Davison, and A. Fitzgibbon. Kinectfusion: real-time dynamic 3d surface reconstruction and interaction. In ACM SIGGRAPH 2011, page 23. ACM, 2011.

    Google Scholar 

  6. N. Silberman and R. Fergus. Indoor scene segmentation using a structured light sensor. In IEEE International Conference on Computer Vision Workshops, pages 601–608. IEEE, 2011.

    Google Scholar 

  7. Yue Gao, You Yang, Yi Zhen, and Qionghai Dai. Depth error elimination for rgb-d cameras. ACM Transactions on Intelligent Systems and Technology, 6(2):13, 2015.

    Article  Google Scholar 

  8. J. Preis, M. Kessel, M. Werner, and C. Linnhoff-Popien. Gait recognition with kinect. In 1st International Workshop on Kinect in Pervasive Computing, pages 1–6, 2012.

    Google Scholar 

  9. J.L. Raheja, A. Chaudhary, and K. Singal. Tracking of fingertips and centers of palm using kinect. In Third International Conference on Computational Intelligence Modelling and Simulation, pages 248–252. IEEE, 2011.

    Google Scholar 

  10. W. Xu and E.J. Lee. Gesture recognition based on 2d and 3d feature by using kinect device. In International Conference on Information and Security Assurance, 2012.

    Google Scholar 

  11. L. Shao, L. Ji, Y. Liu, and J. Zhang. Human action segmentation and recognition via motion and shape analysis. Pattern Recognition Letters, 33(4):438–445, 2012.

    Article  Google Scholar 

  12. Jianfeng Li and Shigang Li. Eye-model-based gaze estimation by rgb-d camera. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 592–596, 2014.

    Google Scholar 

  13. Zhenbao Liu, Jinxin Huang, Junwei Han, Shuhui Bu, and Jianfeng Lv. (in press) human motion tracking by multiple rgbd cameras. IEEE Transactions on Circuits and Systems for Video Technology, 2016.

    Google Scholar 

  14. J. Sturm, S. Magnenat, N. Engelhard, F. Pomerleau, F. Colas, W. Burgard, D. Cremers, and R. Siegwart. Towards a benchmark for rgb-d slam evaluation. In Proc. of the RGB-D Workshop on Advanced Reasoning with Depth Cameras at Robotics: Science and Systems Conference, volume 2, page 3, 2011.

    Google Scholar 

  15. C. Li, H. Ma, C. Yang, and M. Fu. Teleoperation of a virtual icub robot under framework of parallel system via hand gesture recognition. In IEEE International Conference on Fuzzy Systems, pages 1469–1474. IEEE, 2014.

    Google Scholar 

  16. L. Bo, X. Ren, and D. Fox. Unsupervised feature learning for rgb-d based object recognition. Experimental Robotics, pages 387–402, 2013.

    Google Scholar 

  17. L. Spinello and K.O. Arras. People detection in rgb-d data. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3838–3843. IEEE, 2011.

    Google Scholar 

  18. Z. Zhang. Microsoft kinect sensor and its effect. Multimedia, IEEE, 19(2):4–10, 2012.

    Article  Google Scholar 

  19. L. Cruz, D. Lucio, and L. Velho. Kinect and rgbd images: Challenges and applications. In SIBGRAPI Conference on Graphics, Patterns and Images Tutorials, pages 36–49. IEEE, 2012.

    Google Scholar 

  20. Jungong Han, Ling Shao, Dong Xu, and Jamie Shotton. Enhanced computer vision with microsoft kinect sensor: A review. Cybernetics, IEEE Transactions on, 43(5):1318–1334, 2013.

    Article  Google Scholar 

  21. P. Dollár, V. Rabaud, G. Cottrell, and S. Belongie. Behavior recognition via sparse spatio-temporal features. In Proceeding of Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pages 65–72. Beijing, 2005.

    Google Scholar 

  22. H. Zhu and C. Pun. Hand gesture recognition with motion tracking on spatial-temporal filtering. In Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, pages 273–278. ACM, 2011.

    Google Scholar 

  23. H. Farid and E.P. Simoncelli. Optimally rotation-equivariant directional derivative kernels. In Computer Analysis of Images and Patterns, pages 207–214. Springer, 1997.

    Google Scholar 

  24. Z. Ren, J. Yuan, C. Li, and W. Liu. Minimum near-convex decomposition for robust shape representation. In IEEE International Conference on Computer Vision, pages 303–310. IEEE, 2011.

    Google Scholar 

  25. Z. Ju, Y. Wang, W. Zeng, S. Chen, and H. Liu. Depth and rgb image alignment for hand gesture segmentation using kinect. In International Conference on Machine Learning and Cybernetics, volume 2, pages 913–919. IEEE, 2013.

    Google Scholar 

  26. D.S. Alexiadis, P. Kelly, P. Daras, N.E. O’Connor, T. Boubekeur, and M.B. Moussa. Evaluating a dancer’s performance using kinect-based skeleton tracking. In Proceedings of the 19th ACM international conference on Multimedia, pages 659–662. ACM, 2011.

    Google Scholar 

  27. S. Matyunin, D. Vatolin, Y. Berdnikov, and M. Smirnov. Temporal filtering for depth maps generated by kinect depth camera. In 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video, pages 1–4. IEEE, 2011.

    Google Scholar 

  28. R. Tara, P. Santosa, and T. Adji. Hand segmentation from depth image using anthropometric approach in natural interface development. International Journal of Scientific and Engineering Research, 3(5):1–4, 2012.

    Google Scholar 

  29. H. Liang, J. Yuan, and D. Thalmann. 3d fingertip and palm tracking in depth image sequences. In Proceedings of the 20th ACM international conference on Multimedia, pages 785–788. ACM, 2012.

    Google Scholar 

  30. Z. Ren, J. Meng, J. Yuan, and Z. Zhang. Robust hand gesture recognition with kinect sensor. In Proceedings of the 19th ACM international conference on Multimedia, pages 759–760. ACM, 2011.

    Google Scholar 

  31. C. Keskin, F. Kıraç, Y.E. Kara, and L. Akarun. Real time hand pose estimation using depth sensors. In Consumer Depth Cameras for Computer Vision, pages 119–137. Springer, 2013.

    Google Scholar 

  32. M. Tang. Recognizing hand gestures with microsoft’s kinect. Palo Alto: Department of Electrical Engineering of Stanford University:[sn], 2011.

    Google Scholar 

  33. H. Bay, T. Tuytelaars, and L. Van Gool. Surf: Speeded up robust features. In Computer vision–ECCV 2006, pages 404–417. Springer, 2006.

    Google Scholar 

  34. A. Ferreira, W.C. Celeste, F.A. Cheein, T.F. Bastos-Filho, M. Sarcinelli-Filho, and R. Carelli. Human-machine interfaces based on emg and eeg applied to robotic systems. Journal of NeuroEngineering and Rehabilitation, 5(1):1, 2008.

    Article  Google Scholar 

  35. Zhou Ren, Junsong Yuan, Jingjing Meng, and Zhengyou Zhang. Robust part-based hand gesture recognition using kinect sensor. IEEE transactions on multimedia, 15(5):1110–1120, 2013.

    Article  Google Scholar 

  36. Z. Ju and H. Liu. Fuzzy gaussian mixture models. Pattern Recognition, 45(3):1146–1158, 2012.

    Article  MATH  Google Scholar 

  37. Z. Ju and H. Liu. A unified fuzzy framework for human hand motion recognition. IEEE Transactions on Fuzzy Systems, 19(5):901–913, 2011.

    Article  Google Scholar 

  38. Z. Ju, C. Yang, Z. Li, L. Cheng, and H. Ma. Teleoperation of humanoid baxter robot using haptic feedback. In International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, pages 1–6. IEEE, 2014.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, University of Portsmouth, Portsmouth, UK

    Honghai Liu & Zhaojie Ju

  2. School of Automation, Shenyang Aerospace University, Shenyang, China

    Xiaofei Ji

  3. Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    Chee Seng Chan

  4. Mathematics and Physical Sciences, University of Exeter, Exeter, UK

    Mehdi Khoury

Authors
  1. Honghai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhaojie Ju
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaofei Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chee Seng Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mehdi Khoury
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Honghai Liu .

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer-Verlag GmbH Germany

About this chapter

Cite this chapter

Liu, H., Ju, Z., Ji, X., Chan, C.S., Khoury, M. (2017). A Novel Approach to Extract Hand Gesture Feature in Depth Images. In: Human Motion Sensing and Recognition. Studies in Computational Intelligence, vol 675. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-53692-6_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-53692-6_9

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-53690-2

  • Online ISBN: 978-3-662-53692-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation