Multimedia Tools and Applications

, Volume 77, Issue 9, pp 10553–10568 | Cite as

Hierarchical topology based hand pose estimation from a single depth image

Article
First Online:
Received:
Revised:
Accepted:
  • 205 Downloads

Abstract

Hand pose estimation benefits large human computer interaction applications. The hand pose has high dimensions of freedom (dof) for joints, and various hand poses are flexible. Hand pose estimation is still a challenge problem. Since hand joints on the hand skeleton topology model have strict relationships between each other, we propose a hierarchical topology based approach to estimate 3D hand poses. First, we determine palm positions and palm orientations by detecting hand fingertips and calculating their directions in depth images. It is the global topology of hand poses. Moreover, we define connection relationships of finger joints as the local topology of hand model. Based on hierarchical topology, we extract angle features to describe hand poses, and adopt the regression forest algorithm to estimate 3D coordinates of hand joints. We further use freedom forrest algorithm to refine ambiguous poses in estimation to solve error accumulation problem. The hierarchical topology based approach ensures estimated hand poses in a reasonable topology, and improves estimation accuracy. We evaluate our approach on two public databases, and experiments illustrate its efficiency. Compared with state-of-the-art approaches, our approach improves estimation accuracy.

Keywords

Hand pose estimation Regression forest Hierarchical topology Pose refinement 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This research is supported by the Natural Science Foundation of China (NSFC) under grant No. 61305043 and grant No. 61673088. It is also supported by the Natural Science Foundation of China (NSFC) under grant No.61572108.

References

  1. 1.
    Albrecht I, Haber J, Seidel H (2003) Construction and animation of anatomically based human hand models ACM SIGGRAPH, pp 98–109Google Scholar
  2. 2.
    Braort A, Gherbi R, Gibet S, Richardson J, Teil D (2001) Gesture-based communication in human-computer interaction Gesture workshopGoogle Scholar
  3. 3.
    Breiman L (2001) Random forests. Mach Learn 45(1):5–32CrossRefMATHGoogle Scholar
  4. 4.
    Brown JA, Capson DW (2012) A framework for 3d model-based visual tracking using a gpu-accelerated particle filter. IEEE Trans Vis Comput Graph 18(1):68–80CrossRefGoogle Scholar
  5. 5.
    Cheng H, Yang L, Liu Z (2016) A survey on 3d hand gesture recognition. IEEE Trans Circuits Syst Video Technol 26(9):1659–1673CrossRefGoogle Scholar
  6. 6.
    Chi X, Li C (2013) Efficient hand pose estimation from a single depth image ICCVGoogle Scholar
  7. 7.
    Erola A, Bebisa G, Nicolescua M, Boyleb RD, Twomblyb X (2007) Vision-based hand pose estimation: a review. Comput Vis Image Underst 108(1-2):52–73CrossRefGoogle Scholar
  8. 8.
    Ge L, Liang H, Yuan J, Thalmann D (2016) Robust 3d hand pose estimation in single depth images: from single-view cnn to multi-view cnns CVPRGoogle Scholar
  9. 9.
    Jonathan T, Murphy S, Yann L, Ken P (2014) Real-time continuous pose recovery of human hands using convolutional networks. ACM Trans Graph:1935–1946Google Scholar
  10. 10.
    Keskin C, Kiac F, Kara YE, Akarun L (2011) Real time hand pose estimation using depth sensors ICCV WorkshopsGoogle Scholar
  11. 11.
    Keskin C, Kirac F, Kara YE (2013) Real time hand pose estimation using depth sensors. Consumer Depth Cameras for Computer Vision, pp 119–137Google Scholar
  12. 12.
    Keskin C, Kirac F, Kara YE, Akarun L (2012) Hand pose estimation and hand shape classification using multi-layered randomized decision forests ECCVGoogle Scholar
  13. 13.
    Liang H, Yuan J, Thalmann D, Zhang Z (2013) Model-based hand pose estimation via spatial-temporal hand parsing and 3d fingertip localization. Vis Comput 29:837–848CrossRefGoogle Scholar
  14. 14.
    Lien C (2005) A scalable model-based hand posture analysis system. Mach Vis Appl 16(3):157–169CrossRefGoogle Scholar
  15. 15.
    Luo Y, Yang Y, Shen F, Huang Z, Zhou P, Shen HT (2017) Robust discrete code modeling for supervised hashing. Pat Recogn. doi: 10.1016/j.patcog.2017.02.034
  16. 16.
    Lu S, Metaxas D, Samaras D, Oliensis J (2003) Using multiple cues for hand tracking and model refinement CVPRGoogle Scholar
  17. 17.
    Nakagawa Y, Kihara K, Lu Hea (2016) Super resolving of the depth map for 3d reconstruction of underwater terrain using kinect ICPADS, pp 1237–1240Google Scholar
  18. 18.
    Oberweger M, Wohlhart P, Lepetit V (2015) Hands deep in deep learning for hand pose estimation Computer ScienceGoogle Scholar
  19. 19.
    Oikonomidis I, Kyriazis N, Argyros A (2011) Efficient model-based 3d tracking of hand articulations using kinect BMVCGoogle Scholar
  20. 20.
    Oikonomidis I, Lourakis M, Argyros AA (2014) Evolutionary quasi-random search for hand articulations tracking CVPRGoogle Scholar
  21. 21.
    Qian C, Sun X, Wei Y, Tang X (2014) Realtime and robust hand tracking from depth CVPRGoogle Scholar
  22. 22.
    Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43(1):1–54CrossRefGoogle Scholar
  23. 23.
    Sato Y, Saito M, Koik H (2001) Real-time input of 3d pose and gestures of a users hand and its applications for hci IEEE Virtual reality conferenceGoogle Scholar
  24. 24.
    Serikawa S, Lu H (2014) Underwater image dehazing using joint trilateral filter. Comput Electr Eng 40(1):41–50CrossRefGoogle Scholar
  25. 25.
    Sharp T, Keskin C, Robertson D, Taylor J, Shotton J, Kim D, Rhemann C, Leichter I, Vinnikov A, Wei Y (2015) Accurate, robust, and flexible real-time hand tracking ACM Conference on human factors in computing systems, pp 3633–3642Google Scholar
  26. 26.
    Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2013) Real-time human pose recognition in parts from single depth images. Machine Learning for Computer Vision 411:119–135CrossRefGoogle Scholar
  27. 27.
    Sridhar S, Rhodin H, Seidel HP (2014) Real-time hand tracking using a sum of anisotropic gaussians model IEEE International conference on 3d visionGoogle Scholar
  28. 28.
    Srinath S, Antti O, Christian T (2013) Interactive markerless articulated hand motion tracking using rgb and depth data ICCVGoogle Scholar
  29. 29.
    Srinath S, Franziska M, Antti O, Christian T (2015) Fast and robust hand tracking using detection-guided optimization CVPRGoogle Scholar
  30. 30.
    Suarez J, Murphy R (2012) Hand gesture recognition with depth images: a review. IEEE international Symposium on Robot and human Interactive Communication, pp 411–417Google Scholar
  31. 31.
    Sudderth A, Mandel M, Freeman W (2004) Visual hand tracking using nonparametric belief propagation CVPRWGoogle Scholar
  32. 32.
    Sun X, Wei Y, Liang S, Tang X, Sun J (2015) Cascaded hand pose regression CVPRGoogle Scholar
  33. 33.
    Supancic J, Rogez G, Yang Y, Ramanan D, Shotton J (2015) Depth-based hand pose estimation: data, methods, and challenges ICCVGoogle Scholar
  34. 34.
    Tang D, Chang H, Tejani A, Kim T (2014) Latent regression forest: Structured estimation of 3d articulated hand posture CVPRGoogle Scholar
  35. 35.
    Tang D, Jonathan T, Kohli P, Keskin C (2015) Opening the black box: Hierarchical sampling optimization for estimating human hand pose ICCVGoogle Scholar
  36. 36.
    Yang Y, Ma Z, Yang Y, Nie F, Shen HT (2015) Multitask spectral clustering by exploring intertask correlation. IEEE Trans Cybern 45(5):1083–1094CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.School of Automation EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina
  2. 2.School of Computer Science and EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina
  3. 3.The 16th InstituteChina Aerospace Science and Technology CorporationXi’anChina

Personalised recommendations