Efficient Hand Articulations Tracking Using Adaptive Hand Model and Depth Map

  • Byeongkeun KangEmail author
  • Yeejin Lee
  • Truong Q. Nguyen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9474)


Real-time hand articulations tracking is important for many applications such as interacting with virtual/augmented reality devices. However, most of existing algorithms highly rely on expensive and high power-consuming GPUs to achieve real-time processing. Consequently, these systems are inappropriate for mobile and wearable devices. In this paper, we propose an efficient hand tracking system which does not require high performance GPUs.

In our system, we track hand articulations by minimizing discrepancy between depth map from sensor and computer-generated hand model. We also re-initialize hand pose at each frame using finger detection and classification. Our contributions are: (a) propose adaptive hand model to consider different hand shapes of users without generating personalized hand model; (b) improve the highly efficient re-initialization for robust tracking and automatic initialization; (c) propose hierarchical random sampling of pixels from each depth map to improve tracking accuracy while limiting required computations. To the best of our knowledge, it is the first system that achieves both automatic hand model adjustment and real-time tracking without using GPUs.


Particle Swarm Optimization Iterative Close Point Wearable Device Hand Shape Finger Length 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108, 52–73 (2007)CrossRefGoogle Scholar
  2. 2.
    Athitsos, V., Sclaroff, S.: Estimating 3D hand pose from a cluttered image. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2003)Google Scholar
  3. 3.
    Wang, R.Y., Popović, J.: Real-time hand-tracking with a color glove. ACM Trans. Graph. 28, 63 (2009)Google Scholar
  4. 4.
    Tang, D., Yu, T.H., Kim, T.K.: Real-time articulated hand pose estimation using semi-supervised transductive regression forests. In: 2013 IEEE International Conference on Computer Vision (ICCV) (2013)Google Scholar
  5. 5.
    Tompson, J., Stein, M., Lecun, Y., Perlin, K.: Real-time continuous pose recovery of human hands using convolutional networks. ACM Trans. Graph. 33, 63 (2014)CrossRefGoogle Scholar
  6. 6.
    Stenger, B., Thayananthan, A., Torr, P., Cipolla, R.: Model-based hand tracking using a hierarchical Bayesian filter. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1372–1384 (2006)CrossRefGoogle Scholar
  7. 7.
    Rehg, J.M., Kanade, T.: Visual tracking of high dof articulated structures: an application to human hand tracking. In: Computer Vision ECCV 1994 (1994)Google Scholar
  8. 8.
    Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Markerless and efficient 26-dof hand pose recovery. In: Proceedings of the 10th Asian Conference on Computer Vision (2011)Google Scholar
  9. 9.
    Oikonomidis, I., Kyriazis, N., Argyros, A.: Efficient model-based 3D tracking of hand articulations using kinect. In: Proceedings of the British Machine Vision Conference (2011)Google Scholar
  10. 10.
    Qian, C., Sun, X., Wei, Y., Tang, X., Sun, J.: Realtime and robust hand tracking from depth. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  11. 11.
    Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., Freedman, D., Kohli, P., Krupka, E., Fitzgibbon, A., Izadi, S.: Accurate, robust, and flexible real-time hand tracking. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015. ACM (2015)Google Scholar
  12. 12.
    Kang, B., Tripathi, S., Nguyen, T.: Real-time sign language fingerspelling recognition using convolutional neural networks from depth map. In: 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR) (2015)Google Scholar
  13. 13.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Fluids Eng. 82, 35–45 (1960)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringUC San DiegoLa JollaUSA

Personalised recommendations