A Real Time Two-Level Method for Fingertips Tracking and Number Identification in a Video

  • Ouissem Ben HeniaEmail author
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 76)


This paper presents a real time method to estimate the number of fingers observed in a video. The method tracks the fingertips and exploits the shape of the hand contour to determine the number of fingers observed in a sequence of images. The first step of the proposed method is to detect the hand observed in the input image by segmentation into foreground and background areas using skin colour detection method. The foreground corresponds to the area representing the hand to be tracked. Due to the problem of the lighting variation, HSL colour space was used to represent the colour. The second step consists of computing the hand contour. Then a convex Hull and convexity defects are calculated to detect the fingertips. Principal components analysis (PCA) [13] method is applied on the convex hull to deal with the cases in which only one finger is observed in the image or when the hand is closed. The proposed method could be used to produce different Human Computer Interaction systems (HCI). Experimental results obtained from real images demonstrate the potential of the method.


Hand tracking PCA technique Convex hull 


  1. 1.
    Ben Henia, O., Hariti, M., Bouakaz, S.: A two-step minimization algorithm for model-based hand tracking. In: WSCG (2010)Google Scholar
  2. 2.
  3. 3.
  4. 4.
    Ben Henia, O., Bouakaz, S.: A new depth-based function for 3D hand motion tracking. In: Proceedings of the International Conference on Computer Vision Theory and Applications (VISIGRAPP 2011), pp. 653–658 (2011)Google Scholar
  5. 5.
    Bray, M., Koller-Meier, E., Mueller, P., Van Gool, L., Schraudolph, N.N.: 3D hand tracking by rapid stochastic gradient descent using a skinning model. In: Chambers, A., Hilton, A. (eds.) 1st European Conference on Visual Media Production (CVMP), pp. 59–68. IEEE, March 2004Google Scholar
  6. 6.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)CrossRefGoogle Scholar
  7. 7.
    Delamarre, Q., Faugeras, O.: Finding pose of hand in video images: A stereo-based approach. In: IEEE Proceedings of the third International Conference on Automatic Face and Gesture Recognition, pp. 585–590. IEEE Computer Society (1998)Google Scholar
  8. 8.
    Dhawan, A., Honrao, V.: Implementation of hand detection based techniques for human computer interaction. Int. J. Comput. Appl. 72(17), 6–13 (2013)Google Scholar
  9. 9.
    Graham, R., Francesyao, F.: Finding the convex hull of a simple polygon. J. Algorithms 4(4), 324–331 (1983)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Heap, T., Hogg, D.: Towards 3D hand tracking using a deformable model. In: Face and Gesture Recognition, pp. 140–145 (1996)Google Scholar
  11. 11.
    Ben Henia, O., Bouakaz, S.: 3D hand model animation with a new data-driven method. In: Proceedings of the Workshop on Digital Media and Digital Content Management, DMDCM 2011, pp. 72–76. IEEE Computer Society, Washington, DC (2011)Google Scholar
  12. 12.
    Ike, T., Kishikawa, N., Stenger, B.: A real-time hand gesture interface implemented on a multi-core processor. In: MVA, pp. 9–12 (2007)Google Scholar
  13. 13.
    Jolliffe, I.T.: Principal Component Analysis. Springer, New York (2002)zbMATHGoogle Scholar
  14. 14.
    Kato, M., Chen, Y.-W., Gang, X.: Articulated hand motion tracking using ica-based motion analysis and particle filtering. J. Multimedia 1(3), 52–60 (2006)CrossRefGoogle Scholar
  15. 15.
    Liang, H., Yuan, J., Thalmann, D.: 3D fingertip and palm tracking in depth image sequences. In: Proceedings of the 20th ACM International Conference on Multimedia, MM 2012, pp. 785–788. ACM, New York (2012)Google Scholar
  16. 16.
    Kato, M., Xu, G.: Occlusion-free hand motion tracking by multiple cameras and particle filtering with prediction. IJCSNS Int. J. Comput. Sci. Netw. Secur. 6(10), 58–65 (2006)Google Scholar
  17. 17.
    Montalvão, J., Molina, L., Canuto, J.: Robust hand image processing for biometric application. Pattern Anal. Appl. 13(4), 397–407 (2010)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Oka, K., Sato, Y., Koike, H.: Real-time fingertip tracking and gesture recognition. IEEE Comput. Graph. Appl. 22(6), 64–71 (2002)CrossRefGoogle Scholar
  19. 19.
    Qian, C., Sun, X., Wei, Y., Tang, X., Sun, J.: Realtime and robust hand tracking from depth, June 2014Google Scholar
  20. 20.
    Rosales, R., Athitsos, V., Sigal, L., Sclaroff, S.: 3D hand pose reconstruction using specialized mappings. In: ICCV, pp. 378–385 (2001)Google Scholar
  21. 21.
    Shimada, N., Kimura, K., Shirai, Y.: Real-time 3-D hand posture estimation based on 2-D appearance retrieval using monocular camera. In: Proceedings of the IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems (RATFG-RTS 2001), p. 23. IEEE Computer Society, Washington, DC (2001)Google Scholar
  22. 22.
    Silanon, K., Suvonvorn, N.: Fingertips tracking based active contour for general HCI application, pp. 309–316. Springer, Singapore (2014)Google Scholar
  23. 23.
    Sridhar, S., Feit, A.M., Theobalt, C., Oulasvirta, A.: Investigating the dexterity of multi-finger input for mid-air text entry. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015, pp. 3643–3652. ACM, New York (2015)Google Scholar
  24. 24.
    Sridhar, S., Mueller, F., Oulasvirta, A., Theobalt, C.: Fast and robust hand tracking using detection-guided optimization. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  25. 25.
    Sridhar, S., Oulasvirta, A., Theobalt, C.: Interactive markerless articulated hand motion tracking using RGB and depth data. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2013Google Scholar
  26. 26.
    Sridhar, S., Rhodin, H., Seidel, H.-P., Oulasvirta, A., Theobalt, C.: Real-time hand tracking using a sum of anisotropic gaussians model. In: Proceedings of the International Conference on 3D Vision (3DV), December 2014Google Scholar
  27. 27.
    Stenger, B., Mendonca, P.R.S., Cipolla, R.: Model-based 3D tracking of an articulated hand. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 2, pp. II–310–II–315 (2001)Google Scholar
  28. 28.
    Wang, R.Y., Popović, J.: Real-time hand-tracking with a color glove. In: ACM SIGGRAPH 2009 papers, pp. 1–8. ACM, New York (2009)Google Scholar
  29. 29.
    Wu, Y., Lin, J., Huang, T.S.: Analyzing and capturing articulated hand motion in image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27(12), 1910–1922 (2005)CrossRefGoogle Scholar
  30. 30.
    Wu, Y., Lin, J.Y., Huang, T.S.: Capturing natural hand articulation. In: ICCV, pp. 426–432 (2001)Google Scholar
  31. 31.
    Yeo, H.-S., Lee, B.-G., Lim, H.: Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware. Multimedia Tools Appl. 74(8), 2687–2715 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Department of Computer Science, College of Computer ScienceKing Khaled University AbhaAbhaKingdom of Saudi Arabia

Personalised recommendations