Robust Fingertip Tracking with Improved Kalman Filter

  • Chunyang Wang
  • Bo Yuan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8588)


This paper presents a novel approach to reliably tracking multiple fingertips simultaneously using a single optical camera. The proposed technique uses the skin color model to extract the hand region and identifies fingertips via curvature detection. It can remove different types of interfering points through the cross product of vectors and the distance transform. Finally, an improved Kalman filter is employed to predict the locations of fingertips in the current image frame and this information is exploited to associate fingertips with those in the previous image frame to build a complete trajectory. Experimental results show that this method can achieve robust continuous fingertip tracking in a real-time manner.


multi-fingertip tracking curvature detection distance transform Kalman filter data association 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wang, L., Hu, W., Tan, T.: Recent developments in human motion analysis. Pattern Recognition 36, 585–601 (2003)CrossRefGoogle Scholar
  2. 2.
    Oka, K., Sato, Y., Koike, H.: Real-time fingertip tracking and gesture recognition. Computer Graphics and Applications 22, 64–71 (2002)CrossRefGoogle Scholar
  3. 3.
    Xie, Q., Liang, G., Tang, C., Wu, X.: A fast and robust fingertips tracking algorithm for vision-based multi-touch interaction. In: 10th IEEE International Conference on Control and Automation, pp. 1346–1351 (2013)Google Scholar
  4. 4.
    Nakamura, T., Takahashi, S., Tanaka, J.: Double-crossing: A new interaction technique for hand gesture interfaces. In: Lee, S., Choo, H., Ha, S., Shin, I.C. (eds.) APCHI 2008. LNCS, vol. 5068, pp. 292–300. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Jaward, M., Mihaylova, L., Canagarajah, N., Bull, D.: A data association algorithm for multiple object tracking in video sequences. In: The IEE Seminar on Target Tracking: Algorithms and Applications, pp. 129–136 (2006)Google Scholar
  6. 6.
    Letessier, J., Bérard, F.: Visual tracking of bare fingers for interactive surfaces. In: 17th Annual ACM Symposium on User Interface Software and Technology, pp. 119–122 (2004)Google Scholar
  7. 7.
    Wang, X.Y., Zhang, X.W., Dai, G.Z.: An approach to tracking deformable hand gesture for real-time interaction. Journal of Software 18, 2423–2433 (2007)CrossRefGoogle Scholar
  8. 8.
    Zarit, B.D., Super, B.J., Quek, F.K.: Comparison of five color models in skin pixel classification. In: International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 58–63 (1999)Google Scholar
  9. 9.
    An, J.H., Hong, K.S.: Finger gesture-based mobile user interface using a rear-facing camera. In: 2011 IEEE International Conference on Consumer Electronics, pp. 303–304 (2011)Google Scholar
  10. 10.
    Gasparini, F., Schettini, R.: Skin segmentation using multiple thresholding. In: Internet Imaging VII, SPIE, vol. 6061, p. 60610F (2006)Google Scholar
  11. 11.
    Chai, D., Ngan, K.N.: Face segmentation using skin-color map in videophone applications. IEEE Trans. on Circuits and Systems for Video Technology 9, 551–564 (1999)CrossRefGoogle Scholar
  12. 12.
    Lee, T., Hollerer, T., Handy, A.R.: Markerless inspection of augmented reality objects using fingertip tracking. In: 11th IEEE International Symposium on Wearable Computers, pp. 83–90 (2007)Google Scholar
  13. 13.
    Lee, B., Chun, J.: Manipulation of virtual objects in marker-less AR system by fingertip tracking and hand gesture recognition. In: 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human, pp. 1110–1115 (2009)Google Scholar
  14. 14.
    Pan, Z., Li, Y., Zhang, M., Sun, C., Guo, K., Tang, X., Zhou, S.Z.: A real-time multi-cue hand tracking algorithm based on computer vision. In: 2010 IEEE Virtual Reality Conference, pp. 219–222 (2010)Google Scholar
  15. 15.
    Liao, Y., Zhou, Y., Zhou, H., Liang, Z.: Fingertips detection algorithm based on skin colour filtering and distance transformation. In: 12th International Conference on Quality Software, pp. 276–281 (2012)Google Scholar
  16. 16.
    Han, C.Z., Zhu, H., Duan, Z.S.: Multi-source information fusion, 2nd edn. Press of Tsinghua University, Beijing (2010)Google Scholar
  17. 17.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. Journal of Basic Engineering 82, 35–45 (1960)CrossRefGoogle Scholar
  18. 18.
    Schneider, N., Gavrila, D.M.: Pedestrian Path Prediction with Recursive Bayesian Filters: A Comparative Study. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 174–183. Springer, Heidelberg (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Chunyang Wang
    • 1
  • Bo Yuan
    • 1
  1. 1.Intelligent Computing Lab, Division of Informatics Graduate School at ShenzhenTsinghua UniversityShenzhenP.R. China

Personalised recommendations