Real-Time Hand Gesture Detection and Recognition Using Boosted Classifiers and Active Learning

  • Hardy Francke
  • Javier Ruiz-del-Solar
  • Rodrigo Verschae
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4872)


In this article a robust and real-time hand gesture detection and recognition system for dynamic environments is proposed. The system is based on the use of boosted classifiers for the detection of hands and the recognition of gestures, together with the use of skin segmentation and hand tracking procedures. The main novelty of the proposed approach is the use of innovative training techniques - active learning and bootstrap -, which allow obtaining a much better performance than similar boosting-based systems, in terms of detection rate, number of false positives and processing time. In addition, the robustness of the system is increased due to the use of an adaptive skin model, a color-based hand tracking, and a multi-gesture classification tree. The system performance is validated in real video sequences.


Hand gesture recognition hand detection skin segmentation hand tracking active learning bootstrap Adaboost nested cascade classifiers 


  1. 1.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-Based Object Tracking. IEEE Trans. on Pattern Anal. Machine Intell. 25(5), 564–575 (2003)CrossRefGoogle Scholar
  2. 2.
    Liu, X.: Hand gesture recognition using depth data. In: Proc. 6th Int. Conf. on Automatic Face and Gesture Recognition, Seoul, Korea, pp. 529–534 (2004)Google Scholar
  3. 3.
    Kolsch, M., Turk, M.: Robust hand detection. In: Proc. 6th Int. Conf. on Automatic Face and Gesture Recognition, Seoul, Korea, pp. 614–619 (2004)Google Scholar
  4. 4.
    Binh, N.D., Shuichi, E., Ejima, T.: Real-Time Hand Tracking and Gesture Recognition System. In: Proc. GVIP 2005, Cairo, Egypt, pp. 19–21 (2005)Google Scholar
  5. 5.
    Manresa, C., Varona, J., Mas, R., Perales, F.: Hand Tracking and Gesture Recognition for Human-Computer Interaction. Electronic letters on computer vision and image analysis 5(3), 96–104 (2005)Google Scholar
  6. 6.
    Fang, Y., Wang, K., Cheng, J., Lu, H.: A Real-Time Hand Gesture Recognition Method. In: Proc. 2007 IEEE Int. Conf. on Multimedia and Expo, pp. 995–998 (2007)Google Scholar
  7. 7.
    Chen, Q., Georganas, N.D., Petriu, E.M.: Real-time Vision-based Hand Gesture Recognition Using Haar-like Features. In: IMTC 2007. Proc. Instrumentation and Measurement Technology Conf, Warsaw, Poland (2007)Google Scholar
  8. 8.
    Angelopoulou, A., García-Rodriguez, J., Psarrou, A.: Learning 2D Hand Shapes using the Topology Preserving model GNG. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 313–324. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Ong, E.-J., Bowden, R.: A boosted classifier tree for hand shape detection. In: Proc. 6th Int. Conf. on Automatic Face and Gesture Recognition, Seoul, Korea, pp. 889–894 (2004)Google Scholar
  10. 10.
    Wimmer, M., Radig, B.: Adaptive Skin Color Classificator, Int. Journal on Graphics, Vision and Image Processing. Special Issue on Biometrics 2, 39–42 (2006)Google Scholar
  11. 11.
    Verschae, R., Ruiz-del-Solar, J., Correa, M.: A Unified Learning Framework for object Detection and Classification using Nested Cascades of Boosted Classifiers, Machine Vision and Applications (in press) Google Scholar
  12. 12.
    Schapire, R.E., Singer, Y.: Improved Boosting Algorithms using Confidence-rated Predictions. Machine Learning 37(3), 297–336 (1999)zbMATHCrossRefGoogle Scholar
  13. 13.
    Wu, B., Ai, H., Huang, C., Lao, S.: Fast rotation invariant multi-view face detection based on real Adaboost. In: Proc. 6th Int. Conf. on Automatic Face and Gesture Recognition, Seoul, Korea, pp. 79–84 (2004)Google Scholar
  14. 14.
    Abramson, Y., Freund, Y.: Active learning for visual object detection, UCSD Technical Report CS2006-0871 (November 19, 2006) Google Scholar
  15. 15.
    Fröba, B., Ernst, A.: Face detection with the modified census transform. In: Proc. 6th Int. Conf. on Automatic Face and Gesture Recognition, Seoul, Korea, pp. 91–96 (2004)Google Scholar
  16. 16.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 511–518 (2001)Google Scholar
  17. 17.
    Sung, K., Poggio, T.: Example-Based Learning for Viewed-Based Human Face Deteccion. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 39–51 (1998)CrossRefGoogle Scholar
  18. 18.
    The Gesture Recognition Home Page (August 2007), Available at:
  19. 19.
    Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  20. 20.
    IDIAP hand gesture database (August 2007), Available at:
  21. 21.
    RoboCup @Home Official website (August 2007), Available at:
  22. 22.
    UChile RoboCup Teams official website (August 2007), Available at:

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Hardy Francke
    • 1
  • Javier Ruiz-del-Solar
    • 1
  • Rodrigo Verschae
    • 1
  1. 1.Department of Electrical Engineering, Universidad de Chile 

Personalised recommendations