Real-Time Hand Gesture Recognition for Human Robot Interaction

  • Mauricio Correa
  • Javier Ruiz-del-Solar
  • Rodrigo Verschae
  • Jong Lee-Ferng
  • Nelson Castillo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5949)


In this article a hand gesture recognition system that allows interacting with a service robot, in dynamic environments and in real-time, is proposed. The system detects hands and static gestures using cascade of boosted classifiers, and recognize dynamic gestures by computing temporal statistics of the hand’s positions and velocities, and classifying these features using a Bayes classifier. The main novelty of the proposed approach is the use of context information to adapt continuously the skin model used in the detection of hand candidates, to restrict the image’s regions that need to be analyzed, and to cut down the number of scales that need to be considered in the hand-searching and gesture-recognition processes. The system performance is validated in real video sequences. In average the system recognized static gestures in 70% of the cases, dynamic gestures in 75% of them, and it runs at a variable speed of 5-10 frames per second.


dynamic hand gesture recognition static hand gesture recognition context human robot interaction RoboCup @Home 


  1. 1.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-Based Object Tracking. IEEE Trans. on Pattern Anal. Machine Intell. 25(5), 564–575 (2003)CrossRefGoogle Scholar
  2. 2.
    Liu, X., Fujimura, K.: Hand gesture recognition using depth data. In: Proc. 6th Int. Conf. on Automatic Face and Gesture Recognition, Seoul, Korea, pp. 529–534 (2004)Google Scholar
  3. 3.
    Kolsch, M., Turk, M.: Robust hand detection. In: Proc. 6th Int. Conf. on Automatic Face and Gesture Recognition, Seoul, Korea, pp. 614–619 (2004)Google Scholar
  4. 4.
    Dang Binh, N., Shuichi, E., Ejima, T.: Real-Time Hand Tracking and Gesture Recognition System. In: Proc. GVIP 2005, Cairo, Egypt, pp. 19–21 (2005)Google Scholar
  5. 5.
    Manresa, C., Varona, J., Mas, R., Perales, F.: Hand Tracking and Gesture Recognition for Human-Computer Interaction. Electronic letters on computer vision and image analysis 5(3), 96–104 (2005)Google Scholar
  6. 6.
    Fang, Y., Wang, K., Cheng, J., Lu, H.: A Real-Time Hand Gesture Recognition Method. In: Proc. 2007 IEEE Int. Conf. on Multimedia and Expo, pp. 995–998 (2007)Google Scholar
  7. 7.
    Chen, Q., Georganas, N.D., Petriu, E.M.: Real-time Vision-based Hand Gesture Recognition Using Haar-like Features. In: Proc. Instrumentation and Measurement Technology Conf. – IMTC 2007, Warsaw, Poland (2007)Google Scholar
  8. 8.
    Angelopoulou, A., García-Rodriguez, J., Psarrou, A.: Learning 2D Hand Shapes using the Topology Preserving model GNG. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 313–324. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Ruiz-del-Solar, J., Verschae, R.: Skin Detection using Neighborhood Information. In: 6th Int. Conf. on Face and Gesture Recognition – FG 2004, Seoul, Korea, May 2004, pp. 463–468 (2004)Google Scholar
  10. 10.
    Francke, H., Ruiz-del-Solar, J., Verschae, R.: Real-time Hand Gesture Detection and Recognition using Boosted Classifiers and Active Learning. In: Mery, D., Rueda, L. (eds.) PSIVT 2007. LNCS, vol. 4872, pp. 533–547. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Verschae, R., Ruiz-del-Solar, J., Correa, M.: A Unified Learning Framework for object Detection and Classification using Nested Cascades of Boosted Classifiers. Machine Vision and Applications 19(2), 85–103 (2008)CrossRefGoogle Scholar
  12. 12.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 511–518 (2001)Google Scholar
  13. 13.
    Torralba, A., Sinha, P.: On Statistical Context Priming for Object Detection. In: Int. Conf. on Computer Vision – ICCV 2001, vol. 1, pp. 763–770 (2001)Google Scholar
  14. 14.
    Cameron, D., Barnes, N.: Knowledge-based autonomous dynamic color calibration. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup 2003. LNCS (LNAI), vol. 3020, pp. 226–237. Springer, Heidelberg (2004)Google Scholar
  15. 15.
    Jüngel, M., Hoffmann, J., Lötzsch, M.: A real time auto adjusting vision system for robotic soccer. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup 2003. LNCS (LNAI), vol. 3020, pp. 214–225. Springer, Heidelberg (2004)Google Scholar
  16. 16.
    Oliva, A.: Gist of the Scene, Neurobiology of Attention, pp. 251–256. Elsevier, San Diego (2003)Google Scholar
  17. 17.
    Strat, T.: Employing contextual information in computer vision. In: Proc. of DARPA Image Understanding Workshop, pp. 217–229 (1993)Google Scholar
  18. 18.
    Palma-Amestoy, R., Guerrero, P., Ruiz-del-Solar, J., Garretón, C.: Bayesian Spatiotemporal Context Integration Sources in Robot Vision Systems. In: Iocchi, L., Matsubara, H., Weitzenfeld, A., Zhou, C. (eds.) RoboCup 2008. LNCS (LNAI), vol. 5399, pp. 212–224. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  19. 19.
    Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  20. 20.
    RoboCup @Home Official website (January 2009),
  21. 21.
    UChile RoboCup Teams official website (January 2009),
  22. 22.
    Jones, M.J., Rehg, J.M.: Statistical Color Models with Application to Skin Detection. Int. Journal of Computer Vision 46(1), 81–96 (2002)zbMATHCrossRefGoogle Scholar
  23. 23.
    Alon, J., Athitsos, V., Yuan, Q., Sclaroff, S.: A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation. IEEE Trans. on Pattern Anal. Machine Intell. (in press, electrically available on July 28, 2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Mauricio Correa
    • 1
    • 2
  • Javier Ruiz-del-Solar
    • 1
    • 2
  • Rodrigo Verschae
    • 1
  • Jong Lee-Ferng
    • 1
  • Nelson Castillo
    • 1
  1. 1.Department of Electrical EngineeringUniversidad de Chile 
  2. 2.Center for Mining TechnologyUniversidad de Chile 

Personalised recommendations