Vision-Based Approach for Real-Time Hand Detection and Gesture Recognition

  • Rayane El SibaiEmail author
  • Chady Abou Jaoude
  • Jacques Demerjian
Conference paper


The need for interaction between humans and computer devices has advanced so much in recent years. Human-computer interaction (HCI) is the direct manipulation of graphic objects, in particular 3D objects, using several predefined gestures. A gesture can be defined as a physical movement of the hands, face, eyes, and body of the human and is an essential component of the language generation process. In HCI, the use of hand gestures provides an intuitive, attractive, and natural alternative for the interaction between the user and the computer. In this chapter, we present and discuss several vision-based approaches for real-time hand detection which represents the main challenge in real-time HCI applications. Then, we propose a new vision-based approach based on skin color detection for real-time hand detection and gesture recognition. We try to reduce as much as possible the constraints and limitations of the existing approaches. Our method for hand segmentation detects the user’s hand(s), even if the user’s face or other people are viewed by the camera, and can know if the user is doing a gesture using one hand or two hands. Also, our method is robust to the scene’s colors and illumination conditions, as it corrects the video’s colors and brightness before performing the hand segmentation. We also study the performance of hand detection using three color spaces. We show that the RGB mask was the best one. Our system can recognize several hand gestures and allows the user to manipulate a 2D image in real time.


  1. 1.
    Jayashree R Pansare, Shravan H Gawande, and Maya Ingle. Real-time static hand gesture recognition for American sign language (ASL) in complex background. Journal of Signal and Information Processing, 3(03):364, 2012.Google Scholar
  2. 2.
    Annamária R Várkonyi-Kóczy and Balázs Tusor. Human–computer interaction for smart environment applications using fuzzy hand posture and gesture models. IEEE Transactions on Instrumentation and Measurement, 60(5):1505–1514, 2011.CrossRefGoogle Scholar
  3. 3.
    Maureen Schultz, Janet Gill, Sabiha Zubairi, Ruth Huber, and Fred Gordin. Bacterial contamination of computer keyboards in a teaching hospital. Infection Control & Hospital Epidemiology, 24(4):302–303, 2003.CrossRefGoogle Scholar
  4. 4.
    Lawrence Y Deng, Jason C Hung, Huan-Chao Keh, Kun-Yi Lin, Yi-Jen Liu, Nan-Ching Huang, et al. Real-time hand gesture recognition by shape context based matching and cost matrix. JNW, 6(5):697–704, 2011.Google Scholar
  5. 5.
    Kui Liu and Nasser Kehtarnavaz. Real-time robust vision-based hand gesture recognition using stereo images. Journal of Real-Time Image Processing, 11(1):201–209, 2016.CrossRefGoogle Scholar
  6. 6.
    TB Patil, Aakash Jain, Supriya C Sawant, Debnath Bhattacharyya, and Hye-Jin Kim. Virtual interactive hand gestures recognition system in real time environment. International Journal of Database Theory and Application, 9(7):39–50, 2016.CrossRefGoogle Scholar
  7. 7.
    Hasup Lee, Yoshisuke Tateyama, and Tetsuro Ogi. Hand gesture recognition using blob detection for immersive projection display system. World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering, 6(2):260–263, 2012.Google Scholar
  8. 8.
    Mokhtar M Hasan and Pramod K Mishra. Real time fingers and palm locating using dynamic circle templates. International Journal of Computer Applications, 41(6), 2012.Google Scholar
  9. 9.
    Hui-Shyong Yeo, Byung-Gook Lee, and Hyotaek Lim. Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware. Multimedia Tools and Applications, 74(8):2687–2715, 2015.CrossRefGoogle Scholar
  10. 10.
    Manuj Paliwal, Gaurav Sharma, Dina Nath, Astitwa Rathore, Himanshu Mishra, and Soumik Mondal. A dynamic hand gesture recognition system for controlling vlc media player. In Advances in Technology and Engineering (ICATE), 2013 International Conference on, pages 1–4. IEEE, 2013.Google Scholar
  11. 11.
    Pushkar Dhawale, Masood Masoodian, and Bill Rogers. Bare-hand 3d gesture input to interactive systems. In Proceedings of the 7th ACM SIGCHI New Zealand chapter’s international conference on Computer-human interaction: design centered HCI, pages 25–32. ACM, 2006.Google Scholar
  12. 12.
    chandar Subash, Amalraj Willson, and sambandam Gnana. Real-time actuation of cylindrical manipulator model in opengl based on hand gestures recognized using open cvs. International Journal of Modern Engineering Research, pages 3497–3501, 2012.Google Scholar
  13. 13.
    Dong-Luong Dinh, Sungyoung Lee, and Tae-Seong Kim. Hand number gesture recognition using recognized hand parts in depth images. Multimedia Tools and Applications, 75(2):1333–1348, 2016.CrossRefGoogle Scholar
  14. 14.
    Dariu M Gavrila and Larry S Davis. 3-d model-based tracking of humans in action: a multi-view approach. In Computer Vision and Pattern Recognition, 1996. Proceedings CVPR’96, 1996 IEEE Computer Society Conference on, pages 73–80. IEEE, 1996.Google Scholar
  15. 15.
    Andrew Blake, Ben North, and Michael Isard. Learning multi-class dynamics. In Advances in neural information processing systems, pages 389–395, 1999.Google Scholar
  16. 16.
    Yoichi Sato, Yoshinori Kobayashi, and Hideki Koike. Fast tracking of hands and fingertips in infrared images for augmented desk interface. In Automatic Face and Gesture Recognition, 2000. Proceedings. Fourth IEEE International Conference on, pages 462–467. IEEE, 2000.Google Scholar
  17. 17.
    Erdem Yoruk, Ender Konukoglu, Bülent Sankur, and Jérôme Darbon. Shape-based hand recognition. IEEE transactions on image processing, 15(7):1803–1815, 2006.CrossRefGoogle Scholar
  18. 18.
    Yuntao Cui and John J Weng. Hand sign recognition from intensity image sequences with complex backgrounds. In Automatic Face and Gesture Recognition, 1996., Proceedings of the Second International Conference on, pages 259–264. IEEE, 1996.Google Scholar
  19. 19.
    Brian Funt, Kobus Barnard, and Lindsay Martin. Is machine colour constancy good enough? Computer Vision—ECCV’98, pages 445–459, 1998.Google Scholar
  20. 20.
    Mohamed Abdou Berbar. Novel colors correction approaches for natural scenes and skin detection techniques. International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS, 11(2):1–10, 2011.Google Scholar
  21. 21.
    Fayin Li and Harry Wechsler. Open set face recognition using transduction. IEEE transactions on pattern analysis and machine intelligence, 27(11):1686–1697, 2005.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Rayane El Sibai
    • 1
    Email author
  • Chady Abou Jaoude
    • 2
  • Jacques Demerjian
    • 3
  1. 1.Université Pierre et Marie CurieParisFrance
  2. 2.TICKET Lab, Faculty of EngineeringAntonine UniversityBaabdaLebanon
  3. 3.LARIFA-EDST LaboratoryFaculty of Sciences, Lebanese UniversityFanarLebanon

Personalised recommendations