Real Time Head Nod and Shake Detection Using HMMs

  • Yeon Gu Kang
  • Hyun Jea Joo
  • Phill Kyu Rhee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4253)


This paper discusses a technique of detecting a head nod and shake. The proposed system is composed of face detection, eye detection and head nod and head shake detection. We use motion segmentation algorithm that makes use of differencing to detect moving people’s faces. The novelty of this paper comes from the differencing in real time input images, preprocessing to remove noises (morphological operator and so on), detecting edge lines and restoration, finding the face area and cutting the head candidate. Moreover, we adopt K-means algorithm for finding head. Eye detection extracts the location of eyes from the detected face region. It is performed at the region close to a pair of eyes for real-time eye detecting. Head nod and shake can be detected by HMMs those are adapted by a directional vector. The HMMs vector can also be used to determine neutral as well as head nod and head shake. These techniques are implemented on a lot of images and a notable success is notified.


Face Detection Automatic Face Sinusoid Pattern Head Location Head Gesture 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ekman, P., Huang, T., Sejnowski, T., Hager, J.: Final Report to NSF of the planning Workshop on Facial Expression Understanding, Technical report, National Science Foundation, Human Interaction Lab., UCSF, CA 94143 (1993)Google Scholar
  2. 2.
    Kapoor, A., Picard, R.W.: A Real-Time Head Nod and Shake Detector. In: Proceedings from the Workshop on Perceptive User Interfaces, November (2001)Google Scholar
  3. 3.
    Tan, W., Rong, G.: A real-time head nod and shake detector using HMMs. Expert Systems with Applications 25, 461–466 (2003)CrossRefGoogle Scholar
  4. 4.
    Davis, J.W., Vakes, S.: A perceptual user interface for recognizing head gesture acknowledgements. In: ACM workshop on perceptual user interfaces (2001)Google Scholar
  5. 5.
    Toyama, K.: Look, ma—no hands! Hands free cursor control with real-time 3D face tracking. In: Proceedings of workshop on perceptual user interfaces, pp. 49–54 (1998)Google Scholar
  6. 6.
    Chen, Q., Wu, H., Fukumoto, T., Yachida, M.: 3D head pose estimation without feature tracking. In: IEEE International Conference on Automatic Face and Gesture Recognition (1998)Google Scholar
  7. 7.
    Heinzman, J., Zelinsky, A.: 3-D facial pose and gaze point estimation using a robust real-time tracking paradigm. In: IEEE International Conference on Automatic Face and Gesture Recognition (1998)Google Scholar
  8. 8.
    Kawato, S., Ohya, J.: Real-time detection of nodding and head-shaking by directly detecting and tracking the between-eyes. In: Fourth IEEE international conference on automatic face and gesture recognition (2000)Google Scholar
  9. 9.
    Pitas, I.: Digital image processing algorithms. Prentice-Hall, Englewood Cliffs (1993)Google Scholar
  10. 10.
    Lumina, R.L., Shapiro, G., Zuniga, O.: A New Connected Components Algorithm for Virtual Memory Computers. Computer Vision, Graphics, and Image Processing 22, 287–300 (1983)CrossRefGoogle Scholar
  11. 11.
    Jung, J.N., Nam, M.-Y., Rhee, P.K.: Adaptive Eye Location Using FuzzyART. In: Wang, L., Chen, K., S. Ong, Y. (eds.) ICNC 2005. LNCS, vol. 3611, pp. 109–118. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Rabiner, L.: A tutorial on hidden markov models and Selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–285 (1989)CrossRefGoogle Scholar
  13. 13.
    Nefian, A.V., Hayes III, M.H.: Face detection and recognition using hidden Markov models. In: IEEE International Conference on Image Processing (1998)Google Scholar
  14. 14.
    Baum, L., Petrie, T., Weiss, N.: A maximization techniques occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Stat. 41(1), 164–171 (1970)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yeon Gu Kang
    • 1
  • Hyun Jea Joo
    • 1
  • Phill Kyu Rhee
    • 1
  1. 1.Dept. of Computer Science & Engineering Inha UniversityIncheonSouth Korea

Personalised recommendations