Combined Head, Lips, Eyebrows, and Eyelids Tracking Using Adaptive Appearance Models

  • Fadi Dornaika
  • Javier Orozco
  • Jordi Gonzàlez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4069)


The ability to detect and track human heads and faces in video sequences is useful in a great number of applications, such as human-computer interaction and gesture recognition. Recently, we have proposed a real-time tracker that simultaneously tracks the 3D head pose and facial actions associated with the lips and the eyebrows in monocular video sequences. The developed approach relies on Online Appearance Models where the facial texture is learned during the tracking. This paper extends our previous work in two directions. First, we show that by adopting a non-occluded facial texture model more accurate and stable 3D head pose parameters can be obtained. Second, unlike previous approaches to eyelid tracking, we show that the Online Appearance Models can be used for this purpose. Neither color information nor intensity edges are used by our proposed approach. Moreover, our eyelids tracking does not rely on any eye feature extraction which may lead to erroneous results whenever the eye feature detector fails. Experiments on real videos show the feasibility and usefulness of the proposed approach.


Gesture Recognition Appearance Model Facial Action Active Appearance Model Eyelid Motion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(6), 681–684 (2001)CrossRefGoogle Scholar
  2. 2.
    Cascia, M., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3D models. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(4), 322–336 (2000)CrossRefGoogle Scholar
  3. 3.
    Ahlberg, J.: An active model for facial feature tracking. EURASIP Journal on Applied Signal Processing 2002(6), 566–571 (2002)CrossRefGoogle Scholar
  4. 4.
    Matthews, I., Baker, S.: Active appearance models revisited. International Journal of Computer Vision 60(2), 135–164 (2004)CrossRefGoogle Scholar
  5. 5.
    Dornaika, F., Davoine, F.: On appearance based face and facial action tracking. IEEE Transactions on Circuits and Systems for Video Technology (in press)Google Scholar
  6. 6.
    Sirohey, S., Rosenfeld, A., Duric, Z.: A method of detecting and tracking irises and eyelids in video. Pattern Recognition 35(6), 1389–1401 (2002)MATHCrossRefGoogle Scholar
  7. 7.
    Liu, H., Wu, Y., Zha, H.: Eye states detection from color facial image sequence. In: SPIE International Conference on Image and Graphics, vol. 4875, pp. 693–698 (2002)Google Scholar
  8. 8.
    Tian, Y., Kanade, T., Cohn, J.F.: Dual-state parametric eye tracking. In: International Conference on Automatic Face and Gesture Recognition (2000)Google Scholar
  9. 9.
    Zhu, J., Yang, J.: Subpixel eye gaze tracking. In: International Conference on Automatic Face and Gesture Recognition (2002)Google Scholar
  10. 10.
    Ahlberg, J.: Model-based coding: Extraction, coding, and evaluation of face model parameters. PhD thesis, No. 761, Linköping University, Sweden (2002)Google Scholar
  11. 11.
    Jepson, A., Fleet, D., El-Maraghi, T.: Robust online appearance models for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(10), 1296–1311 (2003)CrossRefGoogle Scholar
  12. 12.
    Grauman, K., Betke, M., Gips, J., Bradski, G.R.: Communication via eye blinks - Detection and duration analysis in real time. In: International Conference on Computer Vision and Pattern Recognition (2001)Google Scholar
  13. 13.
    Moriyama, T., Kanade, T., Cohn, J., Xiao, J., Ambadar, Z., Gao, J., Imamura, H.: Automatic recognition of eye blinking in spontaneously occuring behavior. In: International Conference on Pattern Recognition (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Fadi Dornaika
    • 1
  • Javier Orozco
    • 1
  • Jordi Gonzàlez
    • 1
  1. 1.Computer Vision CenterBellaterra, BarcelonaSpain

Personalised recommendations