Skip to main content
SpringerLink
Log in

Real time 3D face and facial feature tracking

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Detecting and tracking human faces in video sequences is useful in a number of applications such as gesture recognition and human-machine interaction. In this paper, we show that online appearance models (holistic approaches) can be used for simultaneously tracking the head, the lips, the eyebrows, and the eyelids in monocular video sequences. Unlike previous approaches to eyelid tracking, we show that the online appearance models can be used for this purpose. Neither color information nor intensity edges are used by our proposed approach. More precisely, we show how the classical appearance-based trackers can be upgraded in order to deal with fast eyelid movements. The proposed eyelid tracking is made robust by avoiding eye feature extraction. Experiments on real videos show the usefulness of the proposed tracking schemes as well as their enhancement to our previous approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://www.prima.inrialpes.fr/FGnet/html/benchmarks.html.

  2. The exact value depends on the camera-intrinsic parameters and the absolute depth.

  3. http://www.science.enotes.com/science-fact-finder/human-body/how-often-does-human-eye-blink.

References

  1. Ahlberg, J.: Real-time facial feature tracking using an active model with fast image warping. In: International Workshop on Very Low Bitrate Video (VLBV). Athens, Greece (2001)

  2. Ahlberg, J.: An active model for facial feature tracking. EURASIP J. Appl. Signal Process. 2002(6), 566–571 (2002)

    Article  Google Scholar 

  3. Ahlberg, J: Model-based coding: extraction, coding, and evaluation of face model parameters. Ph.D. thesis, No. 761, Linköping University, Sweden (2002)

  4. Cascia, M., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: an approach based on registration of texture-mapped 3D models. IEEE Trans. Pattern Anal. Mach. Intell. 22(4), 322–336 (2000)

    Article  Google Scholar 

  5. Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6): 681–684 (2001)

    Article  Google Scholar 

  6. Dornaika, F., Davoine, F.: On appearance based face and facial action tracking. IEEE Trans. Circuits Syst. Video Technol. 16(9):1107–1124 (2006)

    Article  Google Scholar 

  7. Dornaika, F., Orozco, J., Gonzalez, J.: Combined head, lips, eyebrows, and eyelids tracking using adaptive appearance models. In: LNCS 4069. IV Conference on Articulated Motion and Deformable Objects, pp. 110–119 (2006)

  8. Grauman, K., Betke, M., Gips, J., Bradski, G.R.: Communication via eye blinks: detection and duration analysis in real time. In: International Conference on Computer Vision and Pattern Recognition (2001)

  9. Jepson, A., Fleet, D., El-Maraghi, T. Robust online appearance models for visual tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(10):1296–1311 (2003)

    Article  Google Scholar 

  10. Lee, D.: Effective Gaussian mixture learning for video background subtraction. IEEE Trans. Pattern Anal. Mach. Intell. 27(5):827–832 (2005)

    Article  Google Scholar 

  11. Liu, H., Wu, Y., Zha, H.: Eye states detection from color facial image sequence. In: SPIE international conference on image and graphics, vol. 4875, pp. 693–698 (2002)

  12. Matthews, I., Baker, S.: Active appearance models revisited. Int. J. Comput. Vis. 60(2):135–164 (2004)

    Article  Google Scholar 

  13. Moriyama, T., Kanade, T., Cohn, J., Xiao, J., Ambadar, Z., Gao, J., Imamura, H.: Automatic recognition of eye blinking in spontaneously occuring behavior. In: International Conference on Pattern Recognition (2002)

  14. Sirohey, S., Rosenfeld, A., Duric, Z.: A method of detecting and tracking irises and eyelids in video. Pattern Recognit. 35(6):1389–1401 (2002)

    Article  MATH  Google Scholar 

  15. Tan, H., Zhang, Y.J.: Detecting eye blink states by tracking iris and eyelids. Pattern Recognit. Lett. 27(6):667–675 (2006)

    Article  Google Scholar 

  16. Tian, Y., Kanade, T., Cohn, J.F.: Dual-state parametric eye tracking. In: International Conference on Automatic Face and Gesture Recognition (2000)

  17. Uzunova, V.I.: An eyelids and eye corners detection and tracking method for rapid iris tracking. Master’s thesis, University of Magdeburg, 2005

  18. Zhou, S., Chellappa, R., Mogghaddam, B.: Visual tracking and recognition using appearance-adaptive models in particle filters. IEEE Trans. Image Process. 13(11):1473–1490 (2004)

    Article  Google Scholar 

  19. Zhu, J., Yang, J.: Subpixel eye gaze tracking. In: International Conference on Automatic Face and Gesture Recognition (2002)

Download references

Acknowledgments

The authors thank Dr. Franck Davoine from CNRS, Compiegne, France, for providing the video sequence shown in Figure 10.

Author information

Authors and Affiliations

  1. Institut Géographique National Laboratoire MATIS, 2 Avenue Pasteur, 94165, Saint Mandé, France

    Fadi Dornaika

  2. Computer Vision Center, Campus UAB, 08193, Bellaterra, Barcelona, Spain

    Javier Orozco

Authors
  1. Fadi Dornaika
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Javier Orozco
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fadi Dornaika.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dornaika, F., Orozco, J. Real time 3D face and facial feature tracking. J Real-Time Image Proc 2, 35–44 (2007). https://doi.org/10.1007/s11554-007-0032-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-007-0032-2

Keywords

Search

Navigation