Multi-cue Facial Feature Detection and Tracking

  • Jingying Chen
  • Bernard Tiddeman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5099)


An efficient and robust facial feature detection and tracking system is presented in this paper. The system is capable of locating a human face automatically. Six facial feature points (pupils, nostrils and mouth corners) are detected and tracked using multiple cues including facial feature intensity and its probability distribution, geometric characteristics and motion information. In addition, in order to improve the robustness of the tracking system, a simple facial feature model is employed to estimate the relative face poses. This system has the advantage of automatically detecting the facial features and recovering the features lost during the tracking process. Encouraging results have been obtained using the proposed system.


Feature Point Facial Feature Search Window British Machine Vision Facial Feature Point 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Barreto, J., Menezes, P., Dias, J.: Human-robot interaction based on haar-like features and eigenfaces. In: Proceedings of the International Conference on Robotics and Automation, New Orleans, pp. 1888–1893 (2004)Google Scholar
  2. 2.
    Fasel, B., Luettin, J.: Automatic Facial Expression Analysis: A Survey. Pattern Recognition 36(1), 259–275 (2003)zbMATHCrossRefGoogle Scholar
  3. 3.
    Tiddeman, B., Perrett, D.: Moving Facial Image Transformations using Static 2D Prototypes. In: Proceedings of the 9-th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2001 (WSCG 2001), Plzen, Czech Republic, February 5-9 (2001)Google Scholar
  4. 4.
    Chen, J., Tiddeman, B.: A stereo head pose tracking system. In: Proceedings of the 5th IEEE International Symposium on Signal Processing and Information Technology, Greece, December 18-21, 2005, pp. 258–263 (2005)Google Scholar
  5. 5.
    Yang, J., Stiefelhagen, R., Meier, U., Waibel, A.: Real time face and facial feature tracking and applications. In: Proceedings of the International Conference on Auditory-Visual Speech Processing AVSP 1998, pp. 207–212 (1998)Google Scholar
  6. 6.
    Stiefelhagen, R., Meier, U., Yang, J.: Real-time lip-tracking for lip reading. In: Proceedings of the Eurospeech 1997, 5th European Conference on Speech Communication and Technology, Rhodos, Greece (1997)Google Scholar
  7. 7.
    Tian, Y., Kanade, T., Cohn, J.F.: Recognizing upper face action unit for facial expression analysis. In: Proceedings of the International Conference on Computer Vision and Pattern recognition, South Caroline, USA, June 2000, pp. 294–301 (2000)Google Scholar
  8. 8.
    Kapoor, A., Picard, R.W.: Real-Time, Fully Automatic Upper Facial Feature Tracking. In: Proceedings of the 5th International Conference on Automatic Face and Gesture Recognition, Washinton DC, USA, May 2002, pp. 10–15 (2002)Google Scholar
  9. 9.
    Matsumoto, Y., Zelinsky, A.: An algorithm for real time stereo vision implementation of head pose and gaze direction measurement. In: Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, France, pp. 499–505 (2000)Google Scholar
  10. 10.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(6), 681–685 (2001)CrossRefGoogle Scholar
  11. 11.
    Matthews, I., Baker, S.: Active Appearance Models Revisited, Technical report: CMU-RI-TR-03-02, the Robotics Institute Carnegie Mellon University (2002)Google Scholar
  12. 12.
    Cristinacce, D., Cootes, T.: Feature detection and tracking with constrained local models. In: Proceedings of British Machine Vision Conference, UK, pp. 929–938 (2006)Google Scholar
  13. 13.
    Viola, P., Jones, M.: Robust real time object detection. In: Proceedings of the 2nd International Workshop on Statistical and Computational Theories of Vision-Modeling, Learning, Computing and Sampling, Vancouver, Canada (July 2001)Google Scholar
  14. 14.
    Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. International Journal of Computer Vision 61, 55–79 (2005)CrossRefGoogle Scholar
  15. 15.
    Bourel, F., Chibelushi, C.C., Low, A.A.: Robust Facial Feature Tracking. In: Proceedings of the Eleventh British Machine Vision Conference, Bristol, UK (September 2000)Google Scholar
  16. 16.
    Feng, G., Yuen, P.: Multi-cue eye detection on grey intensity image. Pattern Recognition 34, 1033–1046 (2001)zbMATHCrossRefGoogle Scholar
  17. 17.
    Kanade, T.: Picture processing by computer complex and recognition of human faces. Technical report, Kyoto University (1973)Google Scholar
  18. 18.
    Peng, K., Chen, L., Ruan, S., Kukharev, G.: A Robust Algorithm for Eye Detection on Gray Intensity Face without Spectacles. Journal of Computer Science & Technology 5(3), 127–132 (2005)Google Scholar
  19. 19.
    Lucas, B., Kanade, T.: An interactive image registration technique with an application in stereovision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  20. 20.
    Trucco, E., Verri, A.: Introductory techniques for 3-d computer vision. Prentice-Hall, New Jersey (1998)Google Scholar
  21. 21.
    Zhu, Z., Ji, Q.: 3D Face Pose Tracking From an Uncalibrated Monocular Camera. In: Proceedings of the 17th International Conference on Pattern Recognition, UK, pp. 400–403 (2004)Google Scholar
  22. 22.
    Yao, P., Evans, G., Calway, A.: Using affine correspondence to estimate 3-d facial pose. In: Proceedings of the International Conference on Image Processing, Greece, pp. 919–922 (2001)Google Scholar
  23. 23.
    Fischler, M.A., Bolles, R.C.: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Comm. of the ACM 24, 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  24. 24.
    Jesorsky, O., Kirchberg, K.J., Frischholz, R.W.: Robust face detection using Hausdorff distance. In: Proceedings of the 3rd International Conference on Audio and Video-based Biometric Person Authentication, Halmstad, Sweden, pp. 90–95 (2001)Google Scholar
  25. 25.
    Hamouz, M., Kittler, J., Kamarainen, J.K., Kalvioinen, H.: Affine invariant face detection and localization using GMM-based feature detectos and enhanced appearance model. In: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 67–72 (2004)Google Scholar
  26. 26.
    Cristinacce, D., Cootes, T., Scott, I.: A multi-stage approach to facial feature detection. In: Proceedings of British Machine Vision Conference, UK, pp. 277–286 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Jingying Chen
    • 1
  • Bernard Tiddeman
    • 1
  1. 1.School of Computer ScienceUniversity of St AndrewsFifeScotland, UK

Personalised recommendations