Multiple Facial Feature Tracking Using Multi-cue Based Prediction Model

  • Congyong Su
  • Hong Zhou
  • Li Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3179)


It is challenging to track multiple facial features simultaneously in video while rich facial expressions are presented in a human face. To accurately predict the positions of multiple facial features’ contours is important and difficult. This paper proposes a multi-cue prediction model based tracking algorithm. In the prediction model, CAMSHIFT is used to track the face in video in advance, and facial features’ spatial constraint is utilized to roughly obtain the positions of facial features. Second order autoregressive process (ARP) based dynamic model is combined with Bayesian network based dynamic model. Incorporating ARP’s quickness into graphical model’s accurateness, we obtain the fusion of the prediction. Finally the prediction model and the measurement model are integrated into the framework of Kalman filter. The experimental results show that our algorithm can accurately track multiple facial features with varied facial expressions.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. IJCV (1987)Google Scholar
  2. 2.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. PAMI (2001)Google Scholar
  3. 3.
    Bradski, G.R.: Real time face and object tracking as a component of a perceptual user interface. In: Proc. WACV, pp. 214–219 (1998)Google Scholar
  4. 4.
    Kapoor, A., Picard, R.W.: Real-time fully automatic upper facial feature tracking. In: Proc. FG, pp. 8–13 (2002)Google Scholar
  5. 5.
    Gu, H., Ji, Q., Zhu, Z.: Active facial tracking for fatigue detection. In: Proc. WACV, pp. 137–142 (2002)Google Scholar
  6. 6.
    de Boor, C.: A practical guide to splines. Springer, Heidelberg (1978)MATHGoogle Scholar
  7. 7.
    Blake, A., Isard, M.: 3d position, attitude and shape input using video tracking of hands and lips. In: Proc. Siggraph, pp. 185–192 (1994)Google Scholar
  8. 8.
    Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
  9. 9.
    Murphy, K., Weiss, Y.: The factored frontier algorithm for approximate inference in dbns. In: Proc. UAI, pp. 378–385 (2001)Google Scholar
  10. 10.
    Yedidia, J.S., Freeman, W.T.: Understanding belief propagation and its generalizations. In: Proc. IJCAI (2001)Google Scholar
  11. 11.
    Press, W., Teukolsky, S.A., Vetterling, W.T., et al.: Numerical recipes in C, 2nd edn. Cambridge University Press, Cambridge (1992)MATHGoogle Scholar
  12. 12.
    Kanade, T., Cohn, J., Tian, Y.L.: Comprehensive database for facial expression analysis. In: Proc. FG, pp. 46–53 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Congyong Su
    • 1
  • Hong Zhou
    • 2
  • Li Huang
    • 1
  1. 1.College of Computer ScienceZhejiang UniversityHangzhouChina
  2. 2.Department of Instrumentation Science and EngineeringZhejiang UniversityHangzhouChina

Personalised recommendations