3D Facial Pose Tracking in Uncalibrated Videos

  • Gaurav Aggarwal
  • Ashok Veeraraghavan
  • Rama Chellappa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3776)


This paper presents a method to recover the 3D configuration of a face in each frame of a video. The 3D configuration consists of the 3 translational parameters and the 3 orientation parameters which correspond to the yaw, pitch and roll of the face, which is important for applications like face modeling, recognition, expression analysis, etc. The approach combines the structural advantages of geometric modeling with the statistical advantages of a particle-filter based inference. The face is modeled as the curved surface of a cylinder which is free to translate and rotate arbitrarily. The geometric modeling takes care of pose and self-occlusion while the statistical modeling handles moderate occlusion and illumination variations. Experimental results on multiple datasets are provided to show the efficacy of the approach. The insensitivity of our approach to calibration parameters (focal length) is also shown.


Illumination Change Cylindrical Model Face Tracking State Transition Model Orthographic Projection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Lanitis, A., Taylor, C., Cootes, T.: Automatic interpretation and coding of face images using flexible models. IEEE Trans. on Pattern Analysis and Machine Intelligence 19, 743–756 (1997)CrossRefGoogle Scholar
  2. 2.
    Yuille, A.L., Cohen, D.S., Hallinan, P.W.: Feature extraction from faces using deformable templates. In: International Conference on Pattern Recognition (1994)Google Scholar
  3. 3.
    Zhou, S., Chellappa, R., Moghaddam, B.: Visual tracking and recognition using appearance-adaptive models in particle filters. IEEE Trans. on Image Processing 11, 1434–1456 (2004)Google Scholar
  4. 4.
    Hager, G.D., Belhumeur, P.N.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. on Pattern Analysis and Machine Intelligence 20, 1025–1039 (1998)CrossRefGoogle Scholar
  5. 5.
    Jebara, T.S., Pentland, A.: Parameterized structure from motion for 3D adaptive feedback tracking of faces. In: IEEE Conference on Computer Vision and Pattern Recognition, San Juan, PR (1997)Google Scholar
  6. 6.
    Pighin, F., Szeliski, R., Salesin, H.: Resynthesizing facial animation through 3D model-based tracking. In: Seventh International Conference on Computer Vision, Kerkyra, Greece, pp. 143–150 (1999)Google Scholar
  7. 7.
    Lu, L., Dai, X., Hager, G.: A particle filter without dynamics for robust 3D face tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, Washington, D.C. (2004)Google Scholar
  8. 8.
    Moon, H., Chellappa, R., Rosenfeld, A.: 3d object tracking using shape-encoded particle propagation. In: International Conference on Computer Vision (2001)Google Scholar
  9. 9.
    Cascia, M.L., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3D models. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 322–336 (2000)CrossRefGoogle Scholar
  10. 10.
    Birchfield, S.: An elliptical head tracker. In: Proceedings of the 31st Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, California, pp. 1710–1714 (1997)Google Scholar
  11. 11.
    Fieguth, P., Terzopoulos, D.: Color-based tracking of heads and other mobile objects at video frame rates. In: IEEE Conference on Computer Vision and Pattern Recognition, San Juan, PR (1997)Google Scholar
  12. 12.
    Doucet, A., Freitas, N.D., Gordon, N.: Sequential Monte Carlo methods in practice. Springer, New York (2001)zbMATHGoogle Scholar
  13. 13.
    Gordon, N.J., Salmond, D.J., Smith, A.F.M.: Novel approach to nonlinear/non-gaussian bayesian state estimation. In: IEE Proceedings on Radar and Signal Processing, vol. 140, pp. 107–113 (1993)Google Scholar
  14. 14.
    Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: Robust online appearance models for visual tracking. IEEE Trans. on Pattern Analysis and Machine Intelligence 25, 1296–1311 (2003)CrossRefGoogle Scholar
  15. 15.
    Lee, K.C., Ho, J., Yang, M.H., Kriegman, D.: Video-based face recognition using probabilistic appearance manifolds. In: IEEE Conference on Computer Vision and Pattern Recognition (2003)Google Scholar
  16. 16.
    Li, B., Chellappa, R.: Face verification through tracking facial features. Journal of the Optical Society of America A 18, 2969–2981 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Gaurav Aggarwal
    • 1
  • Ashok Veeraraghavan
    • 1
  • Rama Chellappa
    • 1
  1. 1.University of MarylandCollege ParkUSA

Personalised recommendations