Robust Parameterized Component Analysis

Theory and Applications to 2D Facial Modeling
  • Fernando De la Torre
  • Michael J. Black
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2353)


Principal Component Analysis (PCA) has been successfully applied to construct linear models of shape, graylevel, and motion. In particular, PCA has been widely used to model the variation in the appearance of people’s faces. We extend previous work on facial modeling for tracking faces in video sequences as they undergo significant changes due to facial expressions. Here we develop person-specific facial appearance models (PSFAM), which use modular PCA to model complex intra-person appearance changes. Such models require aligned visual training data; in previous work, this has involved a time consuming and error-prone hand alignment and cropping process. Instead, we introduce parameterized component analysis to learn a subspace that is invariant to affine (or higher order) geometric transformations. The automatic learning of a PSFAM given a training image sequence is posed as a continuous optimization problem and is solved with a mixture of stochastic and deterministic techniques achieving sub-pixel accuracy. We illustrate the use of the 2D PSFAM model with several applications including video-conferencing, realistic avatar animation and eye tracking.


Training Image Motion Parameter Appearance Model Geometric Transformation Facial Animation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    S. Avidan. Support vector tracking. In Conference on Computer Vision and Pattern Recognition, 2001.Google Scholar
  2. 2.
    M. Betke and N. Makris. Fast object recognition in noisy images using simulated annealing. In International Conference Computer Vision, pages 523–530, 1994.Google Scholar
  3. 3.
    M. J. Black, D. J. Fleet, and Y. Yacoob. Robustly estimating changes in image appearance. Computer Vision andImage Understanding, 78(1):8–31, 2000.CrossRefGoogle Scholar
  4. 4.
    M. J. Black and A. D. Jepson. Eigentracking: Robust matching and tracking of objects using view-based representation. International Journal of Computer Vision, 26(1):63–84, 1998.CrossRefGoogle Scholar
  5. 5.
    M. J. Black and Y. Yacoob. Recognizing facial expressions in image sequences using local parameterized models of image motion. International Journal of Computer Vision, 25(1):23–48, 1997.CrossRefGoogle Scholar
  6. 6.
    A. Blake and M. Isard. Active Contours. Springer Verlag, 1998.Google Scholar
  7. 7.
    A. Blake and A. Zisserman. Visual Reconstruction. MIT Press series, Massachusetts, 1987.Google Scholar
  8. 8.
    M. Brand. Voice puppetry. In SIGGRAPH, pages 21–28, 1999.Google Scholar
  9. 9.
    R. Cipolla and A. Pentland. Computer vision for Human-Machine Interaction. Cambridge university press, 1998.Google Scholar
  10. 10.
    T. F. Cootes, G. J. Edwards, and C. J. Taylor. Active appearance models. In European Conference Computer Vision, pages 484–498, 1998.Google Scholar
  11. 11.
    T. F. Cootes and C. J. Taylor. Statistical models of appearance for com-puter vision. In World Wide Web Publication, February 2001. (Available from
  12. 12.
    F. de la Torre. Automatic learning of appearance face models. In Second International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems, pages 32–39, 2001.Google Scholar
  13. 13.
    F. de la Torre and M. J. Black. Dynamic coupled component analysis. In Computer Vision and Pattern Recognition, 2001.Google Scholar
  14. 14.
    F. de la Torre and M. J. Black. Robust principal component analysis for computer vision. In International Conference on Computer Vision, pages 362–369, 2001.Google Scholar
  15. 15.
    F. de la Torre, S. Gong, and S. McKenna. View alignment with dynamically updated affine tracking. In Int. Conf. on Automatic Face and Gesture Recognition, pages 510–515, 1998.Google Scholar
  16. 16.
    F. de la Torre, J. Vitrià, P. Radeva, and J. Melenchón. Eigenfiltering for flexible eigentracking. In International Conference on Pattern Recognition, pages 1118–1121, Barcelona, 2000.Google Scholar
  17. 17.
    F. de la Torre, Y. Yacoob, and L. Davis. A probabilisitc framework for rigid and non-rigid appearance based tracking and recognition. In Int. Conf. on Automatic Face and Gesture Recognition, pages 491–498, 2000.Google Scholar
  18. 18.
    K. I. Diamantaras. Principal Component Neural Networks (Therory and Applications). John Wiley & Sons, 1996.Google Scholar
  19. 19.
    G. J. Edwards, A. Lanitis, C. Taylor, and T. F. Cootes. Statistical models of face images-improving specificity. Image and Vision Computing, 16:203–211, 1998.CrossRefGoogle Scholar
  20. 20.
    G. J. Edwards, C. J. Taylor, and T.F. Cootes. Improving identitification performance by integrating evidence from sequences. In Computer Vision and Pattern Recognition, pages 486–491, 1999.Google Scholar
  21. 21.
    B. J. Frey and N. Jojic. Transformation-invariant clustering and dimensionality reduction. Submitted to IEEE Transaction on Pattern Analysis and Machine Intelligence, 2000.Google Scholar
  22. 22.
    S. Geman and D. McClure. Statistical methods for tomographic image reconstruction. Bulletin of the International Statistical Institute, LII:4:5, 1987.Google Scholar
  23. 23.
    S. Gong, S. Mckenna, and A. Psarrou. Dynamic Vision: From Images to Face Recognition. Imperial College Press, 2000.Google Scholar
  24. 24.
    G. Hager and P. Belhumeur. Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10):1025–1039, 1998.CrossRefGoogle Scholar
  25. 25.
    F. Hampel, E. Ronchetti, P. Rousseeuw, and W. Stahel. Robust Statistics: The Approach Based on Influence Functions. Wiley, New York., 1986.zbMATHGoogle Scholar
  26. 26.
    T. Jebara, K. Russell, and A. Pentland. Mixtures of eigenfeatures for real-time structure from texture. In International Conference on Computer Vision, 1998.Google Scholar
  27. 27.
    A. Lanitis, A. Hill, T. F. Cootes, and C. J. Taylor. Locating facial feature using genetic algorithms. In International Conference on Digital Signal Processing, pages 520–525, 1995.Google Scholar
  28. 28.
    G. Li. Robust regression. In D. C. Hoaglin, F. Mosteller, and J. W. Tukey, editors, Exploring Data, Tables, Trends and Shapes. John Wiley & Sons, 1985.Google Scholar
  29. 29.
    M. Mitchell. An Introduction to Genetic Algorithms. MIT Press, 1996.Google Scholar
  30. 30.
    B. Moghaddam and A. Pentland. Probabilistic visual learning for object representation. Pattern Analysis and Machine Intelligence, 19(7): 137–143, July 1997.Google Scholar
  31. 31.
    S. K. Nayar and T. Poggio. Early Visual Learning. Oxford University Press, 1996.Google Scholar
  32. 32.
    R. P. N. Rao. Development of localized oriented receptive fields by learning a translation-invariant code for natural images. Network: Comput. Neural Systems, 9:219–234, 1998.zbMATHCrossRefGoogle Scholar
  33. 33.
    H. Schweitzer. Optimal eigenfeature selection by optimal image registration. In Conference on Computer Vision and Pattern Recognition, pages 219–224, 1999.Google Scholar
  34. 34.
    M. Turk and A. Pentland. Eigenfaces for recognition. Journal Cognitive Neuroscience, 3(1):71–86, 1991.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Fernando De la Torre
    • 1
  • Michael J. Black
    • 2
  1. 1.Department of Communications and Signal Theory, La Salle School of EngineeringUniversitat Ramon LLullBarcelonaSpain
  2. 2.Department of Computer ScienceBrown UniversityProvidenceUSA

Personalised recommendations