A Generative Shape Regularization Model for Robust Face Alignment

  • Leon Gu
  • Takeo Kanade
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5302)


In this paper, we present a robust face alignment system that is capable of dealing with exaggerating expressions, large occlusions, and a wide variety of image noises. The robustness comes from our shape regularization model, which incorporates constrained nonlinear shape prior, geometric transformation, and likelihood of multiple candidate landmarks in a three-layered generative model. The inference algorithm iteratively examines the best candidate positions and updates face shape and pose. This model can effectively recover sufficient shape details from very noisy observations. We demonstrate the performance of this approach on two public domain databases and a large collection of real-world face photographs.


Local Binary Pattern Deformable Model Inference Algorithm Kernel Principal Component Analysis Candidate Position 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Terzopoulos, D., Witkin, A., Kass, M.: Snakes: Active contour models. In: International Conference on Computer Vision, pp. 259–268 (1987)Google Scholar
  2. 2.
    Grenander, U., Chow, Y., Keenan, D.M.: Hands: a pattern theoretic study of biological shapes. Springer, New York (1991)CrossRefzbMATHGoogle Scholar
  3. 3.
    Yuille, A.L., Hallinan, P.W., Cohen, D.S.: Feature extraction from faces using deformable templates. International Journal of Computer Vision 8, 99–111 (1992)CrossRefGoogle Scholar
  4. 4.
    Cootes, T.F., Taylor, C., Cooper, D., Graham, J.: Active shape models - their training and their applications. Computer Vision and Image Understanding (1995)Google Scholar
  5. 5.
    Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking. International Journal of Computer Vision 29, 5–28 (1998)CrossRefGoogle Scholar
  6. 6.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 681–685 (2001)CrossRefGoogle Scholar
  7. 7.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d-faces. In: ACM SIGGRAPH (1999)Google Scholar
  8. 8.
    Cootes, T., Taylor, C.: A mixture model for representing shape variation. IVC 17, 567–573 (1999)CrossRefGoogle Scholar
  9. 9.
    Twining, C., Taylor, C.: Kernel principal component analysis and the construction of non-linear active shape models. In: British Machine Vision Conference (2001)Google Scholar
  10. 10.
    Zhou, Y., Zhang, W., Tang, X., Shum, H.: A bayesian mixture model for multi-view face alignment. In: Proceedings of Computer Vision and Pattern Recognition (2005)Google Scholar
  11. 11.
    Gu, L., Kanande, T.: 3d alignment of face in a single image. In: Computer Vision and Pattern Recognition (2006)Google Scholar
  12. 12.
    Zhang, Z., Liu, Z., Adler, D., Cohen, M.F., Hanson, E., Shan, Y.: Robust and rapid generation of animated faces from video images - a model-based modeling approach. International Jornal of Computer Vision (2004)Google Scholar
  13. 13.
    Liang, L., Wen, F., Xu, Y.Q., tang, X., Shum, H.Y.: Accurate face alignment using shape constrained markov network. In: Computer Vision and Pattern Recognition (2006)Google Scholar
  14. 14.
    Amit, Y., Kong, A.: Graphical templates for model registration. IEEE Trans. Pattern Anal. Mach. Intell. 18, 225–236 (1996)CrossRefGoogle Scholar
  15. 15.
    Coughlan, J., Yuille, A., English, C., Snow, D.: Efficient optimization of a deformable template using dynamic programming. In: Computer Vision and Pattern Recognition, pp. 747–752 (1998)Google Scholar
  16. 16.
    Coughlan, J., Ferreira, S.: Finding deformable shapes using loopy belief propagation. In: Tistarelli, M., Bigun, J., Jain, A.K. (eds.) ECCV 2002. LNCS, vol. 2359, pp. 453–468. Springer, Heidelberg (2002)Google Scholar
  17. 17.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face recognition with local binary patterns. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 469–481. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Cristinacce, D., Cootes, T.: Boosted regression active shape models. In: British Machine Vision Conference, pp. 880–889 (2007)Google Scholar
  19. 19.
    Zhou, Y., Gu, L., Zhang, H.: Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In: Computer Vision and Pattern Recognition, pp. 109–116 (2003)Google Scholar
  20. 20.
    Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analysers. Neural Computation 11, 443–482 (1999)CrossRefGoogle Scholar
  21. 21.
    Rowley, H., Baluja, S., Kanade, T.: Rotation invariant neural network-based face detection. In: Computer Vision and Pattern Recognition, pp. 38–44 (1998)Google Scholar
  22. 22.
    Gross, R., Matthews, I., Cohn, J., Baker, S.: Guide to the cmu multi-pie face database. Technical report, CMU RI (2002)Google Scholar
  23. 23.
    Martinez, A., Benavente, R.: The ar face database. CVC Technical Report 24 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Leon Gu
    • 1
  • Takeo Kanade
    • 1
  1. 1.Computer Science DepartmentCarnegie Mellon UniversityUSA

Personalised recommendations