International Journal of Computer Vision

, Volume 107, Issue 2, pp 177–190 | Cite as

Face Alignment by Explicit Shape Regression

  • Xudong CaoEmail author
  • Yichen Wei
  • Fang Wen
  • Jian Sun


We present a very efficient, highly accurate, “Explicit Shape Regression” approach for face alignment. Unlike previous regression-based approaches, we directly learn a vectorial regression function to infer the whole facial shape (a set of facial landmarks) from the image and explicitly minimize the alignment errors over the training data. The inherent shape constraint is naturally encoded into the regressor in a cascaded learning framework and applied from coarse to fine during the test, without using a fixed parametric shape model as in most previous methods. To make the regression more effective and efficient, we design a two-level boosted regression, shape indexed features and a correlation-based feature selection method. This combination enables us to learn accurate models from large training data in a short time (20 min for 2,000 training images), and run regression extremely fast in test (15 ms for a 87 landmarks shape). Experiments on challenging data show that our approach significantly outperforms the state-of-the-art in terms of both accuracy and efficiency.


Face alignment Shape indexed feature Correlation based feature selection Non-parametric shape constraint Tow-level boosted regression 


  1. Belhumeur, P., Jacobs, D., Kriegman, D., & Kumar, N. (2011). Localizing parts of faces using a concensus of exemplars. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  2. Bingham, E., & Mannila, H. (2001). Random projection in dimensionality reduction: Applications to image and text data. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) .Google Scholar
  3. Cootes, T., Edwards, G., & Taylor, C. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681–685.CrossRefGoogle Scholar
  4. Cootes, T., Taylor, C., Cooper, D., Graham, J., et al. (1995). Active shape models-their training and application. Computer Vision and Image Understanding, 61(1), 38–59.CrossRefGoogle Scholar
  5. Cristinacce, D., & Cootes, T. (2006). Feature detection and tracking with constrained local models. In British Machine Vision Conference (BMVC).Google Scholar
  6. Cristinacce, D., & Cootes, T. (2007). Boosted regression active shape models. In British Machine Vision Conference (BMVC).Google Scholar
  7. Dollar, P., Welinder, P., & Perona, P. (2010). Cascaded pose regression. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  8. Duffy, N., & Helmbold, D. P. (2002). Boosting methods for regression. Machine Learning, 47(2–3), 153–200.CrossRefzbMATHGoogle Scholar
  9. Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12), 3736–3745.CrossRefMathSciNetGoogle Scholar
  10. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189–1232.CrossRefzbMATHMathSciNetGoogle Scholar
  11. Huang, G., Mattar, M., Berg, T., Learned-Miller, E. et al. (2008) Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on Faces in’Real-Life’Images: Detection, Alignment, and Recognition.Google Scholar
  12. Jesorsky, O., Kirchberg, K. J., & Frischholz, R. W. (2001). Robust face detection using the hausdorff distance (pp. 90–95). New York: Springer.Google Scholar
  13. Jolliffe, I. (2005). Principal component analysis. Wiley Online Library.Google Scholar
  14. Le, V., Brandt, J., Lin, Z., Bourdev, L., & Huang, T. (2012). Interactive facial feature localization. In European Conference on Computer Vision.Google Scholar
  15. Liang, L., Xiao, R., Wen, F., & Sun, J. (2008). Face alignment via component-based discriminative search. In European Conference on Computer Vision (ECCV).Google Scholar
  16. Matthews, I., & Baker, S. (2004). Active appearance models revisited. International Journal of Computer Vision, 60(2), 135–164.CrossRefGoogle Scholar
  17. Milborrow, S., & Nicolls, F. (2008). Locating facial features with an extended active shape model. In European Conference on Computer Vision (ECCV).Google Scholar
  18. Ozuysal, M., Calonder, M., Lepetit, V., & Fua, P. (2010). Fast keypoint recognition using random ferns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3), 448–461. Google Scholar
  19. Saragih, J., & Goecke, R. (2007). A nonlinear discriminative approach to aam fitting. In International Conference on Computer Vision (ICCV) .Google Scholar
  20. Sauer, P., & Cootes, C. T. T. (2011). Accurate regression procedures for active appearance models. In British Machine Vision Conference (BMVC).Google Scholar
  21. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., et al. (2011). Real-time human pose recognition in parts from single depth images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  22. Sun, Y., Wang, X., & Tang, X. (2013). Deep convolutional network cascade for facial point detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  23. Tropp, J., & Gilbert, A. (2007). Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53(12), 4655–4666.CrossRefMathSciNetGoogle Scholar
  24. Valstar, M., Martinez, B., Binefa, X., & Pantic, M. (2010). Facial point detection using boosted regression and graph models. In IEEE Conference on Computeer Vision and Pattern Recognition (CVPR).Google Scholar
  25. Vukadinovic, D., & Pantic, M. (2005). Fully automatic facial feature point detection using gabor feature based boosted classifiers. International Conference on Systems, Man and Cybernetics, 2, 1692–1698.CrossRefGoogle Scholar
  26. Xiong, X., De la Torre, F. (2013) Supervised descent method and its applications to face alignment. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  27. Zhou, S. K., & Comaniciu, D. (2007). Shape regression machine. In Information Processing in Medical Imaging, (pp. 13–25). Heidelberg: Springer.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Visual Computing Group, Microsoft Research AsiaBeijingPeople’s Republic of China

Personalised recommendations