Sign-Correlation Partition Based on Global Supervised Descent Method for Face Alignment

  • Yongqiang ZhangEmail author
  • Shuang Liu
  • Xiaosong Yang
  • Daming Shi
  • Jian Jun Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10113)


Face alignment is an essential task for facial performance capture and expression analysis. As a complex nonlinear problem in computer vision, face alignment across poses is still not studied well. Although the state-of-the-art Supervised Descent Method (SDM) has shown good performance, it learns conflict descent direction in the whole complex space due to various poses and expressions. Global SDM has been presented to deal with this case by domain partition in feature and shape PCA spaces for face tracking and pose estimation. However, it is not suitable for the face alignment problem due to unknown ground truth shapes. In this paper we propose a sign-correlation subspace method for the domain partition of global SDM. In our method only one reduced low dimensional subspace is enough for domain partition, thus adjusting the global SDM efficiently for face alignment. Unlike previous methods, we analyze the sign correlation between features and shapes, and project both of them into a mutual sign-correlation subspace. Each pair of projected shape and feature keep sign consistent in each dimension of the subspace, so that each hyperoctant holds the condition that one general descent exists. Then a set of general descent directions are learned from the samples in different hyperoctants. Our sign-correlation partition method is validated in the public face datasets, which includes a range of poses. It indicates that our methods can reveal their latent relationships to poses. The comparison with state-of-the-art methods for face alignment demonstrates that our method outperforms them especially in uncontrolled conditions with various poses, while keeping comparable speed.


Face Image Shape Space Active Appearance Model Active Shape Model Face Tracking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by Harbin Institute of Technology Scholarship Fund 2016 and National Centre for Computer Animation, Bournemouth University.

Supplementary material (19 kb)
Supplementary material 1 (zip 18 KB) (2.2 mb)
Supplementary material 2 (zip 2296 KB)


  1. 1.
    Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models—their training and application. Comput. Vis. Image Underst. 61, 38–59 (1995)CrossRefGoogle Scholar
  2. 2.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 681–685 (2001)CrossRefGoogle Scholar
  3. 3.
    Romdhani, S.: A multi-view nonlinear active shape model using kernel PCA. In: British Machine Vision Conference, pp. 483–492 (1999)Google Scholar
  4. 4.
    Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models. BMVC 41, 929–938 (2006)zbMATHGoogle Scholar
  5. 5.
    Gonzalezmora, J., Torre, F.D.L., Murthi, R., Guil, N., Zapata, E.L.: Bilinear active appearance models. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  6. 6.
    Lee, H.S., Kim, D.: Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1102–1116 (2009)CrossRefGoogle Scholar
  7. 7.
    Saragih, J.M., Lucey, S., Cohn, J.F.: Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vision 91, 200–215 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Valstar, M., Martinez, B., Binefa, X., Pantic, M.: Facial point detection using boosted regression and graph models, pp. 2729–2736 (2010)Google Scholar
  9. 9.
    Dollar, P., Welinder, P., Perona, P.: Cascaded pose regression, vol. 238, pp. 1078–1085. IEEE (2010)Google Scholar
  10. 10.
    Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression US Patent Application number 13/728,584 (2012)Google Scholar
  11. 11.
    Xiong, X., De, la Torre, F.: Supervised descent method and its applications to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)Google Scholar
  12. 12.
    Burgosartizzu, X.P., Perona, P., Dollar, P.: Robust face landmark estimation under occlusion. In: IEEE International Conference on Computer Vision, pp. 1513–1520 (2013)Google Scholar
  13. 13.
    Xing, J., Niu, Z., Huang, J., Hu, W., Yan, S.: Towards multi-view and partially-occluded face alignment. In: Computer Vision and Pattern Recognition, pp. 1829–1836 (2014)Google Scholar
  14. 14.
    Yang, H., He, X., Jia, X., Patras, I.: Robust face alignment under occlusion via regional predictive power estimation. IEEE Trans. Image Process. 24, 2393–2403 (2015)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Yan, J., Lei, Z., Yi, D., Li, S.Z.: Learn to combine multiple hypotheses for accurate face alignment. In: IEEE International Conference on Computer Vision Workshops, pp. 392–396 (2013)Google Scholar
  16. 16.
    Zhang, J., Shan, S., Kan, M., Chen, X.: Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 1–16. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10605-2_1 Google Scholar
  17. 17.
    Fan, X., Wang, H., Luo, Z., Li, Y., Hu, W., Luo, D.: Fiducial facial point extraction using a novel projective invariant. IEEE Trans. Image Process. 24, 1164–1177 (2015)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Liu, L., Hu, J., Zhang, S., Deng, W.: Extended supervised descent method for robust face alignment. In: Jawahar, C.V., Shan, S. (eds.) ACCV 2014. LNCS, vol. 9010, pp. 71–84. Springer, Cham (2015). doi: 10.1007/978-3-319-16634-6_6 Google Scholar
  19. 19.
    Martinez, B., Valstar, M.F.: L 2,1-based regression and prediction accumulation across views for robust facial landmark detection. Image Vis. Comput. 45, 371–382 (2015)Google Scholar
  20. 20.
    Lee, D., Park, H., Yoo, C.D.: Face alignment using cascade Gaussian process regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4204–4212 (2015)Google Scholar
  21. 21.
    Martinez, B., Pantic, M.: Facial landmarking for in-the-wild images with local inference based on global appearance. Image Vis. Comput. 36, 40–50 (2015)CrossRefGoogle Scholar
  22. 22.
    Lindner, C., Bromiley, P.A., Ionita, M.C., Cootes, T.F.: Robust and accurate shape model matching using random forest regression-voting. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1862–1874 (2015)CrossRefGoogle Scholar
  23. 23.
    Yang, H., Patras, I.: Fine-tuning regression forests votes for object alignment in the wild. IEEE Trans. Image Process. 24, 619–631 (2014). A Publication of the IEEE Signal Processing SocietyMathSciNetCrossRefGoogle Scholar
  24. 24.
    Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: Conference on Computer Vision and Pattern Recognition, pp. 3476–3483 (2013)Google Scholar
  25. 25.
    Zhou, E., Fan, H., Cao, Z., Jiang, Y., Yin, Q.: Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: IEEE International Conference on Computer Vision Workshops, pp. 386–391 (2013)Google Scholar
  26. 26.
    Tzimiropoulos, G.: Project-out cascaded regression with an application to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  27. 27.
    Xiong, X., De la Torre, F.: Global supervised descent method. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2664–2673 (2015)Google Scholar
  28. 28.
    Feng, Z.H., Huber, P., Kittler, J., Christmas, W., Wu, X.J.: Random cascaded-regression copse for robust facial landmark detection. IEEE Sig. Process. Lett. 22, 76–80 (2015)CrossRefGoogle Scholar
  29. 29.
    Yang, H., Jia, X., Patras, I., Chan, K.P.: Random subspace supervised descent method for regression problems in computer vision. IEEE Sig. Process. Lett. 22, 1816–1820 (2015)CrossRefGoogle Scholar
  30. 30.
    Zhu, S., Li, C., Loy, C.C., Tang, X.: Face alignment by coarse-to-fine shape searching. In: CVPR, pp. 4998–5006 (2015)Google Scholar
  31. 31.
    Feng, Z.H., Hu, G., Kittler, J., Christmas, W., Wu, X.J.: Cascaded collaborative regression for robust facial landmark detection trained using a mixture of synthetic and real images with dynamic weighting. IEEE Trans. Image Process. 24, 3425–3440 (2015)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)Google Scholar
  33. 33.
    Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 fps via regressing local binary features. IEEE Trans. Image Process. 25, 1685–1692 (2014)Google Scholar
  34. 34.
    Zhang, Z., Zhang, W., Ding, H., Liu, J., Tang, X.: Hierarchical facial landmark localization via cascaded random binary patterns. Pattern Recogn. 48, 1277–1288 (2014)CrossRefGoogle Scholar
  35. 35.
    Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: European Conference on Computer Vision, pp. 94–108 (2014)Google Scholar
  36. 36.
    Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: A semi-automatic methodology for facial landmark annotation. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 896–903 (2013)Google Scholar
  37. 37.
    Kostinger, M., Wohlhart, P., Roth, P.M., Bischof, H.: Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization. In: IEEE International Conference on Computer Vision Workshops, ICCV 2011 Workshops, Barcelona, Spain, pp. 2144–2151, 6–13 November 2011Google Scholar
  38. 38.
    Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments (2008)Google Scholar
  39. 39.
    Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2930–2940 (2013)CrossRefGoogle Scholar
  40. 40.
    Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.S.: Interactive facial feature localization. In: European Conference on Computer Vision, pp. 679–692 (2012)Google Scholar
  41. 41.
    Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 31–37 (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yongqiang Zhang
    • 1
    Email author
  • Shuang Liu
    • 2
  • Xiaosong Yang
    • 2
  • Daming Shi
    • 1
  • Jian Jun Zhang
    • 2
  1. 1.Harbin Institute of TechnologyHarbinChina
  2. 2.Bournemouth UniversityPooleUK

Personalised recommendations