Soft Computing

, Volume 23, Issue 1, pp 241–249 | Cite as

Sign correlation subspace for face alignment

  • Dansong ChengEmail author
  • Yongqiang Zhang
  • Feng Tian
  • Ce Liu
  • Xiaofang Liu
Methodologies and Application


Face alignment is an essential task for facial performance capture and expression analysis. Current methods such as random subspace supervised descent method, stage-wise relational dictionary and coarse-to-fine shape searching can ease multi-pose face alignment problem, but no method can deal with the multiple local minima problem directly. In this paper, we propose a sign correlation subspace method for domain partition in only one reduced low-dimensional subspace. Unlike previous methods, we analyze the sign correlation between features and shapes and project both of them into a mutual sign correlation subspace. Each pair of projected shape and feature keeps their signs consistent in each dimension of the subspace, so that each hyper octant holds the condition that one general descent exists. Then a set of general descents are learned from the samples in different hyperoctants. Requiring only the feature projection for domain partition, our proposed method is effective for face alignment. We have validated our approach with the public face datasets which include a range of poses. The validation results show that our method can reveal their latent relationships to poses. The comparison with state-of-the-art methods demonstrates that our method outperforms them, especially in uncontrolled conditions with various poses, while enjoying the comparable speed.


Sign correlation Sparse representation Supervised descent method Face alignment 



This research was supported by the National Natural Science Foundation of China (Grant No. 61402133)

Compliance with ethical standards

Conflict of interest

All authors declare that there is no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. IEEE Trans Pattern Anal Mach Intell 35(12):2930–40CrossRefGoogle Scholar
  2. Burgosartizzu XP, Perona P, Dollar P (2013) Robust face landmark estimation under occlusion. In: IEEE international conference on computer vision, pp 1513–1520Google Scholar
  3. Cao X, Wei Y, Wen F, Sun J (2012) Face alignment by explicit shape regression. US Patent App. 13/728,584Google Scholar
  4. Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models-their training and application. Comput Vis Image Underst 61(1):38–59CrossRefGoogle Scholar
  5. Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685CrossRefGoogle Scholar
  6. Cristinacce D, Cootes TF (2006) Feature detection and tracking with constrained local models. BMVC 41:929–938zbMATHGoogle Scholar
  7. Dollar P, Welinder P, Perona P (2010) Cascaded pose regression. IEEE 238(6):1078–1085Google Scholar
  8. Fan X, Wang H, Luo Z, Li Y, Hu W, Luo D (2015) Fiducial facial point extraction using a novel projective invariant. IEEE Trans Image Process 24(3):1164–77MathSciNetCrossRefGoogle Scholar
  9. Feng ZH, Huber P, Kittler J, Christmas W, Wu XJ (2015a) Random cascaded-regression copse for robust facial landmark detection. IEEE Signal Process Lett 22(1):76–80CrossRefGoogle Scholar
  10. Feng ZH, Hu G, Kittler J, Christmas W, Wu XJ (2015b) Cascaded collaborative regression for robust facial landmark detection trained using a mixture of synthetic and real images with dynamic weighting. IEEE Trans Image Process 24(11):3425–3440MathSciNetCrossRefGoogle Scholar
  11. Gonzalezmora J, Torre FDL, Murthi R, Guil N, Zapata EL (2007) Bilinear active appearance models. In: IEEE international conference on computer vision, pp 1–8Google Scholar
  12. Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environmentsGoogle Scholar
  13. Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: Computer vision and pattern recognition, pp 1867–1874Google Scholar
  14. Kostinger M, Wohlhart P, Roth PM, Bischof H (2011) Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. In: IEEE international conference on computer vision workshops, ICCV 2011 workshops, Barcelona, Spain, Nov 6–13, 2011, pp 2144–2151Google Scholar
  15. Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. In: European conference on computer vision, pp 679–692Google Scholar
  16. Lee HS, Kim D (2009) Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Trans Pattern Anal Mach Intell 31(6):1102–16CrossRefGoogle Scholar
  17. Lee D, Park H, Yoo CD (2015) Face alignment using cascade gaussian process regression trees. In: IEEE conference on computer vision and pattern recognition, pp 4204–4212Google Scholar
  18. Lindner C, Bromiley PA, Ionita MC, Cootes TF (2015) Robust and accurate shape model matching using random forest regression-voting. IEEE Trans Pattern Anal Mach Intell 37(9):1862–74CrossRefGoogle Scholar
  19. Liu L, Hu J, Zhang S, Deng W (2014) Extended supervised descent method for robust face alignment. Springer, BerlinGoogle Scholar
  20. Martinez B, Pantic M (2015a) Facial landmarking for in-the-wild images with local inference based on global appearance. Image Vis Comput 36(C):40–50CrossRefGoogle Scholar
  21. Martinez B, Valstar MF (2015b) L 2,1-based regression and prediction accumulation across views for robust facial landmark detection. Image Vis Comput 45(4):371–382Google Scholar
  22. Ramanan D (2015) Face detection, pose estimation, and landmark localization in the wild. In: IEEE conference on computer vision and pattern recognition, pp 31–37Google Scholar
  23. Ren S, Cao X, Wei Y, Sun J (2014) Face alignment at 3000 FPS via regressing local binary features. In: IEEE transactions on image processing, pp 1685–1692Google Scholar
  24. Romdhani S (1999) A multi-view nonlinear active shape model using kernel PCA. In: British machine vision conference, pp 483–492Google Scholar
  25. Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) A semi-automatic methodology for facial landmark annotation. In: IEEE conference on computer vision and pattern recognition workshops, pp 896–903Google Scholar
  26. Saragih JM, Lucey S, Cohn JF (2011) Deformable model fitting by regularized landmark mean-shift. Int J Comput Vis 91(2):200–215MathSciNetCrossRefzbMATHGoogle Scholar
  27. Snchez-Lozano E, Martinez B, Valstar MF (2016) Cascaded regression with sparsified feature covariance matrix for facial landmark detection. Pattern Recognit Lett 73(C):19–25CrossRefGoogle Scholar
  28. Sun Y, Wang X, Tang X (2013) Deep convolutional network cascade for facial point detection. In: Conference on computer vision and pattern recognition, pp 3476–3483Google Scholar
  29. Tzimiropoulos G (2015) Project-out cascaded regression with an application to face alignment. In: IEEE conference on computer vision and pattern recognitionGoogle Scholar
  30. Valstar M, Martinez B, Binefa X, Pantic M (2010) Facial point detection using boosted regression and graph models, pp 2729–2736Google Scholar
  31. Xing J, Niu Z, Huang J, Hu W, Yan S (2014) Towards multi-view and partially-occluded face alignment. In: Computer vision and pattern recognition, pp 1829–1836Google Scholar
  32. Xiong X, De la Torre F (2013) Supervised descent method and its applications to face alignment. In: IEEE conference on computer vision and pattern recognition, pp 532–539Google Scholar
  33. Xiong X, la Torre FD (2015) Global supervised descent method. In: IEEE conference on computer vision and pattern recognition, pp 2664–2673Google Scholar
  34. Yang H, Patras I (2014) Fine-tuning regression forests votes for object alignment in the wild. IEEE Trans Image Process A Publ IEEE Signal Process Soc 24(2):619–31MathSciNetCrossRefGoogle Scholar
  35. Yang H, He X, Jia X, Patras I (2015a) Robust face alignment under occlusion via regional predictive power estimation. IEEE Trans Image Process 24(8):2393–403MathSciNetCrossRefGoogle Scholar
  36. Yang H, Jia X, Patras I, Chan KP (2015b) Random subspace supervised descent method for regression problems in computer vision. IEEE Signal Process Lett 22(10):1816–1820CrossRefGoogle Scholar
  37. Yan J, Lei Z, Yi D, Li SZ (2013) Learn to combine multiple hypotheses for accurate face alignment. In: IEEE international conference on computer vision workshops, pp 392–396Google Scholar
  38. Zhang Z, Zhang W, Ding H, Liu J, Tang X (2014a) Hierarchical facial landmark localization via cascaded random binary patterns. Pattern Recognit 48(4):1277–1288CrossRefGoogle Scholar
  39. Zhang Z, Luo P, Loy CC, Tang X (2014b) Facial landmark detection by deep multi-task learning. In: European conference on computer vision, pp 94–108Google Scholar
  40. Zhang J, Shan S, Kan M, Chen X (2014c) Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Computer vision-ECCV. Springer, pp 1–16Google Scholar
  41. Zhou E, Fan H, Cao Z, Jiang Y, Yin Q (2013) Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: IEEE international conference on computer vision workshops, pp 386–391Google Scholar
  42. Zhu S, Li C, Loy CC, Tang X (2015) Face alignment by coarse-to-fine shape searching. In: CVPR, pp 4998–5006Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Dansong Cheng
    • 1
    Email author
  • Yongqiang Zhang
    • 1
  • Feng Tian
    • 3
  • Ce Liu
    • 1
  • Xiaofang Liu
    • 2
  1. 1.School of Computer Science and TechnologyHarbin Institute of TechnologyHarbinPeople’s Republic of China
  2. 2.School of Electrical Engineering and AutomationHarbin Institute of TechnologyHarbinPeople’s Republic of China
  3. 3.Faculty of Science and TechnologyBournemouth UniversityPooleUK

Personalised recommendations