Abstract
Face alignment is an essential task for facial performance capture and expression analysis. Current methods such as random subspace supervised descent method, stage-wise relational dictionary and coarse-to-fine shape searching can ease multi-pose face alignment problem, but no method can deal with the multiple local minima problem directly. In this paper, we propose a sign correlation subspace method for domain partition in only one reduced low-dimensional subspace. Unlike previous methods, we analyze the sign correlation between features and shapes and project both of them into a mutual sign correlation subspace. Each pair of projected shape and feature keeps their signs consistent in each dimension of the subspace, so that each hyper octant holds the condition that one general descent exists. Then a set of general descents are learned from the samples in different hyperoctants. Requiring only the feature projection for domain partition, our proposed method is effective for face alignment. We have validated our approach with the public face datasets which include a range of poses. The validation results show that our method can reveal their latent relationships to poses. The comparison with state-of-the-art methods demonstrates that our method outperforms them, especially in uncontrolled conditions with various poses, while enjoying the comparable speed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. IEEE Trans Pattern Anal Mach Intell 35(12):2930–40
Burgosartizzu XP, Perona P, Dollar P (2013) Robust face landmark estimation under occlusion. In: IEEE international conference on computer vision, pp 1513–1520
Cao X, Wei Y, Wen F, Sun J (2012) Face alignment by explicit shape regression. US Patent App. 13/728,584
Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models-their training and application. Comput Vis Image Underst 61(1):38–59
Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685
Cristinacce D, Cootes TF (2006) Feature detection and tracking with constrained local models. BMVC 41:929–938
Dollar P, Welinder P, Perona P (2010) Cascaded pose regression. IEEE 238(6):1078–1085
Fan X, Wang H, Luo Z, Li Y, Hu W, Luo D (2015) Fiducial facial point extraction using a novel projective invariant. IEEE Trans Image Process 24(3):1164–77
Feng ZH, Huber P, Kittler J, Christmas W, Wu XJ (2015a) Random cascaded-regression copse for robust facial landmark detection. IEEE Signal Process Lett 22(1):76–80
Feng ZH, Hu G, Kittler J, Christmas W, Wu XJ (2015b) Cascaded collaborative regression for robust facial landmark detection trained using a mixture of synthetic and real images with dynamic weighting. IEEE Trans Image Process 24(11):3425–3440
Gonzalezmora J, Torre FDL, Murthi R, Guil N, Zapata EL (2007) Bilinear active appearance models. In: IEEE international conference on computer vision, pp 1–8
Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environments
Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: Computer vision and pattern recognition, pp 1867–1874
Kostinger M, Wohlhart P, Roth PM, Bischof H (2011) Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. In: IEEE international conference on computer vision workshops, ICCV 2011 workshops, Barcelona, Spain, Nov 6–13, 2011, pp 2144–2151
Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. In: European conference on computer vision, pp 679–692
Lee HS, Kim D (2009) Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Trans Pattern Anal Mach Intell 31(6):1102–16
Lee D, Park H, Yoo CD (2015) Face alignment using cascade gaussian process regression trees. In: IEEE conference on computer vision and pattern recognition, pp 4204–4212
Lindner C, Bromiley PA, Ionita MC, Cootes TF (2015) Robust and accurate shape model matching using random forest regression-voting. IEEE Trans Pattern Anal Mach Intell 37(9):1862–74
Liu L, Hu J, Zhang S, Deng W (2014) Extended supervised descent method for robust face alignment. Springer, Berlin
Martinez B, Pantic M (2015a) Facial landmarking for in-the-wild images with local inference based on global appearance. Image Vis Comput 36(C):40–50
Martinez B, Valstar MF (2015b) L 2,1-based regression and prediction accumulation across views for robust facial landmark detection. Image Vis Comput 45(4):371–382
Ramanan D (2015) Face detection, pose estimation, and landmark localization in the wild. In: IEEE conference on computer vision and pattern recognition, pp 31–37
Ren S, Cao X, Wei Y, Sun J (2014) Face alignment at 3000 FPS via regressing local binary features. In: IEEE transactions on image processing, pp 1685–1692
Romdhani S (1999) A multi-view nonlinear active shape model using kernel PCA. In: British machine vision conference, pp 483–492
Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) A semi-automatic methodology for facial landmark annotation. In: IEEE conference on computer vision and pattern recognition workshops, pp 896–903
Saragih JM, Lucey S, Cohn JF (2011) Deformable model fitting by regularized landmark mean-shift. Int J Comput Vis 91(2):200–215
Snchez-Lozano E, Martinez B, Valstar MF (2016) Cascaded regression with sparsified feature covariance matrix for facial landmark detection. Pattern Recognit Lett 73(C):19–25
Sun Y, Wang X, Tang X (2013) Deep convolutional network cascade for facial point detection. In: Conference on computer vision and pattern recognition, pp 3476–3483
Tzimiropoulos G (2015) Project-out cascaded regression with an application to face alignment. In: IEEE conference on computer vision and pattern recognition
Valstar M, Martinez B, Binefa X, Pantic M (2010) Facial point detection using boosted regression and graph models, pp 2729–2736
Xing J, Niu Z, Huang J, Hu W, Yan S (2014) Towards multi-view and partially-occluded face alignment. In: Computer vision and pattern recognition, pp 1829–1836
Xiong X, De la Torre F (2013) Supervised descent method and its applications to face alignment. In: IEEE conference on computer vision and pattern recognition, pp 532–539
Xiong X, la Torre FD (2015) Global supervised descent method. In: IEEE conference on computer vision and pattern recognition, pp 2664–2673
Yang H, Patras I (2014) Fine-tuning regression forests votes for object alignment in the wild. IEEE Trans Image Process A Publ IEEE Signal Process Soc 24(2):619–31
Yang H, He X, Jia X, Patras I (2015a) Robust face alignment under occlusion via regional predictive power estimation. IEEE Trans Image Process 24(8):2393–403
Yang H, Jia X, Patras I, Chan KP (2015b) Random subspace supervised descent method for regression problems in computer vision. IEEE Signal Process Lett 22(10):1816–1820
Yan J, Lei Z, Yi D, Li SZ (2013) Learn to combine multiple hypotheses for accurate face alignment. In: IEEE international conference on computer vision workshops, pp 392–396
Zhang Z, Zhang W, Ding H, Liu J, Tang X (2014a) Hierarchical facial landmark localization via cascaded random binary patterns. Pattern Recognit 48(4):1277–1288
Zhang Z, Luo P, Loy CC, Tang X (2014b) Facial landmark detection by deep multi-task learning. In: European conference on computer vision, pp 94–108
Zhang J, Shan S, Kan M, Chen X (2014c) Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Computer vision-ECCV. Springer, pp 1–16
Zhou E, Fan H, Cao Z, Jiang Y, Yin Q (2013) Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: IEEE international conference on computer vision workshops, pp 386–391
Zhu S, Li C, Loy CC, Tang X (2015) Face alignment by coarse-to-fine shape searching. In: CVPR, pp 4998–5006
Acknowledgements
This research was supported by the National Natural Science Foundation of China (Grant No. 61402133)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that there is no conflict of interest.
Human and animal rights
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Cheng, D., Zhang, Y., Tian, F. et al. Sign correlation subspace for face alignment. Soft Comput 23, 241–249 (2019). https://doi.org/10.1007/s00500-018-3389-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-018-3389-1