Advertisement

Multimedia Systems

, Volume 25, Issue 5, pp 475–485 | Cite as

Salient-points-guided face alignment

  • Yangyang Hao
  • Hengliang Zhu
  • Kai Wu
  • Xiao Lin
  • Lizhuang MaEmail author
Special Issue Paper

Abstract

Regression-based face alignment approach is fast and accurate but is always limited by the initial face. Aim at limitation of initialization in regression methods, this paper presents a novel two-stage framework named salient-points-guided face alignment. In first stage, we use cascade regression framework to train a salient points (eye centers, nose, mouth corners) localization model. Then the salient points information is used as a guidance for searching the similar faces from training set. In second stage, leveraging the similar faces to generate the initial face for all points regression. In order to give more comprehensive comparison, a new evaluation metric is proposed. Considering the global distance between estimated face and ground-truth, the new evaluation metric is defined as sum of the global distance and the widely used average point-to-point distance. The results show that our approach can achieve state-of-the-art performance (12% higher than the human performance on COFW) and the new evaluation metric is more reasonable.

Keywords

Face alignment Salient points Initial face Evaluation metric 

Notes

Acknowledgements

This work has been partially funded by the National Natural Science Foundation of China (No. 61133009, 61472245, U1304616, 61502220).

References

  1. 1.
    Ashraf, A.B., Lucey, S., Cohn, J.F., Chen, T.: The painful face: pain expression recognition using active appearance models. Image Vis Comput 27(12), 1788–1796 (2007)CrossRefGoogle Scholar
  2. 2.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3444–3451 (2013)Google Scholar
  3. 3.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Incremental face alignment in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1859–1866 (2014)Google Scholar
  4. 4.
    Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 545–552 (2013)CrossRefGoogle Scholar
  5. 5.
    Burgos-Artizzu, X.P., Perona, P., Dollr, P.: Robust face landmark estimation under occlusion. In: IEEE International Conference on Computer Vision (ICCV), pp. 1513–1520 (2013)Google Scholar
  6. 6.
    Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 107(2), 117–190 (2012)MathSciNetGoogle Scholar
  7. 7.
    Chen, C., Dantcheva, A., Ross, A.: Automatic facial makeup detection with application in face recognition. In: International Conference on Biometrics (ICB), pp. 1–8 (2013)Google Scholar
  8. 8.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. The IEEE Transactions on Pattern Analysis and Machine Intelligence pp. 681–685 (2001)CrossRefGoogle Scholar
  9. 9.
    Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)CrossRefGoogle Scholar
  10. 10.
    Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models. BMVC 41, 929–938 (2006)zbMATHGoogle Scholar
  11. 11.
    Dantone, M., Gall, J., Fanelli, G., Van Gool, L.: Real-time facial feature detection using conditional regression forests. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2578–2585 (2012)Google Scholar
  12. 12.
    Guo, D., Sim, T.: Digital face makeup by example. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 73–79 (2009)Google Scholar
  13. 13.
    Dollar, P., Welinder, P., Perona, P.: Cascaded pose regression. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1078–1085 (2010)Google Scholar
  14. 14.
    Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Joseph Tan, D., Holzer, S., Navab, N., Ilic, S.: Deformable template tracking in 1ms. In: Proceedings of the British Machine Vision Conference(BMVC), p. 43C56 (2014)Google Scholar
  16. 16.
    Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)Google Scholar
  17. 17.
    Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.: Interactive facial feature localization. ECCV 7574, 679–692 (2012)Google Scholar
  18. 18.
    Lee, D., Park, H., Yoo, C.: Face alignment using cascade gaussian process regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4204–4212 (2015)Google Scholar
  19. 19.
    Liang, L., Xiao, R., Wen, F., Sun, J.: Face alignment via component-based discriminative search. ECCV 5303, 72–85 (2008)Google Scholar
  20. 20.
    Lucey, S., Wang, Y., Cox, M., Sridharan, S., Cohn, J.F.: Efficient constrained local model fitting for non-rigid face alignment. Image Vis. Comput. 27(12), 1804–1813 (2009)CrossRefGoogle Scholar
  21. 21.
    Matthews, I., Baker, S.: Active appearance models revisited. Int. J. Comput. Vis. 60(2), 135–164 (2004)CrossRefGoogle Scholar
  22. 22.
    Messer, K., Matas, J., Kittler, J., Jonsson, K.: Xm2vtsdb: The extended m2vts database. In: Second International Conference on Audio- and Video-based Biometric Person Authentication (AVBPA), pp. 72–77 (2000)Google Scholar
  23. 23.
    Ozuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 448–461 (2010)CrossRefGoogle Scholar
  24. 24.
    Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 fps via regressing local binary features. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1685–1692 (2014)Google Scholar
  25. 25.
    Saragih, J.M., Lucey, S., Cohn, J.F.: Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vis. 91(2), 200–215 (2011)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Smith, B.M., Brandt, J., Lin, Z., Zhang, L.: Nonparametric context modeling of local appearance for pose- and expression-robust facial landmark localization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1741–1748 (2014)Google Scholar
  27. 27.
    Smith, B.M., Zhang, L.: Joint face alignment with non-parametric shape models. Computer Vision–ECCV, pp. 43–56. Springer, New York (2012)Google Scholar
  28. 28.
    Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3476–3483 (2013)Google Scholar
  29. 29.
    Tzimiropoulos, G.: Project-out cascaded regression with an application to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  30. 30.
    Wang, Y., Lucey, S., Cohn, J.F.: Enforcing convexity for improved alignment with constrained local models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  31. 31.
    Wu, T., Turaga, P., Chellappa, R.: Age estimation and face verification across aging using landmarks. IEEE Trans. Inf. Forensics Secur. 7(6), 1780–1788 (2012)CrossRefGoogle Scholar
  32. 32.
    Xiong, X., De la Torre, F.: Global supervised descent method. In: Computer Vision and Pattern Recognition, pp. 2664–2673 (2015)Google Scholar
  33. 33.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)Google Scholar
  34. 34.
    Yang, H., Mou, W., Zhang, Y., Patras, I., Gunes, H., Robinson, P.: Face alignment assisted by head pose estimation. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 130.1–130.13 (2015)Google Scholar
  35. 35.
    Yang, H., Patras, I.: Sieving regression forest votes for facial feature detection in the wild. In: IEEE International Conference on Computer Vision (ICCV), pp. 1936–1943 (2013)Google Scholar
  36. 36.
    Zhang, J., Shan, S., Kan, M., Chen, X.: Coarse-to-fine auto-encoder networks (cfan) for real-time face alignment. In: ECCV, pp. 1–16 (2014)Google Scholar
  37. 37.
    Zhang, Z., Luo, P., Chen, C.L., Tang, X.: Facial landmark detection by deep multi-task learning. In: ECCV, pp. 94–108 (2014)CrossRefGoogle Scholar
  38. 38.
    Zhu, S., Li, C., Loy, C.C., Tang, X.: Face alignment by coarse-to-fine shape searching. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  39. 39.
    Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879–2886 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Yangyang Hao
    • 1
  • Hengliang Zhu
    • 1
  • Kai Wu
    • 1
  • Xiao Lin
    • 1
  • Lizhuang Ma
    • 1
    Email author
  1. 1.School of Electronic Information and Electrical EngineeringShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations