Advertisement

Facial Landmark Detection Under Large Pose

  • Yangyang Hao
  • Hengliang Zhu
  • Zhiwen Shao
  • Xin Tan
  • Lizhuang Ma
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11304)

Abstract

Facial landmark detection is a necessary step in many vision tasks and plenty of excellent methods have been proposed to solve this problem. However, for the conditions with large pose and complex expression, these works usually suffer an eclipse. In this paper, we propose a two-stage cascade regression framework using patch-difference features to overcome the above problem. In the first stage, by applying the patch-difference feature and augmenting the large pose samples to the classical shape regression model, salient landmarks (eye centers, nose, mouth corners) can be located precisely. In the second stage, by applying enhanced feature section constraint to the patch-difference feature, multi-landmark detection is achieved. Experimental results show that our algorithm has a significant improvement compared to the classical shape regression method and achieves superior results on COFW dataset.

Keywords

Facial landmark detection Large pose Patch-difference feature Feature section constraint 

Notes

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 61472245) and the Science and Technology Commission of Shanghai Municipality Program (No. 16511101300).

References

  1. 1.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: Computer Vision and Pattern Recognition, pp. 3444–3451. IEEE, New York (2013)Google Scholar
  2. 2.
    Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 545–552 (2013)CrossRefGoogle Scholar
  3. 3.
    Burgos-Artizzu, X.P., Perona, P., Dollár, P.: Robust face landmark estimation under occlusion. In: International Conference on Computer Vision, pp. 1513–1520. IEEE, New York (2013)Google Scholar
  4. 4.
    Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vision 107(2), 117–190 (2012)MathSciNetGoogle Scholar
  5. 5.
    Chen, C., Dantcheva, A., Ross, A.: Automatic facial makeup detection with application in face recognition. In: International Conference on Biometrics, pp. 1–8. IEEE, New York (2013)Google Scholar
  6. 6.
    Guo, D., Sim, T.: Digital face makeup by example. In: Computer Vision and Pattern Recognition, pp. 73–79. IEEE, New York (2009)Google Scholar
  7. 7.
    Dollár, P., Welinder, P., Perona, P.: Cascaded pose regression. In: Computer Vision and Pattern Recognition, pp. 1078–1085. IEEE, New York (2010)Google Scholar
  8. 8.
    Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat., 1189–1232 (2001)Google Scholar
  9. 9.
    Honari, S., Yosinski, J., Vincent, P., Pal, C.: Recombinator networks: learning coarse-to-fine feature aggregation. In: Computer Vision and Pattern Recognition, pp. 5743–5752. IEEE Computer Society, New York (2016)Google Scholar
  10. 10.
    Jain, A.K.: Fundamentals of Digital Image Processing. Prentice Hall (1989)Google Scholar
  11. 11.
    Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Computer Vision and Pattern Recognition, pp. 1867–1874. IEEE, New York (2014)Google Scholar
  12. 12.
    Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.S.: Interactive facial feature localization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7574, pp. 679–692. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33712-3_49CrossRefGoogle Scholar
  13. 13.
    Lee, D., Park, H., Yoo, C.: Face alignment using cascade Gaussian process regression trees. In: Computer Vision and Pattern Recognition, pp. 4204–4212. IEEE, New York (2015)Google Scholar
  14. 14.
    Ramirez Rivera, A., Castillo, R., Chae, O.: Local directional number pattern for face analysis: face and expression recognition. IEEE Trans. Image Process. 22(5), 1740–1752 (2013)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 fps via regressing local binary features. In: Computer Vision and Pattern Recognition, pp. 1685–1692. IEEE, New York (2014)Google Scholar
  16. 16.
    Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: Computer Vision and Pattern Recognition, pp. 3476–3483. IEEE Computer Society, New York (2013)Google Scholar
  17. 17.
    Trigeorgis, G., Snape, P., Nicolaou, M.A., Antonakos, E., Zafeiriou, S.: Mnemonic descent method: a recurrent process applied for end-to-end face alignment. In: Computer Vision and Pattern Recognition, pp. 4177–4187. IEEE, New York (2016)Google Scholar
  18. 18.
    Tzimiropoulos, G.: Project-out cascaded regression with an application to face alignment. In: Computer Vision and Pattern Recognition. IEEE, New York (2015)Google Scholar
  19. 19.
    Xiao, S., et al.: Recurrent 3D–2D dual learning for large-pose facial landmark detection. In: IEEE International Conference on Computer Vision, pp. 1642–1651. IEEE Computer Society, New York (2017)Google Scholar
  20. 20.
    Xiao, S., Feng, J., Xing, J., Lai, H., Yan, S., Kassim, A.: Robust facial landmark detection via recurrent attentive-refinement networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 57–72. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_4CrossRefGoogle Scholar
  21. 21.
    Xiong, X., De la Torre, F.: Global supervised descent method. In: Computer Vision and Pattern Recognition, pp. 2664–2673. IEEE Computer Society, New York (2015)Google Scholar
  22. 22.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: Computer Vision and Pattern Recognition, pp. 532–539. IEEE, New York (2013)Google Scholar
  23. 23.
    Zhang, J., Shan, S., Kan, M., Chen, X.: Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 1–16. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10605-2_1CrossRefGoogle Scholar
  24. 24.
    Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Learning deep representation for face alignment with auxiliary attributes. IEEE Trans. Pattern Anal. Mach. Intell. 38(5), 918–930 (2016)CrossRefGoogle Scholar
  25. 25.
    Zhou, E., Fan, H., Cao, Z., Jiang, Y.: Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: International Conference on Computer Vision Workshops, pp. 386–391. IEEE Computer Society, New York (2014)Google Scholar
  26. 26.
    Zhu, S., Li, C., Loy, C.C., Tang, X.: Face alignment by coarse-to-fine shape searching. In: Computer Vision and Pattern Recognition. IEEE, New York (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Yangyang Hao
    • 1
  • Hengliang Zhu
    • 1
  • Zhiwen Shao
    • 1
  • Xin Tan
    • 1
  • Lizhuang Ma
    • 1
    • 2
  1. 1.Department of Computer Science and EngineeringShanghai Jiao Tong UniversityShanghaiChina
  2. 2.Department of Computer Science and Software EngineeringEast China Normal UniversityShanghaiChina

Personalised recommendations