Skip to main content
SpringerLink
Log in

Robust Face Frontalization in Unconstrained Images

  • Conference paper
  • First Online:
Pattern Recognition (CCPR 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 662))

Included in the following conference series:

Abstract

The goal of face frontalization is to recover frontal facing views of faces appearing in single unconstrained images. The previous works mainly focus on how to achieve the frontal facing views effectively. However, they ignore the influences of the face images with occlusion. To overcome the problem, this paper presents a novel but simple scheme for robust face frontalization with only a single 3D model. We employ the same scheme with T. Hassner’s work to render the non-frontal facing view to the frontal facing view and estimate the invisible (self-occlusion) region. Subsequently, we compute the differences of the local patches around each fixed facial feature points between the average face (male average face or female average face) and test images for occlusion detection. Finally, we combine the proposed local face symmetry strategy and the Poisson image editing to fill the invisible region and occlusion region. Experimental results demonstrate advantages of the proposed method over the previous work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.humansensing.cs.cmu.edu/intraface.

References

  1. Ding, C.: Multi-task pose-invariant face recognition. IEEE Trans. Image Process. 24(3), 980–993 (2015)

    Article  MathSciNet  Google Scholar 

  2. Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1063–1074 (2003)

    Article  Google Scholar 

  3. Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Computer Graphics Proceedings of SIGGRAPH 2002, pp. 187–194 (2002)

    Google Scholar 

  4. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface : closing the gap to human-level performance in face verification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1701–1708 (2013)

    Google Scholar 

  5. Sun, Z.-l., Lam, K.-M.: Depth estimation of face images using the nonlinear least-squares model. IEEE Trans. Image Process. 22(1), 895–898 (2013)

    Article  MathSciNet  Google Scholar 

  6. Zhu, X., Lei, Z., Yan, J., Yi, D.: High-fidelity pose and expression normalization for face recognition in the wild. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 787–796 (2015)

    Google Scholar 

  7. Sagonas, C., Panagakis, Y., Zafeiriou, S., Pantic, M.: Robust statistical face frontalization. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 4321–4329 (2015)

    Google Scholar 

  8. Zhu, Z., Luo, P., Wang, X., Tang, X.: Recover canonical view faces in the wild with deep neural networks. arXiv preprint arXiv:1404.3543 (2014)

  9. Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4295–4304 (2015)

    Google Scholar 

  10. Xiong, X., De la Torre, F.: Supervised descent method and its application to face alignment. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 532–539 (2013)

    Google Scholar 

  11. Zhu, X., Ramanan, D.: Face detection, pose estimation and landmark localization in the wild. In: Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012

    Google Scholar 

  12. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, June, 2014

    Google Scholar 

  13. Perez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. (TOG) 22, 313–318 (2003)

    Article  Google Scholar 

  14. Hassner, T.: Viewing real-world faces in 3D. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 3607–3614 (2013)

    Google Scholar 

  15. Kemelmacher-Shlizerman, I., Basri, R.: 3D face reconstruction from a single image using a single reference face shape. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 394–405 (2011)

    Article  Google Scholar 

Download references

Acknowledgment

This work was partially supported by the National Science Fund of China under Grant Nos. 91420201, 61472187, 61502235, 61233011 and 61373063, the Key Project of Chinese Ministry of Education under Grant No. 313030, the 973 Program No. 2014CB349303, and Program for Changjiang Scholars and Innovative Research Team in University.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

    Yuhan Zhang, Jianjun Qian & Jian Yang

Authors
  1. Yuhan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianjun Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuhan Zhang .

Editor information

Editors and Affiliations

  1. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Tieniu Tan

  2. Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China

    Xuelong Li

  3. Chinese Academy of Sciences, Institute of Computing Technology, Beijing, China

    Xilin Chen

  4. Tsinghua University , Beijing, China

    Jie Zhou

  5. Nanjing University of Science and Technology, Nanjing, China

    Jian Yang

  6. University of Electronic Science and Technology, Chengdu, Sichuan, China

    Hong Cheng

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Zhang, Y., Qian, J., Yang, J. (2016). Robust Face Frontalization in Unconstrained Images. In: Tan, T., Li, X., Chen, X., Zhou, J., Yang, J., Cheng, H. (eds) Pattern Recognition. CCPR 2016. Communications in Computer and Information Science, vol 662. Springer, Singapore. https://doi.org/10.1007/978-981-10-3002-4_19

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-3002-4_19

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-3001-7

  • Online ISBN: 978-981-10-3002-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation