Illumination-Recovered Pose Normalization for Unconstrained Face Recognition

  • Zhongjun WuEmail author
  • Weihong Deng
  • Zhanfu An
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10113)


Identifying subjects with pose variations is still considered as one of the most challenging problems in face recognition, despite the great progress achieved in unconstrained face recognition in recent years. Pose problem is essentially a misalignment problem together with self-occlusion (information loss). In this paper, we propose a continuous identity-preserving face pose normalization method and produce natural results in terms of preserving the illumination condition of the query face, based on only five fiducial landmarks. “Raw” frontalization is performed by aligning a generic 3D face model into the query face and rendering it at frontal pose, with an accurate self-occlusion part estimation based on face borderline detection. Then we apply Quotient Image as a face symmetrical feature which is robust to illumination to fill the self-occlusion part. Natural normalization result is obtained where the self-occlusion part keeps the illumination conditions of the query face. Large scale face recognition experiments on LFW and MultiPIE achieve comparative results with state-of-the-art methods, verifying effectiveness of proposed method, with advantage of being database-independent and suitable both for face identification and face verification.


Face Recognition Query Image Input Face Illumination Normalization Quotient Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was partially sponsored by supported by the NSFC (National Natural Science Foundation of China) under Grant No. 61375031, No. 61573068, No. 61471048, and No.61273217, the Fundamental Research Funds for the Central Universities under Grant No. 2014ZD03-01, This work was also supported by Beijing Nova Program, CCF-Tencent Open Research Fund, and the Program for New Century Excellent Talents in University.


  1. 1.
    Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The feret evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 22, 1090–1104 (2000)CrossRefGoogle Scholar
  2. 2.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28, 807–813 (2010)CrossRefGoogle Scholar
  3. 3.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07–49, University of Massachusetts, Amherst (2007)Google Scholar
  4. 4.
    Ding, C., Tao, D.: A comprehensive survey on pose-invariant face recognition. arXiv preprint arXiv:1502.04383 (2015)
  5. 5.
    Li, A., Shan, S., Gao, W.: Coupled bias-variance tradeoff for cross-pose face recognition. IEEE Trans. Image Process. 21, 305–315 (2012)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chai, X., Shan, S., Chen, X., Gao, W.: Locally linear regression for pose-invariant face recognition. IEEE Trans. Image Process. 16, 1716–1725 (2007)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Zhu, Z., Luo, P., Wang, X., Tang, X.: Deep learning identity-preserving face space. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 113–120. IEEE (2013)Google Scholar
  8. 8.
    Kan, M., Shan, S., Chang, H., Chen, X.: Stacked progressive auto-encoders (spae) for face recognition across poses. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1883–1890. IEEE (2014)Google Scholar
  9. 9.
    Zhu, Z., Luo, P., Wang, X., Tang, X.: Multi-view perceptron: a deep model for learning face identity and view representations. In: Advances in Neural Information Processing Systems, pp. 217–225 (2014)Google Scholar
  10. 10.
    Yim, J., Jung, H., Yoo, B., Choi, C., Park, D., Kim, J.: Rotating your face using multi-task deep neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 676–684 (2015)Google Scholar
  11. 11.
    Yi, D., Lei, Z., Li, S.: Towards pose robust face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3539–3545 (2013)Google Scholar
  12. 12.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1063–1074 (2003)CrossRefGoogle Scholar
  13. 13.
    Prabhu, U., Heo, J., Savvides, M.: Unconstrained pose-invariant face recognition using 3D generic elastic models. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1952–1961 (2011)CrossRefGoogle Scholar
  14. 14.
    Asthana, A., Marks, T.K., Jones, M.J., Tieu, K.H., Rohith, M.: Fully automatic pose-invariant face recognition via 3D pose normalization. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 937–944. IEEE (2011)Google Scholar
  15. 15.
    Zhu, X., Lei, Z., Yan, J., Yi, D., Li, S.Z.: High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 787–796 (2015)Google Scholar
  16. 16.
    Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4295–4304 (2015)Google Scholar
  17. 17.
    Shashua, A., Riklin-Raviv, T.: The quotient image: class-based re-rendering and recognition with varying illuminations. IEEE Trans. Pattern Anal. Mach. Intell. 23, 129–139 (2001)CrossRefGoogle Scholar
  18. 18.
    Ding, L., Ding, X., Fang, C.: Continuous pose normalization for pose-robust face recognition. Sig. Process. Lett. 19, 721–724 (2012). IEEECrossRefGoogle Scholar
  19. 19.
    Foley, J.D.: Computer Graphics: Principles and Practice, vol. 12110. Addison-Wesley Professional, Reading (1996)Google Scholar
  20. 20.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. (TOG) 22, 313–318. ACM (2003)Google Scholar
  21. 21.
    Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2009, pp. 296–301. IEEE (2009)Google Scholar
  22. 22.
    Chen, D., Cao, X., Wen, F., Sun, J.: Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3025–3032 (2013)Google Scholar
  23. 23.
    Cao, Q., Ying, Y., Li, P.: Similarity metric learning for face recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2408–2415 (2013)Google Scholar
  24. 24.
    Wolf, L., Hassner, T., Taigman, Y.: Similarity scores based on background samples. In: Zha, H., Taniguchi, R., Maybank, S. (eds.) ACCV 2009. LNCS, vol. 5995, pp. 88–97. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-12304-7_9 CrossRefGoogle Scholar
  25. 25.
    Huang, G., Mattar, M., Lee, H., Learned-Miller, E.G.: Learning to align from scratch. In: Advances in Neural Information Processing Systems, pp. 764–772 (2012)Google Scholar
  26. 26.
    Deng, W., Hu, J., Zhou, X., Guo, J.: Equidistant prototypes embedding for single sample based face recognition with generic learning and incremental learning. Pattern Recogn. 47, 3738–3749 (2014)CrossRefGoogle Scholar
  27. 27.
    Li, S., Liu, X., Chai, X., Zhang, H., Lao, S., Shan, S.: Morphable displacement field based image matching for face recognition across pose. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 102–115. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33718-5_8 CrossRefGoogle Scholar
  28. 28.
    Du, S., Ward, R.: Wavelet-based illumination normalization for face recognition. In: IEEE International Conference on Image Processing, ICIP 2005, vol. 2, p. II-954. IEEE (2005)Google Scholar
  29. 29.
    Chen, W., Er, M.J., Wu, S.: Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36, 458–466 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Beijing University of Posts and TelecommunicationsBeijingChina

Personalised recommendations