Face Recognition Using a Unified 3D Morphable Model

  • Guosheng Hu
  • Fei Yan
  • Chi-Ho Chan
  • Weihong Deng
  • William Christmas
  • Josef Kittler
  • Neil M. Robertson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9912)


We address the problem of 3D-assisted 2D face recognition in scenarios when the input image is subject to degradations or exhibits intra-personal variations not captured by the 3D model. The proposed solution involves a novel approach to learn a subspace spanned by perturbations caused by the missing modes of variation and image degradations, using 3D face data reconstructed from 2D images rather than 3D capture. This is accomplished by modelling the difference in the texture map of the 3D aligned input and reference images. A training set of these texture maps then defines a perturbation space which can be represented using PCA bases. Assuming that the image perturbation subspace is orthogonal to the 3D face model space, then these additive components can be recovered from an unseen input image, resulting in an improved fit of the 3D face model. The linearity of the model leads to efficient fitting. Experiments show that our method achieves very competitive face recognition performance on Multi-PIE and AR databases. We also present baseline face recognition results on a new data set exhibiting combined pose and illumination variations as well as occlusion.


3D morphable model Face recognition 



This work was sponsored by EPSRC project ‘Signal processing in a networked battlespace’ under contract EP/K014307/1, ‘FACER2VM’ under EP/N007743/1, NSFC project under Grant 61375031 and 61573068. The support from EPSRC and the MOD University Defence Research Collaboration (UDRC) in Signal Processing is gratefully acknowledged.


  1. 1.
    Hu, G., Chan, C.H., Yan, F., Christmas, W., Kittler, J.: Robust face recognition by an Albedo based 3D morphable model. In: 2014 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–8. IEEE (2014)Google Scholar
  2. 2.
    Zhu, Z., Luo, P., Wang, X., Tang, X.: Deep learning identity preserving face space. In: Proceedings of ICCV, vol. 1, p. 2 (2013)Google Scholar
  3. 3.
    Niinuma, K., Han, H., Jain, A.K.: Automatic multi-view face recognition via 3D model based pose regularization. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8. IEEE (2013)Google Scholar
  4. 4.
    Prabhu, U., Heo, J., Savvides, M.: Unconstrained pose-invariant face recognition using 3D generic elastic models. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1952–1961 (2011)CrossRefGoogle Scholar
  5. 5.
    Zhang, X., Gao, Y., Leung, M.K.: Recognizing rotated faces from frontal and side views: an approach toward effective use of mugshot databases. IEEE Trans. Inf. Forensics Secur. 3(4), 684–697 (2008)CrossRefGoogle Scholar
  6. 6.
    Asthana, A., Marks, T.K., Jones, M.J., Tieu, K.H., Rohith, M.: Fully automatic pose-invariant face recognition via 3D pose normalization. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 937–944. IEEE (2011)Google Scholar
  7. 7.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell., 25(9), 1063–1074 (2003)CrossRefGoogle Scholar
  8. 8.
    Aldrian, O., Smith, W.A.: Inverse rendering of faces with a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1080–1093 (2013)CrossRefGoogle Scholar
  9. 9.
    Rodriguez, J.T.: 3D face modelling for 2D+3D face recognition. Ph.D. thesis. Surrey University, Guildford, UK (2007)Google Scholar
  10. 10.
    Chu, B., Romdhani, S., Chen, L.: 3D-aided face recognition robust to expression and pose variations. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1907–1914. IEEE (2014)Google Scholar
  11. 11.
    Amberg, B., Knothe, R., Vetter, T.: Expression invariant 3D face recognition with a morphable model. In: 8th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2008, pp. 1–6. IEEE (2008)Google Scholar
  12. 12.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194 (1999)Google Scholar
  13. 13.
    Zhang, L., Samaras, D.: Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 351–363 (2006)CrossRefGoogle Scholar
  14. 14.
    Zhu, X., Lei, Z., Liu, X., Shi, H., Li, S.Z.: Face alignment across large poses: a 3D solution. arXiv preprint arXiv:1511.07212 (2015)
  15. 15.
    Zhu, X., Lei, Z., Yan, J., Yi, D., Li, S.Z.: High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 787–796 (2015)Google Scholar
  16. 16.
    Romdhani, S., Vetter, T.: Efficient, robust and accurate fitting of a 3D morphable model. In: Proceedings of 9th IEEE International Conference on Computer Vision, pp. 59–66. IEEE (2003)Google Scholar
  17. 17.
    Kang, B., Byun, H., Kim, D.: Multi-resolution 3D morphable models and its matching method. In: 19th International Conference on Pattern Recognition, pp. 1–4. IEEE (2008)Google Scholar
  18. 18.
    Romdhani, S., Blanz, V., Vetter, T.: Face identification by fitting a 3D morphable model using linear shape and texture error functions. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 3–19. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  19. 19.
    Romdhani, S., Vetter, T.: Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 986–993. IEEE (2005)Google Scholar
  20. 20.
    Huber, P., Feng, Z.H., Christmas, W., Kittler, J., Rätsch, M.: Fitting 3D morphable models using local features. In: ICIP (2015)Google Scholar
  21. 21.
    Zhu, X., Yan, J., Yi, D., Lei, Z., Li, S.Z.: Discriminative 3D morphable model fitting. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 1, pp. 1–8. IEEE (2015)Google Scholar
  22. 22.
    Huber, P., Hu, G., Tena, R., Mortazavian, P., Koppen, W.P., Christmas, W.J., Rtsch, M., Kittler, J.: A multi resolution 3D morphable face model and fitting framework. In: Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications,pp. 79–86 (2016)Google Scholar
  23. 23.
    Chen, D., Cao, X., Wang, L., Wen, F., Sun, J.: Bayesian face revisited: a joint formulation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 566–579. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  24. 24.
    Deng, W., Hu, J., Guo, J.: Extended SRC: undersampled face recognition via intraclass variant dictionary. IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1864–1870 (2012)CrossRefGoogle Scholar
  25. 25.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)CrossRefGoogle Scholar
  26. 26.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)CrossRefGoogle Scholar
  27. 27.
    Martinez, A.M.: The AR face database. CVC Technical report 24 (1998)Google Scholar
  28. 28.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07–49. University of Massachusetts, Amherst (2007)Google Scholar
  29. 29.
    Sharma, A., Kumar, A., Daume, H., Jacobs, D.W.: Generalized multiview analysis: a discriminative latent space. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2160–2167. IEEE (2012)Google Scholar
  30. 30.
    Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)CrossRefzbMATHGoogle Scholar
  31. 31.
    Kan, M., Shan, S., Chang, H., Chen, X.: Stacked progressive auto-encoders (spae) for face recognition across poses. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1883–1890 (2013)Google Scholar
  32. 32.
    Li, S., Liu, X., Chai, X., Zhang, H., Lao, S., Shan, S.: Morphable displacement field based image matching for face recognition across pose. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part I. LNCS, vol. 7572, pp. 102–115. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  33. 33.
    Chen, D., Cao, X., Wen, F., Sun, J.: Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3025–3032 (2013)Google Scholar
  34. 34.
    Li, A., Shan, S., Gao, W.: Coupled bias-variance tradeoff for cross-pose face recognition. IEEE Trans. Image Process. 21(1), 305–315 (2012)CrossRefMathSciNetGoogle Scholar
  35. 35.
    Zhu, Z., Luo, P., Wang, X., Tang, X.: Deep learning multi-view representation for face recognition. arXiv preprint arXiv:1406.6947 (2014)

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Guosheng Hu
    • 1
    • 2
  • Fei Yan
    • 2
  • Chi-Ho Chan
    • 2
  • Weihong Deng
    • 3
  • William Christmas
    • 2
  • Josef Kittler
    • 2
  • Neil M. Robertson
    • 1
    • 4
  1. 1.AnyvisionBelfastUK
  2. 2.CVSSPUniversity of SurreyGuildfordUK
  3. 3.Beijing University of Posts and TelecommunicationsBeijingChina
  4. 4.ECITQueen’s University of BelfastBelfastUK

Personalised recommendations