Advertisement

Machine Vision and Applications

, Volume 29, Issue 3, pp 375–391 | Cite as

Annotated face model-based alignment: a robust landmark-free pose estimation approach for 3D model registration

  • Yuhang Wu
  • Shishir K. Shah
  • Ioannis A. Kakadiaris
Original Paper
  • 681 Downloads

Abstract

Registering a 3D facial model onto a 2D image is important for constructing pixel-wise correspondences between different facial images. The registration is based on a 3 \(\times \) 4 dimensional projection matrix, which is obtained from pose estimation. Conventional pose estimation approaches employ facial landmarks to determine the coefficients inside the projection matrix and are sensitive to missing or incorrect landmarks. In this paper, a landmark-free pose estimation method is presented. The method can be used to estimate the matrix when facial landmarks are not available. Experimental results show that the proposed method outperforms several landmark-free pose estimation methods and achieves competitive accuracy in terms of estimating pose parameters. The method is also demonstrated to be effective as part of a 3D-aided face recognition pipeline (UR2D), whose rank-1 identification rate is competitive to the methods that use landmarks to estimate head pose.

Keywords

Pose estimation Face alignment Model registration Face recognition 

Abbreviations

GIS

Geometry image space

AFM

Annotated face model

T-AFM

Texture of annotated face model

RDD

Rotation determined decomposition

TBB

Target bounding box

SDM

Supervised descent method

GSDM

Global supervised descent method

RSSDM

Random subspace supervised descent method

2dSC

Two-dimensional sparse coding

G3D

Generic 3D model

PS3D

Personalized 3D model

E-AFMA

Ex-annotated face model-based alignment

AFMA

Annotated face model-based alignment

References

  1. 1.
    3dMD 3dMD: 3D imaging systems and software (2012). http://www.3dmd.com/
  2. 2.
    Abiantun, R., Prabhu, U., Savvides, M.: Sparse feature extraction for pose-tolerant face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36(10), 2061–2073 (2014)CrossRefGoogle Scholar
  3. 3.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: Proceedings of IEEE conference on computer vision and pattern recognition, Portland, OR, pp. 3444–3451 (2013)Google Scholar
  4. 4.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Incremental face alignment in the wild. In: Proceedings of IEEE conference on computer vision and pattern recognition, Columbus, OH, pp. 1859–1866 (2014)Google Scholar
  5. 5.
    Asthana, A., Zafeiriou, S., Tzimiropoulos, G., Cheng, S., Pantic, M.: From pixels to response maps: discriminative image filtering for face alignment in the wild. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1312–1320 (2015)CrossRefGoogle Scholar
  6. 6.
    Ba, S.O., Odobez, J.M.: Recognizing visual focus of attention from head pose in natural meetings. IEEE Int. Conf. Syst. Man Cybern. 39(1), 16–33 (2009)CrossRefGoogle Scholar
  7. 7.
    Balasubramanian, V., Ye, J., Panchanathan, S.: Biased manifold embedding: a framework for person-independent head pose estimation. In: Proceedings of IEEE conference on computer vision and pattern recognition, Minneapolis, MN, pp. 1–7 (2007)Google Scholar
  8. 8.
    Bouaziz, S., Wang, Y., Pauly, M.: Online modeling for realtime facial animation. ACM Trans. Graph. 32(4), 40 (2013)CrossRefMATHGoogle Scholar
  9. 9.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of IEEE conference on computer vision and pattern recognition, San Diego, CA, pp. 886–893 (2005)Google Scholar
  10. 10.
    Divvala, SK., Efros, AA., Hebert, M.: How important are deformable parts in the deformable parts model? In: Proceedings of European conference on computer vision (workshop), Florence, Italy, pp. 31–40 (2012)Google Scholar
  11. 11.
    Dou, P., Wu, Y., Shishir, S.K., Kakadiaris, I.A.: Benchmarking 3D pose estimation for face recognition. In: Proceedings of IEEE international conference on pattern recognition, Stockholm, Sweden, pp. 190–195 (2014)Google Scholar
  12. 12.
    Dou, P., Zhang, L., Wu, Y., Shah, S.K., Kakadiaris, I.A.: Pose-robust face signature for multi-view face recognition. In: Proceedings of IEEE international conference on biometrics: theory, applications and systems, Arlington, VA, pp. 1–8 (2015)Google Scholar
  13. 13.
    Drucker, H., Burges, C., Kaufman, L., Smola, A., Vapnik, V.: Support vector regression machines. In: Advances in neural information processing systems, Denver, CO, pp. 155–161 (1997)Google Scholar
  14. 14.
    Geng, X., Xia, Y.: Head pose estimation based on multivariate label distribution. In: Proceedings of IEEE conference on computer vision and pattern recognition, Columbus, OH, pp. 1837–1842 (2014)Google Scholar
  15. 15.
    Gourier, N., Hall, D., Crowley, J.: Estimating face orientation from robust detection of salient facial structures. In: Proceedings of international workshop on visual observation of deictic gestures, Cambridge, UK, pp. 1–9 (2004)Google Scholar
  16. 16.
    Guo, G., Fu, Y., Dyer, CR., Huang, T.: Head pose estimation: classification or regression? In: Proceedings of international conference on pattern recognition, Tampa, FL, pp. 1–4 (2008)Google Scholar
  17. 17.
    Hara, K., Chellappa, R.: Growing regression forests by classification: applications to object pose estimation. In: Proceedings of European conference on computer vision, Zurich, Switzerland (2014)Google Scholar
  18. 18.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)MATHGoogle Scholar
  19. 19.
    Hsu, G., Peng, H.: Face recognition across poses using a single 3D reference model. In: Proceedings of IEEE conference on computer vision and pattern recognition, Portland, OR, pp. 869–874 (2013)Google Scholar
  20. 20.
    Huang, D., Storer, M., De la Torre, F., Bischof, H.: Supervised local subspace learning for continuous head pose estimation. In: Proceedings of IEEE conference on computer vision and pattern recognition, Colorado Springs, CO, pp. 2921–2928 (2011)Google Scholar
  21. 21.
    Jeni, L., Cohn, J., Kanade, T.: Dense 3D face alignment from 2D videos in real-time. In: Proceedings of IEEE international conference and workshops on automatic face and gesture recognition, Ljubljana, Slovenia, vol. 1, pp. 1–8 (2015)Google Scholar
  22. 22.
    Kakadiaris, I.A., Passalis, G., Toderici, G., Murtuza, M., Lu, Y., Karampatziakis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 640–649 (2007)CrossRefGoogle Scholar
  23. 23.
    Kakadiaris, I.A., Toderici, G., Evangelopoulos, G., Passalis, G., Zhao, X., Shah, S.K., Theoharis, T.: 3D–2D face recognition with pose and illumination normalization. Comput. Vis. Image Underst. 154, 137–151 (2017)CrossRefGoogle Scholar
  24. 24.
    Kemelmacher-Shlizerman, I., Seitz, S.: Face reconstruction in the wild. In: Proceedings of IEEE international conference on computer vision, Barcelona, Spain, pp. 1746–1753 (2011)Google Scholar
  25. 25.
    Ma, B., Li, A., Chai, X., Shan, S.: CovGa: a novel descriptor based on symmetry of regions for head pose estimation. Neurocomputing 143, 97–108 (2014)CrossRefGoogle Scholar
  26. 26.
    Masi, I., Lisanti, G., Bagdanov, A., Pala, P., Bimbo, A.: Using 3D models to recognize 2D faces in the wild. In: Proceedings of IEEE conference on computer vision and pattern recognition, Portland, OR, pp. 775–780 (2013)Google Scholar
  27. 27.
    Phillips, P.J., Scruggs, W.T., O’Toole, A.J., Flynn, P.J., Bowyer, K.W., Schott, C.L., Sharpe, M.: FRVT 2006 and ICE 2006 large-scale experimental results. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 831–846 (2010)CrossRefGoogle Scholar
  28. 28.
    Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: Proceedings of IEEE conference on computer vision and pattern recognition, Columbus, OH, pp. 1701–1708 (2014)Google Scholar
  29. 29.
    Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans. Image Process. 19(6), 1635–1650 (2010)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B. 58(1), 267–288 (1996)Google Scholar
  31. 31.
    Toderici, G., Passalis, G., Zafeiriou, S., Tzimiropoulos, G., Petrou, M., Theoharis, T., Kakadiaris, I.A.: Bidirectional relighting for 3D-aided 2D face recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition, San Francisco, CA, pp. 2721–2728 (2010)Google Scholar
  32. 32.
    Toderici, G., Evangelopoulos, G., Fang, T., Theoharis, T., Kakadiaris, I.A.: UHDB11 database for 3D–2D face recognition. In: Proceedings of Pacific-Rim symposium on image and video technology, Guanajuato, Mexico, pp. 73–86 (2013)Google Scholar
  33. 33.
    Vu, P.V., Chandler, D.: A fast wavelet-based algorithm for global and local image sharpness estimation. IEEE Signal Process. Lett. 19(7), 423–426 (2012)CrossRefGoogle Scholar
  34. 34.
    Wagner, A., Wright, J., Ganesh, A., Zhou, Z., Mobahi, H., Ma, Y.: Toward a practical face recognition system: robust alignment and illumination by sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 372–386 (2012)CrossRefGoogle Scholar
  35. 35.
    Wang, C., Song, X.: Robust head pose estimation via supervised manifold learning. Neural Netw. 53, 15–25 (2014)CrossRefMATHGoogle Scholar
  36. 36.
    Weise, T., Bouaziz, S., Li, H., Pauly, M.: Realtime performance-based facial animation. ACM Trans. Graph. 30(4), 77 (2011)CrossRefGoogle Scholar
  37. 37.
    Wu, Y., Xu, X., Shah, S.K., Kakadiaris, I.A.: Towards fitting a 3D dense facial model to a 2D image: a landmark-free approach. In: Proceedings of international conference on biometrics: theory, applications and systems, Arlington, VA, pp. 1–8 (2015)Google Scholar
  38. 38.
    Wu, Y., Shah, S.K., Kakadiaris, I.A.: Rendering or normalization? An analysis of the 3D-aided pose-invariant face recognition. In: Proceedings of IEEE international conference on identity, security and behavior analysis, Sendai, Japan, pp. 1–8 (2016)Google Scholar
  39. 39.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: Proceedings of IEEE conference on computer vision and pattern recognition, Portland, OR, pp. 532–539 (2013)Google Scholar
  40. 40.
    Xiong, X., De la Torre, F.: Global supervised descent method. In: Proceedings of IEEE conference on computer vision and pattern recognition, Boston, MA, pp. 2664–2673 (2015)Google Scholar
  41. 41.
    Yang, H., Jia, X., Patras, I., Chan, K.P.: Random subspace supervised descent method for regression problems in computer vision. IEEE Trans. Signal Process. Lett. 22(10), 1816–1820 (2015)CrossRefGoogle Scholar
  42. 42.
    Yang, H., Mou, W., Zhang, Y., Patras, I., Gunes, H., Robinson, P.: Face alignment assisted by head pose estimation. In: Proceedings of British machine vision conference, Swansea, UK, pp. 1–13 (2015)Google Scholar
  43. 43.
    Zhao, X., Kim, T.K., Luo, W.: Unified face analysis by iterative multi-output random forests. In: Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, Columbus, OH, pp. 1765–1772 (2014)Google Scholar
  44. 44.
    Zhen, X., Wang, Z., Yu, M., Li, S.: Supervised descriptor learning for multi-output regression. In: Proceedings of IEEE conference on computer vision and pattern recognition, Boston, MA, pp. 1211–1218 (2015)Google Scholar
  45. 45.
    Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: Proceedings of IEEE conference on computer vision and pattern recognition, Providence, RI, pp. 2879–2886 (2012)Google Scholar
  46. 46.
    Zhu, X., Lei, Z., Yan, J., Yi, D., Li, S.: High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of IEEE conference on computer vision and pattern recognition, Boston, MA, pp. 787–796 (2015)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Yuhang Wu
    • 1
  • Shishir K. Shah
    • 1
  • Ioannis A. Kakadiaris
    • 1
  1. 1.Computational Biomedicine Lab, Department of Computer ScienceUniversity of HoustonHoustonUSA

Personalised recommendations