Advertisement

Shape Augmented Regression for 3D Face Alignment

  • Chao Gou
  • Yue Wu
  • Fei-Yue Wang
  • Qiang Ji
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9914)

Abstract

2D face alignment has been an active topic and is becoming mature for real applications. However, when large head pose exists, 2D annotated points lose geometric correspondence with respect to actual 3D location. In addition, local appearance varies more dramatically when subjects are with large pose or under various illuminations. 3D face alignment from 2D images is a promising solution to tackle this problem. 3D face alignment aims to estimate the 3D face shape which is consistent across all poses. In this paper, we propose a novel 3D face alignment method. This method consists of two steps. First, we perform 2D landmark detection based on the shape augmented regression. Second, we estimate the 3D shape using the detected 2D landmarks and 3D deformable model. Experimental results on benchmark database demonstrate its preferable performances.

Keywords

Shape augmented regression 3D face alignment 

Notes

Acknowledgments

This work was completed when the first author visited Rensselaer Polytechnic Institute (RPI), supported by a scholarship from University of Chinese Academy of Sciences (UCAS). The authors would like to acknowledge support from UCAS and RPI. This work was also supported in part by National Science Foundation under the grant 1145152 and by the National Natural Science Foundation of China under Grant 61304200 and 61533019.

References

  1. 1.
    Wu, Y., Ji, Q.: Constrained joint cascade regression framework for simultaneous facial action unit recognition and facial landmark detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2016)Google Scholar
  2. 2.
    Wagner, A., Wright, J., Ganesh, A., Zhou, Z., Mobahi, H., Ma, Y.: Toward a practical face recognition system: robust alignment and illumination by sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 372–386 (2012)CrossRefGoogle Scholar
  3. 3.
    Narayanan, A., Kaimal, R.M., Bijlani, K.: Estimation of driver head yaw angle using a generic geometric model. IEEE Trans. Intell. Transp. Syst. PP(99), 1–15 (2016)Google Scholar
  4. 4.
    Roth, J., Tong, Y., Liu, X.: Adaptive 3D face reconstruction from unconstrained photo collections. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2016)Google Scholar
  5. 5.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 532–539. IEEE (2013)Google Scholar
  6. 6.
    Wu, Y., Ji, Q.: Shape augmented regression method for face alignment. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 26–32 (2015)Google Scholar
  7. 7.
    Tulyakov, S., Sebe, N.: Regressing a 3D face shape from a single image. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3748–3755. IEEE (2015)Google Scholar
  8. 8.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 6, 681–685 (2001)CrossRefGoogle Scholar
  9. 9.
    Matthews, I., Baker, S.: Active appearance models revisited. Int. J. Comput. Vis. 60(2), 135–164 (2004)CrossRefGoogle Scholar
  10. 10.
    Lucey, S., Navarathna, R., Ashraf, A.B., Sridharan, S.: Fourier lucas-kanade algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1383–1396 (2013)CrossRefGoogle Scholar
  11. 11.
    Saragih, J.M., Lucey, S., Cohn, J.F.: Face alignment through subspace constrained mean-shifts. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1034–1041. IEEE (2009)Google Scholar
  12. 12.
    Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models. 1(2), 3 (2006)Google Scholar
  13. 13.
    Lindner, C., Bromiley, P.A., Ionita, M.C., Cootes, T.F.: Robust and accurate shape model matching using random forest regression-voting. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1862–1874 (2015)CrossRefGoogle Scholar
  14. 14.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM Press/Addison-Wesley Publishing Co. (1999)Google Scholar
  15. 15.
    Gu, L., Kanade, T.: 3D alignment of face in a single image. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 1, pp. 1305–1312. IEEE (2006)Google Scholar
  16. 16.
    Jeni, L.A., Cohn, J.F., Kanade, T.: Dense 3D face alignment from 2D videos in real-time. In: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 1, pp. 1–8. IEEE (2015)Google Scholar
  17. 17.
    Cao, C., Weng, Y., Lin, S., Zhou, K.: 3D shape regression for real-time facial animation. ACM Trans. Graph. (TOG) 32(4), 41 (2013)CrossRefzbMATHGoogle Scholar
  18. 18.
    Jourabloo, A., Liu, X.: Pose-invariant 3D face alignment. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3694–3702 (2015)Google Scholar
  19. 19.
    Jourabloo, A., Liu, X.: Large-pose face alignment via CNN-based dense 3D model fitting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2016)Google Scholar
  20. 20.
    Zhu, X., Lei, Z., Liu, X., Shi, H., Li, S.Z.: Face alignment across large poses: a 3D solution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2016)Google Scholar
  21. 21.
    Jeni, L.A., Cohn, J.F., Kanade, T.: Dense 3D face alignment from 2D video for real-time use. Image and Vision Computing (2016)Google Scholar
  22. 22.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  23. 23.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)CrossRefGoogle Scholar
  24. 24.
    Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: 8th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2008, pp. 1–6. IEEE (2008)Google Scholar
  25. 25.
    Zhang, X., Yin, L., Cohn, J.F., Canavan, S., Reale, M., Horowitz, A., Liu, P., Girard, J.M.: BP4D-spontaneous: a high-resolution spontaneous 3D dynamic facial expression database. Image Vis. Comput. 32(10), 692–706 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.ECSE, Rensselaer Polytechnic InstituteTroyUSA
  3. 3.Qingdao Academy of Intelligent IndustriesQingdaoChina

Personalised recommendations