Advertisement

The First 3D Face Alignment in the Wild (3DFAW) Challenge

  • László A. Jeni
  • Sergey Tulyakov
  • Lijun Yin
  • Nicu Sebe
  • Jeffrey F. Cohn
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9914)

Abstract

2D alignment of face images works well provided images are frontal or nearly so and pitch and yaw remain modest. In spontaneous facial behavior, these constraints often are violated by moderate to large head rotation. 3D alignment from 2D video has been proposed as a solution. A number of approaches have been explored, but comparisons among them have been hampered by the lack of common test data. To enable comparisons among alternative methods, The 3D Face Alignment in the Wild (3DFAW) Challenge, presented for the first time, created an annotated corpus of over 23,000 multi-view images from four sources together with 3D annotation, made training and validation sets available to investigators, and invited them to test their algorithms on an independent test-set. Eight teams accepted the challenge and submitted test results. We report results for four that provided necessary technical descriptions of their methods. The leading approach achieved prediction consistency error of 3.48 %. Corresponding result for the lowest ranked approach was 5.9 %. The results suggest that 3D alignment from 2D video is feasible on a wide range of face orientations. Differences among methods are considered and suggest directions for further research.

Keywords

3D alignment from 2D video Head rotation Prediction consistency error Faces in-the-wild 

Notes

Acknowledgements

This work was supported in part by US National Institutes of Health grant MH096951 to the University of Pittsburgh and by US National Science Foundation grants CNS-1205664 and CNS-1205195 to the University of Pittsburgh and the University of Binghamton. Neither agency was involved in the planning or writing of the work.

References

  1. 1.
    de B. Zavan, F.H., Nascimento, A.C.P., e Silva, L.P., Bellon, O.R.P., Silva, L.: 3d face alignment in the wild: A landmark-free, nose-based approach. In: 2016 European Conference on Computer Vision Workshops (ECCVW) (2016)Google Scholar
  2. 2.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. SIGGRAPH (1999). http://dx.doi.org/10.1145/311535.311556
  3. 3.
    Bulat, A., Tzimiropoulos, G.: Two-stage convolutional part heatmap regression for the 1st 3d face alignment in the wild (3DFAW) challenge. In: European Conference on Computer Vision Workshops (ECCVW) (2016)Google Scholar
  4. 4.
    Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2887–2894, June 2012Google Scholar
  5. 5.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)CrossRefGoogle Scholar
  6. 6.
    Cristinacce, D., Cootes, T.: Automatic feature localisation with constrained local models. Pattern Recogn. 41(10), 3054–3067 (2008). http://dx.doi.org/10.1016/j.patcog.2008.01.024 CrossRefzbMATHGoogle Scholar
  7. 7.
    Dimitrijevic, M., Ilic, S., Fua, P.: Accurate face models from uncalibrated and ill-lit video sequences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). vol. 2, pp. II-1034-II-1041, June 2004Google Scholar
  8. 8.
    Dollar, P., Welinder, P., Perona, P.: Cascaded pose regression. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1078–1085, June 2010Google Scholar
  9. 9.
    Ekman, P., Friesen, W., Hager, J.: Facial Action Coding System (FACS): Manual. A Human Face, Salt Lake City (2002)Google Scholar
  10. 10.
    Gou, C., Wu, Y., Wang, F.Y., Ji, Q.: Shape augmented regression for 3d face alignment. In: 2016 European Conference on Computer Vision Workshops (ECCVW) (2016)Google Scholar
  11. 11.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)CrossRefGoogle Scholar
  12. 12.
    Gu, L., Kanade, T.: 3d alignment of face in a single image. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. (CVPR) 1, 1305–1312 (2006)Google Scholar
  13. 13.
    Jeni, L.A., Cohn, J.F., Kanade, T.: Dense 3d face alignment from 2d videos in real-time. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) (2015). http://zface.org
  14. 14.
    Jeni, L.A., Cohn, J.F., Kanade, T.: Dense 3d face alignment from 2d video for real-time use. Image Vis. Comput. 28(5), 807–813 (2016)Google Scholar
  15. 15.
    Jourabloo, A., Liu, X.: Large-pose face alignment via cnn-based dense 3d model fitting. In: CVPR (2016)Google Scholar
  16. 16.
    Matthews, I., Baker, S.: Active appearance models revisited. Int. J. Comput. Vis. 60(2), 135–164 (2004)CrossRefGoogle Scholar
  17. 17.
    Piotraschke, M., Blanz, V.: Automated 3d face reconstruction from multiple images using quality measures. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3418–3427 (2016)Google Scholar
  18. 18.
    Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 fps via regressing local binary features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1685–1692, June (2014)Google Scholar
  19. 19.
    Roth, J., Tong, Y., Liu, X.: Adaptive 3d face reconstruction from unconstrained photo collections. In: CVPR (2016)Google Scholar
  20. 20.
    Sánchez-Escobedo, D., Castelán, M., Smith, W.A.: Statistical 3d face shape estimation from occluding contours. Comput. Vis. Image Underst. 142, 111–124 (2016)CrossRefGoogle Scholar
  21. 21.
    Saragih, J.M., Lucey, S., Cohn, J.F.: Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vis. 91(2), 200–215 (2011). http://dx.doi.org/10.1007/s11263-010-0380-4 MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Tulyakov, S., Sebe, N.: Regressing a 3d face shape from a single image. In: IEEE International Conference on Computer Vision (ICCV), pp. 3748–3755. IEEE (2015)Google Scholar
  23. 23.
    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: Large-scale scene recognition from abbey to zoo. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3485–3492. IEEE (2010)Google Scholar
  24. 24.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 532–539, June (2013)Google Scholar
  25. 25.
    Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3d dynamic facial expression database. In: 8th IEEE International Conference on Automatic Face Gesture Recognition, 2008. FG 2008, pp. 1–6, Sept 2008Google Scholar
  26. 26.
    Zhang, X., Yin, L., Cohn, J.F., Canavan, S., Reale, M., Horowitz, A., Liu, P., Girard, J.M.: Bp. 4d-spontaneous: a high-resolution spontaneous 3d dynamic facial expression database. Image Vis. Comput. 32(10), 692–706 (2014). best of Automatic Face and Gesture Recognition 2013CrossRefGoogle Scholar
  27. 27.
    Zhang, Z., Liu, Z., Adler, D., Cohen, M.F., Hanson, E., Shan, Y.: Robust and rapid generation of animated faces from video images: A model-based modeling approach. Int. J. Comput. Vis. 58(2), 93–119 (2004). http://dx.doi.org/10.1023/B: VISI.0000015915.50080.85 CrossRefGoogle Scholar
  28. 28.
    Zhao, R., Wang, Y., Benitez-Quiroz, C.F., Liu, Y., Martinez, A.M.: Fast & precise face alignment and 3d shape recovery from a single image. In: 2016 European Conference on Computer Vision Workshops (ECCVW) (2016)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • László A. Jeni
    • 1
  • Sergey Tulyakov
    • 2
  • Lijun Yin
    • 3
  • Nicu Sebe
    • 2
  • Jeffrey F. Cohn
    • 1
    • 4
  1. 1.Robotics InstituteCarnegie Mellon UniversityPittsburghUSA
  2. 2.DISIUniversity of TrentoTrentoItaly
  3. 3.Department of Computer ScienceState University of New York at BinghamtonBinghamtonUSA
  4. 4.Department of PsychologyThe University of PittsburghPittsburghUSA

Personalised recommendations