Fitting a 3D Morphable Model to Edges: A Comparison Between Hard and Soft Correspondences

  • Anil BasEmail author
  • William A. P. Smith
  • Timo Bolkart
  • Stefanie Wuhrer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10117)


In this paper we explore the problem of fitting a 3D morphable model to single face images using only sparse geometric features (edges and landmark points). Previous approaches to this problem are based on nonlinear optimisation of an edge-derived cost that can be viewed as forming soft correspondences between model and image edges. We propose a novel approach, that explicitly computes hard correspondences. The resulting objective function is non-convex but we show that a good initialisation can be obtained efficiently using alternating linear least squares in a manner similar to the iterated closest point algorithm. We present experimental results on both synthetic and real images and show that our approach outperforms methods that use soft correspondence and other recent methods that rely solely on geometric features.


Face Shape Iterate Close Point Active Shape Model Edge Cost Orthographic Projection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1063–1074 (2003)CrossRefGoogle Scholar
  2. 2.
    Ichim, A.E., Bouaziz, S., Pauly, M.: Dynamic 3D avatar creation from hand-held video input. ACM Trans. Graph. 34, 45 (2015)CrossRefGoogle Scholar
  3. 3.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH (1999)Google Scholar
  4. 4.
    Romdhani, S., Vetter, T.: Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In: CVPR (2005)Google Scholar
  5. 5.
    Huber, P., Feng, Z., Christmas, W., Kittler, J., Rätsch, M.: Fitting 3D morphable models using local features. In: ICIP (2015)Google Scholar
  6. 6.
    Amberg, B., Blake, A., Fitzgibbon, A., Romdhani, S., Vetter, T.: Reconstructing high quality face-surfaces using model based stereo. In: ICCV (2007)Google Scholar
  7. 7.
    Keller, M., Knothe, R., Vetter, T.: 3D reconstruction of human faces from occluding contours. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2007. LNCS, vol. 4418, pp. 261–273. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-71457-6_24 CrossRefGoogle Scholar
  8. 8.
    Breuer, P., Kim, K., Kienzle, W., Schölkopf, B., Blanz, V.: Automatic 3D face reconstruction from single images or video. In: Proceedings of the FG, pp. 1–8 (2008)Google Scholar
  9. 9.
    Aldrian, O., Smith, W.A.P.: Inverse rendering of faces with a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1080–1093 (2013)CrossRefGoogle Scholar
  10. 10.
    Amberg, B., Vetter, T.: Optimal landmark detection using shape models and branch and bound. In: Proceedings of the ICCV (2011)Google Scholar
  11. 11.
    Schönborn, S., Forster, A., Egger, B., Vetter, T.: A monte carlo strategy to integrate detection and model-based face analysis. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 101–110. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40602-7_11 CrossRefGoogle Scholar
  12. 12.
    Cootes, T.F., Taylor, C.J., Cooper, D., Graham, J.: Active shape models - their training and application. Comput. Vis. Image Underst. 61, 38–59 (1995)CrossRefGoogle Scholar
  13. 13.
    Atkinson, G.A., Smith, M.L., Smith, L.N., Farooq, A.R.: Facial geometry estimation using photometric stereo and profile views. In: Tistarelli, M., Nixon, M.S. (eds.) ICB 2009. LNCS, vol. 5558, pp. 1–11. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-01793-3_1 CrossRefGoogle Scholar
  14. 14.
    Moghaddam, B., Lee, J., Pfister, H., Machiraju, R.: Model-based 3D face capture with shape-from-silhouettes. In: Proceedings of the FG (2003)Google Scholar
  15. 15.
    Lüthi, M., Albrecht, T., Vetter, T.: Probabilistic modeling and visualization of the flexibility in morphable models. In: Hancock, E.R., Martin, R.R., Sabin, M.A. (eds.) Mathematics of Surfaces 2009. LNCS, vol. 5654, pp. 251–264. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-03596-8_14 CrossRefGoogle Scholar
  16. 16.
    Fitzgibbon, A.W.: Robust registration of 2D and 3D point sets. Image Vis. Comput. 21, 1145–1153 (2003)CrossRefGoogle Scholar
  17. 17.
    Cashman, T.J., Fitzgibbon, A.W.: What shape are dolphins? Building 3D morphable models from 2D images. IEEE Trans. Pattern Anal. Mach. Intell. 35, 232–244 (2013)CrossRefGoogle Scholar
  18. 18.
    Zhu, X., Lei, Z., Yan, J., Yi, D., Li, S.Z.: High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the CVPR, pp. 787–796 (2015)Google Scholar
  19. 19.
    Sánchez-Escobedo, D., Castelán, M., Smith, W.: Statistical 3D face shape estimation from occluding contours. Comput. Vis. Image Underst. 142, 111–124 (2016)CrossRefGoogle Scholar
  20. 20.
    Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 239–256 (1992)CrossRefGoogle Scholar
  21. 21.
    Suwajanakorn, S., Kemelmacher-Shlizerman, I., Seitz, S.M.: Total moving face reconstruction. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 796–812. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10593-2_52 Google Scholar
  22. 22.
    Cao, C., Bradley, D., Zhou, K., Beeler, T.: Real-time high-fidelity facial performance capture. ACM Trans. Graph. 34, 46 (2015)CrossRefGoogle Scholar
  23. 23.
    Dementhon, D.F., Davis, L.S.: Model-based object pose in 25 lines of code. Int. J. Comput. Vis. 15, 123–141 (1995)CrossRefGoogle Scholar
  24. 24.
    Brunton, A., Salazar, A., Bolkart, T., Wuhrer, S.: Review of statistical shape spaces for 3D data with comparative analysis for human faces. Comput. Vis. Image Underst. 128, 1–17 (2014)CrossRefGoogle Scholar
  25. 25.
    Coleman, T., Li, Y.: An interior, trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6, 418–445 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: Proceedings of the CVPR (2012)Google Scholar
  27. 27.
    Grant, M., Boyd, S., Ye, Y.: Disciplined convex programming. In: Liberti, L., Maculan, N. (eds.) Global Optimization: From Theory to Implementation, pp. 155–210. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  28. 28.
    Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: Proceedings of the AVSS (2009)Google Scholar
  29. 29.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1615–1618 (2003)CrossRefGoogle Scholar
  30. 30.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07–49, University of Massachusetts, Amherst (2007)Google Scholar
  31. 31.
    David, P., DeMenthon, D., Duraiswami, R., Samet, H.: SoftPOSIT: simultaneous pose and correspondence determination. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 698–714. Springer, Heidelberg (2002). doi: 10.1007/3-540-47977-5_46 CrossRefGoogle Scholar
  32. 32.
    Moreno-Noguer, F., Lepetit, V., Fua, P.: Pose priors for simultaneously solving alignment and correspondence. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5303, pp. 405–418. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88688-4_30 CrossRefGoogle Scholar
  33. 33.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings of the 3DIM (2001)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Anil Bas
    • 1
    Email author
  • William A. P. Smith
    • 1
  • Timo Bolkart
    • 2
  • Stefanie Wuhrer
    • 3
  1. 1.Department of Computer ScienceUniversity of YorkYorkUK
  2. 2.Multimodal Computing and InteractionSaarland UniversitySaarbrückenGermany
  3. 3.Morpheo TeamInria Grenoble Rhône-AlpesGrenobleFrance

Personalised recommendations