Multilinear Wavelets: A Statistical Shape Space for Human Faces

  • Alan Brunton
  • Timo Bolkart
  • Stefanie Wuhrer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8689)

Abstract

We present a statistical model for 3D human faces in varying expression, which decomposes the surface of the face using a wavelet transform, and learns many localized, decorrelated multilinear models on the resulting coefficients. Using this model we are able to reconstruct faces from noisy and occluded 3D face scans, and facial motion sequences. Accurate reconstruction of face shape is important for applications such as tele-presence and gaming. The localized and multi-scale nature of our model allows for recovery of fine-scale detail while retaining robustness to severe noise and occlusion, and is computationally efficient and scalable. We validate these properties experimentally on challenging data in the form of static scans and motion sequences. We show that in comparison to a global multilinear model, our model better preserves fine detail and is computationally faster, while in comparison to a localized PCA model, our model better handles variation in expression, is faster, and allows us to fix identity parameters for a given subject.

Keywords

Statistical shape models human faces multilinear model wavelets 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-319-10590-1_20_MOESM1_ESM.avi (34.9 mb)
Electronic Supplementary Material (AVI 35,747 KB)

References

  1. 1.
    Amberg, B., Knothe, R., Vetter, T.: Expression invariant 3D face recognition with a morphable model. In: FG, pp. 1–6 (2008)Google Scholar
  2. 2.
    Arya, A., Mount, D.: Approximate nearest neighbor queries in fixed dimensions. In: SODA, pp. 271–280 (1993), http://www.cs.umd.edu/~mount/ANN/
  3. 3.
    Bertram, M., Duchaineau, M., Hamann, B., Joy, K.I.: Generalized B-Spline subdivision-surface wavelets for geometry compression. TVCG 10(3), 326–338 (2004)Google Scholar
  4. 4.
    Blanz, V., Scherbaum, K., Seidel, H.P.: Fitting a morphable model to 3D scans of faces. In: ICCV, pp. 1–8 (2007)Google Scholar
  5. 5.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH, pp. 187–194 (1999)Google Scholar
  6. 6.
    Bolkart, T., Brunton, A., Salazar, A., Wuhrer, S.: Statistical 3D shape models of human faces (2013), http://statistical-face-models.mmci.uni-saarland.de/
  7. 7.
    Bolkart, T., Wuhrer, S.: Statistical analysis of 3D faces in motion. In: 3DV, pp. 103–110 (2013)Google Scholar
  8. 8.
    Brunton, A., Shu, C., Lang, J., Dubois, E.: Wavelet model-based stereo for fast, robust face reconstruction. In: CRV, pp. 347–354 (2011)Google Scholar
  9. 9.
    Chen, Y., Liu, Z., Zhang, Z.: Tensor-based human body modeling. In: CVPR, pp. 105–112 (2013)Google Scholar
  10. 10.
    Creusot, C., Pears, N., Austin, J.: A machine-learning approach to keypoint detection and landmarking on 3D meshes. IJCV 102(1-3), 146–179 (2013)CrossRefGoogle Scholar
  11. 11.
    Dale, K., Sunkavalli, K., Johnson, M.K., Vlasic, D., Matusik, W., Pfister, H.: Video face replacement. TOG 30(6), 130:1–130:10 (2011)Google Scholar
  12. 12.
    Davatzikos, C., Tao, X., Shen, D.: Hierarchical active shape models, using the wavelet transform. TMI 22(3), 414–423 (2003)Google Scholar
  13. 13.
    ter Haar, F.B., Veltkamp, R.C.: 3D face model fitting for recognition. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 652–664. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Kakadiaris, I., Passalis, G., Toderici, G., Murtuza, M., Lu, Y., Karamelpatzis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: An annotated deformable model approach. TPAMI 29(4), 640–649 (2007)CrossRefGoogle Scholar
  15. 15.
    Kobbelt, L., Campagna, S., Vorsatz, J., Seidel, H.P.: Interactive multi-resolution modeling on arbitrary meshes. In: SIGGRAPH 1998, pp. 105–114 (1998)Google Scholar
  16. 16.
    Lathauwer, L.D.: Signal processing based on multilinear algebra. Ph.D. thesis, K.U. Leuven, Belgium (1997)Google Scholar
  17. 17.
    Lecron, F., Boisvert, J., Mahmoudi, S., Labelle, H., Benjelloun, M.: Fast 3D spine reconstruction of postoperative patients using a multilevel statistical model. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part II. LNCS, vol. 7511, pp. 446–453. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Li, Y., Tan, T.S., Volkau, I., Nowinski, W.: Model-guided segmentation of 3D neuroradiological image using statistical surface wavelet model. In: CVPR, pp. 1–7 (2007)Google Scholar
  19. 19.
    Liu, D., Nocedal, J.: On the limited memory method for large scale optimization. Math. Prog.: Ser. A, B 45(3), 503–528 (1989)CrossRefMATHMathSciNetGoogle Scholar
  20. 20.
    Mpiperis, I., Malassiotis, S., Strintzis, M.G.: Bilinear models for 3-D face and facial expression recognition. TIFS 3, 498–511 (2008)Google Scholar
  21. 21.
    Nain, D., Haker, S., Bobick, A., Tannenbaum, A.R.: Multiscale 3D shape analysis using spherical wavelets. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3750, pp. 459–467. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  22. 22.
    Patel, A., Smith, W.: 3D morphable face models revisited. In: CVPR, pp. 1327–1334 (2009)Google Scholar
  23. 23.
    Salazar, A., Wuhrer, S., Shu, C., Prieto, F.: Fully automatic expression-invariant face correspondence. MVAP 25(4), 859–879 (2014)Google Scholar
  24. 24.
    Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3D face analysis. In: Schouten, B., Juul, N.C., Drygajlo, A., Tistarelli, M. (eds.) BIOID 2008. LNCS, vol. 5372, pp. 47–56. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  25. 25.
    Schröder, P., Sweldens, W.: Spherical wavelets: Efficiently representing functions on the sphere. In: SIGGRAPH, pp. 161–172 (1995)Google Scholar
  26. 26.
    De Smet, M., Van Gool, L.: Optimal regions for linear model-based 3D face reconstruction. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 276–289. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  27. 27.
    Sweldens, W.: The lifting scheme: A custom-design construction of biorthogonal wavelets. Appl. Comp. Harm. Anal. 3(2), 186–200 (1996)CrossRefMATHMathSciNetGoogle Scholar
  28. 28.
    Tam, G., Cheng, Z.Q., Lai, Y.K., Langbein, F., Liu, Y., Marshall, D., Martin, R., Sun, X.F., Rosin, P.: Registration of 3D point clouds and meshes: A survey from rigid to nonrigid. TVCG 19(7), 1199–1217 (2013)Google Scholar
  29. 29.
    Vasilescu, M.A.O., Terzopoulos, D.: Multilinear analysis of image ensembles: TensorFaces. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part I. LNCS, vol. 2350, pp. 447–460. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  30. 30.
    Vlasic, D., Brand, M., Pfister, H., Popović, J.: Face transfer with multilinear models. TOG 24(3), 426–433 (2005)CrossRefGoogle Scholar
  31. 31.
    Yang, F., Bourdev, L., Wang, J., Shechtman, E., Metaxas, D.: Facial expression editing in video using a temporally-smooth factorization. In: CVPR, pp. 861–868 (2012)Google Scholar
  32. 32.
    Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: FG, pp. 1–6 (2008)Google Scholar
  33. 33.
    Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: FG, pp. 211–216 (2006)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Alan Brunton
    • 1
  • Timo Bolkart
    • 2
    • 3
  • Stefanie Wuhrer
    • 2
  1. 1.Fraunhofer Institute for Computer Graphics Research IGDGermany
  2. 2.Cluster of Excellence MMCISaarland UniversityGermany
  3. 3.Saarbrücken Graduate School of Computer ScienceGermany

Personalised recommendations