Eigen Appearance Maps of Dynamic Shapes

  • Adnane BoukhaymaEmail author
  • Vagia Tsiminaki
  • Jean-Sébastien Franco
  • Edmond Boyer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9905)


We address the problem of building efficient appearance representations of shapes observed from multiple viewpoints and in several movements. Multi-view systems now allow the acquisition of spatio-temporal models of such moving objects. While efficient geometric representations for these models have been widely studied, appearance information, as provided by the observed images, is mainly considered on a per frame basis, and no global strategy yet addresses the case where several temporal sequences of a shape are available. We propose a per subject representation that builds on PCA to identify the underlying manifold structure of the appearance information relative to a shape. The resulting eigen representation encodes shape appearance variabilities due to viewpoint and motion, with Eigen textures, and due to local inaccuracies in the geometric model, with Eigen warps. In addition to providing compact representations, such decompositions also allow for appearance interpolation and appearance completion. We evaluate their performances over different characters and with respect to their ability to reproduce compelling appearances in a compact way.


Optical Flow Input Texture Appearance Variation Texture Space Projection Coefficient 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., Sullivan, S.: High-quality streamable free-viewpoint video. ACM Trans. Graph. 34, 69:1–69:13 (2015)CrossRefGoogle Scholar
  2. 2.
    Nishino, K., Sato, Y., Ikeuchi, K.: Eigen-texture method: appearance compression and synthesis based on a 3D model. IEEE Trans. Pattern Anal. Mach. Intell. 23, 1257–1265 (2001)CrossRefGoogle Scholar
  3. 3.
    Debevec, P.E., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In: ACM SIGGRAPH 1996 (1996)Google Scholar
  4. 4.
    Lempitsky, V.S., Ivanov, D.V.: Seamless mosaicing of image-based texture maps. In: CVPR (2007)Google Scholar
  5. 5.
    Waechter, M., Moehrle, N., Goesele, M.: Let there be color! Large-scale texturing of 3D reconstructions. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 836–850. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10602-1_54 Google Scholar
  6. 6.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: ACM SIGGRAPH 1996 (1999)Google Scholar
  7. 7.
    Carranza, J., Theobalt, C., Magnor, M.A., Seidel, H.P.: Free-viewpoint video of human actors. ACM Trans. Graph. 22, 569–577 (2003)CrossRefGoogle Scholar
  8. 8.
    Zitnick, C., Kang, S., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. In: ACM SIGGRAPH 2004 (2004)Google Scholar
  9. 9.
    Eisemann, M., De Decker, B., Magnor, M., Bekaert, P., de Aguiar, E., Ahmed, N., Theobalt, C., Sellent, A.: Floating textures. Comput. Graph Forum (Proc. of Eurographics) 27, 409–418 (2008)CrossRefGoogle Scholar
  10. 10.
    Tung, T.: Simultaneous super-resolution and 3D video using graph-cuts (2008)Google Scholar
  11. 11.
    Tsiminaki, V., Franco, J.S., Boyer, E.: High resolution 3D shape texture from multiple videos. In: CVPR (2014)Google Scholar
  12. 12.
    Goldlücke, B., Aubry, M., Kolev, K., Cremers, D.: A super-resolution framework for high-accuracy multiview reconstruction. Int. J. Comput. Vis. 106, 172–191 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 98:1–98:10 (2008)CrossRefGoogle Scholar
  14. 14.
    Cagniart, C., Boyer, E., Ilic, S.: Free-from mesh tracking: a patch-based approach. In: CVPR (2010)Google Scholar
  15. 15.
    Volino, M., Casas, D., Collomosse, J., Hilton, A.: Optimal representation of multiple view video. In: BMVC (2014)Google Scholar
  16. 16.
    Boukhayma, A., Boyer, E.: Video based animation synthesis with the essential graph. In: 3DV (2015)Google Scholar
  17. 17.
    Casas, D., Tejera, M., Guillemaut, J.Y., Hilton, A.: 4D parametric motion graphs for interactive animation. In: ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (2012)Google Scholar
  18. 18.
    Casas, D., Volino, M., Collomosse, J., Hilton, A.: 4D video textures for interactive character appearance. Comput. Graph. Forum (Proc. of Eurographics) 33, 371–380 (2014)CrossRefGoogle Scholar
  19. 19.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3, 71–86 (1991)CrossRefGoogle Scholar
  20. 20.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)CrossRefGoogle Scholar
  21. 21.
    Allain, B., Franco, J.S., Boyer, E.: An efficient volumetric framework for shape tracking. In: CVPR (2015)Google Scholar
  22. 22.
    Sanchez Prez, J., Meinhardt-Llopis, E., Facciolo, G.: TV-L1 optical flow estimation. Image Process. On Line 3, 137–150 (2013)CrossRefGoogle Scholar
  23. 23.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)CrossRefGoogle Scholar
  24. 24.
    Chen, T., Zhu, J.Y., Shamir, A., Hu, S.M.: Motion-aware gradient domain video composition. IEEE Trans. Image Process. 22(7), 2532–2544 (2013)CrossRefGoogle Scholar
  25. 25.
    Linz, C., Lipski, C., Magnor, M.: Multi-image interpolation based on graph-cuts and symmetric optical flow (2010)Google Scholar
  26. 26.
    Mahajan, D., Huang, F.C., Matusik, W., Ramamoorthi, R., Belhumeur, P.N.: Moving gradients: a path-based method for plausible image interpolation. ACM Trans. Graph. 28(3), 1–11 (2009)CrossRefGoogle Scholar
  27. 27.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)CrossRefGoogle Scholar
  28. 28.
    Xu, D., Zhang, H., Wang, Q., Bao, H.: Poisson shape interpolation. In: ACM Symposium on Solid and Physical Modeling (2005)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Adnane Boukhayma
    • 1
    Email author
  • Vagia Tsiminaki
    • 1
  • Jean-Sébastien Franco
    • 1
  • Edmond Boyer
    • 1
  1. 1.LJKUniversité Grenoble Alpes, Inria Grenoble Rhône-AlpesGrenobleFrance

Personalised recommendations