International Symposium on Visual Computing

Advances in Visual Computing pp 92-101 | Cite as

Time-Varying Surface Reconstruction of an Actor’s Performance

  • Ludovic Blache
  • Mathieu Desbrun
  • Céline Loscos
  • Laurent Lucas
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9474)


We propose a fully automatic time-varying surface reconstruction of an actor’s performance captured from a production stage through omnidirectional video. The resulting mesh and its texture can then directly be edited in post-production. Our method makes no assumption on the costumes or accessories present in the recording. We take as input a raw sequence of volumetric static poses reconstructed from video sequences acquired in a multi-viewpoint chroma-key studio. The first frame is chosen as the reference mesh. An iterative approach is applied throughout the sequence in order to induce a deformation of the reference mesh for all input frames. At first, a pseudo-rigid transformation adjusts the pose to match the input visual hull as closely as possible. Then, local deformation is added to reconstruct fine details. We provide examples of actors’ performance inserted into virtual scenes, including dynamic interaction with the environment.



M. Desbrun is partially funded by the National Science Foundation (CCF-1011944 grant), and gratefully acknowledges being hosted by the TITANE team in the context of an INRIA International Chair. We would like to thank our partner XD Productions. This work has been carried out thanks to the support of the RECOVER3D project, funded by the Investissements d’Avenir program. Some of the captured performance data were provided courtesy of the Max-Planck-Center for Visual Computing and Communication (MPI Informatik/Stanford) and Morpheo research team of INRIA.


  1. 1.
    de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 98:1–98:10 (2008)CrossRefGoogle Scholar
  2. 2.
    Vlasic, D., Baran, I., Matusik, W., Popović, J.: Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27, 97:1–97:09 (2008)CrossRefGoogle Scholar
  3. 3.
    Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., Seidel, H.P.: Motion capture using joint skeleton tracking and surface estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1746–1753 (2009)Google Scholar
  4. 4.
    Li, H., Luo, L., Vlasic, D., Peers, P., Popović, J., Pauly, M., Rusinkiewicz, S.: Temporally coherent completion of dynamic shapes. ACM Trans. Graph. 31, 2:1–2:11 (2012)Google Scholar
  5. 5.
    Starck, J., Hilton, A.: Correspondence labelling for wide-timeframe free-form surface matching. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1–8 (2007)Google Scholar
  6. 6.
    Varanasi, K., Zaharescu, A., Boyer, E., Horaud, R.: Temporal surface tracking using mesh evolution. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 30–43. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  7. 7.
    Tung, T., Matsuyama, T.: Dynamic surface matching by geodesic mapping for 3D animation transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1402–1409 (2010)Google Scholar
  8. 8.
    Petit, B., Letouzey, A., Boyer, E., Franco, J.S.: Surface flow from visual cues. In: International Workshop on Vision, Modeling and Visualization (VMV), pp. 1–8 (2011)Google Scholar
  9. 9.
    Nobuhara, S., Matsuyama, T.: Heterogeneous deformation model for 3D shape and motion recovery from multi-viewpoint images. In: Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission, pp. 566–573 (2004)Google Scholar
  10. 10.
    Blache, L., Loscos, C., Nocent, O., Lucas, L.: 3d volume matching for mesh animation of moving actors. In: EG Workshop on 3D Object Retrieval, pp. 69–76 (2014)Google Scholar
  11. 11.
    Bouaziz, S., Tagliasacchi, A., Pauly, M.: Dynamic 2D/3D registration. In: Holzschuch, N., Myszkowski, K. (eds.) Eurographics Tutorial. The Eurographics Association (2014)Google Scholar
  12. 12.
    Sorkine, O., Alexa, M.: As-rigid-as-possible surface modeling. In: Proceedings of Symposium on Geometry Processing, pp. 109–116 (2007)Google Scholar
  13. 13.
    Laurentini, A.: Visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994)CrossRefGoogle Scholar
  14. 14.
    Saito, T., Toriwaki, J.I.: New algorithms for euclidean distance transformation of an n-dimensional digitized picture with applications. Pattern Recogn. 27, 1551–1565 (1994)CrossRefGoogle Scholar
  15. 15.
    Bojsen-Hansen, M., Li, H., Wojtan, C.: Tracking surfaces with evolving topology. ACM Trans. Graph. 31, 53:1–53:10 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Ludovic Blache
    • 1
  • Mathieu Desbrun
    • 2
    • 3
  • Céline Loscos
    • 1
  • Laurent Lucas
    • 1
  1. 1.University of Reims Champagne-ArdenneReimsFrance
  2. 2.CaltechPasadenaUSA
  3. 3.INRIASophia-AntipolisFrance

Personalised recommendations