Globally Optimal Spatio-temporal Reconstruction from Cluttered Videos

  • Ehsan Aganj
  • Jean-Philippe Pons
  • Renaud Keriven
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5996)


We propose a method for multi-view reconstruction from videos adapted to dynamic cluttered scenes under uncontrolled imaging conditions. Taking visibility into account and being based on a global optimization of a true spatio-temporal energy, it offers several desirable properties: no need for silhouettes, robustness to noise, independent from any initialization, no heuristic force, reduced flickering results, etc. Results on real-world data proves the potential of what is, to our knowledge, the only globally optimal spatio-temporal multi-view reconstruction method.


Point Cloud Voronoi Diagram Delaunay Triangulation Temporal Plane Normalize Cross Correlation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aganj, E., Pons, J.P., Ségonne, F., Keriven, R.: Spatio-temporal shape from silhouette using four-dimensional Delaunay meshing. In: ICCV (2007)Google Scholar
  2. 2.
    Ahmed, N., de Aguiar, E., Theobalt, C., Magnor, M., Seidel, H.P.: Automatic generation of personalized human avatars from multi-view video. In: Proc. VRST 2005, pp. 257–260 (2005)Google Scholar
  3. 3.
    Ahmed, N., Theobalt, C., Rössl, C., Thrun, S., Seidel, H.P.: Dense correspondence finding for parametrization-free animation reconstruction from video. In: CVPR (2008)Google Scholar
  4. 4.
    de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. In: ACM SIGGRAPH (2008)Google Scholar
  5. 5.
    de Aguiar, E., Theobalt, C., Stoll, C., Seidel, H.P.: Marker-less deformable mesh tracking for human shape and motion capture. In: CVPR (2007)Google Scholar
  6. 6.
    Furukawa, Y., Ponce, J.: Dense 3D motion capture from synchronized video streams. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  7. 7.
    Goldlücke, B., Magnor, M.: Space-time isosurface evolution for temporally coherent 3D reconstruction. In: CVPR, vol. 1, pp. 350–355 (2004)Google Scholar
  8. 8.
    Neumann, J., Aloimonos, Y.: Spatio-temporal stereo using multi-resolution subdivision surfaces. IJCV 47(1-3), 181–193 (2002)zbMATHCrossRefGoogle Scholar
  9. 9.
    Pons, J.-P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. IJCV 72(2), 179–193 (2007)CrossRefGoogle Scholar
  10. 10.
    Starck, J., Hilton, A.: Surface capture for performance based animation. IEEE Computer Graphics and Applications 27(3), 21–31 (2007)CrossRefGoogle Scholar
  11. 11.
    Theobalt, C., Ahmed, N., Lensch, H., Magnor, M., Seidel, H.P.: Seeing people in different light-joint shape, motion, and reflectance capture. TVCG 13(4), 663–674 (2007)Google Scholar
  12. 12.
    Varanasi, K., Zaharescu, A., Boyer, E., Horaud, R.P.: Temporal surface tracking using mesh evolution. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 30–43. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Vedula, S., Baker, S., Kanade, T.: Image-based spatio-temporal modeling and view interpolation of dynamic events. ACM TOG 24(2), 240–261 (2005)Google Scholar
  14. 14.
    Vedula, S., Baker, S., Seitz, S., Kanade, T.: Shape and motion carving in 6D. In: CVPR (2000)Google Scholar
  15. 15.
    Vlasic, D., Baran, I., Matusik, W., Popović, J.: Articulated mesh animation from multi-view silhouettes. ACM Transactions on Graphics 27(3) (2008)Google Scholar
  16. 16.
    Laurentini, A.: The visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16(2), 150–162 (1994)CrossRefGoogle Scholar
  17. 17.
    Labatut, P., Pons, J.P., Keriven, R.: Efficient multi-view reconstruction of large-scale scenes using interest points, delaunay triangulation and graph cuts. In: ICCV (2007)Google Scholar
  18. 18.
    Cheung, G.K.M., Baker, S., Kanade, T.: Shape-from-silhouette across time part i: Theory and algorithms. IJCV 62(3), 221–247 (2004)CrossRefGoogle Scholar
  19. 19.
    Boissonnat, J.D., Oudot, S.: Provably good sampling and meshing of surfaces. Graphical Models 67, 405–451 (2005)zbMATHCrossRefGoogle Scholar
  20. 20.
    Boissonnat, J.D., Yvinec, M.: Algorithmic Geometry. Cambridge University Press, Cambridge (1998)zbMATHGoogle Scholar
  21. 21.
    de Berg, M., van Kreveld, M., Overmars, M., Schwarzkopf, O.: Computational Geometry, Algorithms and Applications. Springer, Heidelberg (1997)zbMATHGoogle Scholar
  22. 22.
    Ford, L.R., Fulkerson, D.R.: Flows in Networks. Princeton University Press, Princeton (1962)zbMATHGoogle Scholar
  23. 23.
    Greig, D.M., Porteous, B.T., Seheult, A.H.: Exact maximum a posteriori estimation for binary imagesGoogle Scholar
  24. 24.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimizedvia graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)CrossRefGoogle Scholar
  25. 25.
    Kirsanov, D., Gortler, S.J.: A discrete global minimization algorithm for continuous variational problems. Harvard computer science technical report: Tr-14-04. Technical report, Cambridge, MA (07/2004) (2004)Google Scholar
  26. 26.
    Guéziec, A., Hummel, R.: Exploiting triangulated surface extraction using tetrahedral decomposition. TVCG 1(4), 328–342 (1995)Google Scholar
  27. 27.
    Boissonnat, J.D., Devillers, O., Teillaud, M., Yvinec, M.: Triangulations in CGAL. In: Annual Symposium on Computational Geometry, pp. 11–18 (2000)Google Scholar
  28. 28.
    Barber, C.B., Dobkin, D.P., Huhdanpaa, H.: The quickhull algorithm for convex hulls. ACM Transactions on Mathematical Software 22, 469–483 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    White, R., Crane, K., Forsyth, D.: Capturing and animating occluded cloth. In: SIGGRAPH (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Ehsan Aganj
    • 1
  • Jean-Philippe Pons
    • 2
  • Renaud Keriven
    • 1
  1. 1.IMAGINEÉcole des Ponts ParisTechMarne-la-ValléeFrance
  2. 2.CSTBSophia-Antipolis, CedexFrance

Personalised recommendations