Advertisement

Integration of Multiview Stereo and Silhouettes Via Convex Functionals on Convex Domains

  • Kalin Kolev
  • Daniel Cremers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5302)

Abstract

We propose a convex framework for silhouette and stereo fusion in 3D reconstruction from multiple images. The key idea is to show that the reconstruction problem can be cast as one of minimizing a convex functional where the exact silhouette consistency is imposed as a convex constraint that restricts the domain of admissible functions. As a consequence, we can retain the original stereo-weighted surface area as a cost functional without heuristic modifications by balloon terms or other strategies, yet still obtain meaningful (nonempty) global minimizers. Compared to previous methods, the introduced approach does not depend on initialization and leads to a more robust numerical scheme by removing the bias near the visual hull boundary. We propose an efficient parallel implementation of this convex optimization problem on a graphics card. Based on a photoconsistency map and a set of image silhouettes we are therefore able to compute highly-accurate and silhouette-consistent reconstructions for challenging real-world data sets in less than one minute.

Keywords

Admissible Function Visual Hull Reprojection Error Stereo Information Multiview Stereo 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Baumgart, B.: Geometric modeling for computer vision. PhD thesis, Department of Computer Science, Stanford University, USA (1974)Google Scholar
  2. 2.
    Laurentini, A.: The visual hull concept for visual-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 150–162 (1994)CrossRefGoogle Scholar
  3. 3.
    Martin, W.N., Aggarwal, J.K.: Volumetric descriptions of objects from multiple views. IEEE Transactions on Pattern Analysis and Machine Intelligence 5, 150–158 (1983)CrossRefGoogle Scholar
  4. 4.
    Cipolla, R., Giblin, P.: Visual motion of curves and surfaces. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  5. 5.
    Franco, J.S., Boyer, E.: Exact polyhedral visual hulls. In: Proceedings of the Fourteenth British Machine Vision Conference, Norwich, UK, pp. 329–338 (2003)Google Scholar
  6. 6.
    Yezzi, A., Soatto, S.: Stereoscopic segmentation. International Journal of Computer Vision 53, 31–43 (2003)CrossRefGoogle Scholar
  7. 7.
    Seitz, S., Dyer, C.: Photorealistic scene reconstruction by voxel coloring. In: Proc. International Conference on Computer Vision and Pattern Recognition, pp. 1067–1073 (1997)Google Scholar
  8. 8.
    Faugeras, O., Keriven, R.: Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Transactions on Image Processing 7, 336–344 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Duan, Y., Yang, L., Qin, H., Samaras, D.: Shape reconstruction from 3D and 2D data using PDE-based deformable surfaces. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 238–251. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Lempitsky, V., Boykov, Y., Ivanov, D.: Oriented visibility for multiview reconstruction. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 226–238. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Matsumoto, Y., Fujimura, K., Kitamura, T.: Shape-from-silhouette/stereo and its application to 3D digitizer. In: Proceedings of Discrete Geometry for Computing Imagery, pp. 177–190 (1999)Google Scholar
  12. 12.
    Vogiatzis, G., Torr, P., Cippola, R.: Multi-view stereo via volumetric graph-cuts. In: Proc. International Conference on Computer Vision and Pattern Recognition, pp. 391–399 (2005)Google Scholar
  13. 13.
    Cross, G., Zisserman, A.: Surface reconstruction from multiple views using apparent contours and surface texture. In: Confluence of Computer Vision and Computer Graphics, Norwell, MA, USA, pp. 25–47. Kluwer Academic Publishers, Dordrecht (2000)CrossRefGoogle Scholar
  14. 14.
    Esteban, C.H., Schmitt, F.: Silhouette and stereo fusion for 3D object modeling. Computer Vision and Image Understanding 96, 367–392 (2004)CrossRefGoogle Scholar
  15. 15.
    Furukawa, Y., Ponce, J.: Carved visual hulls for image-based modeling. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 564–577. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Gargallo, P., Prados, E., Sturm, P.: Minimizing the reprojection error in surface reconstruction from images. In: Proceedings of the International Conference on Computer Vision, Janeiro, Brazil. IEEE Computer Society Press, Los Alamitos (2007)Google Scholar
  17. 17.
    Isidoro, J., Sclaroff, S.: Stochastic refinement of the visual hull to satisfy photometric and silhouette consistency constraints. In: Proc. International Conference on Computer Vision, Washington, DC, USA, pp. 1335–1342 (2003)Google Scholar
  18. 18.
    Tran, S., Davis, L.: 3D surface reconstruction using graph cuts with surface constraints. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 219–231. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Sinha, S., Mordohai, P., Pollefeys, M.: Multiview stereo via graph cuts on the dual of an adaptive tetrahedral mesh. In: Proc. International Conference on Computer Vision, Rio de Janeiro, Brazil (2007)Google Scholar
  20. 20.
    Sinha, S., Pollefeys, M.: Multi-view reconstruction using photo-consistency and exact silhouette constraints: A maximum-flow formulation. In: Proc. International Conference on Computer Vision, Washington, DC, USA, pp. 349–356. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  21. 21.
    Hernández, C., Vogiatzis, G., Cipolla, R.: Probabilistic visibility for multi-view stereo. In: Proc. International Conference on Computer Vision and Pattern Recognition, Minneapolis, Minnesota, USA. IEEE Computer Society, Los Alamitos (2007)Google Scholar
  22. 22.
    Kolev, K., Klodt, M., Brox, T., Cremers, D.: Propagated photoconsistency and convexity in variational multiview 3D reconstruction. In: Workshop on Photometric Analysis for Computer Vision, Rio de Janeiro, Brazil (2007)Google Scholar
  23. 23.
    Boykov, Y., Lempitsky, V.: From photohulls to photoflux optimization. In: Proc. British Machine Vision Conference, vol. 3, pp. 1149–1158 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Kalin Kolev
    • 1
  • Daniel Cremers
    • 1
  1. 1.Department of Computer ScienceUniversity of BonnGermany

Personalised recommendations