International Journal of Computer Vision

, Volume 66, Issue 2, pp 141–161 | Cite as

A Surface Reconstruction Method Using Global Graph Cut Optimization

Article

Abstract

Surface reconstruction from multiple calibrated images has been mainly approached using local methods, either as a continuous optimization problem driven by level sets, or by discrete volumetric methods such as space carving. We propose a direct surface reconstruction approach which starts from a continuous geometric functional that is minimized up to a discretization by a global graph-cut algorithm operating on a 3D embedded graph. The method is related to the stereo disparity computation based on graph-cut formulation, but fundamentally different in two aspects. First, existing stereo disparity methods are only interested in obtaining layers of constant disparity, while we focus on high resolution surface geometry. Second, most of the existing graph-cut algorithms only reach approximate solutions, while we guarantee a global minimum. The whole procedure is consistently incorporated into a voxel representation that handles both occlusions and discontinuities. We demonstrate our algorithm on real sequences, yielding remarkably detailed surface geometry up to 1/10th of a pixel.

Keywords

graph flow graph cut 3D reconstruction from calibrated cameras discontinuities self-occlusions occlusions global minimum 

References

  1. Ahuja, R.K., Magnanti, T.L., and Orlin, J.B. 1993. Network Flows: Theory, Algorithms, and Applications. Prentice Hall.Google Scholar
  2. Aubert, G. and Kornprobst, P. 2002. Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, sen Applied Mathematical Sciences. Springer-Verlag, Vol. 147.Google Scholar
  3. Black, M.J., Sapiro, G., Marimont, D., and Heeger, D. 1998. Robust anisotropic diffusion. IEEE Transactions on Image Processing.Google Scholar
  4. Blake, A. and Zisserman, A. 1987. Visual reconstruction, sen Artificial Intelligence. MIT Press: Cambridge.Google Scholar
  5. Boykov, Y. and Kolmogorov, V. 2003. Computing geodesies and minimal surfaces via graph cuts. In International Conference on Computer Vision.Google Scholar
  6. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).Google Scholar
  7. Boykov, Y. and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).Google Scholar
  8. Buehler, C., Gortler, S., Cohen, M., and McMillan, L. 2002. Minimal surfaces for stereo. In European Conference on Computer Vision (ECCV 02).Google Scholar
  9. Caselles, V.R. Sapiro, G. 1997. Geodesic active contours. International Journal of Computer Vision.Google Scholar
  10. Catté, F., Lions, P.-L., Morel, J.-M., and Coll, T. 1992. Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis.Google Scholar
  11. Cherkassky, B.V. and Goldberg, A.V. 1997. On implementing the push-relabel method for the maximum flow problem. Algorithmica, 19, 4, pp. 390–410.CrossRefMathSciNetGoogle Scholar
  12. Faugeras, O. and Keriven, R. 1998. Variational principles, surface evolution, PDE's, level set methods and the stereo problem. Transactions on Image Processing.Google Scholar
  13. Ford, L. and Fulkerson, D. 1962. Flows in Networks. Princeton University Press.Google Scholar
  14. Ishikawa, H. 2003. Exact optimization for markov random fields with convex priors. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).Google Scholar
  15. Ishikawa, H. and Geiger, D. 1998. Occlusions, discontinuities, and epipolar lines in stereo. In European Conference on Computer Vision (ECCV 98).Google Scholar
  16. Ishikawa, H. 2000. Global optimization using embedded graphs. Ph.D. dissertation, New York University.Google Scholar
  17. Isidoro, J. and Sclaroff, S. 2003. Stochastic refinement of the visual hull to satisfy photometric and silhouette consistency constraints. In Proc. Int. Conf. on Computer Vision. IEEE. pp. 1335–1342.Google Scholar
  18. Kim, J., Kolmogorov, V., and Zabih, R. 2003. Visual correspondence using energy minimization and mutual information. In International Conference on Computer Vision.Google Scholar
  19. Kolmogorov, V. and Zabih, R. 2002. Multi-camera scene reconstruction via graph cuts. In European Conference on Computer Vision.Google Scholar
  20. Kolmogorov, V. and Zabih, R. 2001. Computing visual correspondence with occlusions using graph cuts. In International Conference on Computer Vision.Google Scholar
  21. Kolmogorov, V. and Zabih R. 2002. What energy functions can be minimized via graph cuts? In European Conference on Computer Vision.Google Scholar
  22. Kutulakos, K.N. 2000. Approximate n-view stereo. In European Conference on Computer Vision (ECCV 00).Google Scholar
  23. Kutulakos, K. and Seitz, S. 1999. A theory of shape by space carving. In Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV-99), pp. 307–314.Google Scholar
  24. Lhuillier, M. and Quan, L. 2002. Quasi-dense reconstruction from image sequence. In European Conference on Computer Vision (ECCV 02).Google Scholar
  25. Lhuillier M. and Quan, L. 2003. Surface reconstruction by integrating 3d and 2d data of multiple views. In Proc. of Int. Conf. on Computer Vision. IEEE.Google Scholar
  26. Museth, K., Breen, D., Whitaker, R., and Barr, A. 2002. Level set surface editing operators. ACM Transactions on Graphics (Siggraph 02), 21(3):330–338.Google Scholar
  27. Osher, S. and Sethian, J. 1988. Fronts propagating with curvature-dependent speed: Algorithms based on hamilton-jacobi formulations. Journal of Computational Physics.Google Scholar
  28. Paris, S. and Sillion, F. 2003. Robust acquisition of 3d informations from short image sequences. Graphical Models, 65, 4. pp. 222–238.Google Scholar
  29. Roy, S. 1999. Stereo without epipolar lines: A maximum-flow formulation. Int. Journal of Computer Vision, 34(2/3) 147–162.Google Scholar
  30. Roy, S. and Cox, I. 1998. A maximum-flow formulation of the n-camera stereo correspondence problem. In IEEE International Conference on Computer Vision.Google Scholar
  31. Saito, H. and Kanade, T. 1999. Shape reconstruction in projective grid space from large number of images. In Computer Vision and Pattern Recognition (CVPR-99) pp. 49–54.Google Scholar
  32. Seitz, S.M. and Dyer, C.R. 1999. Photorealistic scene reconstruction by voxel coloring. IJCV.Google Scholar
  33. Sethian, J. 1999. Level Set Methods and Fast Marching Methods. Cambridge University Press.Google Scholar
  34. Slabaugh, G., Culbertson, B., Malzbender, T., and Schafer, R. 2001. A survey of methods for volumetric scene reconstruction from photographs. In VolumeGraphics 01.Google Scholar
  35. Slabaugh, G., Malzbender, T., and Culbertson, W.B. 2000. Volumetric warping for voxel coloring on an infinite domain. In 3D Structure from Images---SMILE 2000.Google Scholar
  36. Szeliski, R. Golland, P. 1998. Stereo matching with transparency and matting. In International Conference on Computer Vision (ICCV 98).Google Scholar
  37. Terzopoulos, D., Witkin, A., and Kass, M. 1988. Constraints on deformable models: Recovering 3d shape and nonrigid motions. Artificial Intelligence.Google Scholar
  38. Ulvklo, M., Knutsson, H., and Granlund, G.H. 1998. Depth segmentation and occluded scene reconstruction using ego-motion. SPIE Visual Information Processing.Google Scholar
  39. Veksler, O. 1999. Efficient graph-based energy minimization methods in computer vision. Ph.D. dissertation, Cornell University.Google Scholar

Copyright information

© Springer Science + Business Media, Inc. 2006

Authors and Affiliations

  • Sylvain Paris
    • 1
  • François X. Sillion
    • 2
  • Long Quan
    • 3
  1. 1.MIT CSAILCambridgeUSA
  2. 2.ARTIS, GRAVIR/IMAG-INRIASaint IsmierFrance
  3. 3.Department of Computer ScienceHong Kong University of Science and TechnologyHong KongChina

Personalised recommendations