Abstract
In this article, we introduce a new global optimization method to the field of multiview 3D reconstruction. While global minimization has been proposed in a discrete formulation in form of the maxflow-mincut framework, we suggest the use of a continuous convex relaxation scheme. Specifically, we propose to cast the problem of 3D shape reconstruction as one of minimizing a spatially continuous convex functional. In qualitative and quantitative evaluation we demonstrate several advantages of the proposed continuous formulation over the discrete graph cut solution. Firstly, geometric properties such as weighted boundary length and surface area are represented in a numerically consistent manner: The continuous convex relaxation assures that the algorithm does not suffer from metrication errors in the sense that the reconstruction converges to the continuous solution as the spatial resolution is increased. Moreover, memory requirements are reduced, allowing for globally optimal reconstructions at higher resolutions.
We study three different energy models for multiview reconstruction, which are based on a common variational template unifying regional volumetric terms and on-surface photoconsistency. The three models use data measurements at increasing levels of sophistication. While the first two approaches are based on a classical silhouette-based volume subdivision, the third one relies on stereo information to define regional costs. Furthermore, this scheme is exploited to compute a precise photoconsistency measure as opposed to the classical estimation. All three models are compared on standard data sets demonstrating their advantages and shortcomings. For the third one, which gives the most accurate results, a more exhaustive qualitative and quantitative evaluation is presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Appleton, B., & Talbot, H. (2005). Globally optimal geodesic active contours. Journal of Mathematical Imaging and Vision, 23(1), 67–86.
Appleton, B., & Talbot, H. (2006). Globally minimal surfaces by continuous maximal flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 106–118.
Boykov, Y., & Lempitsky, V. (2006). From photohulls to photoflux optimization. In Proceedings of the British machine vision conference (Vol. 3, pp. 1149–1158).
Bresson, X., Esedoḡlu, S., Vandergheynst, P., Thiran, J. P., & Osher, S. (2005). Global minimizers of the active contour/snake model (Technical Report CAM-05-04). Department of Mathematics, University of California at Los Angeles, CA, USA.
Caselles, V., Kimmel, R., & Sapiro, G. (1995). Geodesic active contours. In Proceedings of the fifth international conference on computer vision (pp. 694–699). Cambridge: IEEE Computer Society.
Chambolle, A. (2005). Total variation minimization and a class of binary MRF models. In EMMCVPR (pp. 136–152).
Chan, T., Esedoḡlu, S., & Nikolova, M. (2006). Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics, 66(5), 1632–1648.
Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In SIGGRAPH ’96: proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 303–312). New York: ACM.
Duan, Y., Yang, L., Qin, H., & Samaras, D. (2004). Shape reconstruction from 3D and 2D data using pde-based deformable surfaces. In Proceedings of the European conference on computer vision (pp. 238–251).
Faugeras, O., & Keriven, R. (1998). Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344.
Greig, D., Porteous, B., & Seheult, A. (1989). Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society B, 51(2), 271–279.
Hernandez, C., & Schmitt, F. (2004). Silhouette and stereo fusion for 3D object modeling. Computer Vision and Image Understanding, 96(3), 367–392.
Hernández, C., Vogiatzis, G., & Cipolla, R. (2007). Probabilistic visibility for multi-view stereo. In Proceedings of the international conference on computer vision and pattern recognition. Minneapolis: IEEE Computer Society.
Hornung, A., & Kobbelt, L. (2006). Hierarchical volumetric multi-view stereo reconstruction of manifold surfaces based on dual graph embedding. In Proceedings of the international conference on computer vision and pattern recognition (pp. 503–510), New York, NY, USA.
Hu, T. C. (1969). Integer programming and network flows. Reading: Addison-Wesley.
Kass, M., Witkin, A., & Terzopoulos, D. (1988). Snakes: active contour models. International Journal of Computer Vision, 1, 321–331.
Kichenassamy, S., Kumar, A., Olver, P., Tannenbaum, A., & Yezzi, A. (1995). Gradient flows and geometric active contour models. In Proceedings of the fifth international conference on computer vision (pp. 810–815). Cambridge: IEEE Computer Society.
Kirsanov, D., & Gortler, S. (2004). A discrete global minimization algorithm for continuous variational problems. In Harvard computer science technical report, TR-14-04.
Klodt, M., Schoenemann, T., Kolev, K., Schikora, M., & Cremers, D. (2008). An experimental comparison of discrete and continuous shape optimization methods. In European conference on computer vision (ECCV), Marseille, France.
Kolev, K., Brox, T., & Cremers, D. (2006). Robust variational segmentation of 3D objects from multiple views. In LNCS : Vol. 4174. Pattern recognition (Proc. DAGM) (pp. 688–697). Berlin: Springer.
Kolev, K., Klodt, M., Brox, T., & Cremers, D. (2007a). Propagated photoconsistency and convexity in variational multiview 3D reconstruction. In: Workshop on photometric analysis for computer vision, Rio de Janeiro, Brazil.
Kolev, K., Klodt, M., Brox, T., Esedoglu, S., & Cremers, D. (2007b). Continuous global optimization in multiview 3D reconstruction. In LNCS : Vol. 4679. Energy minimization methods in computer vision and pattern recognition (EMMCVPR) (pp. 441–452), Ezhou, China. Berlin: Springer.
Kolmogorov, V., & Boykov, Y. (2004). An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(9), 1124–1137.
Kolmogorov, V., & Zabih, R. (2002). What energy functions can be minimized via graph cuts? In Proceedings of the European conference on computer vision (pp. 65–81), London, UK.
Kutulakos, K. N., & Seitz, S. M. (2000). A theory of shape by space carving. International Journal of Computer Vision, 38(3), 199–218.
Labatut, P., Pons, J.-P., & Keriven, R. (2007). Efficient multi-view reconstruction of large-scale scenes using interest points, Delaunay triangulation and graph cuts. In Proceedings of the international conference on computer vision, Rio de Janeiro, Brazil.
Lempitsky, V., Boykov, Y., & Ivanov, D. (2006). Oriented visibility for multiview reconstruction. In LNCS : Vol. 3953. Proceedings of the European conference on computer vision (pp. 226–238). Berlin: Springer.
Mumford, D., & Shah, J. (1989). Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 42, 577–685.
Pons, J.-P., Keriven, R., & Faugeras, O. (2007). Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision, 72(2), 179–193.
Rudin, L. I., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60, 259–268.
Seitz, S., & Dyer, C. (1997). Photorealistic scene reconstruction by voxel coloring. In Proceedings of the international conference on computer vision and pattern recognition (pp. 1067–1073).
Seitz, S., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of the international conference on computer vision and pattern recognition (pp. 519–528). Washington: IEEE Computer Society.
Sethian, J. A. (1996). Level set methods. Cambridge: Cambridge University Press.
Sinha, S., Mordohai, P., & Pollefeys, M. (2007). Multiview stereo via graph cuts on the dual of an adaptive tetrahedral mesh. In Proceedings of the international conference on computer vision, Rio de Janeiro, Brazil.
Soatto, S., Yezzi, A. J., & Jin, H. (2003). Tales of shape and radiance in multi-view stereo. In Proceedings of the international conference on computer vision (p. 974). Washington: IEEE Computer Society.
Strang, G. (1983). Maximal flow through a domain. Mathematical Programming, 26, 123–243.
Tran, S., & Davis, L. (2006). 3D surface reconstruction using graph cuts with surface constraints. In LNCS : Vol. 3952. Proceedings of the European conference on computer vision (pp. 219–231). Berlin: Springer.
Vogiatzis, G., Esteban, C. H., Torr, P. H. S., & Cipolla, R. (2007). Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2241–2246.
Vogiatzis, G., Torr, P., & Cippola, R. (2005). Multi-view stereo via volumetric graph-cuts. In Proceedings of the international conference on computer vision and pattern recognition (pp. 391–399).
Zach, C., Pock, T., & Bischof, H. (2007). A globally optimal algorithm for robust TV-L1 range image integration. In: Proceedings of the international conference on computer vision, Rio de Janeiro, Brazil.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kolev, K., Klodt, M., Brox, T. et al. Continuous Global Optimization in Multiview 3D Reconstruction. Int J Comput Vis 84, 80–96 (2009). https://doi.org/10.1007/s11263-009-0233-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-009-0233-1