Abstract
We cast the problem of shape reconstruction of a scene as the global region segmentation of a collection of calibrated images. We assume that the scene is composed of a number of smooth surfaces and a background, both of which support smooth Lambertian radiance functions. We formulate the problem in a variational framework, where the solution (both the shape and radiance of the scene) is a minimizer of a global cost functional which combines a geometric prior on shape, a smoothness prior on radiance and a data fitness score. We estimate the shape and radiance via an alternating minimization: The radiance is computed as the solutions of partial differential equations defined on the surface and the background. The shape is estimated using a gradient descent flow, which is implemented using the level set method. Our algorithm works for scenes with smooth radiances as well as fine homogeneous textures, which are known challenges to traditional stereo algorithms based on local correspondence.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bertalmio, M., Cheng, L., Osher, S., and Sapiro, G. (2001). Variational problems and partial differential equations on implicit surfaces. J. Comput. Phys. 174, 759-780.
Caselles, V., Catte, F., Coll, T., and Dibos, F. (1993). A geometric model for active contours in image processing. Numer. Math. 66, 1-31.
Caselles, V., Kimmel, R., and Sapiro, G. (1997). Geodesic active contours. Int. J. Comput. Vision 22, 61-79.
Chakraborty, A., and Duncan, J. (1999). Game-Theoretic Integration for image segmentation. IEEE Trans. Pattern Anal. Machine Intell. 21, 12-30.
Chakraborty, A., Staib, L., and Duncan, J. (1996). Deformable boundary finding in medical images by integrating gradient and region information. IEEE Trans. Medical Imaging 15, 859-870.
Chan, T., and Vese, L. (2001). Active contours without edges. IEEE Trans. on Image Processing 10, 266-277.
Cipolla, R., and Blake, A. (1992). Surface shape from the deformation of apparent contours. Int. J. Comput. Vision 9.
Cohen, L. (1991). On active contour models and balloons. CVGIP: Image Understanding 53, 211-218.
Crandall, M. G., Ishii, H., and Lions, P. L. (1992). User's guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. (N.S.) 27, 1-67.
Crandall, M. G., and Lions, P. L. (1984). Two approximations of solutions of Hamilton–Jacobi equations. Math. Comp. 43, 1-19.
Faugeras, O. (1993). Three Dimensional Vision, a Geometric Viewpoint, MIT Press.
Faugeras, O., and Keriven, R. (1998). Variational principles, surface evolution pdes, level set methods and the stereo problem. IEEE Trans. Image Process. 7, 336-344.
Horn, B., and Brooks, M. (eds.) (1989). Shape from Shading, MIT Press.
Jin, H., Yezzi, A., and Soatto, S. (2000). Stereoscopic shading: Integrating shape cues in a variational framework. Proc. of the IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pp. 169-176.
Kass, M., Witkin, A., and Terzopoulos, D. (1987). Snakes: Active contour models. Int. J. Comput. Vision 1, 321-331.
Kimmel, R. (2002). 3D shape reconstruction from autostereograms and stereo. J. Vis. Commun. Image Repres. 13, 324-333.
Kreyszig, E. (1991). Differential Geometry, Dover.
Kutulakos, K., and Seitz, S. (2000). A theory of shape by space carving. Int. J. Comput. Vision 38, 199-218.
Mumford, D., and Shah, J. (1989). Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math. 42, 577-685.
Osher, S., and Sethian, J. (1988). Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi equations. J. Comput. Phys. 79, 12-49.
Osher, S., and Shu, C.-W. (1991). High-order essentially nonoscillatory schemes for Hamilton–Jacobi equations. SIAM J. Numer. Anal. 28, 907-922.
Ronfard, R. (1994). Region-based strategies for active contour models. Int. J. Comput. Vision 13, 229-251.
Souganidis, P. E. (1985). Approximation schemes for viscosity solutions of Hamilton–Jacobi equations. J. Differential Equations 59, 1-43.
Tek, H., and Kimia, B. (1995). Image segmentation by reaction diffusion bubbles. Proc. Int. Conf. Computer Vision, pp. 156-162.
Terzopoulos, D., and Witkin, A. (1988). Constraints on deformable models: Recovering shape and non-rigid motion. Artificial Intelligence 36, 91-123.
Tsai, Y.-H., Cheng, L.-T., Burchard, P., Osher, S., and Sapiro, G. (2002). Dynamic Visibility in an Implicit Framework, UCLA CAM Report, 02-06.
Yezzi, A., and Soatto, S. (2001). Stereoscopic segmentation. Proc. Int. Conf. Computer Vision, 56-66.
Yezzi, A., Tsai, A., and Willsky, A. (1999). A statistical approach to image segmentation for bimodal and trimodal imagery. Proc. Int. Conf. Computer Vision, 898-903.
Zhu, S., and Yuille, A. (1996). Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Trans. Pattern Anal. Machine Intelligence 18, 884-900.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Jin, H., Yezzi, A.J., Tsai, YH. et al. Estimation of 3D Surface Shape and Smooth Radiance from 2D Images: A Level Set Approach. Journal of Scientific Computing 19, 267–292 (2003). https://doi.org/10.1023/A:1025308109816
Issue Date:
DOI: https://doi.org/10.1023/A:1025308109816