Robust Variational Segmentation of 3D Objects from Multiple Views

  • Kalin Kolev
  • Thomas Brox
  • Daniel Cremers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4174)


We propose a probabilistic formulation of 3D segmentation given a series of images from calibrated cameras. Instead of segmenting each image separately in order to build a 3D surface consistent with these segmentations, we compute the most probable surface that gives rise to the images. Additionally, our method can reconstruct the mean intensity and variance of the extracted object and background. Although it is designed for scenes, where the objects can be distinguished visually from the background (i.e. images of piecewise homogeneous regions), the proposed algorithm can also cope with noisy data. We carry out the numerical implementation in the level set framework. Our experiments on synthetic data sets reveal favorable results compared to state-of-the-art methods, in particular in terms of robustness to noise and initialization.


Computer Vision Input Image Multiple View Jacobi Formulation Visual Hull 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Broadhurst, A., Drummond, T.W., Cipolla, R.: A probabilistic framework for space carving. In: Proc. International Conference on Computer Vision, pp. 388–393 (July 2001)Google Scholar
  2. 2.
    Chan, T., Vese, L.: Active contours without edges. IEEE Transactions on Image Processing 10(2), 266–277 (2001)MATHCrossRefGoogle Scholar
  3. 3.
    Cipolla, R., Blake, A.: Surface shape from the deformation of apparent contours. International Journal of Computer Vision 9(2), 83–112 (1992)CrossRefGoogle Scholar
  4. 4.
    Dervieux, A., Thomasset, F.: A finite element method for the simulation of Rayleigh–Taylor instability. In: Rautman, R. (ed.) Approximation Methods for Navier–Stokes Problems. Lecture Notes in Mathematics, vol. 771, pp. 145–158. Springer, Berlin (1979)CrossRefGoogle Scholar
  5. 5.
    Duan, Y., Yang, L., Qin, H., Samaras, D.: Shape reconstruction from 3D and 2D data using PDE-based deformable surfaces. In: Pajdla, T., Matas, J.G. (eds.) ECCV 2004. LNCS, vol. 3023, pp. 238–251. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Faugeras, O., Keriven, R.: Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Transactions on Image Processing 7(3), 336–344 (1998)MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Garcia, B., Brunet, P.: 3D reconstruction with projective octrees and epipolar geometry. In: Proc. International Conference on Computer Vision, pp. 1067–1072 (January 1998)Google Scholar
  8. 8.
    Horn, B., Brooks, M.: Shape from shading. MIT Press, Cambridge (1989)Google Scholar
  9. 9.
    Jin, H., Cremers, D., Yezzi, A., Soatto, S.: Shedding light on stereoscopic segmentation. In: Davis, L. (ed.) Proc. International Conference on Computer Vision and Pattern Recognition, Washington, DC, vol. 1, pp. 36–42 (2004)Google Scholar
  10. 10.
    Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. International Journal of Computer Vision 38(3), 199–218 (2000)MATHCrossRefGoogle Scholar
  11. 11.
    Laurentini, A.: The visual hull concept for visual-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(2), 150–162 (1994)CrossRefGoogle Scholar
  12. 12.
    Malik, J., Rosenholtz, R.: A differential method for computing local shape-from-texture for planar and curved surfaces. In: Computer Vision and Pattern Recognition Conference, pp. 267–273 (June 1993)Google Scholar
  13. 13.
    Martin, W.N., Aggarwal, J.K.: Volumetric descriptions of objects from multiple views. IEEE Transactions on Pattern Analysis and Machine Intelligence 5(2), 150–158 (1983)CrossRefGoogle Scholar
  14. 14.
    Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi formulations. Journal of Computational Physics 79, 12–49 (1988)MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Potmesil, M.: Generating octree models of 3D objects from their silhouettes from a sequence of images. Computer Vision, Graphics, and Image Processing 40(1), 1–29 (1987)CrossRefGoogle Scholar
  16. 16.
    Rousson, M., Brox, T., Deriche, R.: Active unsupervised texture segmentation on a diffusion based feature space. In: Proc. International Conference on Computer Vision and Pattern Recognition, Madison, WI, pp. 699–704 (June 2003)Google Scholar
  17. 17.
    Seitz, S., Dyer, C.: Complete scene structure from four point correspondences. In: Proc. International Conference on Computer Vision, pp. 330–337 (June 1995)Google Scholar
  18. 18.
    Seitz, S., Dyer, C.: Photorealistic scene reconstruction by voxel coloring. In: Proc. International Conference on Computer Vision and Pattern Recognition, pp. 1067–1073 (June 1997)Google Scholar
  19. 19.
    Szeliski, R.: Rapid octree construction from image sequences. Computer Vision, Graphics, and Image Processing 58(1), 23–32 (1993)CrossRefGoogle Scholar
  20. 20.
    Vaillant, R., Faugeras, O.: Using extremal boundaries for 3D object modelling. IEEE Transactions on Pattern Analysis and Machine Intelligence 14(2), 157–173 (1992)CrossRefGoogle Scholar
  21. 21.
    Yezzi, A., Soatto, S.: Stereoscopic segmentation. International Journal of Computer Vision 53(1), 31–43 (2003)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Kalin Kolev
    • 1
  • Thomas Brox
    • 1
  • Daniel Cremers
    • 1
  1. 1.CVPR GroupUniversity of BonnBonnGermany

Personalised recommendations