International Journal of Computer Vision

, Volume 57, Issue 3, pp 179–199 | Cite as

Methods for Volumetric Reconstruction of Visual Scenes

  • Gregory G. Slabaugh
  • W. Bruce Culbertson
  • Thomas Malzbender
  • Mark R. Stevens
  • Ronald W. Schafer
Article

Abstract

In this paper, we present methods for 3D volumetric reconstruction of visual scenes photographed by multiple calibrated cameras placed at arbitrary viewpoints. Our goal is to generate a 3D model that can be rendered to synthesize new photo-realistic views of the scene. We improve upon existing voxel coloring/space carving approaches by introducing new ways to compute visibility and photo-consistency, as well as model infinitely large scenes. In particular, we describe a visibility approach that uses all possible color information from the photographs during reconstruction, photo-consistency measures that are more robust and/or require less manual intervention, and a volumetric warping method for application of these reconstruction methods to large-scale scenes.

scene reconstruction voxel coloring space carving photo-consistency histogram intersection volumetric warping 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Beardsley, P., Torr, P., and Zisserman, A. 1996. 3D model acquisition from extended image sequences. In Proc. European Conference on Computer Vision, pp. 683–695.Google Scholar
  2. Bhotika, R., Fleet, D., and Kutulakos, K. 2002.Aprobabilistic theory of occupancy and emptiness. In Proc. European Conference on Computer Vision, vol. 3, pp. 112–132.Google Scholar
  3. Blinn, J. and Newell, M. 1976. Texture and reflection on computer generated images. Communications of ACM, 19(10):542–547.Google Scholar
  4. Bolles, R., Baker, H., and Marimont, D. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1(1):7–55.Google Scholar
  5. Broadhurst, A. and Cipolla, R. 2001. A probabilistic framework for space carving. In Proc. International Conference on Computer Vision, pp. 388–393.Google Scholar
  6. Carceroni, R. and Kutulakos, K. 2001. Multi-view scene capture by surfel sampling: From video streams to non-rigid 3D motion, shape & reflectance. In Proc. International Conference on Computer Vision, pp. 60–67.Google Scholar
  7. Chhabra, V., 2001. Reconstructing specular objects withi imagebased rendering using color caching. Master's Thesis,Worchester Polytechnic Institute.Google Scholar
  8. Colosimo, A., Sarti, A., and Tubaro, S. 2001. Image-based object modeling: A multi-resolution level-set approach. In Proc. International Conference on Image Processing, vol. 2, pp. 181–184.Google Scholar
  9. Culbertson, W.B., Malzbender, T., and Slabaugh, G. 1999. Generalized voxel coloring. In Proc. ICCV Workshop, Vision Algorithms Theory and Practice, Springer-Verlag Lecture Notes in Computer Science 1883, pp. 100–115.Google Scholar
  10. De Bonet, J. and Viola, P. 1999. Roxels: Responsibility weighted 3D volume reconstruction. In Proc. International Conference on Computer Vision, pp. 418–425.Google Scholar
  11. Debevec, P., Taylor, C., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry and imagebased approach. In Proc. SIGGRAPH, pp. 11–20.Google Scholar
  12. Dyer, C. 2001.Volumetric scene reconstruction from multiple views. In Foundations of Image Analysis, L.S. Davis (ed.), Kluwer: Boston, MA, pp. 469–489.Google Scholar
  13. Eisert, P., Steinbach, E., and Girod, B. 1999. Multi-hypothesis, volumetric reconstruction of 3-D objects from multiple calibrated camera views. In Proc. of the International Conference on Acoustics, Speech, and Signal Processing, pp. 3509–3512.Google Scholar
  14. Eisert, P., Steinbach, E., and Girod, B. 2000. Automatic reconstruction of 3-D stationary objects from multiple uncalibrated camera views. IEEE Transactions on Circuits and Systems for Video Technology, 10(2):261–277.Google Scholar
  15. Faugeras, O. and Lustman, F. 1988. Motion and structure from motion in a piecewise-planar environment. Journal of Pattern Recognition and Artificial Intelligence, 2(3):485–508.Google Scholar
  16. Faugeras, O. and Keriven, R. 1998. Complete dense stereovision using level set methods. IEEE Transactions on Image Processing, 7(3):336–344.Google Scholar
  17. Fua, P. and Leclerc, Y. 1995. Object-centered surface reconstruction: Combining multi-image stereo and shading. International Journal of Computer Vision, 16(1):35–56.Google Scholar
  18. Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The Lumigraph. In Proc. SIGGRAPH, pp. 43–54.Google Scholar
  19. Greene, N. 1986. Environment mapping and other applications of world projections. IEEE Computer Graphics and Applications, 21–29.Google Scholar
  20. Jebara, T., Azarbayejani, A., and Pentland, A. 1999. 3D structure from 2D motion. IEEE Signal Processing Magazine, 16(3):66–84.Google Scholar
  21. Johansson, B. 1999. View synthesis and 3D reconstruction of piecewise planar scenes using intersection lines between the planes. In Proc. International Conference on Computer Vision, vol. 1, pp. 54–59.Google Scholar
  22. Kimura, M., Satio, H., and Kanade, T. 1999. 3D voxel construction based on epipolar geometry. In Proc. International Conference on Image Processing, pp. 135–139.Google Scholar
  23. Kutulakos, K. and Seitz, S. 1999.A theory of shape by space carving. International Conference on Computer Vision, vol. 1, pp. 307–314.Google Scholar
  24. Kutulakos, K. and Seitz, S. 2000.A theory of shape by space carving. International Journal of Computer Vision, 38(3):199–218.Google Scholar
  25. Laurentini, A. 1994. The visual Hull concept for Silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2):150–162.Google Scholar
  26. Levoy, M. and Hanrahan, P. 1996. Light field rendering. In Proc. SIGGRAPH, pp. 31–42.Google Scholar
  27. Livingston, M. and Sobel, I. 2001. A new scale arbitration algorithm for image sequences applied to cylindrical photographs. HP Laboratories technical report HPL-2001-226R1.Google Scholar
  28. Longuet-Higgins, H. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293(10):133–135.Google Scholar
  29. Matusik, W., Buehler, C., Raskar, R., Gortler, S., and McMillan, L. 2000. Image-based visual Hulls. In Proc. SIGGRAPH, pp. 367–374.Google Scholar
  30. Max, N. 1996. Hierarchical rendering of trees from precomputed multi-layer Z-buffers. In Proc. Eurographics Rendering Workshop, pp. 165–174.Google Scholar
  31. McMillan, M. and Bishop, G. 1995. Plenoptic modeling: An imagebased rendering system. In Proc. SIGGRAPH, pp. 39–46.Google Scholar
  32. Narayanan, P., Rander, P., and Kanade, T. 1998. Constructing virtual worlds using dense stereo. In Proc. International Conference on Computer Vision.Google Scholar
  33. Özün, O. 2002. Comparison of photo-consistency measures used in the voxel coloring algorithm. Masters Thesis, Middle East Technical University.Google Scholar
  34. Pfister, H., Zwicker, M., Van Baar, J., and Gross, M. 2000. Surfels: Surface elements as rendering primitives. In Proc. SIGGRAPH, pp. 335–342.Google Scholar
  35. Pollefeys, M., Koch, R., Vergauwen, M., and Van Gool, L. 1999. Hand-held acquisition of 3D models with a video camera. In Proc. 2nd International Conference on 3-D Digital Imaging and Modeling, pp. 14–23.Google Scholar
  36. Prock, A. and Dyer, C. 1998. Towards real-time voxel coloring. In Proc. DARPA Image Understanding Workshop, pp. 315–321.Google Scholar
  37. Saito, H. and Kanade, T. 1999. Shape reconstruction in projective grid space from large number of images. In Proc. CVPR, pp. 49–54.Google Scholar
  38. Savarese, S., Rushmeier, H., Bernardini, F., and Perona, P. 2001. Shadow carving. In Proc. International Conference on Computer Vision, vol 1, pp. 190–197.Google Scholar
  39. Seitz, S. and Dyer, C. 1999. Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision, 35(2):151–173.Google Scholar
  40. Shade, J., Gortler, S., He, L., and Szeliski, R. 1998. Layered depth images. In Proc. SIGGRAPH, pp. 231–242.Google Scholar
  41. Shum, H.Y., Han, M., and Szeliski, R. 1998. Interactive construction of 3D models from panoramic mosaics. In Proc. CVPR, pp. 427–433.Google Scholar
  42. Slabaugh, G., Malzbender, T., and Culbertson, W.B. 2000. Volumetric warping for voxel coloring on an infinite domain. In Proc. 3D Structure from Multiple Images for Large Scale Environments (SMILE), pp. 41–50.Google Scholar
  43. Slabaugh, G., Culbertson, W.B., Malzbender, T., and Schafer, R. 2001. A survey of volumetric scene reconstruction methods from photographs. In Proc. International Workshop on Volume Graphics, pp. 81–100.Google Scholar
  44. Slabaugh, G., Schafer, R., and Hans, M. 2002. Image-based photo Hulls. In Proc. 1st International Symposium on 3D Processing, Visualization, and Transmission, pp. 704–708.Google Scholar
  45. Slabaugh, G. 2002. Novel volumetric scene reconstruction methods for new view synthesis. Ph.D. Thesis, Georgia Institute of Technology.Google Scholar
  46. Steinbach, E., Eisert, P., Betz, A., and Girod, B. 2000. 3-D reconstruction of real world objects using extended voxels. In Proc. International Conference on Image Processing, vol. I, pp. 569–572.Google Scholar
  47. Stevens, M., Culbertson, W.B., and Malzbender, T. 2002. A histogram-based color consistency test for voxel coloring. In Proc. International Conference on Pattern Recognition, vol. 4, pp. 118–121.Google Scholar
  48. Stevens, M. 1999. Reasoning about object appearance in the context of a scene. Ph.D. Thesis, Colorado State University.Google Scholar
  49. Szeliski, R. 1993. Rapid octree construction from image sequences. Computer Vision, Graphics and Image Processing: Image Understanding, 58(1):23–32.Google Scholar
  50. Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proc. International Conference on Computer Vision, pp. 517–524.Google Scholar
  51. Tsai, R. 1987. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal of Robotics and Automation, RA-3(4):323–344.Google Scholar
  52. Vedula, S., Baker, S., Seitz, S., and Kanade, T. 2000 Shape and motion carving in 6D. In Proc. CVPR, vol. 2, pp. 592–598.Google Scholar
  53. Weghorst, H., Hooper, G., and Greenberg, D.P. 1984. Improving computational methods for ray tracing. ACM Transactions on Graphics, 3(1):52–69.Google Scholar
  54. Yezzi, A. and Soatto, S., 2001. Stereoscopic segmentation. In Proc. International Conference on Computer Vision, pp. 59–66.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Gregory G. Slabaugh
    • 1
  • W. Bruce Culbertson
    • 2
  • Thomas Malzbender
    • 2
  • Mark R. Stevens
    • 3
  • Ronald W. Schafer
    • 4
  1. 1.Intelligent Vision and Reasoning DepartmentSiemens Corporate ResearchPrincetonUSA
  2. 2.Visual Computing DepartmentHewlett-Packard LaboratoriesPalo AltoUSA
  3. 3.Charles River Analytics Inc.CambridgeUSA
  4. 4.Center for Signal and Image ProcessingGeorgia Institute of TechnologyAtlantaUSA

Personalised recommendations