Advertisement

International Journal of Computer Vision

, Volume 35, Issue 2, pp 151–173 | Cite as

Photorealistic Scene Reconstruction by Voxel Coloring

  • Steven M. Seitz
  • Charles R. Dyer
Article

Abstract

A novel scene reconstruction technique is presented, different from previous approaches in its ability to cope with large changes in visibility and its modeling of intrinsic scene color and texture information. The method avoids image correspondence problems by working in a discretized scene space whose voxels are traversed in a fixed visibility ordering. This strategy takes full account of occlusions and allows the input cameras to be far apart and widely distributed about the environment. The algorithm identifies a special set of invariant voxels which together form a spatial and photometric reconstruction of the scene, fully consistent with the input images. The approach is evaluated with images from both inward-facing and outward-facing cameras.

scene reconstruction multi-baseline stereo invariants voxel representations image correspondence occlusion photorealism 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Avidan, S. and Shashua, A. 1997. Novel view synthesis in tensor space. In Proc. Computer Vision and Pattern Recognition Conf., pp. 1034–1040.Google Scholar
  2. Baker, H.H. and Bolles, R.C. 1989. Generalizing epipolar-plane image analysis on the spatiotemporal surface. Int. J. of Computer Vision, 3(1):33–49.Google Scholar
  3. Beardsley, P., Torr, P., and Zisserman, A. 1996. 3D model acquisition from extended image sequences. In Proc. European Conf. on Computer Vision, pp. 683–695.Google Scholar
  4. Belhumeur, P.N. and Mumford, D. 1992. A Bayesian treatment of the stereo correspondence problem using half-occluded regions. In Proc. Computer Vision and Pattern Recognition Conf., pp. 506–512.Google Scholar
  5. Beymer, D. and Poggio, T. 1996. Image representations for visual learning. Science, (272):1905–1909.Google Scholar
  6. Bolles, R.C., Baker, H.H., and Marimont, D.H. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. Int. J. of Computer Vision, 1(1):7–55.Google Scholar
  7. Brelstaff, G. and Blake, A. 1988. Detecting specular reflections using lambertian constraints. In Proc. 2nd Int. Conf. on Computer Vision, pp. 297–302.Google Scholar
  8. Chen, S.E. and Williams, L. 1993. View interpolation for image synthesis. In Proc. SIGGRAPH 93, pp. 279–288.Google Scholar
  9. Collins, R.T. 1996. A space-sweep approach to true multi-image matching. In Proc. Computer Vision and Pattern Recognition Conf., pp. 358–363.Google Scholar
  10. Collins, R.T. 1997. Multi-image focus of attention for rapid site model construction. In Proc. Computer Vision and Pattern Recognition Conf., pp. 575–581.Google Scholar
  11. Curless, B. and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proc. SIGGRAPH 96, pp. 303–312.Google Scholar
  12. Debevec, P.E., Taylor, C.J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In Proc. SIGGRAPH 96, pp. 11–20.Google Scholar
  13. Faugeras, O.D. and Keriven, R. 1998. Variational principles, surface evolution, PDE's, level set methods and the stereo problem. IEEE Trans. on Image Processing, 7(3):336–344.Google Scholar
  14. Floyd, R.W. and Steinberg, L. 1975. An adaptive algorithm for spatial gray scale. In SID 75, Int. Symp. Dig. Tech. Papers, p. 36.Google Scholar
  15. Freund, J.E. 1992. Mathematical Statistics. Prentice Hall: Englewood Cliffs, NJ.Google Scholar
  16. Fua, P. and Leclerc, Y.G. 1995. Obejct-centered surface reconstruction: Combining multi-image stereo and shading. Int. J. of Computer Vision, 16:35–56.Google Scholar
  17. Geiger, D., Landendorf, B., and Yuille, A. 1992. Occlusions and binocular stereo. In Proc. European Conf. on Computer Vision, pp. 425–433.Google Scholar
  18. Healey, G.E., Shafer, S.A., and Wolff, L.B. (Eds.). 1992. Physics-Based Vision: Principles and Practice, COLOR. Jones and Bartlett: Boston, MA, pp. 205–299.Google Scholar
  19. Kanade, T., Rander, P., and Narayanan, P.J. 1997. Virtualized reality: Constructing virtual worlds from real scenes. IEEE Multimedia, 4(1):34–46.Google Scholar
  20. Kang, S.B. and Szeliski, R. 3D Scene data recovery using omnidirectional multibaseline stereo. In Proc. Computer Vision and Pattern Recognition Conf., pp. 364–370.Google Scholar
  21. Katayama, A., Tanaka, K., Oshino, T., and Tamura, H. 1995. A viewpoint dependent stereoscopic display using interpolation of multiviewpoint images. In Proc. SPIE, Vol. 2409A, pp. 21–30.Google Scholar
  22. Klinker, G.J. and Shafer, S.A. 1990. A physical approach to color image understanding. Int. J. of Computer Vision, 4(7):7–38.Google Scholar
  23. Kutulakos, K.N. and Dyer, C.R. 1995. Global surface reconstruction by purposive control of observer motion. Artificial Intelligence, 78:147–177.Google Scholar
  24. Kutulakos, K.N. and Seitz, S.M. 1999. A theory of shape by space carving. In Proc. 7th Int. Conf. on Computer Vision.Google Scholar
  25. Laur, D. and Hanrahan, P. 1991. Hierarchical splatting:A progressive refinement algorithm for volume rendering. In Proc. SIGGRAPH 91.Google Scholar
  26. Laurentini, A. 1995. How far 3D shapes can be understood from 2D Silhouettes. IEEE Trans. on Pattern Analysis and Machine Intelligence, 17(2):188–195.Google Scholar
  27. Laurentini, A. 1997. How many 2D silhouettes does it take to reconstruct a 3D object. Computer Vision and Image Understanding, 67(1):81–87.Google Scholar
  28. Laveau, S. and Faugeras, O. 1994. 3D scene representation as a collection of images. In Proc. Int. Conf. on Pattern Recognition, pp. 689–691.Google Scholar
  29. Martin, W.N. and Aggarwal, J.K. 1991. Volumetric description of objects from multiple views. IEEE Trans. on Pattern Analysis and Machine Intelligence, 5(2):150–158.Google Scholar
  30. McMillan, L. and Bishop, G. 1995. Plenoptic modeling. In Proc. SIGGRAPH 95, pp. 39–46.Google Scholar
  31. Moezzi, S., Katkere, A., Kuramura, D.Y., and Jain, R. 1996. Reality modeling and visualization from multiple video sequences. IEEE Computer Graphics and Applications, 16(6):58–63.Google Scholar
  32. Nakamura, Y., Matsuura, T., Satoh, K., and Ohta, Y. 1996. Occlusion detectable stereo: occlusion patterns in camera matrix. In Proc. Computer Vision and Pattern Recognition Conf., pp. 371–378.Google Scholar
  33. Narayanan, P.J., Rander, P.W., and Kanade, T. 1998. Constructing virtual worlds using dense stereo. In Proc. Sixth Int. Conf. on Computer Vision, pp. 3–10.Google Scholar
  34. Poggio, T., Torre, V., and Koch, C. 1985. Computational vision and regularization theory. Nature, 317:314–319.Google Scholar
  35. Pollefeys, M., Koch, R., and Gool, L.V. 1998. Self-calibration and metric reconstruction in spite of varying unknown internal camera parameters. In Proc. Sixth Int. Conf. on Computer Vision, pp. 90–91.Google Scholar
  36. Roy, S. and Cox, I.J. 1998. A maximum-flow formulation of the Ncamera stereo correspondence problem. In Proc. Sixth Int. Conf. on Computer Vision, pp. 492–499.Google Scholar
  37. Samet, H. 1984. The quadtree and related hierarchical data structures. ACM Computing Surveys, 16:187–260.Google Scholar
  38. Scharstein, D. 1996. Stereo vision for view synthesis. In Proc. Computer Vision and Pattern Recognition Conf., pp. 852–858.Google Scholar
  39. Seitz, S.M. and Dyer, C.R. 1995. Complete structure from four point correspondences. In Proc. Fifth Int. Conf. on Computer Vision, pp. 330–337.Google Scholar
  40. Seitz, S.M. and Dyer, C.R. 1996. View morphing. In Proc. SIGGRAPH 96, pp. 21–30.Google Scholar
  41. Seitz, S.M. and Dyer, C.R. 1997. Photorealistic scene reconstruction by voxel coloring. In Proc. Image Understanding Workshop, pp. 1067–1073.Google Scholar
  42. Seitz, S.M. and Kutulakos, K.N. 1998. Plenoptic image editing. In Proc. Sixth Int. Conf. on Computer Vision, pp. 17–24.Google Scholar
  43. Sethian, J.A. 1996. Level Set Methods. Cambridge University Press: Cambridge, UK.Google Scholar
  44. Sullivan, S. and Ponce, J. 1998. Automatic model construction, pose estimation, and object recognition from photographs using triangular splines. In Proc. Sixth Int. Conf. on Computer Vision, pp. 510–516.Google Scholar
  45. Szeliski, R. 1993. Rapid octree construction from image sequences. Computer Vision, Graphics, and Image Processing: Image Understanding, 1(58):23–32.Google Scholar
  46. Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proc. Sixth Int. Conf. on Computer Vision, pp. 517–524.Google Scholar
  47. Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. Int. J. of Computer Vision, 9(2):137–154.Google Scholar
  48. Tsai, R.Y. 1987. A versatile camera calibration technique for highaccuracy 3D machine vision metrology using off-the-shelf cameras and lenses. IEEE Trans. Robotics and Automation, 3(4):323–344.Google Scholar
  49. Westover, L. 1990. Footprint evaluation for volume rendering. In Proc. SIGGRAPH 90, pp. 367–376.Google Scholar
  50. Zitnick, C.L. and Webb, J.A. 1996. Multi-baseline stereo using surface extraction. Technical Report CMU-CS–96–196, Carnegie Mellon University, Pittsburgh, PA.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Steven M. Seitz
    • 1
  • Charles R. Dyer
    • 2
  1. 1.The Robotics InstituteCarnegie Mellon UniversityPittsburgh
  2. 2.Department of Computer SciencesUniversity of WisconsinMadison

Personalised recommendations