Advertisement

International Journal of Computer Vision

, Volume 25, Issue 2, pp 167–183 | Cite as

3-D Scene Data Recovery Using Omnidirectional Multibaseline Stereo

  • Sing Bing Kang
  • Richard Szeliski
Article

Abstract

A traditional approach to extracting geometric information from a large scene is to compute multiple 3-D depth maps from stereo pairs or direct range finders, and then to merge the 3-D data. However, the resulting merged depth maps may be subject to merging errors if the relative poses between depth maps are not known exactly. In addition, the 3-D data may also have to be resampled before merging, which adds additional complexity and potential sources of errors.

This paper provides a means of directly extracting 3-D data covering a very wide field of view, thus by-passing the need for numerous depth map merging. In our work, cylindrical images are first composited from sequences of images taken while the camera is rotated 360° about a vertical axis. By taking such image panoramas at different camera locations, we can recover 3-D data of the scene using a set of simple techniques: feature tracking, an 8-point structure from motion algorithm, and multibaseline stereo. We also investigate the effect of median filtering on the recovered 3-D point distributions, and show the results of our approach applied to both synthetic and real scenes.

omnidirectional stereo multibaseline stereo panoramic structure from motion 8-point algorithm scene modeling 3-D median filtering 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ayache, N. 1991. Artificial Vision for Mobile Robots: Stereo Vision and Multisensory Perception. MIT Press: Cambridge, Massachusetts.Google Scholar
  2. Azarbayejani, A. and Pentland, A. P. 1995. Recursive estimation of motion, structure, and focal length. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(6):562-575.Google Scholar
  3. Barnard, S. T. and Fischler, M. A. 1982. Computational stereo. Computing Surveys, 14(4):553-572.Google Scholar
  4. Bolles, R. C., Baker, H. H., and Marimont, D. H. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1:7-55.Google Scholar
  5. Deriche, R., Zhang, Z., Luong, Q.-T., and Faugeras, O. 1994. Robust recovery of the epipolar geometry for an uncalibrated stereo rig. In Third European Conference on Computer Vision (ECCV'94), Springer-Verlag: Stockholm, Sweden, pp. 567-576.Google Scholar
  6. Dhond, U. R. and Aggarwal, J. K. 1989. Structure from stereo-A review. IEEE Transactions on Systems, Man, and Cybernetics, 19(6):1489-1510.Google Scholar
  7. Faugeras, O. D. 1992. What can be seen in three dimensions with an uncalibrated stereo rig? In Second European Conference on Computer Vision (ECCV'92), Springer-Verlag: Santa Margherita Liguere, Italy, pp. 563-578.Google Scholar
  8. Faugeras, O. 1993. Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press: Cambridge, Massachusetts.Google Scholar
  9. Ferrie, F. P. and Levine, M. D. 1987. Integrating information from multiple views. In IEEE Workshop on Computer Vision, IEEE Computer Society, pp. 117-122.Google Scholar
  10. Freedman, D. H. 1995. A camera for near, far, and wide. Discover, 16(48):48.Google Scholar
  11. Hartley, R. 1995. In defence of the 8-point algorithm. In Fifth International Conference on Computer Vision (ICCV'95), IEEE Computer Society Press, Cambridge, Massachusetts, pp. 1064- 1070.Google Scholar
  12. Higuchi, K., Hebert, M., and Ikeuchi, K. 1993. Building 3-D models from unregistered range images. Technical Report CMU-CS-93-214, Carnegie Mellon University.Google Scholar
  13. Ishiguro, H., Yamamoto, M., and Tsuji, S. 1992. Omnidirectional stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14:257-262.Google Scholar
  14. Kang, S. B., Johnson, A., and Szeliski, R. 1995. Extraction of concise and realistic 3-D models from real data. Technical Report 95/7, Digital Equipment Corporation, Cambridge Research Lab.Google Scholar
  15. Kang, S. B., Webb, J., Zitnick, L., and Kanade, T. 1995. A multi-baseline stereo system with active illumination and real-time image acquisition. In Fifth International Conference on Computer Vision (ICCV'95), Cambridge, Massachusetts, pp. 88-93.Google Scholar
  16. Kang, S. B. and Weiss, R. 1996. Characterization of errors in compositing panoramic images. Technical Report 96/2, Digital Equipment Corporation, Cambridge Research Lab.Google Scholar
  17. Kolb, C. E. 1994. Rayshade User's Guide and Reference Manual.Google Scholar
  18. Krishnan, A. and Ahuja, N. 1996. Panoramic image acquisition. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'96), San Francisco, California.Google Scholar
  19. Kuglin, C. D. and Hines, D. C. 1975. The phase correlation image alignment method. In IEEE 1975 Conference on Cybernetics and Society, New York, pp. 163-165.Google Scholar
  20. Longuet-Higgins, H. C. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133-135.Google Scholar
  21. McMillan, L. and Bishop, G. 1995. Plenoptic modeling: An image-based rendering system. Computer Graphics (SIGGRAPH'95), pp. 39-46.Google Scholar
  22. Murray, D. W. 1995. Recovering range using virtual multicamera stereo. Computer Vision and Image Understanding, 61(2):285- 291.Google Scholar
  23. Okutomi, M. and Kanade, T. 1993. A multiple baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(4):353-363.Google Scholar
  24. Parvin, B. and Medioni, G. 1992. B-rep from unregistered multiple range images. In IEEE Int'l Conference on Robotics and Automation, IEEE Society, pp. 1602-1607.Google Scholar
  25. Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T. 1992. Numerical Recipes in C: The Art of Scientific Computing. Second edition, Cambridge University Press: Cambridge, England.Google Scholar
  26. Shabana, A. A. 1989. Dynamics of Multibody Systems. J. Wiley: New York.Google Scholar
  27. Shashua, A. 1994. Projective structure from uncalibrated images: Structure from motion and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(8):778-790.Google Scholar
  28. Shi, J. and Tomasi, C. 1994. Good features to track. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), IEEE Computer Society, Seattle, Washington, pp. 593-600.Google Scholar
  29. Shum, H.-Y., Ikeuchi, K., and Reddy, R. 1994. Principal component analysis with missing data and its application to object modeling. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), IEEE Computer Society, Seattle, Washington, pp. 560-565.Google Scholar
  30. Stein, G. 1995. Accurate internal camera calibration using rotation, with analysis of sources of error. In Fifth International Conference on Computer Vision (ICCV'95), Cambridge, Massachusetts, pp. 230-236.Google Scholar
  31. Szeliski, R. 1994. Image mosaicing for tele-reality applications. Technical Report 94/2, Digital Equipment Corporation, Cambridge Research Lab.Google Scholar
  32. Szeliski, R. 1996. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, pp. 22-30.Google Scholar
  33. Szeliski, R. and Coughlan, J. 1994. Hierarchical spline-based image registration. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), IEEE Computer Society, Seattle, Washington, pp. 194-201.Google Scholar
  34. Szeliski, R. and Kang, S. B. 1994. Recovering 3D shape and motion from image streams using nonlinear least squares. Journal of Visual Communication and Image Representation, 5(1):10- 28.Google Scholar
  35. Szeliski, R., Kang, S. B., and Shum, H.-Y. 1995. A parallel feature tracker for extended image sequences. In IEEE International Symposium on Computer Vision, Coral Gables, Florida, pp. 241- 246.Google Scholar
  36. Taylor, C. J., Debevec, P. E., and Malik, J. 1996. Reconstructing polyhedral models of architectural scenes from photographs. In Fourth European Conference on Computer Vision (ECCV'96), Springer-Verlag: Cambridge, England, pp. 659-668.Google Scholar
  37. Tian, T. Y., Tomasi, C., and Heeger, D. J. 1996. Comparison of approaches to egomotion computation. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 315-320.Google Scholar

Copyright information

© Kluwer Academic Publishers 1997

Authors and Affiliations

  • Sing Bing Kang
    • 1
  • Richard Szeliski
    • 2
  1. 1.Cambridge Research LabDigital Equipment CorporationCambridge
  2. 2.Microsoft CorporationRedmond

Personalised recommendations