International Journal of Computer Vision

, Volume 28, Issue 2, pp 155–174 | Cite as

Stereo Matching with Nonlinear Diffusion

  • Daniel Scharstein
  • Richard Szeliski
Article

Abstract

One of the central problems in stereo matching (and other image registration tasks) is the selection of optimal window sizes for comparing image regions. This paper addresses this problem with some novel algorithms based on iteratively diffusing support at different disparity hypotheses, and locally controlling the amount of diffusion based on the current quality of the disparity estimate. It also develops a novel Bayesian estimation technique, which significantly outperforms techniques based on area-based matching (SSD) and regular diffusion. We provide experimental results on both synthetic and real stereo image pairs.

stereo matching variable-sized support region nonlinear diffusion Bayesian estimation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arnold, R.D. 1983. Automated stereo perception. Technical Report AIM-351, Artificial Intelligence Laboratory, Stanford University.Google Scholar
  2. Baker, H.H. 1980. Edge based stereo correlation. In Image Understanding Workshop, L.S. Baumann (Ed.), Science Applications International Corporation, pp. 168–175.Google Scholar
  3. Barnard, S.T. 1989. Stochastic stereo matching over scale. International Journal of Computer Vision, 3(1):17–32.Google Scholar
  4. Barnard, S.T. and Fischler, M.A. 1982. Computational stereo. ACM Computing Surveys, 14(4):553–572.Google Scholar
  5. Belhumeur, P.N. and Mumford, D. 1992. A Bayesian treatment of the stereo correspondence problem using half-occluded regions. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'92), Champaign-Urbana, IL, IEEE Computer Society Press, pp. 506–512.Google Scholar
  6. Besag, J. 1986. On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society, B-48(3):259–302.Google Scholar
  7. Black, M.J. and Rangarajan, A. 1996. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision, 19(1):57– 91.Google Scholar
  8. Blake, A. and Zisserman, A. 1987.Visual Reconstruction.MITPress: Cambridge, MA.Google Scholar
  9. Bolles, R.C., Baker, H.H., and Marimont, D.H. 1987. Epipolarplane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1:7– 55.Google Scholar
  10. Cover, T.M. and Thomas, J.A. 1991. Elements of Information Theory. John Wiley & Sons: New York.Google Scholar
  11. Cox, I.J. 1994. A maximum likelihood n-camera stereo algorithm. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), Seattle, WA, IEEE Computer Society Press, pp. 733–739.Google Scholar
  12. Dhond, U.R. and Aggarwal, J.K. 1989. Structure from stereo—A review. IEEE Transactions on Systems, Man, and Cybernetics, 19(6):1489–1510.Google Scholar
  13. Fua, P. 1993. A parallel stereo algorithm that produces dense depth maps and preserves image features. Machine Vision and Applications, 6:35–49.Google Scholar
  14. Geiger, D. and Girosi, F. 1991. Mean field theory for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(5):401–412.Google Scholar
  15. Geiger, D., Ladendorf, B., and Yuille, A. 1992. Occlusions and binocular stereo. In Second European Conference on Computer Vision (ECCV'92), Santa Margherita Ligure, Italy, LNCS 588, Springer-Verlag, pp. 425–433.Google Scholar
  16. Geman, S. and Geman, D. 1984. Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6):721– 741.Google Scholar
  17. Grimson, W.E.L. 1985. Computational experiments with a feature based stereo algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7(1):17–34.Google Scholar
  18. Hoff, W. and Ahuja, N. 1989. Surfaces from stereo: Integrating feature matching, disparity estimation, and contour detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(2):121–136.Google Scholar
  19. Huber, P.J. 1981. Robust Statistics. John Wiley & Sons: New York, NY.Google Scholar
  20. Intille, S.S. and Bobick, A.F. 1994. Disparity-space images and large occlusion stereo. In Third European Conference on Computer Vision (ECCV';94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 179–186.Google Scholar
  21. Jenkin, M.R.M., Jepson, A.D., and Tsotsos, J.K. 1991. Techniques for disparity measurement. CVGIP: Image Understanding, 53(1):14–30.Google Scholar
  22. Jones, D.G. and Malik, J. 1992. A computational framework for determining stereo correspondence from a set of linear spatial filters. In Second European Conference on Computer Vision (ECCV'92), Santa Margherita Ligure, Italy, LNCS 588, Springer-Verlag, pp. 395–410.Google Scholar
  23. Kanade, T. 1994. Development of a video-rate stereo machine. In Image Understanding Workshop, Monterey, CA, Morgan Kaufmann Publishers, pp. 549–557.Google Scholar
  24. Kanade, T. and Okutomi, M. 1994. A stereo matching algorithm with an adaptive window: Theory and experiment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(9):920– 932.Google Scholar
  25. Kang, S.B., Webb, J., Zitnick, L., and Kanade, T. 1995. A multibaseline stereo system with active illumination and real-time image acquisition. In Fifth International Conference on Computer Vision (ICCV’95), MIT, Cambridge, MA, IEEE Computer Society Press, pp. 88–93.Google Scholar
  26. Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application in stereo vision. In Seventh International Joint Conference on Artificial Intelligence (IJCAI-81), Vancouver, pp. 674–679.Google Scholar
  27. Marr, D. and Poggio, T. 1976. Cooperative computation of stereo disparity. Science, 194:283–287.Google Scholar
  28. Marr, D.C. and Poggio, T. 1979. A computational theory of human stereo vision. Proceedings of the Royal Society of London, B204:301–328.Google Scholar
  29. Marroquin, J., Mitter, S., and Poggio, T. 1987. Probabilistic solution of ill-posed problems in computational vision. Journal of the American Statistical Association, 82(397):76– 89.Google Scholar
  30. Matthies, L., Szeliski, R., and Kanade, T. 1989. Kalman filter-based algorithms for estimating depth from image sequences. International Journal of Computer Vision, 3:209–236.Google Scholar
  31. Nordström, N. 1990. Biased anisotropic diffusion—A unified regularization and diffusion approach to edge detection. In First European Conference on Computer Vision (ECCV'90), Antibes, France, LNCS 427, Springer-Verlag, pp. 18–27.Google Scholar
  32. Ohta, Y. and Kanade, T. 1985. Stereo by intra-and interscanline search using dynamic programming. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7(2):139– 154.Google Scholar
  33. Okutomi, M. and Kanade, T. 1992. A locally adaptive window for signal matching. International Journal of Computer Vision, 7(2):143–162.Google Scholar
  34. Okutomi, M. and Kanade, T. 1993. A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(4):353–363.Google Scholar
  35. Olsen, S.I. 1990. Stereo correspondence by surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(3):309–314.Google Scholar
  36. Parisi, G. 1988. Statistical Field Theory. Addison-Wesley.Google Scholar
  37. Perona, P. and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629–639.Google Scholar
  38. Peterson, C. and Söderberg, B. 1989. A new method of mapping optimization problems onto neural networks. International Journal of Neural Systems, 1(1):3.Google Scholar
  39. Pollard, S.B., Mayhew, J.E.W., and Frisby, J.P. 1985. PMF: A stereo correspondence algorithm using a disparity gradient limit. Perception, 14:449–470.Google Scholar
  40. Prazdny, K. 1985. Detection of binocular disparities. Biological Cybernetics, 52(2):93–99.Google Scholar
  41. Proesmans, M., VanGool, L.J., Pauwels, E., and Oosterlinck, A. 1994. Determination of optical flow and its discontinuities using nonlinear diffusion. In Third European Conference on Computer Vision (ECCV'94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 295–304.Google Scholar
  42. Quam, L.H. 1984. Hierarchicalwarp stereo. In Image Understanding Workshop, New Orleans, Louisiana, Science Applications International Corporation, pp. 149–155.Google Scholar
  43. Ryan, T.W., Gray, R.T., and Hunt, B.R. 1980. Prediction of correlation errors in stereo-pair images. Optical Engineering, 19(3):312– 322.Google Scholar
  44. Scharstein, D. 1994. Matching images by comparing their gradient fields. In 12th International Conference on Pattern Recognition (ICPR’94), Jerusalem, Israel, vol. 1, pp. 572–575.Google Scholar
  45. Seitz, P. 1989. Using local orientation information as image primitive for robust object recognition. In SPIE Visual Communications and Image Processing IV, vol. 1199, pp. 1630–1639.Google Scholar
  46. Shah, J. 1993. A nonlinear diffusion model for discontinuous disparity and half-occlusion in stereo. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'93), New York, NY, IEEE Computer Society Press, pp. 34–40.Google Scholar
  47. Stewart, C.V., Flatland, R.Y., and Bubna, K. 1996. Geometric constraints and stereo disparity computation. International Journal of Computer Vision, 20(3):143–168.Google Scholar
  48. Szeliski, R. 1989. Bayesian Modeling of Uncertainty in Low-Level Vision. Kluwer Academic Publishers: Boston, MA.Google Scholar
  49. Szeliski, R. and Hinton, G. 1985. Solving random-dot stereograms using the heat equation. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’85), San Francisco, CA, IEEE Computer Society Press, pp. 284– 288.Google Scholar
  50. Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Sixth International Conference on Computer Vision (ICCV'98), Bombay, India, IEEE Computer Society Press.Google Scholar
  51. Terzopoulos, D. 1986. Regularization of inverse visual problems involving discontinuities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(4):413–424.Google Scholar
  52. Tian, Q. and Huhns, M.N. 1986. Algorithms for subpixel registration. Computer Vision, Graphics, and Image Processing, 35:220– 233.Google Scholar
  53. Witkin, A., Terzopoulos, D., and Kass, M. 1987. Signal matching through scale space. International Journal of Computer Vision, 1:133–144.Google Scholar
  54. Yang, Y., Yuille, A., and Lu, J. 1993. Local, global, and multilevel stereo matching. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'93), New York, NY, IEEE Computer Society Press, pp. 274–279.Google Scholar
  55. Yuille, A.L. and Poggio, T. 1984. A generalized ordering constraint for stereo correspondence. A.I. Memo 777, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA.Google Scholar
  56. Zabih, R. and Woodfill, J. 1994. Non-parametric local transforms for computing visual correspondence. In Third European Conference on Computer Vision (ECCV'94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 151– 158.Google Scholar
  57. Zerubia, J. and Chepalla, R. 1993. Mean field annealing using compound Gauss-Markov random fields for edge detection and image estimation. IEEE Transactions on Neural Networks, 4(4).Google Scholar
  58. Zhang, J., Modestino, J.W., and Langan, D.A. 1994. Maximumlikelihood parameter estimation for unsupervised stochastic model-based image segmentation. IEEE Transactions on Image Processing, 3(4):404–420.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Daniel Scharstein
    • 1
  • Richard Szeliski
    • 2
  1. 1.Department of Mathematics and Computer ScienceMiddlebury CollegeMiddlebury
  2. 2.Microsoft ResearchOne Microsoft WayRedmond

Personalised recommendations