Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Stereo Matching with Nonlinear Diffusion

Abstract

One of the central problems in stereo matching (and other image registration tasks) is the selection of optimal window sizes for comparing image regions. This paper addresses this problem with some novel algorithms based on iteratively diffusing support at different disparity hypotheses, and locally controlling the amount of diffusion based on the current quality of the disparity estimate. It also develops a novel Bayesian estimation technique, which significantly outperforms techniques based on area-based matching (SSD) and regular diffusion. We provide experimental results on both synthetic and real stereo image pairs.

This is a preview of subscription content, log in to check access.

References

  1. Arnold, R.D. 1983. Automated stereo perception. Technical Report AIM-351, Artificial Intelligence Laboratory, Stanford University.

  2. Baker, H.H. 1980. Edge based stereo correlation. In Image Understanding Workshop, L.S. Baumann (Ed.), Science Applications International Corporation, pp. 168–175.

  3. Barnard, S.T. 1989. Stochastic stereo matching over scale. International Journal of Computer Vision, 3(1):17–32.

  4. Barnard, S.T. and Fischler, M.A. 1982. Computational stereo. ACM Computing Surveys, 14(4):553–572.

  5. Belhumeur, P.N. and Mumford, D. 1992. A Bayesian treatment of the stereo correspondence problem using half-occluded regions. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'92), Champaign-Urbana, IL, IEEE Computer Society Press, pp. 506–512.

  6. Besag, J. 1986. On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society, B-48(3):259–302.

  7. Black, M.J. and Rangarajan, A. 1996. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision, 19(1):57– 91.

  8. Blake, A. and Zisserman, A. 1987.Visual Reconstruction.MITPress: Cambridge, MA.

  9. Bolles, R.C., Baker, H.H., and Marimont, D.H. 1987. Epipolarplane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1:7– 55.

  10. Cover, T.M. and Thomas, J.A. 1991. Elements of Information Theory. John Wiley & Sons: New York.

  11. Cox, I.J. 1994. A maximum likelihood n-camera stereo algorithm. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), Seattle, WA, IEEE Computer Society Press, pp. 733–739.

  12. Dhond, U.R. and Aggarwal, J.K. 1989. Structure from stereo—A review. IEEE Transactions on Systems, Man, and Cybernetics, 19(6):1489–1510.

  13. Fua, P. 1993. A parallel stereo algorithm that produces dense depth maps and preserves image features. Machine Vision and Applications, 6:35–49.

  14. Geiger, D. and Girosi, F. 1991. Mean field theory for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(5):401–412.

  15. Geiger, D., Ladendorf, B., and Yuille, A. 1992. Occlusions and binocular stereo. In Second European Conference on Computer Vision (ECCV'92), Santa Margherita Ligure, Italy, LNCS 588, Springer-Verlag, pp. 425–433.

  16. Geman, S. and Geman, D. 1984. Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6):721– 741.

  17. Grimson, W.E.L. 1985. Computational experiments with a feature based stereo algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7(1):17–34.

  18. Hoff, W. and Ahuja, N. 1989. Surfaces from stereo: Integrating feature matching, disparity estimation, and contour detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(2):121–136.

  19. Huber, P.J. 1981. Robust Statistics. John Wiley & Sons: New York, NY.

  20. Intille, S.S. and Bobick, A.F. 1994. Disparity-space images and large occlusion stereo. In Third European Conference on Computer Vision (ECCV';94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 179–186.

  21. Jenkin, M.R.M., Jepson, A.D., and Tsotsos, J.K. 1991. Techniques for disparity measurement. CVGIP: Image Understanding, 53(1):14–30.

  22. Jones, D.G. and Malik, J. 1992. A computational framework for determining stereo correspondence from a set of linear spatial filters. In Second European Conference on Computer Vision (ECCV'92), Santa Margherita Ligure, Italy, LNCS 588, Springer-Verlag, pp. 395–410.

  23. Kanade, T. 1994. Development of a video-rate stereo machine. In Image Understanding Workshop, Monterey, CA, Morgan Kaufmann Publishers, pp. 549–557.

  24. Kanade, T. and Okutomi, M. 1994. A stereo matching algorithm with an adaptive window: Theory and experiment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(9):920– 932.

  25. Kang, S.B., Webb, J., Zitnick, L., and Kanade, T. 1995. A multibaseline stereo system with active illumination and real-time image acquisition. In Fifth International Conference on Computer Vision (ICCV’95), MIT, Cambridge, MA, IEEE Computer Society Press, pp. 88–93.

  26. Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application in stereo vision. In Seventh International Joint Conference on Artificial Intelligence (IJCAI-81), Vancouver, pp. 674–679.

  27. Marr, D. and Poggio, T. 1976. Cooperative computation of stereo disparity. Science, 194:283–287.

  28. Marr, D.C. and Poggio, T. 1979. A computational theory of human stereo vision. Proceedings of the Royal Society of London, B204:301–328.

  29. Marroquin, J., Mitter, S., and Poggio, T. 1987. Probabilistic solution of ill-posed problems in computational vision. Journal of the American Statistical Association, 82(397):76– 89.

  30. Matthies, L., Szeliski, R., and Kanade, T. 1989. Kalman filter-based algorithms for estimating depth from image sequences. International Journal of Computer Vision, 3:209–236.

  31. Nordström, N. 1990. Biased anisotropic diffusion—A unified regularization and diffusion approach to edge detection. In First European Conference on Computer Vision (ECCV'90), Antibes, France, LNCS 427, Springer-Verlag, pp. 18–27.

  32. Ohta, Y. and Kanade, T. 1985. Stereo by intra-and interscanline search using dynamic programming. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7(2):139– 154.

  33. Okutomi, M. and Kanade, T. 1992. A locally adaptive window for signal matching. International Journal of Computer Vision, 7(2):143–162.

  34. Okutomi, M. and Kanade, T. 1993. A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(4):353–363.

  35. Olsen, S.I. 1990. Stereo correspondence by surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(3):309–314.

  36. Parisi, G. 1988. Statistical Field Theory. Addison-Wesley.

  37. Perona, P. and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629–639.

  38. Peterson, C. and Söderberg, B. 1989. A new method of mapping optimization problems onto neural networks. International Journal of Neural Systems, 1(1):3.

  39. Pollard, S.B., Mayhew, J.E.W., and Frisby, J.P. 1985. PMF: A stereo correspondence algorithm using a disparity gradient limit. Perception, 14:449–470.

  40. Prazdny, K. 1985. Detection of binocular disparities. Biological Cybernetics, 52(2):93–99.

  41. Proesmans, M., VanGool, L.J., Pauwels, E., and Oosterlinck, A. 1994. Determination of optical flow and its discontinuities using nonlinear diffusion. In Third European Conference on Computer Vision (ECCV'94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 295–304.

  42. Quam, L.H. 1984. Hierarchicalwarp stereo. In Image Understanding Workshop, New Orleans, Louisiana, Science Applications International Corporation, pp. 149–155.

  43. Ryan, T.W., Gray, R.T., and Hunt, B.R. 1980. Prediction of correlation errors in stereo-pair images. Optical Engineering, 19(3):312– 322.

  44. Scharstein, D. 1994. Matching images by comparing their gradient fields. In 12th International Conference on Pattern Recognition (ICPR’94), Jerusalem, Israel, vol. 1, pp. 572–575.

  45. Seitz, P. 1989. Using local orientation information as image primitive for robust object recognition. In SPIE Visual Communications and Image Processing IV, vol. 1199, pp. 1630–1639.

  46. Shah, J. 1993. A nonlinear diffusion model for discontinuous disparity and half-occlusion in stereo. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'93), New York, NY, IEEE Computer Society Press, pp. 34–40.

  47. Stewart, C.V., Flatland, R.Y., and Bubna, K. 1996. Geometric constraints and stereo disparity computation. International Journal of Computer Vision, 20(3):143–168.

  48. Szeliski, R. 1989. Bayesian Modeling of Uncertainty in Low-Level Vision. Kluwer Academic Publishers: Boston, MA.

  49. Szeliski, R. and Hinton, G. 1985. Solving random-dot stereograms using the heat equation. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’85), San Francisco, CA, IEEE Computer Society Press, pp. 284– 288.

  50. Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Sixth International Conference on Computer Vision (ICCV'98), Bombay, India, IEEE Computer Society Press.

  51. Terzopoulos, D. 1986. Regularization of inverse visual problems involving discontinuities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(4):413–424.

  52. Tian, Q. and Huhns, M.N. 1986. Algorithms for subpixel registration. Computer Vision, Graphics, and Image Processing, 35:220– 233.

  53. Witkin, A., Terzopoulos, D., and Kass, M. 1987. Signal matching through scale space. International Journal of Computer Vision, 1:133–144.

  54. Yang, Y., Yuille, A., and Lu, J. 1993. Local, global, and multilevel stereo matching. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'93), New York, NY, IEEE Computer Society Press, pp. 274–279.

  55. Yuille, A.L. and Poggio, T. 1984. A generalized ordering constraint for stereo correspondence. A.I. Memo 777, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA.

  56. Zabih, R. and Woodfill, J. 1994. Non-parametric local transforms for computing visual correspondence. In Third European Conference on Computer Vision (ECCV'94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 151– 158.

  57. Zerubia, J. and Chepalla, R. 1993. Mean field annealing using compound Gauss-Markov random fields for edge detection and image estimation. IEEE Transactions on Neural Networks, 4(4).

  58. Zhang, J., Modestino, J.W., and Langan, D.A. 1994. Maximumlikelihood parameter estimation for unsupervised stochastic model-based image segmentation. IEEE Transactions on Image Processing, 3(4):404–420.

Download references

Author information

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Scharstein, D., Szeliski, R. Stereo Matching with Nonlinear Diffusion. International Journal of Computer Vision 28, 155–174 (1998). https://doi.org/10.1023/A:1008015117424

Download citation

  • stereo matching
  • variable-sized support region
  • nonlinear diffusion
  • Bayesian estimation