Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Shape and the Stereo Correspondence Problem

Abstract

We examine the implications of shape on the process of finding dense correspondence and half-occlusions for a stereo pair of images. The desired property of the disparity map is that it should be a piecewise continuous function which is consistent with the images and which has the minimum number of discontinuities. To zeroth order, piecewise continuity becomes piecewise constancy. Using this approximation, we first discuss an approach for dealing with such a fronto-parallel shapeless world, and the problems involved therein. We then introduce horizontal and vertical slant to create a first order approximation to piecewise continuity. In particular, we emphasize the following geometric fact: a horizontally slanted surface (i.e., having depth variation in the direction of the separation of the two cameras) will appear horizontally stretched in one image as compared to the other image. Thus, while corresponding two images, N pixels on a scanline in one image may correspond to a different number of pixels M in the other image. This leads to three important modifications to existing stereo algorithms: (a) due to unequal sampling, existing intensity matching metrics must be modified, (b) unequal numbers of pixels in the two images must be allowed to correspond to each other, and (c) the uniqueness constraint, which is often used for detecting occlusions, must be changed to an interval uniqueness constraint. We also discuss the asymmetry between vertical and horizontal slant, and the central role of non-horizontal edges in the context of vertical slant. Using experiments, we discuss cases where existing algorithms fail, and how the incorporation of these new constraints provides correct results.

This is a preview of subscription content, log in to check access.

References

  1. Barnard, S.T. 1989. Stochastic stereo matching over scale. Int'l Journal of Computer Vision, 3(1):17–32.

  2. Birchfield, S. and Tomasi, C. 1999. Multiway cut for stereo and motion with slanted surfaces. In Proc. Int'l Conf. Computer Vision, vol. 1, pp. 489–495.

  3. Birchfield, S. and Tomasi, C. 1998. A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans. Pattern Analysis and Machine Intelligence, 20(4):401–406.

  4. Bobick, A.F. and Intille, S.S. 1999. Large occlusion stereo. Int'l Journal of Computer Vision, 33(3):181–200.

  5. Boykov, Y., Veksler, O., and Zabih, R. 1998. A variable window approach to early vision. IEEE Trans. Pattern Analysis and Machine Intelligence, 20(12):1283–1294.

  6. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. In IEEE Trans. Pattern Analysis and Machine Intelligence, 23(11):1222–1239.

  7. Caganello, R. and Rogers, B. 1993. Anisotropies in the perception of sterescopic surfaces: The role of orientation disparity. Vision Research, 33(16):2189–2201.

  8. Devernay, F. and Faugeras, O. 1994. Computing differential properties of 3-D shapes from stereoscopic images without 3-D models. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 208–213.

  9. Egnal, G. and Wildes, R. 2002. Detecting binocular half-occlusions: Empirical comparisons of five approaches. IEEE Trans. Pattern Analysis and Machine Intelligence, 24(8):1127–1133.

  10. Faugeras, O. and Lustman, F. 1988. Motion and structure-from-motion in a piecewise planar environment. Int'l Journal of Pattern Recognition and Artificial Intelligence, 2(3):485–508.

  11. Fusiello, A., Roberto, V., and Trucco, E. 1997. Efficient stereo with multiple windowing. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 858–863.

  12. Geiger, D., Ladendorf, B., and Yuille, A. 1992. Occlusions and binocular stereo. In Proc. European Conf. Computer Vision, pp. 425–433.

  13. Geman, S. and Geman, D. 1984. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans. Pattern Analysis and Machine Intelligence, 6(6):721–741.

  14. Gilliam, B. and Ryan, C. 1992. Perspective, orientation disparity, and anisotropy in sterescopic slant perception. Perception, 21: 427–439.

  15. Hillis, J., Watt, S., Landy, M. and Banks, M. 2004. Slant from texture and disparity cues: Optimal cue combination. Journal of Vision, 4: 967–992.

  16. Kanade, T. and Okutomi, M. 1994. A stereo matching algorithm with an adaptive window: Theory and experiment. IEEE Trans. Pattern Analysis and Machine Intelligence, 16(9):920–932.

  17. Kolmogorov, V. and Zabih, R. 2001. Computing visual correspondence with occlusions using graph cuts. In Proc. Int'l Conf. Computer Vision, pp. 508–515.

  18. Lin, M. and Tomasi, C. 2003. Surfaces with occlusions from layered stereo. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. I–710–I–717.

  19. Marr, D. and Poggio, T. 1979. A computational theory of human stereo vision. Proc. Royal Soc. London B, vol. 204(3):01–328.

  20. Mitchison, G. and McKee, S. 1990. Mechanisms underlying the anisotropy of sterescopic tilt perception. Vision Research, 30(11):1781–1791.

  21. Mulligan, J. and Daniilidis, K. 2000. Predicting disparity windows for real-time stereo. Lecture Notes in Computer Science, 1842: 220–235.

  22. Ogale, A.S. 2004. The compositional character of visual correspondence. Ph.D. dissertation, University of Maryland, College Park.

  23. Ogale, A. and Aloimonos, Y. 2004. Stereo correspondence with slanted surfaces: Critical implications of horizontal slant. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol 1, pp. 568–573.

  24. Ogale, A. and Aloimonos, Y. 2005. Robust contrast invariant stereo correspondence. In Proc. IEEE Conf. on Robotics and Automation.

  25. Ohta, Y. and Kanade, T. 1985. Stereo by intra- and inter-scanline search using dynamic programming. IEEE Trans. Pattern Analysis and Machine Intelligence, 7(2):139–154.

  26. Okutomi, M. and Kanade, T. 1993. A multiple baseline stereo, IEEE Trans. Pattern Analysis and Machine Intelligence, 15(4):353–363.

  27. Okutomi, M., Katayama, Y., and Oka, S. 2002. A simple stereo algorithm to recover precise object boundaries and smooth surfaces. Int'l Journal Computer Vision, 47(1–3):261–273.

  28. Rogers, B. and Graham, M. 1983. Anisotropies in the perception of three-dimensional surfaces. Science, 221:1409–1411.

  29. Roy, S. and Cox, I. 1998. A maximum-flow formulation of the n-camera stereo correspondence problem. In Proc. Int'l Conf. Computer Vision, pp. 492–499.

  30. Ryan, C. and Gilliam, B. 1994. Cue conflict and sterescopic surface slant about horizontal and vertical axes. Perception, 23: 645–658.

  31. Scharstein, D. and Szeliski, R. 1998. Stereo matching with nonlinear diffusion. Int'l Journal of Computer Vision, 28(2):155–174.

  32. Scharstein, taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int'l Journal of Computer Vision, 47(1):7–42.

  33. Szeliski, R. 1990. Bayesian modeling of uncertainty in low-level vision. Int'l Journal of Computer Vision, 5(3):271–302.

  34. Tao, H., Sawhney, H., and Kumar, R. 2001. A global matching framework for stereo computation. Proc. Int'l Conf. Computer Vision 1:532–539.

Download references

Author information

Correspondence to Abhijit S. Ogale.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ogale, A.S., Aloimonos, Y. Shape and the Stereo Correspondence Problem. Int J Comput Vision 65, 147–162 (2005). https://doi.org/10.1007/s11263-005-3672-3

Download citation

Keywords

  • Geometric Fact
  • Depth Variation
  • Uniqueness Constraint
  • Piecewise Constancy
  • Stereo Pair