International Journal of Computer Vision

, Volume 65, Issue 3, pp 147–162 | Cite as

Shape and the Stereo Correspondence Problem

  • Abhijit S. Ogale
  • Yiannis Aloimonos
Article

Abstract

We examine the implications of shape on the process of finding dense correspondence and half-occlusions for a stereo pair of images. The desired property of the disparity map is that it should be a piecewise continuous function which is consistent with the images and which has the minimum number of discontinuities. To zeroth order, piecewise continuity becomes piecewise constancy. Using this approximation, we first discuss an approach for dealing with such a fronto-parallel shapeless world, and the problems involved therein. We then introduce horizontal and vertical slant to create a first order approximation to piecewise continuity. In particular, we emphasize the following geometric fact: a horizontally slanted surface (i.e., having depth variation in the direction of the separation of the two cameras) will appear horizontally stretched in one image as compared to the other image. Thus, while corresponding two images, N pixels on a scanline in one image may correspond to a different number of pixels M in the other image. This leads to three important modifications to existing stereo algorithms: (a) due to unequal sampling, existing intensity matching metrics must be modified, (b) unequal numbers of pixels in the two images must be allowed to correspond to each other, and (c) the uniqueness constraint, which is often used for detecting occlusions, must be changed to an interval uniqueness constraint. We also discuss the asymmetry between vertical and horizontal slant, and the central role of non-horizontal edges in the context of vertical slant. Using experiments, we discuss cases where existing algorithms fail, and how the incorporation of these new constraints provides correct results.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barnard, S.T. 1989. Stochastic stereo matching over scale. Int'l Journal of Computer Vision, 3(1):17–32.MathSciNetGoogle Scholar
  2. Birchfield, S. and Tomasi, C. 1999. Multiway cut for stereo and motion with slanted surfaces. In Proc. Int'l Conf. Computer Vision, vol. 1, pp. 489–495.Google Scholar
  3. Birchfield, S. and Tomasi, C. 1998. A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans. Pattern Analysis and Machine Intelligence, 20(4):401–406.CrossRefGoogle Scholar
  4. Bobick, A.F. and Intille, S.S. 1999. Large occlusion stereo. Int'l Journal of Computer Vision, 33(3):181–200.Google Scholar
  5. Boykov, Y., Veksler, O., and Zabih, R. 1998. A variable window approach to early vision. IEEE Trans. Pattern Analysis and Machine Intelligence, 20(12):1283–1294.CrossRefGoogle Scholar
  6. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. In IEEE Trans. Pattern Analysis and Machine Intelligence, 23(11):1222–1239.Google Scholar
  7. Caganello, R. and Rogers, B. 1993. Anisotropies in the perception of sterescopic surfaces: The role of orientation disparity. Vision Research, 33(16):2189–2201.Google Scholar
  8. Devernay, F. and Faugeras, O. 1994. Computing differential properties of 3-D shapes from stereoscopic images without 3-D models. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 208–213.Google Scholar
  9. Egnal, G. and Wildes, R. 2002. Detecting binocular half-occlusions: Empirical comparisons of five approaches. IEEE Trans. Pattern Analysis and Machine Intelligence, 24(8):1127–1133.CrossRefGoogle Scholar
  10. Faugeras, O. and Lustman, F. 1988. Motion and structure-from-motion in a piecewise planar environment. Int'l Journal of Pattern Recognition and Artificial Intelligence, 2(3):485–508.Google Scholar
  11. Fusiello, A., Roberto, V., and Trucco, E. 1997. Efficient stereo with multiple windowing. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 858–863.Google Scholar
  12. Geiger, D., Ladendorf, B., and Yuille, A. 1992. Occlusions and binocular stereo. In Proc. European Conf. Computer Vision, pp. 425–433.Google Scholar
  13. Geman, S. and Geman, D. 1984. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans. Pattern Analysis and Machine Intelligence, 6(6):721–741.MATHGoogle Scholar
  14. Gilliam, B. and Ryan, C. 1992. Perspective, orientation disparity, and anisotropy in sterescopic slant perception. Perception, 21: 427–439.Google Scholar
  15. Hillis, J., Watt, S., Landy, M. and Banks, M. 2004. Slant from texture and disparity cues: Optimal cue combination. Journal of Vision, 4: 967–992.CrossRefGoogle Scholar
  16. Kanade, T. and Okutomi, M. 1994. A stereo matching algorithm with an adaptive window: Theory and experiment. IEEE Trans. Pattern Analysis and Machine Intelligence, 16(9):920–932.CrossRefGoogle Scholar
  17. Kolmogorov, V. and Zabih, R. 2001. Computing visual correspondence with occlusions using graph cuts. In Proc. Int'l Conf. Computer Vision, pp. 508–515.Google Scholar
  18. Lin, M. and Tomasi, C. 2003. Surfaces with occlusions from layered stereo. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. I–710–I–717.Google Scholar
  19. Marr, D. and Poggio, T. 1979. A computational theory of human stereo vision. Proc. Royal Soc. London B, vol. 204(3):01–328.CrossRefGoogle Scholar
  20. Mitchison, G. and McKee, S. 1990. Mechanisms underlying the anisotropy of sterescopic tilt perception. Vision Research, 30(11):1781–1791.CrossRefGoogle Scholar
  21. Mulligan, J. and Daniilidis, K. 2000. Predicting disparity windows for real-time stereo. Lecture Notes in Computer Science, 1842: 220–235.Google Scholar
  22. Ogale, A.S. 2004. The compositional character of visual correspondence. Ph.D. dissertation, University of Maryland, College Park.Google Scholar
  23. Ogale, A. and Aloimonos, Y. 2004. Stereo correspondence with slanted surfaces: Critical implications of horizontal slant. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol 1, pp. 568–573.Google Scholar
  24. Ogale, A. and Aloimonos, Y. 2005. Robust contrast invariant stereo correspondence. In Proc. IEEE Conf. on Robotics and Automation.Google Scholar
  25. Ohta, Y. and Kanade, T. 1985. Stereo by intra- and inter-scanline search using dynamic programming. IEEE Trans. Pattern Analysis and Machine Intelligence, 7(2):139–154.CrossRefGoogle Scholar
  26. Okutomi, M. and Kanade, T. 1993. A multiple baseline stereo, IEEE Trans. Pattern Analysis and Machine Intelligence, 15(4):353–363.Google Scholar
  27. Okutomi, M., Katayama, Y., and Oka, S. 2002. A simple stereo algorithm to recover precise object boundaries and smooth surfaces. Int'l Journal Computer Vision, 47(1–3):261–273.MATHGoogle Scholar
  28. Rogers, B. and Graham, M. 1983. Anisotropies in the perception of three-dimensional surfaces. Science, 221:1409–1411.Google Scholar
  29. Roy, S. and Cox, I. 1998. A maximum-flow formulation of the n-camera stereo correspondence problem. In Proc. Int'l Conf. Computer Vision, pp. 492–499.Google Scholar
  30. Ryan, C. and Gilliam, B. 1994. Cue conflict and sterescopic surface slant about horizontal and vertical axes. Perception, 23: 645–658.Google Scholar
  31. Scharstein, D. and Szeliski, R. 1998. Stereo matching with nonlinear diffusion. Int'l Journal of Computer Vision, 28(2):155–174.Google Scholar
  32. Scharstein, taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int'l Journal of Computer Vision, 47(1):7–42.MATHGoogle Scholar
  33. Szeliski, R. 1990. Bayesian modeling of uncertainty in low-level vision. Int'l Journal of Computer Vision, 5(3):271–302.Google Scholar
  34. Tao, H., Sawhney, H., and Kumar, R. 2001. A global matching framework for stereo computation. Proc. Int'l Conf. Computer Vision 1:532–539.Google Scholar

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  • Abhijit S. Ogale
    • 1
  • Yiannis Aloimonos
    • 1
  1. 1.Department of Computer Science, Computer Vision Laboratory, Institute for Advanced Computer StudiesUniversity of MarylandUSA

Personalised recommendations