Advertisement

International Journal of Computer Vision

, Volume 6, Issue 3, pp 197–226 | Cite as

A topological stereo matcher

  • Margaret M. Fleck
Article

Abstract

Presented here is a new stereo algorithm that produces dense, high-quality, subpixel disparity maps. It offers two improvements over previous algorithms. First, it does not blur disparity values across sharp changes in depth. Second, it can reconstruct the correct correspondence between two images even when there is substantial vertical displacement between them: this algorithm has been tested with rotations up to 10 degrees and vertical translations up to 16 pixels. Although such image pairs require extra processing time, this ability is vital when exact calibration cannot be maintained.

The new algorithm depends on two new ideas. First, it exploits the fact that the correct vertical disparity field is due to camera misalignment and, thus, has only a few (significant) degrees of freedom. The algorithm passes camera alignment parameters, not raw disparity fields, between scales. Disparities at individual locations can diverge only slightly from this global model, greatly reducing the algorithm's search space.

Second, the new algorithm uses a pre-match filter that prevents two patches of image from matching if they do not have the same (local) topological structure. This constraint subsumes previous “figural continuity” proposals and can be checked by simple, local operations. The filter seems to improve the algorithm's ability to select the correct match from many alternatives and it suppresses intermediate values near sharp changes in disparity. This technique can be extended to other matching tasks, such as motion tracking, analyzing texture periodicity, and evaluating the performance of edge finders.

Keywords

Matching Task Stereo Matcher Correct Match Alignment Parameter Vertical Translation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ayache, N., and Faverjon, B. 1987. Efficient registration of stereo images by matching graph descriptions of edge segments. Intern. J. Comput. Vision 1: 107–131.Google Scholar
  2. Bajcsy, R. 1973. Computer identification of visual surfaces. Comput. Graph. Image Proc. 2: 118–130Google Scholar
  3. Baker, H.H., and Binford, T.O. 1981. Depth from edge and intensity based stereo. Proc. 7th Intern. Joint Conf. Artif. Intell., Vancouver, pp. 631–636.Google Scholar
  4. Barnard, S.T. 1989. Stochastic stereo matching over scale. Intern. J. Comput. Vision 3(1): 17–32.Google Scholar
  5. Barnard, S.T., and Fischler, M.A. 1982. Computational stereo. Computing Surveys 14: 553–572.Google Scholar
  6. Barnard, S.T., and Thompson, W.B. 1980. Disparity analysis of images. IEEE Trans. Patt. Anal. Mach. Intell. 2: 333–340.Google Scholar
  7. Boie, R.A., and Cox, I.J. 1987. Two dimensional optimal edge recognition using matched and Wiener filters for machine vision. Proc. Intern. Conf. Comput Vision, London, pp. 450–456.Google Scholar
  8. Boie, R.A., Cox, I.J., and Rehak, P. 1986. On optimum edge recognition using matched filters. Proc. Conf. Comput. Vision Patt. Recog. Miami Beach, pp. 100–108.Google Scholar
  9. Bolles, R.C., Baker, H.H., and Marimont, D.H. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. Intern. J. Comput. Vision 1: 7–55.Google Scholar
  10. Bovik, A.C., Clark, M., and Geisler, W.S. 1987. Computational texture analysis using localized spatial filtering. Proc. IEEE Comput. Soc. Work. Comput. Vision. pp. 201–206.Google Scholar
  11. Boyer, K.L., and Kak, A.C. 1988. Structural stereopsis for 3-D vision. IEEE Trans. Patt. Anal. Mach. Intell. 10: 144–166.Google Scholar
  12. Bülthoff, H.H., and Mallot, H.A. 1988. Interaction of depth modules: Stereo and shading. J. Opt. Soc. Amer. 5: 1749–1758.Google Scholar
  13. Burt, P., and Julesz, B. 1980a. Modifications of the classical notion of Panum's fusional area. Perception 9: 671–682.Google Scholar
  14. Burt, P., and Julesz, B. 1980b. A disparity gradient limit for binocular fusion. Science 208: 615–617.Google Scholar
  15. Buxton, B.F., and Buxton, H. 1984. Computation of optic flow from the motion of edge features in image sequences. Image and Vision Comput. 2: 59–75.Google Scholar
  16. Callahan, J. 1974. Singularities and plane maps. Amer. Math. Monthly 81: 211–240.Google Scholar
  17. Callahan, J. 1977. Singularities and plane maps II: Sketching catastrophes. Amer. Math. Monthly 84: 765–803.Google Scholar
  18. Callahan, J., and Weiss, R. 1985. A model for deseribing surface shape. Proc. Conf. Comput. Vision Patt. Recog. San Francisco, pp. 240–245.Google Scholar
  19. Canny, J.F. 1983. Finding edges and lines in images. M.S. thesis, Mass. Inst. of Technol., also Artif. Intell. Lab. Techn. Reprt. 720.Google Scholar
  20. Canny, J.F. 1986. A computational approach to edge detection. IEEE Trans. Patt. Anal. Mach. Intell. 8: 679–698.Google Scholar
  21. Cavanagh, P. 1987. Reconstructing the third dimension: Interactions between color, texture, motion, binocular disparity, and shape. Comput. Vision Graph., Image Process. 37: 171–195.Google Scholar
  22. Chen, L. 1985. Topological structure in the perception of apparent motion. Perception 14: 197–208.Google Scholar
  23. Clark, J.J. 1988. Singularity theory and phantom edges in scale space. IEEE Trans. Patt. Anal. Mach. Intell. 10: 720–727.Google Scholar
  24. Clark, J.J. 1989. Authenticating edges produced by zero-crossing algorithms. IEEE Trans. Patt. Anal. Mach. Intell. 11: 43–57.Google Scholar
  25. Day, T., and Muller, J-P. 1989. Digital elevation model production by stereo-matching spot image-pairs: A comparison of algorithms. Image Vision Comput. 7(2): 95–101.Google Scholar
  26. Drumheller, M., and Poggio, T. 1986. On parallel stereo. Proc. IEEE Intern. Conf. Robot. Autom., San Francisco, pp. 1439–1448.Google Scholar
  27. Duwaer, A.L., and van denBrink, G. 1981. Diplopia thresholds and the initiation of vergence eye movements. Vision Research 21: 1727–1737.Google Scholar
  28. Fleck, M.M. 1988a. Representing space for practical reasoning. Image Vision Comput. 6: 75–86.Google Scholar
  29. Fleck, M.M. 1988b. Boundaries and topological algorithms. Ph.D. thesis, Mass. Inst. of Technol., also Artif. Intell. Lab. Techn. Reprt. 1065.Google Scholar
  30. Fleck, M.M. 1989. Spectre: An improved phantom edge finder. Proc. Alvey Vision Conf., pp. 127–132.Google Scholar
  31. Fleck, M.M. 1990a. Topological models for space and time. OUEL Report No. 1860/90, Oxford Univ., Dept. of Eng. Science.Google Scholar
  32. Fleck, M.M. 1990b. Multiple widths yield reliable finite differences. Proc. 3rd Intern. Conf. Comput. Vision, December, Osaka, Japan, pp. 58–61.Google Scholar
  33. Fleet, D.J., and Jepson, A.D. 1989. Computation of normal velocity from local phase information. Proc. IEEE Conf. Comput. Vision Patt. Recog., San Diego, pp. 379–386.Google Scholar
  34. Fram, J.R., and Deutsch, E.S. 1975. On the quantitative evaluation of edge detection schemes and their comparison with human performance. IEEE Trans. Comput. C-24(6): 616–628.Google Scholar
  35. Gennert, M.A. 1987. A computational framework for understanding problems in stereo vision. Sc.D. thesis, MIT, Cambridge, MA.Google Scholar
  36. Gennert, M.A. 1988. Brightness-based stereo matching. Proc. 2nd Intern. Conf. Comput. Vision, Tampa, FL, pp. 139–143.Google Scholar
  37. Gennery, D.B. 1977. A stereo vision system for an autonomous vehicle. Proc. 5th Intern. Joint Conf. Artif. Intell., Cambridge, MA, pp. 576–582.Google Scholar
  38. Gillett, W.E. 1988. Issues in parallel stereo matching. M.S. thesis. Mass. Inst. of Technol., Cambridge, MA.Google Scholar
  39. Grimson, W.E.L. 1981a. From Images to Surfaces: A Computation Study of the Human Early Visual System, MIT Press: Cambridge, MA.Google Scholar
  40. Grimson, W.E.L. 1981b. A computer implementation of a theory of human stereo vision. Phil. Trans. Roy. Soc. London B 292: 217–253.Google Scholar
  41. Grimson, W.E.L. 1985. Computational experiments with a feature based stereo algorithm. IEEE Trans. Patt. Anal. Mach. Intell. 7: 17–34.Google Scholar
  42. Grimson, W.E.L., and Pavlidis, T. 1985. Discontinuity detection for visual surface reconstruction. Comput. Vision, Graph., Image Process. 30: 316–330.Google Scholar
  43. Grünbaum, B., and Shephard, G.C. 1987. Tilings and Patterns. W.H. Freeman: New York.Google Scholar
  44. Hannah, M.J. 1980. Bootstrap stereo. Proc. DARPA Image Underst. Work. College Park, MD, pp. 201–208.Google Scholar
  45. Haralick, R.M. 1984. Digital step edges from zero crossings of second directional differences. IEEE Trans. Patt. Anal. Mach. Intell. 6: 58–68.Google Scholar
  46. Heeger, D.J. 1987. Optical flow using spatiotemporal filters. Intern. J. Comput. Vision 1: 279–301.Google Scholar
  47. Hildreth, E. 1983. The detection of intensity changes by computer and biological vision system. Comut. Vision, Graph., Image Process. 22: 1–27.Google Scholar
  48. Hildreth, E., 1984. The Measurement of Visual Motion. MIT Press: Cambridge, MA.Google Scholar
  49. Hoaglin, D.C., Mosteller, F., and Tukey, J.W., Eds., 1983. Understanding Robust and Exploratory Data Analysis. John Wiley: New York.Google Scholar
  50. Hoff, W., and Ahuja, N. 1989. Surfaces from stereo: Integrating feature matching, disparity estimation, and contour detection. IEEE Trans. Patt. Anal. Mach. Intell. 11: 121–136.Google Scholar
  51. Huertas, A., and Medioni, G. 1986. Detection of intensity changes with subpixel accuracy using Laplacian-Gaussian masks. IEEE Trans. Patt. Anal. Mach. Intell. 8: 651–664.Google Scholar
  52. Jepson, A.D., and Jenkin, M.R.M. 1989. The fast computation of disparity from phase differences. Proc. IEEE Conf. Comput. Vision Patt. Recog., San Diego, pp. 398–403.Google Scholar
  53. Kass, M. 1983. Computing visual correspondence. Proc DARPA Image Underst. Work., Arlington, VA, pp. 54–60.Google Scholar
  54. Kass, M. 1987. Linear image features in stereopsis. Intern. J. Comput. Vision 1: 347–368.Google Scholar
  55. Kass, M., and Witkin, A. 1985. Analyzing oriented patterns. Proc. 9th Intern. Joint Conf. Artif. Intell., Los Angeles, pp. 944–952.Google Scholar
  56. Kass, M., and Witkin, A. 1987. Analyzing oriented patterns. Comput. Vision Graph. Image Process. 37: 362–385.Google Scholar
  57. Koenderink, J.J., and vanDoorn, A.J. 1976. The singularities of the visual mapping. Biological Cybernetics 24: 51–59.Google Scholar
  58. Krol, J.D., and van deGrind, W.A. 1980. The double-nail illusion: Experiments on binocular vision with nails, needles, and pins. Perception 9: 651–669.Google Scholar
  59. Krotkov, E.P. 1986. Visual hyperacuity: Representation and computation of high precision position information. Comput. Vision, Graph., Image Process. 33: 99–115.Google Scholar
  60. Lawton, D. 1983. Processing translational motion sequences. Comput. Vision Graph., Image Process. 22: 116–144.Google Scholar
  61. Levine, M.D., O'Handley, D.A., and Yagi, G.M. 1973. Computer determination of depth maps. Comput. Graph. Image Process. 2: 131–150.Google Scholar
  62. Little, J., Bültoff, and Poggio, T. 1987. Parallel optical flow computation. Proc. DARPA Image Underst. Work., Los Angeles, pp. 915–920.Google Scholar
  63. Little, J., and Gillett, W. 1990. Direct evidence for occlusion in stereo and motion. 1st Europ. Conf. Comput. Vision, Antibes, France, pp. 336–340.Google Scholar
  64. Marr, D., and Hildreth, E. 1980. Theory of edge detection. Phil. Trans. Roy. Soc. London B 207: 187–217.Google Scholar
  65. Marr, D., Palm, G., and Poggio, T. 1978. Analysis of a cooperative stereo algorithm. Biological Cybernetics 28: 223–239.Google Scholar
  66. Marr, D., and Poggio, T. 1976. Cooperative computation of stereo disparity. Science 194: 283–287.Google Scholar
  67. Marr, D., and Poggio, T. 1979. A computational theory of human stereo vision. Phil. Trans. Roy. Soc. London B 204: 301–328.Google Scholar
  68. Matsuyama, T., Miura, S., and Nagao, M. 1983. Structural analysis of natural textures by Fourier transformation. Comput. Vision Graph., Image Process. 24: 347–362.Google Scholar
  69. Mayhew, J.E.W., and Frisby, J.P. 1980. The computation of binocular edges. Perception 9: 69–86.Google Scholar
  70. Mayhew, J.E.W., and Frisby, J.P. 1981. Psychophysical and computational studies towards a theory of human stereopsis. Artificial Intelligence 17: 349–385.Google Scholar
  71. Medioni, G., and Nevatia, R. 1985. Segment-based stereo matching. Comput. Vision, Graph., Image Process. 31: 2–18.Google Scholar
  72. Mohan, R., Medioni, G., and Nevatia, R. 1989. Stereo error detection, correction, and evaluation. IEEE Trans. Patt. Anal. Mach. Intell. 11: 113–120.Google Scholar
  73. Moravec, H.P. 1977. Towards automatic visual obstacle avoidance. Proc. Intern. Joint Conf. Artif. Intell. p. 584.Google Scholar
  74. Moravec, H.P. 1981. Rover visual obstacle avoidance. Proc. 7th Intern. Joint Conf. Artif. Intell., Vancouver, pp. 785–790.Google Scholar
  75. Mori, K., Kidode, M., and Asada, H. 1973. Comput. Graph. Image Process. 2: 393–401.Google Scholar
  76. Mowforth, P., Mayhew, J.E.W., and Frisby, J.P. 1981. Vergence eye movements made in response to spatial-frequency-filtered randomdot stereograms. Perception 10: 299–304.Google Scholar
  77. Nalwa, V.S. 1987. Edge-detector resolution improvement by image interpolation. IEEE Trans. Patt. Anal. Mach. Intell. 9(3): 446–451.Google Scholar
  78. Nalwa, V.S., and Binford, T.O. 1986. On detecting edges. IEEE Trans. Patt. Anal. Mach. Intell. 8: 699–714.Google Scholar
  79. Nevatia, R. 1976. Depth measurement by motion stereo. Comput. Graph. Image Process. 5: 203–214.Google Scholar
  80. Nielsen, K.R.K., and Poggio, T. 1983. Vertical image registration in stereopsis. Mass. Inst. of Technol., Artif. Intell. Lab. Memo 743.Google Scholar
  81. Nishihara, H.K. 1984. Practical real-time imaging stereo matcher. Optical Engineering 23: 536–545.Google Scholar
  82. Ohta, Y., and Kanade, T. 1985. Stereo by intra- and inter-line scanline search using dynamic programming. IEEE Trans. Patt. Anal. Mach. Intell. 7: 139–154.Google Scholar
  83. Otto, G.P., and Chau, T.K.W. 1989. “Region-growing” algorithm for matching of terrain images. Image Vision Comput. 7(2): 83–94.Google Scholar
  84. Pearson, D.E., and Robinson, J.A. 1985. Visual comunication at very low data rates. Proc. IEEE 73: 795–812.Google Scholar
  85. Poggio, G.F., and Poggio, T. 1984. The analysis of stereopsis. Ann. Rev. Neuroscience 7: 379–412.Google Scholar
  86. Pollard, S.B., Mayhew, J.E.W., and Frisby, J.P. 1985. PMF: A stereo correspondence algorithm using a disparity gradient limit. Perception 14: 449–470.Google Scholar
  87. Pratt, W.K. 1978. Digital Image Processing, John Wiley: New York.Google Scholar
  88. Prazdny, K. 1985. The detection of binocular disparities. Biological Cybernetics 52: 93–99.Google Scholar
  89. Quam, L.H. 1984. Hierarchical warp stereo. Proc. DARPA Image Underst. Work., New Orleans, pp. 149–155.Google Scholar
  90. Rosenfeld, A. 1979. Digital topology. Amer. Math. Monthly 86: 621–630.Google Scholar
  91. Rourke, C.P., and Sanderson, B.J. 1982. Introduction to Piecewise-Linear Topology. Springer-Verlag: Berlin.Google Scholar
  92. Rousseeuw, P.J., and Leroy, A.M. 1987. Robust Regression and Outlier Detection. John Wiley: New York.Google Scholar
  93. Schunck, B.G. 1989. Image flow segmentation and estimation by constraint line clustering. IEEE Trans. Patt. Anal. Mach. Intell. 11: 1010–1027.Google Scholar
  94. Scott, G.L. 1988. Local and Global Interpretation of Moving Images. Pitman: London.Google Scholar
  95. Serra, J. 1982. Image Analysis and Mathematical Morphology. Academic Press: New York.Google Scholar
  96. Shahraray, B., and Brown, M.K. 1988. Robust depth estimation from optical flow. Proc. 2nd Intern. Conf. Comput. Vision, Tampa, FL, pp. 641–650.Google Scholar
  97. Sher, D.B. 1987a. A probabilistic approach to low-level vision. Ph.D. thesis, University of Rochester, Dept. Comp. Sci. Techn. Rep. 232.Google Scholar
  98. Sher, D.B. 1987b. Tunable facet model likelihood generators for boundary pixel detection. Proc. IEEE Comput. Soc. Work. Comput. Vision, pp. 35–40.Google Scholar
  99. Spacek, L.A. 1986. Edge detection and motion detection. Image Vision Comput. 4: 43–56.Google Scholar
  100. Stewart, C.V., and Dyer, C.R. 1988. The trinocular general support algorithm: A three-camera stereo algorithm for overcoming binocular matching erros. Proc. 2nd Intern. Conf. Comjput. Vision, Tampa, FL, pp. 134–138.Google Scholar
  101. Thomas, G.A. 1987. Television motion measurement for DATV and other applications. BBC Res. Dept. Report BBC RD 1987/11.Google Scholar
  102. Trivedi, H.P., and Lloyd, S.A. 1985. The role of disparity gradient in stereo vision. Perception 14: 685–690.Google Scholar
  103. Vilnrotter, F.M., Nevatia, R., and Price, K.E. 1986. Structural analysis of natural textures. IEEE Trans. Patt. Anal. Mach. Intell. 8: 76–89.Google Scholar
  104. Voorhees, H., and Poggio, T. 1987. Detecting textons and texture boundaries in natural images. Proc. 1st Intern. Conf. Comput. Vision, London, pp. 250–258.Google Scholar
  105. Watt, R.J., and Morgan, M.J. 1983. Mechanisms responsible for the assessment of visual location: Theory and evidence. Vision Research 23: 97–109.Google Scholar
  106. Witkin, A., Terzopoulos, D., and Kass, M. 1987. Signal matching through scale space. Intern. Comput. Vision 1(2): 133–144.Google Scholar
  107. Williams, D.R. 1988. Topography of the foveal cone mosaic in the living human eye. Vision Research 28(3): 433–454.Google Scholar
  108. Yeshurun, Y., and Schwartz, E.L. 1989. Cepstral filtering on a columnar image architecture: A fast algorithm for binocular stereo segmentation. IEEE Trans. Patt. Anal. Mach. Intell. 11(7): 750–767.Google Scholar
  109. Young, R.A. 1986. Locating industrial parts with subpixel accuracies. Proc. Soc. Photo-Optical Instr. Eng. 728: 2–9.Google Scholar
  110. Zucker, S.W. 1985. Early orientation selection: Tangent fields and the dimensionality of their support. Comput. Vision, Graph., Image Process. 32: 74–103.Google Scholar

Copyright information

© Kluwer Academic Publishers 1991

Authors and Affiliations

  • Margaret M. Fleck
    • 1
  1. 1.Department of Engineering ScienceOxfordUnited Kingdom

Personalised recommendations