A computational theory of human stereo vision

  • D. Marr
  • T. Poggio
  • Ellen C. Hildreth
  • W. Eric L. Grimson


An algorithm is proposed for solving the stereoscopic matching problem. The algorithm consists of five steps: (1) Each image is filtered at different orientations with bar masks of four sizes that increase with eccentricity; the equivalent filters are one or two octaves wide. (2) Zero-crossings in the filtered images, which roughly correspond to edges, are localized. Positions of the ends of lines and edges are also found. (3) For each mask orientation and size, matching takes place between pairs of zero-crossings or terminations of the same sign in the two images, for a range of disparities up to about the width of the mask’s central region. (4) Wide masks can control vergence movements, thus causing small masks to come into correspondence. (5) When a correspondence is achieved, it is stored in a dynamic buffer, called the 2½-D sketch.


Receptive Field Subjective Contour Receptive Field Size False Target Line Spread Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Barlow, H. B., Blakemore, C. & Pettigrew, J. D. 1967 The neural mechanism of binocular depth discrimination. J. Physiol., Load. 193 327–342.Google Scholar
  2. Bishop, P. Q., Henry, G. H. & Smith, C. J. 1971 Binocular interaction fields of single units in the cat striate cortex. J. Physiol., Lond. 216 39–68.Google Scholar
  3. Blakemore, C. & Campbell, F. W. 1969 On the existence of neurons in the human visual system selectively sensitive to the orientation and size of retinal images. J. Physiol., Lond. 203 237–260.Google Scholar
  4. Blomfield, S. 1973 Implicit features and stereoscopy. Nature, new Biol. 245 256.CrossRefGoogle Scholar
  5. Campbell, F. W. & Robson, J. 1968 Application of Fourier analysis to the visibility of gratings. J. Physiol., Lond. 197 551–566.Google Scholar
  6. Clarke, P. G. H., Donaldson, I. M. L. & Whitteridge, D. 1976 Binocular visual mechanisms in cortical areas I and II of the sheep. J. Physiol., Lond. 256 509–526.Google Scholar
  7. Cowan, J. D. 1977 Some remarks on channel bandwidths for visual contrast detection. Neurosci. Res. Progr. Bull. 15 492–517.Google Scholar
  8. Dev, P. 1975 Perception of depth surfaces in random-dot stereograms: a neural model. Int. J. Man-Machine Stud. 7, 511–528.CrossRefGoogle Scholar
  9. Felton, T. B., Richards, W. & Smith, R. A. Jr. 1972 Disparity processing of spatial frequencies in man. J. Physiol., Lond. 225, 349–362.Google Scholar
  10. Fender, D. & Julesz, B. 1967 Extension of Panum’s fusional area in binocularly stabilized vision. J. opt. Soc. Am. 57, 819–830.CrossRefGoogle Scholar
  11. Foley, J. M., Applebaum, T. H. & Richards, W. A. 1975 Stereopsis with large disparities: discrimination and depth magnitude. Vision Res. 15, 417–422.CrossRefGoogle Scholar
  12. Frisby, J. P. & Clatworthy, J. L. 1975 Learning to see complex random-dot stereograms. Perception 4, 173–178.CrossRefGoogle Scholar
  13. Frisby, J. P. & Mayhew, J. E. W. 1979 Spatial frequency selective masking and stereopsis. (In preparation.)Google Scholar
  14. Georgeson, M. A. & Sullivan, G. D. 1975 Contrast constancy: deblurring in human vision by spatial frequency channels. J. Physiol., Lond. 252, 627–656.Google Scholar
  15. Grimson, W. E. L. & Marr, D. 1979 A computer implementation of a theory of human stereo vision. (In preparation.)Google Scholar
  16. von der Heydt, R., Adorjani, Cs., Hanny, P. & Baumgartner, G. 1978 Disparity sensitivity and receptive field incongruity of units in the cat striate cortex. Exp. Brain Res. 31, 523–545.CrossRefGoogle Scholar
  17. Hines, M. 1976 Line spread function variation near the fovea. Vision Res. 16, 567–572.CrossRefGoogle Scholar
  18. Hirai, Y. & Fukushima, K. 1976 An inference upon the neural network finding binocular correspondence. Trans. IECE J59-D, 133–140.Google Scholar
  19. Hubel, D. H. & Wieset, T. N. 1974 Sequence regularity and geometry of orientation columns in monkey striate cortex. J. comp. Neurol. 158, 267–294.CrossRefGoogle Scholar
  20. Jones, R. 1972 Psychophysical and oculomotor responses of manual and stereoanomalous observers to disparate retinal stimulation. Doctoral dissertation, Ohio State University. Dissertation Abstract N. 72–20970.Google Scholar
  21. Julesz, B. 1960 Binocular depth perception of computer-generated patterns. Bell System Tech. J. 39 1125–1162.Google Scholar
  22. Julesz, B. 1963 Towards the automation of binocular depth perception (automap-1). Proceedings of the IFIPS Congres, Munich 1962 (ed. C. M. Popplewell). Amsterdam: North Holland.Google Scholar
  23. Julesz, B. 1971 Foundations of cyclopean perception. The University of Chicago Press.Google Scholar
  24. Julesz, B. & Chang, J. J. 1976 Interaction between pools of binocular disparity detectors tuned to different disparities. Biol. Cybernetics 22 107–120.CrossRefGoogle Scholar
  25. Julesz, B. & Miller, J. E. 1975 Independent spatial-frequency-tuned channels in binocular fusion and rivalry. Perception 4, 125–143.CrossRefGoogle Scholar
  26. Kaufman, L. 1964 On the nature of binocular disparity. Am. J. Psychol. 77, 393–402.CrossRefGoogle Scholar
  27. Leadbetter, M. R. 1969 On the distributions of times between events in a stationary stream of events. J. R. statist. Soc. B 31, 295–302.Google Scholar
  28. Longuet-Higgins, M. S. 1962 The distribution of intervals between zeros of a stationary random function. Phil. Trans. R. Soc. Lond. A 254 557–599.CrossRefGoogle Scholar
  29. Marr, D. 1974 A note on the computation of binocular disparity in a symbolic, low-level visual processor. M.I.T. A.I. Lab. Memo 327.Google Scholar
  30. Marr, D. 1976 Early processing of visual information. Phil. Trans. R. Soc. Lond. B 275 483–524.CrossRefGoogle Scholar
  31. Marr, D. 1977 Representing visual information. AAAS 143rd Annual Meeting. Symposium on Some Mathematical Questions in Biology, February. Published in Lectures on mathematics in the life sciences 10 101–180 (1978) Also available as M.I.T. A.I. Lab. Memo 415.Google Scholar
  32. Marr, D. & Hildreth, E. 1979 Theory of edge detection. (In preparation.)Google Scholar
  33. Marr, D. & Nishihara, H. K. 1978 Representation and recognition of the spatial organization of three-dimensional shapes. Proc. R. Soc. Lond. B 200, 269–294.CrossRefGoogle Scholar
  34. Marr, D., Palm, G. & Poggio, T. 1978 Analysis of a cooperative stereo algorithm. Biol. Cybernetics 28 223–229.CrossRefGoogle Scholar
  35. Marr, D. & Poggio, T. 1976 Cooperative computation of stereo disparity. Science, N.Y. 194, 283–287.CrossRefGoogle Scholar
  36. Marr, D. & Poggio, T. 1977a A theory of human stereo vision. M.I.T. A.I. Lab. Memo 451.Google Scholar
  37. Marr, D. & Poggio, T. 1977b Theory of human stereopsis. J. opt. Soc. Am. 67, 1400.Google Scholar
  38. Mayhew, J. E. W. & Frisby, J. P. 1976 Rivalrous texture stereograms. Nature, Lond. 264 53–56.CrossRefGoogle Scholar
  39. Mitchell, D. E. 1966 Retinal disparity and diplopia. Vision Res. 6, 441–451.CrossRefGoogle Scholar
  40. Nelson, J. I. 1975 Globality and stereoscopic fusion in binocular vision. J. theor. Biol. 49 1–88.Google Scholar
  41. Nelson, J. I., Kato, H. & Bishop, P. O. 1977 Discrimination of orientation and position disparities by binocularly activated neurons in cat striate cortex. J. Neurophysiol. 40 260–283.Google Scholar
  42. Papoulis, A. 1968 Systems and transforms with applications in optics. New York: McGraw Hill.Google Scholar
  43. Pettigrew, J. D., Nikara, T. & Bishop, P. O. 1968 Binocular interaction on single units in cat striate cortex: simultaneous stimulation by single moving slit with receptive fields in correspondence. Exp. Brain Res. 6, 311–410.Google Scholar
  44. Poggio, G. F. & Fischer, B. 1978 Binocular interaction and depth sensitivity of striate and prestriate cortical neurons of the behaving rhesus monkey. J. Neurophysiol. 40 1392–1405.Google Scholar
  45. Rashbass, C. & Westheimer, G. 1961a Disjunctive eye movements. J. Physiol., Lond. 159 339–360.Google Scholar
  46. Rashbass, C. & Westheimer, G. 1961b Independence of conjunctive and disjunctive eye movements. J. Physiol., Lond. 159 361–364.Google Scholar
  47. Rice, S. O. 1945 Mathematical analysis of random noise. Bell Syst. Tech. J. 24 46–156.Google Scholar
  48. Richards, W. 1970 Stereopsis and steroblindness. Exp. Brain Res. 10 380–388.CrossRefGoogle Scholar
  49. Richards, W. 1971 Anomalous stereoscopic depth perception. J. opt. Soc. Am. 61 410–414.CrossRefGoogle Scholar
  50. Richards, W. 1975 Visual space perception. In Handbook of Perception, vol. 5, Seeing, ch. 10, pp. 351–386 (ed E. C. Carterette & M. D. Freidman). New York: Academic Press.Google Scholar
  51. Richards, W. A. 1977 Stereopsis with and without monocular cues. Vision Res. 17 967–969.CrossRefGoogle Scholar
  52. Richards, W. A. & Regan, D. 1973 A stereo field map with implications for disparity processing. Invest. Ophthal. 12 904–909.Google Scholar
  53. Riggs, L. A. & Niehl, E. W. 1960 Eye movements recorded during convergence and divergence. J. opt. Soc. Am. 50 913–920.CrossRefGoogle Scholar
  54. Saye, A. & Frisby, J. P. 1975 The role of monocularly conspicuous features in facilitating stereopsis from random-dot stereograms. Perception 4 159–171.CrossRefGoogle Scholar
  55. Schiller, P. H., Finlay, B. L. & Volman, S. F. 1977 Quantitative studies of single-cell properties in monkey striate cortex. III. Spatial frequency. J. Neurophysiol. 39 1334–1351.Google Scholar
  56. Sperling, G. 1970 Binocular vision: a physical and a neural theory. Am. J. Psychol. 83 461–534.CrossRefGoogle Scholar
  57. Sugie, N. & Suwa, M. 1977 A scheme for binocular depth perception suggested by neuro-physiological evidence. Biol. Cybernetics 26 1–15.CrossRefGoogle Scholar
  58. Waltz, D. 1975 Understanding line drawings of scenes with shadows. In The psychology of computer vision (ed. P. H. Winston), pp. 19–91. New York: McGraw-Hill.Google Scholar
  59. Westheimer, G. & Mitchell, D. E. 1969 The sensory stimulus for disjunctive eye movements. Vision Res. 9, 749–755.CrossRefGoogle Scholar
  60. Williams, R. H. & Fender, D. H. 1977 The synchrony of binocular saccadic eye movements. Vision Res. 17, 303–306.CrossRefGoogle Scholar
  61. Wilson, H. R. 1978a Quantitative characterization of two types of line spread function near the fovea. Vision Res. 18, 971–981.CrossRefGoogle Scholar
  62. Wilson, H. R. 1978b Quantitative prediction of line spread function measurements: implications for channel bandwidths. Vision Res. 18, 493–496.CrossRefGoogle Scholar
  63. Wilson, H. R. & Bergen, J. R. 1979 A four mechanism model for spatial vision. Vision Res. (in the press).Google Scholar
  64. Wilson, H. R. & Giese, S. C. 1977 Threshold visibility of frequency gradient patterns. Vision Res. 17, 1177–1190.CrossRefGoogle Scholar
  65. Wilson, H. R., Phillips, G., Rentschler, I. & Hilz, R. 1979 Spatial probability summation and disinhibition in psychophysically measured line spread functions. Vision Res. (in the press).Google Scholar
  66. Ferster D (1981): A comparison of binocular depth mechanisms in areas 17 and 18 of the cat visual cortex. J Physiol 311: 623–655Google Scholar
  67. Grimson WEL (1981): From Images to Surfaces: A Computational Study of the Human Early Visual System. Cambridge, MA: MIT PressGoogle Scholar
  68. Grimson WEL (1985): Computational experiments with a feature based stereo algorithm. IEEE Trans Patt Anal Machine Intell PAMI 7: 17–34CrossRefGoogle Scholar
  69. Kidd AL, Mayhew JEW, Frisby JP (1979): Texture contours can facilitate stereopsis by initiating vergence eye movements. Nature 280: 829–832CrossRefGoogle Scholar
  70. Marr D, Palm G, Poggio T (1978): Analysis of a cooperative stereo algorithm. Biol Cybern 28: 223–239CrossRefGoogle Scholar
  71. Mayhew JEW, Frisby JP (1979): Convergent disparity discriminations in narrow-bandfiltered random-dot stereograms. Vision Res 19: 63–71CrossRefGoogle Scholar
  72. Mayhew JEW, Frisby JP (1981): Psychophysical and computational studies toward a theory of human stereopsis. Artif Intell 16: 349–385CrossRefGoogle Scholar
  73. Mowforth P, Mayhew JEW, Frisby JP (1981): Vergence eye movements made in response to spatial-frequency-filtered random-dot stereograms. Perception 10: 299–304CrossRefGoogle Scholar
  74. Poggio GF, Poggio T (1984): The analysis of stereopsis. Ann Rev Neurosci 7: 379–412CrossRefGoogle Scholar
  75. Schumer RA, Julesz B (1982): Disparity limits in bandpass random-grating stereograms. Invest Ophthal Visual Sci 22 (Suppl) 272Google Scholar

Copyright information

© Birkhäuser Boston 1991

Authors and Affiliations

  • D. Marr
    • 1
  • T. Poggio
    • 2
  • Ellen C. Hildreth
    • 3
    • 4
  • W. Eric L. Grimson
    • 5
  1. 1.Psychology DepartmentM.I.T.CambridgeUSA
  2. 2.Max-Planck-Institut für Biologische KybernetikTübingenGermany
  3. 3.Brain and Cognitive SciencesArtificial Intelligence LaboratoryUSA
  4. 4.Center for Biological Information ProcessingMassachusetts Institute of TechnologyCambridgeUSA
  5. 5.Electrical Engineering and Computer Sciences, Artificial Intelligence LaboratoryMassachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations