Biological Cybernetics

, Volume 98, Issue 4, pp 305–337 | Cite as

Globally consistent depth sorting of overlapping 2D surfaces in a model using local recurrent interactions

  • Axel Thielscher
  • Heiko Neumann
Open Access
Original Paper


The human visual system utilizes depth information as a major cue to group together visual items constituting an object and to segregate them from items belonging to other objects in the visual scene. Depth information can be inferred from a variety of different visual cues, such as disparity, occlusions and perspective. Many of these cues provide only local and relative information about the depth of objects. For example, at occlusions, T-junctions indicate the local relative depth precedence of surface patches. However, in order to obtain a globally consistent interpretation of the depth relations between the surfaces and objects in a visual scene, a mechanism is necessary that globally propagates such local and relative information. We present a computational framework in which depth information derived from T-junctions is propagated along surface contours using local recurrent interactions between neighboring neurons. We demonstrate that within this framework a globally consistent depth sorting of overlapping surfaces can be obtained on the basis of local interactions. Unlike previous approaches in which locally restricted cell interactions could merely distinguish between two depths (figure and ground), our model can also represent several intermediate depth positions. Our approach is an extension of a previous model of recurrent V1–V2 interaction for contour processing and illusory contour formation. Based on the contour representation created by this model, a recursive scheme of local interactions subsequently achieves a globally consistent depth sorting of several overlapping surfaces. Within this framework, the induction of illusory contours by the model of recurrent V1–V2 interaction gives rise to the figure-ground segmentation of illusory figures such as a Kanizsa square.


Depth Layer Bipole Cell Model Stage Illusory Contour Surface Contour 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Adelson EH, Anandan P (1990) Ordinal characteristics of transparency. AAAI workshop on qualitative vision, pp 77–81Google Scholar
  2. Anderson BL (1997). A theory of illusory lightness and transparency in monocular and binocular images: the role of contour junctions. Perception 26(4): 419–453 PubMedCrossRefGoogle Scholar
  3. Banks MS, Gepshtein S and Landy MS (2004). Why is spatial stereoresolution so low?. J Neurosci 24(9): 2077–2089 PubMedCrossRefGoogle Scholar
  4. Baumann R, v. d. Zwan R and Peterhans E (1997). Figure-ground segregation at contours: a neural mechanism in the visual cortex of the alert monkey. Eur J Neurosci 9(6): 1290–1303 PubMedCrossRefGoogle Scholar
  5. Baumann R, Peterhans E and Zwan R (1997). Figure-ground segregation at contours: a neural mechanism in the visual cortex of the alert monkey. Eur J Neurosci 9: 1290–1303 PubMedCrossRefGoogle Scholar
  6. Bayerl P, Neumann H (2004) Disambiguating visual motion through contextual feedback modulation. Neu Comput 16:2041–2066Google Scholar
  7. Baylis GC and Driver J (2001). Shape-coding in IT cells generalizes over contrast and mirror reversal, but not figure-ground reversal. Nat Neurosci 4(9): 937–942 PubMedCrossRefGoogle Scholar
  8. Boselie F (1994). Local and global factors in visual occlusion. Perception 23: 517–528 PubMedCrossRefGoogle Scholar
  9. Crick F and Koch C (1998). Constraints on cortical and thalamic projections: the no-strong-loops hypothesis. Nature 391: 245–250 PubMedCrossRefGoogle Scholar
  10. Dev P (1975). Perception of depth surfaces in random-dot stereograms: a neural model. Int J Man-Machine Stud 7: 511–528 CrossRefGoogle Scholar
  11. Eckhorn R (2000). Neural mechanisms of visual feature grouping. Neurol Neurochir Pol 34: 27–42 PubMedGoogle Scholar
  12. Engel AK, Fries P and Singer W (2001). Dynamic predictions: oscillations and synchrony in top–down processing. Nat Rev Neurosci 2(10): 704–716 PubMedCrossRefGoogle Scholar
  13. Felleman DJ and Essen DC (1991). Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1: 1–47 PubMedCrossRefGoogle Scholar
  14. Finkel LH and Sajda P (1992). Object discrimination based on depth-from-occulsion. Neural Comput 4: 901–921 CrossRefGoogle Scholar
  15. Francis G, Grossberg S and Mingolla E (1994). Cortical dynamics of feature binding and reset: control of visual persistence. Vis Res 34(8): 1089–1104 PubMedCrossRefGoogle Scholar
  16. Frien A and Eckhorn R (2000). Functional coupling shows stronger stimulus dependency for fast oscillations than for low-frequency components in striate cortex of awake monkey. Eur J Neurosci 12(4): 1466–1478 PubMedCrossRefGoogle Scholar
  17. Fukushima K (2001). Recognition of partly occluded patterns: a neural network model. Biol Cybern 84: 251–259 PubMedCrossRefGoogle Scholar
  18. Gilbert CD and Wiesel TN (1989). Columnar specificity of intrinisic horizontal and corticocortical connections in cat visual cortex. J Neurosci 9(7): 2432–2442 PubMedGoogle Scholar
  19. Grossberg S (1980). How does a brain build a cognitive code. Psychol Rev 87(1): 1–51 PubMedCrossRefGoogle Scholar
  20. Grossberg S (1991). Why do parallel cortical systems exist forthe perception of static form and moving form. Percept Psychophys 49(2): 117–141 PubMedGoogle Scholar
  21. Grossberg S (1993). A solution of the figure-ground problem for biological vision. Neural Netw 6: 463–483 CrossRefGoogle Scholar
  22. Grossberg S and Grunewald A (2002). Temporal dynamics of binocular disparity processing with corticogeniculate interactions. Neural Netw 15: 181–200 CrossRefGoogle Scholar
  23. Grossberg S and Mingolla E (1985). Neural dynamics of perceptual grouping: textures, boundaries and emergent segmentations. Percept Psychophys 38(2): 141–171 PubMedGoogle Scholar
  24. Heider B, Meskenaite V and Peterhans E (2000). Anatomy and physiology of a neural mechanism defining depth order and contrast polarity at illusory contours. Eur J Neurosci 12: 4117–4130 PubMedCrossRefGoogle Scholar
  25. Heider B, Spillmann L and Peterhans E (2002). Stereoscopic illusory contours—cortical neuron responses and human perception. J Cogn Neurosci 14(7): 1018–1029 PubMedCrossRefGoogle Scholar
  26. Heitger F, v.d. Heydt R, Peterhans E, Rosenthaler L and Kübler O (1998). Simulation of neural contour mechanisms: representing anomalous contours. Image Vis Comput 6: 407–421 CrossRefGoogle Scholar
  27. Hirsch JA and Gilbert CD (1991). Synaptic physiology of horizontal connections in the cat’s visual cortex. J Neurosci 11: 1800–1809 PubMedGoogle Scholar
  28. Howard IP (2003). Neurons that respond to more than one depth cue. Trends Neurosci 26(10): 515–517 PubMedCrossRefGoogle Scholar
  29. Howard IP and Duke PA (2003). Monocular transparency generates quantitative depth. Vis Res 43: 2615–2621 PubMedCrossRefGoogle Scholar
  30. Hubel DH and Wiesel TN (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160: 106–154 PubMedGoogle Scholar
  31. Hubel DH and Wiesel TN (1965). Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J Neurophysiol 28: 229–289 PubMedGoogle Scholar
  32. Hupe JM, James AC, Payne BR, Lomber SG, Girard P and Bullier J (1998). Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature 394(6695): 784–787 PubMedCrossRefGoogle Scholar
  33. Kanizsa G (1979). Organization in vision: essays on Gestalt perception. Praeger, New York Google Scholar
  34. Kellman PJ and Shipley TF (1991). A theory of visual interpolation in object perception. Cogn Psychol 23(2): 141–221 CrossRefPubMedGoogle Scholar
  35. Kellman PJ, Yin C and Shipley TF (1998). A common mechanism for illusory and occluded object completion. J Exp Psychol Hum Percept Perform 24(3): 859–869 PubMedCrossRefGoogle Scholar
  36. Kelly F and Grossberg S (2000). Neural dynamics of 3D surface perception: figure-ground separation and lightness perception. Percept Psychophys 62(8): 1596–1618 PubMedGoogle Scholar
  37. Koenderink JJ and v. Doorn A (1982). The shape of smooth objects and the way contours end. Percept Psychophys 11: 129–137 CrossRefGoogle Scholar
  38. Kovacs G, Vogels R and Orban GA (1995). Selectivity of macaque inferior temporal neurons for partially occluded shapes. J Neurosci 15(3): 1984–1997 PubMedGoogle Scholar
  39. Kumaran K, Geiger D and Gurvits L (1996). Illusory surface perception and visual organization. Netw Comput Neural Syst 7: 33–60 CrossRefGoogle Scholar
  40. Liu X, Wang DL (1999) Perceptual organization based on temporal dynamics. In: Paper presented at the IJCNN’99, Washington DC, USAGoogle Scholar
  41. Marr D and Poggio T (1979). A computational theory of human stereo vision. Proc R Soc Lond B 204: 301–328 PubMedGoogle Scholar
  42. McDermott J and Adelson EH (2004a). The geometry of the occluding contour and its effect on motion interpretation. J Vis 4(10): 944–954 PubMedCrossRefGoogle Scholar
  43. McDermott J and Adelson EH (2004b). Junctions and cost functions in motion interpretation. J Vis 4(7): 552–563 PubMedCrossRefGoogle Scholar
  44. Mignard M and Malpeli JG (1991). Paths of information flow through visual cortex. Science 251(4998): 1249–1251 PubMedCrossRefGoogle Scholar
  45. Mumford DB (1994) Neuronal architectures for pattern-theoretic problems. In: Koch C, Davis J (eds) Large-scale neuronal theories of the brain. MIT Press, Cambridge, pp 125–152Google Scholar
  46. Nakayama K, Shimojo S and Ramachandran VS (1990). Transparency: relation to depth, subjective contours, luminance, and neon color spreading. Perception 19(4): 497–513 PubMedCrossRefGoogle Scholar
  47. Nakayama K, Shimojo S and Silverman GH (1989). Stereoscopic depth: its relation to image segmentation, grouping, and the recognition of occluded objects. Perception 18(1): 55–68 PubMedCrossRefGoogle Scholar
  48. Neumann H and Mingolla E (2001). Computational neural models of spatial integration in perceptual grouping. In: Shipley, TF and Kellman, PJ (eds) From fragments to objects—segmentation and grouping in vision., pp 353–400. Elsevier, Amsterdam CrossRefGoogle Scholar
  49. Neumann H and Sepp W (1999). Recurrent V1–V2 interaction in early visual boundary processing. Biol Cybern 81: 425–444 PubMedCrossRefGoogle Scholar
  50. Nishina S, Okada M and Kawato M (2003). Spatio-temporal dynamics of depth propagation on uniform region. Vis Res 42: 2493–2503 CrossRefGoogle Scholar
  51. Ohzawa I, DeAngelis GC and Freeman RD (1997). The neural coding of stereoscopic depth. Neuroreport 8(3): iii–xiii PubMedCrossRefGoogle Scholar
  52. Peterhans E (1997) Functional organization of area V2 in the awake monkey. In: Rockland KS, Kaas JH, Peters A (eds) Extrastriate cortex in primates, vol 12. Plenum Press, New YorkGoogle Scholar
  53. Peterhans E and Heitger F (2001). Simulation of neuronal responses definining depth order and contrast polarity at illusory contours in monkey area V2. J Comput Neurosci 10(2): 195–211 PubMedCrossRefGoogle Scholar
  54. Pianta MJ and Gillam BJ (2003). Paired and unpaired features can be equally effective in human depth perception. Vis Res 43: 1–6 PubMedCrossRefGoogle Scholar
  55. Poggio GF, Gonzalez F and Krause F (1988). Stereoscopic mechanisms in monkey visual cortex: binocular correlation and disparity selectivity. J Neurosci 8(12): 4531–4550 PubMedGoogle Scholar
  56. Przybyszewski AW, Gaska JP, Foote W and Pollen DA (2000). Striate cortex increases contrast gain of macaque LGN neurons. Vis Neurosci 17(4): 485–494 PubMedCrossRefGoogle Scholar
  57. Qiu FT and v.d. Heydt R (2005). Figure and ground in the visual cortex: V2 combines stereoscopic cues with Gestalt rules. Neuron 47(1): 155–166 PubMedCrossRefGoogle Scholar
  58. Regan D, Erkelens CJ and Collewijn H (1986). Visual field defects for vergence eye movements and for stereomotion perception. Invest Ophthalmol Vis Sci 27: 806–819 PubMedGoogle Scholar
  59. Raizada RD and Grossberg S (2003). Towards a theory of the laminar architecture of cerebral cortex: computational clues from the visual system. Cereb Cortex 13: 100–113 PubMedCrossRefGoogle Scholar
  60. Rubin N (2001a). Figure and ground in the brain. Nat Neurosci 4(9): 857–858 PubMedCrossRefGoogle Scholar
  61. Rubin N (2001b). The role of junctions in surface completion and contour matching. Perception 30: 339–366 PubMedCrossRefGoogle Scholar
  62. Sajda P and Finkel LH (1995). Intermediate-level visual representations and the construction of surface perception. J Cogn Neurosci 7(2): 267–291 CrossRefGoogle Scholar
  63. Salin PA and Bullier J (1995). Corticocortical connections in the visual system: structure and function. Physiol Rev 75(1): 107–154 PubMedGoogle Scholar
  64. Sandell JH and Schiller PH (1982). Effect of cooling area 18 on striate cortex cells in the squirrel monkey. J Neurophysiol 48(1): 38–48 PubMedGoogle Scholar
  65. Shadlen MN and Movshon JA (1999). Synchrony unbound: a critical evaluation of the temporal binding hypothesis. Neuron 24: 67–77 PubMedCrossRefGoogle Scholar
  66. Shimojo S and Nakayama K (1990). Real world occlusion constraints and binocular rivalry. Vis Res 30(1): 69–80 PubMedCrossRefGoogle Scholar
  67. Shipley TF and Kellman PJ (1990). The role of discontinuities in the perception of subjective figures. Percept Psychophys 48(3): 259–270 PubMedGoogle Scholar
  68. Shipley TF and Kellman PJ (1992). Strength of visual interpolation depends on the ratio of physically specified to total edge length. Percept Psychophys 52(1): 97–106 PubMedGoogle Scholar
  69. Singh M, Huang X (2003) Computing layered surface representations: an algorithm for detecting and separating transparent overlays. In: Paper presented at the IEEE CVPR’03, Wisconsin USAGoogle Scholar
  70. Smith AT, Singh KD, Williams AL and Greenlee MW (2001). Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex. Cereb Cortex 11: 1182–1190 PubMedCrossRefGoogle Scholar
  71. Sporns O, Tononi G and Edelman GM (1991). Modeling perceptual grouping and figure-ground segregation by means of active reentrant connections. PNAS 88: 129–133 PubMedCrossRefGoogle Scholar
  72. Spratling MW and Johnson MH (2001). Dendritic inhibition enhances neural coding properties. Cereb Cortex 11: 1144–1149 PubMedCrossRefGoogle Scholar
  73. Thielscher A and Neumann H (2003). Neural mechanisms of cortico-cortical interaction in texture boundary detection: a modeling approach. Neuroscience 122: 921–939 PubMedCrossRefGoogle Scholar
  74. Thomas OM, Cumming BG and Parker AJ (2002). A specialization for relative disparity in V2. Nat Neurosci 5(5): 472–478 PubMedCrossRefGoogle Scholar
  75. Tse PU (1999). Volume completion. Cogn Psychol 39: 37–68 CrossRefPubMedGoogle Scholar
  76. Tyler CW and Kontsevich LL (1995). Mechanisms of stereoscopic processing: stereoattention and surface perception in depth reconstruction. Perception 24: 127–153 PubMedCrossRefGoogle Scholar
  77. v. d. Heydt R, Heitger F and Peterhans E (1993). Perception of occluding contours: neural mechanisms and a computational model. Biomed Res 14: 1–6 Google Scholar
  78. v. d. Heydt R, Peterhans E and Baumgartner G (1984). Illusory contours and cortical neuron responses. Science 224(4654): 1260–1262 CrossRefGoogle Scholar
  79. Williams LR and Hanson AR (1996). Perceptual completion of occluded surfaces. Comput Vis Image Understand 64(1): 1–20 CrossRefGoogle Scholar
  80. Williamson JR (1996). Neural network for dynamic binding with graph representation: from, linking and depth-from-occlusion. Neural Comput 8: 1203–1225 PubMedCrossRefGoogle Scholar
  81. Zhou H, Friedman HS and v.d. Heydt R (2000). Coding of border ownership in monkey visual cortex. J Neurosci 20(17): 6594–6611 PubMedGoogle Scholar

Copyright information

© Springer-Verlag 2008

Authors and Affiliations

  1. 1.High-Field Magnetic Resonance CenterMax Planck Institute for Biological CyberneticsTübingenGermany
  2. 2.Department of Neural Information ProcessingUniversity of UlmUlmGermany

Personalised recommendations