Abstract
The human visual system utilizes depth information as a major cue to group together visual items constituting an object and to segregate them from items belonging to other objects in the visual scene. Depth information can be inferred from a variety of different visual cues, such as disparity, occlusions and perspective. Many of these cues provide only local and relative information about the depth of objects. For example, at occlusions, T-junctions indicate the local relative depth precedence of surface patches. However, in order to obtain a globally consistent interpretation of the depth relations between the surfaces and objects in a visual scene, a mechanism is necessary that globally propagates such local and relative information. We present a computational framework in which depth information derived from T-junctions is propagated along surface contours using local recurrent interactions between neighboring neurons. We demonstrate that within this framework a globally consistent depth sorting of overlapping surfaces can be obtained on the basis of local interactions. Unlike previous approaches in which locally restricted cell interactions could merely distinguish between two depths (figure and ground), our model can also represent several intermediate depth positions. Our approach is an extension of a previous model of recurrent V1–V2 interaction for contour processing and illusory contour formation. Based on the contour representation created by this model, a recursive scheme of local interactions subsequently achieves a globally consistent depth sorting of several overlapping surfaces. Within this framework, the induction of illusory contours by the model of recurrent V1–V2 interaction gives rise to the figure-ground segmentation of illusory figures such as a Kanizsa square.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Adelson EH, Anandan P (1990) Ordinal characteristics of transparency. AAAI workshop on qualitative vision, pp 77–81
Anderson BL (1997). A theory of illusory lightness and transparency in monocular and binocular images: the role of contour junctions. Perception 26(4): 419–453
Banks MS, Gepshtein S and Landy MS (2004). Why is spatial stereoresolution so low?. J Neurosci 24(9): 2077–2089
Baumann R, v. d. Zwan R and Peterhans E (1997). Figure-ground segregation at contours: a neural mechanism in the visual cortex of the alert monkey. Eur J Neurosci 9(6): 1290–1303
Baumann R, Peterhans E and Zwan R (1997). Figure-ground segregation at contours: a neural mechanism in the visual cortex of the alert monkey. Eur J Neurosci 9: 1290–1303
Bayerl P, Neumann H (2004) Disambiguating visual motion through contextual feedback modulation. Neu Comput 16:2041–2066
Baylis GC and Driver J (2001). Shape-coding in IT cells generalizes over contrast and mirror reversal, but not figure-ground reversal. Nat Neurosci 4(9): 937–942
Boselie F (1994). Local and global factors in visual occlusion. Perception 23: 517–528
Crick F and Koch C (1998). Constraints on cortical and thalamic projections: the no-strong-loops hypothesis. Nature 391: 245–250
Dev P (1975). Perception of depth surfaces in random-dot stereograms: a neural model. Int J Man-Machine Stud 7: 511–528
Eckhorn R (2000). Neural mechanisms of visual feature grouping. Neurol Neurochir Pol 34: 27–42
Engel AK, Fries P and Singer W (2001). Dynamic predictions: oscillations and synchrony in top–down processing. Nat Rev Neurosci 2(10): 704–716
Felleman DJ and Essen DC (1991). Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1: 1–47
Finkel LH and Sajda P (1992). Object discrimination based on depth-from-occulsion. Neural Comput 4: 901–921
Francis G, Grossberg S and Mingolla E (1994). Cortical dynamics of feature binding and reset: control of visual persistence. Vis Res 34(8): 1089–1104
Frien A and Eckhorn R (2000). Functional coupling shows stronger stimulus dependency for fast oscillations than for low-frequency components in striate cortex of awake monkey. Eur J Neurosci 12(4): 1466–1478
Fukushima K (2001). Recognition of partly occluded patterns: a neural network model. Biol Cybern 84: 251–259
Gilbert CD and Wiesel TN (1989). Columnar specificity of intrinisic horizontal and corticocortical connections in cat visual cortex. J Neurosci 9(7): 2432–2442
Grossberg S (1980). How does a brain build a cognitive code. Psychol Rev 87(1): 1–51
Grossberg S (1991). Why do parallel cortical systems exist forthe perception of static form and moving form. Percept Psychophys 49(2): 117–141
Grossberg S (1993). A solution of the figure-ground problem for biological vision. Neural Netw 6: 463–483
Grossberg S and Grunewald A (2002). Temporal dynamics of binocular disparity processing with corticogeniculate interactions. Neural Netw 15: 181–200
Grossberg S and Mingolla E (1985). Neural dynamics of perceptual grouping: textures, boundaries and emergent segmentations. Percept Psychophys 38(2): 141–171
Heider B, Meskenaite V and Peterhans E (2000). Anatomy and physiology of a neural mechanism defining depth order and contrast polarity at illusory contours. Eur J Neurosci 12: 4117–4130
Heider B, Spillmann L and Peterhans E (2002). Stereoscopic illusory contours—cortical neuron responses and human perception. J Cogn Neurosci 14(7): 1018–1029
Heitger F, v.d. Heydt R, Peterhans E, Rosenthaler L and Kübler O (1998). Simulation of neural contour mechanisms: representing anomalous contours. Image Vis Comput 6: 407–421
Hirsch JA and Gilbert CD (1991). Synaptic physiology of horizontal connections in the cat’s visual cortex. J Neurosci 11: 1800–1809
Howard IP (2003). Neurons that respond to more than one depth cue. Trends Neurosci 26(10): 515–517
Howard IP and Duke PA (2003). Monocular transparency generates quantitative depth. Vis Res 43: 2615–2621
Hubel DH and Wiesel TN (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160: 106–154
Hubel DH and Wiesel TN (1965). Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J Neurophysiol 28: 229–289
Hupe JM, James AC, Payne BR, Lomber SG, Girard P and Bullier J (1998). Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature 394(6695): 784–787
Kanizsa G (1979). Organization in vision: essays on Gestalt perception. Praeger, New York
Kellman PJ and Shipley TF (1991). A theory of visual interpolation in object perception. Cogn Psychol 23(2): 141–221
Kellman PJ, Yin C and Shipley TF (1998). A common mechanism for illusory and occluded object completion. J Exp Psychol Hum Percept Perform 24(3): 859–869
Kelly F and Grossberg S (2000). Neural dynamics of 3D surface perception: figure-ground separation and lightness perception. Percept Psychophys 62(8): 1596–1618
Koenderink JJ and v. Doorn A (1982). The shape of smooth objects and the way contours end. Percept Psychophys 11: 129–137
Kovacs G, Vogels R and Orban GA (1995). Selectivity of macaque inferior temporal neurons for partially occluded shapes. J Neurosci 15(3): 1984–1997
Kumaran K, Geiger D and Gurvits L (1996). Illusory surface perception and visual organization. Netw Comput Neural Syst 7: 33–60
Liu X, Wang DL (1999) Perceptual organization based on temporal dynamics. In: Paper presented at the IJCNN’99, Washington DC, USA
Marr D and Poggio T (1979). A computational theory of human stereo vision. Proc R Soc Lond B 204: 301–328
McDermott J and Adelson EH (2004a). The geometry of the occluding contour and its effect on motion interpretation. J Vis 4(10): 944–954
McDermott J and Adelson EH (2004b). Junctions and cost functions in motion interpretation. J Vis 4(7): 552–563
Mignard M and Malpeli JG (1991). Paths of information flow through visual cortex. Science 251(4998): 1249–1251
Mumford DB (1994) Neuronal architectures for pattern-theoretic problems. In: Koch C, Davis J (eds) Large-scale neuronal theories of the brain. MIT Press, Cambridge, pp 125–152
Nakayama K, Shimojo S and Ramachandran VS (1990). Transparency: relation to depth, subjective contours, luminance, and neon color spreading. Perception 19(4): 497–513
Nakayama K, Shimojo S and Silverman GH (1989). Stereoscopic depth: its relation to image segmentation, grouping, and the recognition of occluded objects. Perception 18(1): 55–68
Neumann H and Mingolla E (2001). Computational neural models of spatial integration in perceptual grouping. In: Shipley, TF and Kellman, PJ (eds) From fragments to objects—segmentation and grouping in vision., pp 353–400. Elsevier, Amsterdam
Neumann H and Sepp W (1999). Recurrent V1–V2 interaction in early visual boundary processing. Biol Cybern 81: 425–444
Nishina S, Okada M and Kawato M (2003). Spatio-temporal dynamics of depth propagation on uniform region. Vis Res 42: 2493–2503
Ohzawa I, DeAngelis GC and Freeman RD (1997). The neural coding of stereoscopic depth. Neuroreport 8(3): iii–xiii
Peterhans E (1997) Functional organization of area V2 in the awake monkey. In: Rockland KS, Kaas JH, Peters A (eds) Extrastriate cortex in primates, vol 12. Plenum Press, New York
Peterhans E and Heitger F (2001). Simulation of neuronal responses definining depth order and contrast polarity at illusory contours in monkey area V2. J Comput Neurosci 10(2): 195–211
Pianta MJ and Gillam BJ (2003). Paired and unpaired features can be equally effective in human depth perception. Vis Res 43: 1–6
Poggio GF, Gonzalez F and Krause F (1988). Stereoscopic mechanisms in monkey visual cortex: binocular correlation and disparity selectivity. J Neurosci 8(12): 4531–4550
Przybyszewski AW, Gaska JP, Foote W and Pollen DA (2000). Striate cortex increases contrast gain of macaque LGN neurons. Vis Neurosci 17(4): 485–494
Qiu FT and v.d. Heydt R (2005). Figure and ground in the visual cortex: V2 combines stereoscopic cues with Gestalt rules. Neuron 47(1): 155–166
Regan D, Erkelens CJ and Collewijn H (1986). Visual field defects for vergence eye movements and for stereomotion perception. Invest Ophthalmol Vis Sci 27: 806–819
Raizada RD and Grossberg S (2003). Towards a theory of the laminar architecture of cerebral cortex: computational clues from the visual system. Cereb Cortex 13: 100–113
Rubin N (2001a). Figure and ground in the brain. Nat Neurosci 4(9): 857–858
Rubin N (2001b). The role of junctions in surface completion and contour matching. Perception 30: 339–366
Sajda P and Finkel LH (1995). Intermediate-level visual representations and the construction of surface perception. J Cogn Neurosci 7(2): 267–291
Salin PA and Bullier J (1995). Corticocortical connections in the visual system: structure and function. Physiol Rev 75(1): 107–154
Sandell JH and Schiller PH (1982). Effect of cooling area 18 on striate cortex cells in the squirrel monkey. J Neurophysiol 48(1): 38–48
Shadlen MN and Movshon JA (1999). Synchrony unbound: a critical evaluation of the temporal binding hypothesis. Neuron 24: 67–77
Shimojo S and Nakayama K (1990). Real world occlusion constraints and binocular rivalry. Vis Res 30(1): 69–80
Shipley TF and Kellman PJ (1990). The role of discontinuities in the perception of subjective figures. Percept Psychophys 48(3): 259–270
Shipley TF and Kellman PJ (1992). Strength of visual interpolation depends on the ratio of physically specified to total edge length. Percept Psychophys 52(1): 97–106
Singh M, Huang X (2003) Computing layered surface representations: an algorithm for detecting and separating transparent overlays. In: Paper presented at the IEEE CVPR’03, Wisconsin USA
Smith AT, Singh KD, Williams AL and Greenlee MW (2001). Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex. Cereb Cortex 11: 1182–1190
Sporns O, Tononi G and Edelman GM (1991). Modeling perceptual grouping and figure-ground segregation by means of active reentrant connections. PNAS 88: 129–133
Spratling MW and Johnson MH (2001). Dendritic inhibition enhances neural coding properties. Cereb Cortex 11: 1144–1149
Thielscher A and Neumann H (2003). Neural mechanisms of cortico-cortical interaction in texture boundary detection: a modeling approach. Neuroscience 122: 921–939
Thomas OM, Cumming BG and Parker AJ (2002). A specialization for relative disparity in V2. Nat Neurosci 5(5): 472–478
Tse PU (1999). Volume completion. Cogn Psychol 39: 37–68
Tyler CW and Kontsevich LL (1995). Mechanisms of stereoscopic processing: stereoattention and surface perception in depth reconstruction. Perception 24: 127–153
v. d. Heydt R, Heitger F and Peterhans E (1993). Perception of occluding contours: neural mechanisms and a computational model. Biomed Res 14: 1–6
v. d. Heydt R, Peterhans E and Baumgartner G (1984). Illusory contours and cortical neuron responses. Science 224(4654): 1260–1262
Williams LR and Hanson AR (1996). Perceptual completion of occluded surfaces. Comput Vis Image Understand 64(1): 1–20
Williamson JR (1996). Neural network for dynamic binding with graph representation: from, linking and depth-from-occlusion. Neural Comput 8: 1203–1225
Zhou H, Friedman HS and v.d. Heydt R (2000). Coding of border ownership in monkey visual cortex. J Neurosci 20(17): 6594–6611
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License ( https://creativecommons.org/licenses/by-nc/2.0 ), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Thielscher, A., Neumann, H. Globally consistent depth sorting of overlapping 2D surfaces in a model using local recurrent interactions. Biol Cybern 98, 305–337 (2008). https://doi.org/10.1007/s00422-008-0211-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00422-008-0211-7