Visual processing of objects makes use of both feedforward and feedback streams of information. However, the nature of feedback signals is largely unknown, as is the identity of the neuronal populations in lower visual areas that receive them. Here, we develop a recurrent neural model to address these questions in the context of contour integration and figure-ground segregation. A key feature of our model is the use of grouping neurons whose activity represents tentative objects (“proto-objects”) based on the integration of local feature information. Grouping neurons receive input from an organized set of local feature neurons, and project modulatory feedback to those same neurons. Additionally, inhibition at both the local feature level and the object representation level biases the interpretation of the visual scene in agreement with principles from Gestalt psychology. Our model explains several sets of neurophysiological results (Zhou et al. Journal of Neuroscience, 20(17), 6594–6611 2000; Qiu et al. Nature Neuroscience, 10(11), 1492–1499 2007; Chen et al. Neuron, 82(3), 682–694 2014), and makes testable predictions about the influence of neuronal feedback and attentional selection on neural responses across different visual areas. Our model also provides a framework for understanding how object-based attention is able to select both objects and the features associated with them.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Ardila, D, Mihalas, S, von der Heydt, R, & Niebur, E (2012). Medial axis generation in a model of perceptual organization. In IEEE CISS-2012 46th Annual conference on information sciences and systems (pp. 1–4). NJ: IEEE, Princeton University.
Baek, K, & Sajda, P (2005). Inferring figure-ground using a recurrent integrate-and-fire neural circuit. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 13(2), 125–130.
Bosking, WH, Zhang, Y, Schofield, B, & Fitzpatrick, D (1997). Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. The Journal of Neuroscience, 17(6), 2112–2127.
Brincat, S, & Connor, C (2004). Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nature Neuroscience, 7, 880–886.
Chen, M, Yan, Y, Gong, X, Gilbert, CD, Liang, H, & Li, W (2014). Incremental integration of global contours through interplay between visual cortical areas. Neuron, 82(3), 682–694.
Cox, MA, Schmid, MC, Peters, AJ, Saunders, RC, Leopold, DA, & Maier, A (2013). Receptive field focus of visual area V4 neurons determines responses to illusory surfaces. Proceedings of the National Academy of Sciences, 110(42), 17,095–17,100.
Craft, E, Schütze, H, Niebur, E, & von der Heydt, R (2007). A neural model of figure-ground organization. Journal of Neurophysiology, 97(6), 4310–4326. pMID: 17442769.
de Oliveira, SC, Thiele, A, & Hoffmann, KP (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. Journal of Neuroscience, 17(23), 9248–9260.
Domijan, D, & Ṡetić, M (2008). A feedback model of figure-ground assignment. Journal of Vision, 8(7), 10–10.
Dong, Y, Mihalas, S, Qiu, F, von der Heydt, R, & Niebur, E (2008). Synchrony and the binding problem in macaque visual cortex. Journal of Vision, 8(7), 1–16. https://doi.org/10.1167/8.7.30. pMC2647779. http://journalofvision.org/8/7/30/.
Duncan, J (1984). Selective attention and the organization of visual information. Journal of Experimental Psychology: General, 113, 501–517.
Egly, R, Driver, J, & Rafal, R (1994). Shifting visual attention between objects and locations: evidence for normal and parietal lesion subjects. Journal of Experimental Psychology: General, 123, 161–177.
Epshtein, B, Lifshitz, I, & Ullman, S (2008). Image interpretation by a single bottom-up top-down cycle. Proceedings of the National Academy of Sciences, 105(38), 14,298–14,303.
Field, DJ, Hayes, A, & Hess, RF (1993). Contour integration by the human visual system: evidence for a local association field. Vision Research, 33(2), 173–193.
Gilbert, C, & Wiesel, T (1989). Columnar specificity of intrinsic horizontal and cortico- cortical connections in cat visual cortex. Journal of Neuroscience, 9, 2432–2442.
Girard, P, Hupé, J, & Bullier, J (2001). Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities. Journal of Neurophysiology, 85, 1328–1331.
Gray, C, König, P, Engel, A, & Singer, W (1989). Oscillatory responses in cat visual cortx exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338, 334–337.
Green, DM, & Swets, JA. (1966). Signal detection theory and psychophysics. New York: Wiley.
Grossberg, S (1994). 3-D vision and figure-ground separation by visual cortex. Perception & Psychophysics, 55, 48–120.
Grossberg, S (1997). Cortical dynamics of three-dimensional figure–ground perception of two-dimensional pictures. Psychological Review, 104(3), 618. pMID 9243966.
He, ZJ, & Nakayama, K (1995). Visual attention to surfaces in three-dimensional space. Proceedings of the National Academy of Sciences USA, 9(24), 11,155–11,159. pMID 7479956.
Hegdé, J, & Van Essen, DC (2007). A comparative study of shape representation in macaque visual areas V2 and V4. Cerebral Cortex, 17(5), 1100–1116.
Hesse, JK, & Tsao, DY (2016). Consistency of border-ownership cells across artificial stimuli, natural stimuli, and stimuli with ambiguous contours. Journal of Neuroscience, 36(44), 11,338–11,349.
Ho, MC, & Yeh, SL (2009). Effects of instantaneous object input and past experience on object-based attention. Acta Psychologica, 132(1), 31–39.
Hochstein, S, & Ahissar, M (2002). View from the top: hierarchies and reverse hierarchies in the visual system. Neuron, 36(5), 791–804.
Hopf, J, Boehler, C, Luck, S, Tsotsos, J, Heinze, H, & Schoenfeld, M (2006). Direct neurophysiological evidence for spatial suppression surrounding the focus of attention in vision. Proceedings of the National Academy of Sciences, 103(4), 1053. pMID 16410356.
Hu, B, von der Heydt, R, & Niebur, E (2015). A neural model for perceptual organization of 3D surfaces. In IEEE CISS-2015 49th Annual Conference on Information Sciences and Systems. https://doi.org/10.1109/CISS.2015.7086906 (pp. 1–6). Baltimore: IEEE Information Theory Society.
Hupé, J, James, A, Payne, B, Lomber, S, Girard, P, & Bullier, J (1998). Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature, 394, 784– 787.
Intriligator, J, & Cavanagh, P (2001). The spatial resolution of visual attention. Cognitive Psychology, 43, 171–216.
Jansen-Amorim, AK, Fiorani, M, & Gattass, R (2012). GABA inactivation of area V4 changes receptive-field properties of V2 neurons in Cebus monkeys. Experimental Neurology, 235(2), 553–562.
Jehee, JF, Lamme, VA, & Roelfsema, PR (2007). Boundary assignment in a recurrent network architecture. Vision Research, 47(9), 1153–1165.
Kikuchi, M, & Akashi, Y (2001). A model of border-ownership coding in early vision. In Dorffner, G, Bischof, H, & Hornik, K (Eds.) ICANN (Vol. 2001, pp. 1069–1074).
Kimchi, R, Yeshurun, Y, & Cohen-Savransky, A (2007). Automatic, stimulus-driven attentional capture by objecthood. Psychonomic Bulletin and Review, 14(1), 166–172.
Koffka, K. (1935). Principles of Gestalt psychology. New York: Harcourt-Brace.
Kreiter, AK, & Singer, W (1992). Oscillatory neuronal responses in the visual cortex of the awake macaque monkey. The European Journal of Neuroscience, 4(4), 369–375.
Lamme, VAF (1995). The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience, 15, 1605–1615.
Lamme, VAF, Zipser, K, & Spekreijse, H (1998). Figure-ground activity in primary visual cortex is suppressed by anesthesia. Proceedings of the National Academy of Sciences of the United States of America, 9(6), 3263–3268.
Layton, OW, Mingolla, E, & Yazdanbakhsh, A (2012). Dynamic coding of border-ownership in visual cortex. Journal of vision, 12(13), 8.
Lee, TS, Mumford, D, Romero, R, & Lamme, VAF (1998). The role of the primary visual cortex in higher level vision. Vision Research, 38, 2429–2452.
Li, Z (1998). A neural model of contour integration in the primary visual cortex. Neural Computation, 10, 903–940.
Li, W, Piëch, V, & Gilbert, CD (2008). Learning to link visual contours. Neuron, 57(3), 442–451.
Martin, AB, & von der Heydt, R (2015). Spike synchrony reveals emergence of proto-objects in visual cortex. The Journal of Neuroscience, 35(17), 6860–6870.
McAdams, CJ, & Maunsell, JHR (1999). Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4. Journal of Neuroscience, 19, 431–441.
Mihalas, S, Dong, Y, von der Heydt, R, & Niebur, E (2011). Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects. Proceedings of the National Academy of Sciences, 108(18), 7583–7588 . pMC3088583.
Motter, BC (1994). Neural correlates of attentive selection for color or luminance in extrastriate area V4. Journal of Neuroscience, 14, 2178–2189.
Nishimura, H, & Sakai, K (2004). Determination of border-ownership based on the surround context of contrast. Neurocomputing, 58-60, 843–848.
Nishimura, H, & Sakai, K (2005). The computational model for border-ownership determination consisting of surrounding suppression and facilitation in early vision. Neurocomputing, 65, 77–83.
Pao, HK, Geiger, D, & Rubin, N (1999). Measuring convexity for figure/ground separation. In 7th international conference on computer vision. Kerkyra.
Pasupathy, A, & Connor, CE (2002). Population coding of shape in area V4. Nature Neuroscience, 5(12), 1332–1338.
Piëch, V, Li, W, Reeke, GN, & Gilbert, CD (2013). Network model of top-down influences on local gain and contextual interactions in visual cortex. Proceedings of the National Academy of Sciences, 110(43), E4108–E4117. pMC3808648.
Polat, U, Mizobe, K, Pettet, MW, Kasamatsu, T, & Norcia, AM (1998). Collinear stimuli regulate visual responses depending on cell’s contrast threshold. Nature, 391, 580–584.
Poort, J, Raudies, F, Wannig, A, Lamme, VA, Neumann, H, & Roelfsema, PR (2012). The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex. Neuron, 75(1), 143–156.
Qiu, FT, & von der Heydt, R (2005). Figure and ground in the visual cortex: V2 combines stereoscopic cues with Gestalt rules. Neuron, 47, 155–166.
Qiu, FT, & von der Heydt, R (2007). Neural representation of transparent overlay. Nature Neuroscience, 10 (3), 283–284.
Qiu, FT, Sugihara, T, & von der Heydt, R (2007). Figure-ground mechanisms provide structure for selective attention. Nature Neuroscience, 10(11), 1492–1499.
Rensink, RA (2000). The dynamic representation of scenes. Visual Cognition, 7(1/2/3), 17–42.
Roelfsema, PR (2006). Cortical algorithms for perceptual grouping. Annual Review of Neuroscience, 29, 203–227.
Roelfsema, PR, Lamme, VAF, & Spekreijse, H (2004). Synchrony and covariation of firing rates in the primary visual cortex during contour grouping. Nature Neuroscience, 7(9), 982–991. https://doi.org/10.1038/nn1304.
Russell, AF, Mihalas, S, von der Heydt, R, Niebur, E, & Etienne-Cummings, R (2014). A model of proto-object based saliency. Vision Research, 94, 1–15.
Sajda, P, & Finkel, L (1995). Intermediate-level visual representations and the construction of surface perception. Journal of Cognitive Neuroscience, 7, 267–291.
Sakai, K, Nishimura, H, Shimizu, R, & Kondo, K (2012). Consistent and robust determination of border ownership based on asymmetric surrounding contrast. Neural Networks, 33, 257– 274.
Scholl, BJ (2001). Objects and attention: the state of the art. Cognition, 80(1-2), 1–46.
Schütze, H, Niebur, E, & von der Heydt, R (2003). Modeling cortical mechanisms of border ownership coding. Journals of Vision, 3(9), 114a.
Singer, W (1999). Neuronal synchrony: a versatile code for the definition of relations? Neuron, 24, 49–65.
Stemmler, M, Usher, M, & Niebur, E (1995). Lateral cortical connections may contribute to both contour completion and redundancy reduction in visual processing. Society for Neuroscience Abstracts, 21(1), 510.
Stettler, DD, Das, A, Bennett, J, & Gilbert, CD (2002). Lateral connectivity and contextual interactions in macaque primary visual cortex. Neuron, 36(4), 739–750.
Sugihara, T, Qiu, FT, & von der Heydt, R (2011). The speed of context integration in the visual cortex. Journal of Neurophysiology, 106(1), 374–385. pMC3129740.
Sundberg, KA, Mitchell, JF, & Reynolds, JH (2009). Spatial attention modulates center-surround interactions in macaque visual area v4. Neuron, 61, 952–963.
Supèr, H, & Lamme, VA (2007). Altered figure-ground perception in monkeys with an extra-striate lesion. Neuropsychologia, 45(14), 3329–3334.
Thiele, A, & Stoner, G (2003). Neuronal synchrony does not correlate with motion coherence in cortical area MT. Nature, 421(6921), 366–370.
Treisman, A (1996). The binding problem. Current Opinion in Neurobiology, 6(2), 171–178.
Treue, S, & Martinez Trujillo, J.C. (1999). Feature-based attention influences motion processing gain in macaque visual cortex. Nature, 399, 575–579.
Treue, S (2001). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24, 295–300.
Tschechne, S, & Neumann, H (2014). Hierarchical representation of shapes in visual cortex—from localized features to figural shape segregation. Frontiers in computational neuroscience, 8, 93.
Tsotsos, JK (2011). A computational perspective on visual attention. MIT Press.
Ullman, S (1984). Visual routines. Cognition, 18, 97–159.
Ullman, S, Gregory, R, & Atkinson, J (1992). Low-level aspects of segmentation and recognition [and discussion]. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences, 337(1281), 371–379.
Ungerleider, LG, Galkin, TW, Desimone, R, & Gattass, R (2007). Cortical connections of area V4 in the macaque. Cerebral Cortex, 18(3), 477–499.
von der Heydt, R, Qiu, FT, & He, ZJ (2003). Neural mechanisms in border ownership assignment: motion parallax and gestalt cues. Journal of Vision, 3(9), 666a.
Wagatsuma, N, von der Heydt, R, & Niebur, E (2016). Spike synchrony generated by modulatory common input through NMDA-type synapses. Journal of Neurophysiology, 116(3), 1418–1433.
Wegener, D, Freiwald, WA, & Kreiter, AK (2004). The influence of sustained selective attention on stimulus selectivity in macaque visual area mt. Journal of Neuroscience, 24(27), 6106–6114.
Wertheimer, M (1923). Untersuchungen zur Lehre von der Gestalt II. Psychologische Forschung, 4, 301–350.
Williford, JR, & von der Heydt, R (2013). Border-ownership coding. Scholarpedia, 8(10), 30,040.
Williford, JR, & von der Heydt, R (2014). Early visual cortex assigns border ownership in natural scenes according to image context. Journal of Vision, 14(10), 588–588.
Williford, JR, & von der Heydt, R (2016). Figure-ground organization in visual cortex for natural scenes. eNeuro, 3(6), ENEURO–0127.
Yen, SC, & Finkel, LH (1998). Extraction of perceptually salient contours by striate cortical networks. Vision Research, 38(5), 719–741.
Zeki, SM (1978). Uniformity and diversity of structure and function in rhesus monkey prestriate visual cortex. The Journal of Physiology, 277(1), 273–290.
Zhang, N, & von der Heydt, R (2010). Analysis of the context integration mechanisms underlying figure–ground organization in the visual cortex. The Journal of Neuroscience, 30(19), 6482–6496. pMC2910339.
Zhaoping, L (2005). Border ownership from intracortical interactions in visual area V2. Neuron, 47, 43–153. pMID 15996554.
Zhou, H, Friedman, HS, & von der Heydt, R (2000). Coding of border ownership in monkey visual cortex. Journal of Neuroscience, 20(17), 6594–6611. pMID 10964965.
Zwickel, T, Wachtler, T, & Eckhorn, R (2007). Coding the presence of visual objects in a recurrent neural network of visual cortex. Bio Systems, 89(1), 216–226.
We would like to thank Justin Killebrew for his help in using the computational cluster in order to run the simulations. We would also like to thank Rüdiger von der Heydt for sharing his deep insights on vision with us.
Conflict of interests
The authors declare that they have no conflict of interest.
Action Editor: B. A. Olshausen
This work is supported by the National Institutes of Health under Grants R01EY027544 and R01DA040990.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Hu, B., Niebur, E. A recurrent neural model for proto-object based contour integration and figure-ground segregation. J Comput Neurosci 43, 227–242 (2017). https://doi.org/10.1007/s10827-017-0659-3