Skip to main content
SpringerLink
Log in

A recurrent neural model for proto-object based contour integration and figure-ground segregation

Journal of Computational Neuroscience volume 43, pages 227–242 (2017)Cite this article

Abstract

Visual processing of objects makes use of both feedforward and feedback streams of information. However, the nature of feedback signals is largely unknown, as is the identity of the neuronal populations in lower visual areas that receive them. Here, we develop a recurrent neural model to address these questions in the context of contour integration and figure-ground segregation. A key feature of our model is the use of grouping neurons whose activity represents tentative objects (“proto-objects”) based on the integration of local feature information. Grouping neurons receive input from an organized set of local feature neurons, and project modulatory feedback to those same neurons. Additionally, inhibition at both the local feature level and the object representation level biases the interpretation of the visual scene in agreement with principles from Gestalt psychology. Our model explains several sets of neurophysiological results (Zhou et al. Journal of Neuroscience, 20(17), 6594–6611 2000; Qiu et al. Nature Neuroscience, 10(11), 1492–1499 2007; Chen et al. Neuron, 82(3), 682–694 2014), and makes testable predictions about the influence of neuronal feedback and attentional selection on neural responses across different visual areas. Our model also provides a framework for understanding how object-based attention is able to select both objects and the features associated with them.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  • Ardila, D, Mihalas, S, von der Heydt, R, & Niebur, E (2012). Medial axis generation in a model of perceptual organization. In IEEE CISS-2012 46th Annual conference on information sciences and systems (pp. 1–4). NJ: IEEE, Princeton University.

  • Baek, K, & Sajda, P (2005). Inferring figure-ground using a recurrent integrate-and-fire neural circuit. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 13(2), 125–130.

    Article  PubMed  Google Scholar 

  • Bosking, WH, Zhang, Y, Schofield, B, & Fitzpatrick, D (1997). Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. The Journal of Neuroscience, 17(6), 2112–2127.

    CAS  PubMed  Google Scholar 

  • Brincat, S, & Connor, C (2004). Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nature Neuroscience, 7, 880–886.

    Article  CAS  PubMed  Google Scholar 

  • Chen, M, Yan, Y, Gong, X, Gilbert, CD, Liang, H, & Li, W (2014). Incremental integration of global contours through interplay between visual cortical areas. Neuron, 82(3), 682–694.

    Article  CAS  PubMed  Google Scholar 

  • Cox, MA, Schmid, MC, Peters, AJ, Saunders, RC, Leopold, DA, & Maier, A (2013). Receptive field focus of visual area V4 neurons determines responses to illusory surfaces. Proceedings of the National Academy of Sciences, 110(42), 17,095–17,100.

    Article  CAS  Google Scholar 

  • Craft, E, Schütze, H, Niebur, E, & von der Heydt, R (2007). A neural model of figure-ground organization. Journal of Neurophysiology, 97(6), 4310–4326. pMID: 17442769.

    Article  PubMed  Google Scholar 

  • de Oliveira, SC, Thiele, A, & Hoffmann, KP (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. Journal of Neuroscience, 17(23), 9248–9260.

    PubMed  Google Scholar 

  • Domijan, D, & Ṡetić, M (2008). A feedback model of figure-ground assignment. Journal of Vision, 8(7), 10–10.

    Article  PubMed  Google Scholar 

  • Dong, Y, Mihalas, S, Qiu, F, von der Heydt, R, & Niebur, E (2008). Synchrony and the binding problem in macaque visual cortex. Journal of Vision, 8(7), 1–16. https://doi.org/10.1167/8.7.30. pMC2647779. http://journalofvision.org/8/7/30/.

  • Duncan, J (1984). Selective attention and the organization of visual information. Journal of Experimental Psychology: General, 113, 501–517.

    Article  CAS  Google Scholar 

  • Egly, R, Driver, J, & Rafal, R (1994). Shifting visual attention between objects and locations: evidence for normal and parietal lesion subjects. Journal of Experimental Psychology: General, 123, 161–177.

    Article  CAS  Google Scholar 

  • Epshtein, B, Lifshitz, I, & Ullman, S (2008). Image interpretation by a single bottom-up top-down cycle. Proceedings of the National Academy of Sciences, 105(38), 14,298–14,303.

    Article  CAS  Google Scholar 

  • Field, DJ, Hayes, A, & Hess, RF (1993). Contour integration by the human visual system: evidence for a local association field. Vision Research, 33(2), 173–193.

    Article  CAS  PubMed  Google Scholar 

  • Gilbert, C, & Wiesel, T (1989). Columnar specificity of intrinsic horizontal and cortico- cortical connections in cat visual cortex. Journal of Neuroscience, 9, 2432–2442.

    CAS  PubMed  Google Scholar 

  • Girard, P, Hupé, J, & Bullier, J (2001). Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities. Journal of Neurophysiology, 85, 1328–1331.

    CAS  PubMed  Google Scholar 

  • Gray, C, König, P, Engel, A, & Singer, W (1989). Oscillatory responses in cat visual cortx exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338, 334–337.

    Article  CAS  PubMed  Google Scholar 

  • Green, DM, & Swets, JA. (1966). Signal detection theory and psychophysics. New York: Wiley.

    Google Scholar 

  • Grossberg, S (1994). 3-D vision and figure-ground separation by visual cortex. Perception & Psychophysics, 55, 48–120.

    Article  CAS  Google Scholar 

  • Grossberg, S (1997). Cortical dynamics of three-dimensional figure–ground perception of two-dimensional pictures. Psychological Review, 104(3), 618. pMID 9243966.

    Article  CAS  PubMed  Google Scholar 

  • He, ZJ, & Nakayama, K (1995). Visual attention to surfaces in three-dimensional space. Proceedings of the National Academy of Sciences USA, 9(24), 11,155–11,159. pMID 7479956.

    Article  Google Scholar 

  • Hegdé, J, & Van Essen, DC (2007). A comparative study of shape representation in macaque visual areas V2 and V4. Cerebral Cortex, 17(5), 1100–1116.

    Article  PubMed  Google Scholar 

  • Hesse, JK, & Tsao, DY (2016). Consistency of border-ownership cells across artificial stimuli, natural stimuli, and stimuli with ambiguous contours. Journal of Neuroscience, 36(44), 11,338–11,349.

    Article  CAS  Google Scholar 

  • Ho, MC, & Yeh, SL (2009). Effects of instantaneous object input and past experience on object-based attention. Acta Psychologica, 132(1), 31–39.

    Article  PubMed  Google Scholar 

  • Hochstein, S, & Ahissar, M (2002). View from the top: hierarchies and reverse hierarchies in the visual system. Neuron, 36(5), 791–804.

    Article  CAS  PubMed  Google Scholar 

  • Hopf, J, Boehler, C, Luck, S, Tsotsos, J, Heinze, H, & Schoenfeld, M (2006). Direct neurophysiological evidence for spatial suppression surrounding the focus of attention in vision. Proceedings of the National Academy of Sciences, 103(4), 1053. pMID 16410356.

    Article  CAS  Google Scholar 

  • Hu, B, von der Heydt, R, & Niebur, E (2015). A neural model for perceptual organization of 3D surfaces. In IEEE CISS-2015 49th Annual Conference on Information Sciences and Systems. https://doi.org/10.1109/CISS.2015.7086906 (pp. 1–6). Baltimore: IEEE Information Theory Society.

  • Hupé, J, James, A, Payne, B, Lomber, S, Girard, P, & Bullier, J (1998). Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature, 394, 784– 787.

    Article  PubMed  Google Scholar 

  • Intriligator, J, & Cavanagh, P (2001). The spatial resolution of visual attention. Cognitive Psychology, 43, 171–216.

    Article  CAS  PubMed  Google Scholar 

  • Jansen-Amorim, AK, Fiorani, M, & Gattass, R (2012). GABA inactivation of area V4 changes receptive-field properties of V2 neurons in Cebus monkeys. Experimental Neurology, 235(2), 553–562.

    Article  CAS  PubMed  Google Scholar 

  • Jehee, JF, Lamme, VA, & Roelfsema, PR (2007). Boundary assignment in a recurrent network architecture. Vision Research, 47(9), 1153–1165.

    Article  PubMed  Google Scholar 

  • Kikuchi, M, & Akashi, Y (2001). A model of border-ownership coding in early vision. In Dorffner, G, Bischof, H, & Hornik, K (Eds.) ICANN (Vol. 2001, pp. 1069–1074).

  • Kimchi, R, Yeshurun, Y, & Cohen-Savransky, A (2007). Automatic, stimulus-driven attentional capture by objecthood. Psychonomic Bulletin and Review, 14(1), 166–172.

    Article  PubMed  Google Scholar 

  • Koffka, K. (1935). Principles of Gestalt psychology. New York: Harcourt-Brace.

    Google Scholar 

  • Kreiter, AK, & Singer, W (1992). Oscillatory neuronal responses in the visual cortex of the awake macaque monkey. The European Journal of Neuroscience, 4(4), 369–375.

    Article  PubMed  Google Scholar 

  • Lamme, VAF (1995). The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience, 15, 1605–1615.

    CAS  PubMed  Google Scholar 

  • Lamme, VAF, Zipser, K, & Spekreijse, H (1998). Figure-ground activity in primary visual cortex is suppressed by anesthesia. Proceedings of the National Academy of Sciences of the United States of America, 9(6), 3263–3268.

    Article  Google Scholar 

  • Layton, OW, Mingolla, E, & Yazdanbakhsh, A (2012). Dynamic coding of border-ownership in visual cortex. Journal of vision, 12(13), 8.

    Article  PubMed  Google Scholar 

  • Lee, TS, Mumford, D, Romero, R, & Lamme, VAF (1998). The role of the primary visual cortex in higher level vision. Vision Research, 38, 2429–2452.

    Article  CAS  PubMed  Google Scholar 

  • Li, Z (1998). A neural model of contour integration in the primary visual cortex. Neural Computation, 10, 903–940.

    Article  CAS  PubMed  Google Scholar 

  • Li, W, Piëch, V, & Gilbert, CD (2008). Learning to link visual contours. Neuron, 57(3), 442–451.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Martin, AB, & von der Heydt, R (2015). Spike synchrony reveals emergence of proto-objects in visual cortex. The Journal of Neuroscience, 35(17), 6860–6870.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • McAdams, CJ, & Maunsell, JHR (1999). Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4. Journal of Neuroscience, 19, 431–441.

    CAS  PubMed  Google Scholar 

  • Mihalas, S, Dong, Y, von der Heydt, R, & Niebur, E (2011). Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects. Proceedings of the National Academy of Sciences, 108(18), 7583–7588 . pMC3088583.

    Article  CAS  Google Scholar 

  • Motter, BC (1994). Neural correlates of attentive selection for color or luminance in extrastriate area V4. Journal of Neuroscience, 14, 2178–2189.

    CAS  PubMed  Google Scholar 

  • Nishimura, H, & Sakai, K (2004). Determination of border-ownership based on the surround context of contrast. Neurocomputing, 58-60, 843–848.

    Article  Google Scholar 

  • Nishimura, H, & Sakai, K (2005). The computational model for border-ownership determination consisting of surrounding suppression and facilitation in early vision. Neurocomputing, 65, 77–83.

    Article  Google Scholar 

  • Pao, HK, Geiger, D, & Rubin, N (1999). Measuring convexity for figure/ground separation. In 7th international conference on computer vision. Kerkyra.

  • Pasupathy, A, & Connor, CE (2002). Population coding of shape in area V4. Nature Neuroscience, 5(12), 1332–1338.

    Article  CAS  PubMed  Google Scholar 

  • Piëch, V, Li, W, Reeke, GN, & Gilbert, CD (2013). Network model of top-down influences on local gain and contextual interactions in visual cortex. Proceedings of the National Academy of Sciences, 110(43), E4108–E4117. pMC3808648.

    Article  Google Scholar 

  • Polat, U, Mizobe, K, Pettet, MW, Kasamatsu, T, & Norcia, AM (1998). Collinear stimuli regulate visual responses depending on cell’s contrast threshold. Nature, 391, 580–584.

    Article  CAS  PubMed  Google Scholar 

  • Poort, J, Raudies, F, Wannig, A, Lamme, VA, Neumann, H, & Roelfsema, PR (2012). The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex. Neuron, 75(1), 143–156.

    Article  CAS  PubMed  Google Scholar 

  • Qiu, FT, & von der Heydt, R (2005). Figure and ground in the visual cortex: V2 combines stereoscopic cues with Gestalt rules. Neuron, 47, 155–166.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Qiu, FT, & von der Heydt, R (2007). Neural representation of transparent overlay. Nature Neuroscience, 10 (3), 283–284.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Qiu, FT, Sugihara, T, & von der Heydt, R (2007). Figure-ground mechanisms provide structure for selective attention. Nature Neuroscience, 10(11), 1492–1499.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Rensink, RA (2000). The dynamic representation of scenes. Visual Cognition, 7(1/2/3), 17–42.

    Article  Google Scholar 

  • Roelfsema, PR (2006). Cortical algorithms for perceptual grouping. Annual Review of Neuroscience, 29, 203–227.

    Article  CAS  PubMed  Google Scholar 

  • Roelfsema, PR, Lamme, VAF, & Spekreijse, H (2004). Synchrony and covariation of firing rates in the primary visual cortex during contour grouping. Nature Neuroscience, 7(9), 982–991. https://doi.org/10.1038/nn1304.

    Article  CAS  PubMed  Google Scholar 

  • Russell, AF, Mihalas, S, von der Heydt, R, Niebur, E, & Etienne-Cummings, R (2014). A model of proto-object based saliency. Vision Research, 94, 1–15.

    Article  PubMed  Google Scholar 

  • Sajda, P, & Finkel, L (1995). Intermediate-level visual representations and the construction of surface perception. Journal of Cognitive Neuroscience, 7, 267–291.

    Article  CAS  PubMed  Google Scholar 

  • Sakai, K, Nishimura, H, Shimizu, R, & Kondo, K (2012). Consistent and robust determination of border ownership based on asymmetric surrounding contrast. Neural Networks, 33, 257– 274.

    Article  PubMed  Google Scholar 

  • Scholl, BJ (2001). Objects and attention: the state of the art. Cognition, 80(1-2), 1–46.

    Article  CAS  PubMed  Google Scholar 

  • Schütze, H, Niebur, E, & von der Heydt, R (2003). Modeling cortical mechanisms of border ownership coding. Journals of Vision, 3(9), 114a.

    Article  Google Scholar 

  • Singer, W (1999). Neuronal synchrony: a versatile code for the definition of relations? Neuron, 24, 49–65.

    Article  CAS  PubMed  Google Scholar 

  • Stemmler, M, Usher, M, & Niebur, E (1995). Lateral cortical connections may contribute to both contour completion and redundancy reduction in visual processing. Society for Neuroscience Abstracts, 21(1), 510.

    Google Scholar 

  • Stettler, DD, Das, A, Bennett, J, & Gilbert, CD (2002). Lateral connectivity and contextual interactions in macaque primary visual cortex. Neuron, 36(4), 739–750.

    Article  CAS  PubMed  Google Scholar 

  • Sugihara, T, Qiu, FT, & von der Heydt, R (2011). The speed of context integration in the visual cortex. Journal of Neurophysiology, 106(1), 374–385. pMC3129740.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sundberg, KA, Mitchell, JF, & Reynolds, JH (2009). Spatial attention modulates center-surround interactions in macaque visual area v4. Neuron, 61, 952–963.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Supèr, H, & Lamme, VA (2007). Altered figure-ground perception in monkeys with an extra-striate lesion. Neuropsychologia, 45(14), 3329–3334.

    Article  PubMed  Google Scholar 

  • Thiele, A, & Stoner, G (2003). Neuronal synchrony does not correlate with motion coherence in cortical area MT. Nature, 421(6921), 366–370.

    Article  CAS  PubMed  Google Scholar 

  • Treisman, A (1996). The binding problem. Current Opinion in Neurobiology, 6(2), 171–178.

    Article  CAS  PubMed  Google Scholar 

  • Treue, S, & Martinez Trujillo, J.C. (1999). Feature-based attention influences motion processing gain in macaque visual cortex. Nature, 399, 575–579.

    Article  CAS  PubMed  Google Scholar 

  • Treue, S (2001). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24, 295–300.

    Article  CAS  PubMed  Google Scholar 

  • Tschechne, S, & Neumann, H (2014). Hierarchical representation of shapes in visual cortex—from localized features to figural shape segregation. Frontiers in computational neuroscience, 8, 93.

    Article  PubMed  PubMed Central  Google Scholar 

  • Tsotsos, JK (2011). A computational perspective on visual attention. MIT Press.

  • Ullman, S (1984). Visual routines. Cognition, 18, 97–159.

    Article  CAS  PubMed  Google Scholar 

  • Ullman, S, Gregory, R, & Atkinson, J (1992). Low-level aspects of segmentation and recognition [and discussion]. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences, 337(1281), 371–379.

    Article  CAS  PubMed  Google Scholar 

  • Ungerleider, LG, Galkin, TW, Desimone, R, & Gattass, R (2007). Cortical connections of area V4 in the macaque. Cerebral Cortex, 18(3), 477–499.

    Article  PubMed  Google Scholar 

  • von der Heydt, R, Qiu, FT, & He, ZJ (2003). Neural mechanisms in border ownership assignment: motion parallax and gestalt cues. Journal of Vision, 3(9), 666a.

    Article  Google Scholar 

  • Wagatsuma, N, von der Heydt, R, & Niebur, E (2016). Spike synchrony generated by modulatory common input through NMDA-type synapses. Journal of Neurophysiology, 116(3), 1418–1433.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wegener, D, Freiwald, WA, & Kreiter, AK (2004). The influence of sustained selective attention on stimulus selectivity in macaque visual area mt. Journal of Neuroscience, 24(27), 6106–6114.

    Article  CAS  PubMed  Google Scholar 

  • Wertheimer, M (1923). Untersuchungen zur Lehre von der Gestalt II. Psychologische Forschung, 4, 301–350.

    Article  Google Scholar 

  • Williford, JR, & von der Heydt, R (2013). Border-ownership coding. Scholarpedia, 8(10), 30,040.

    Article  Google Scholar 

  • Williford, JR, & von der Heydt, R (2014). Early visual cortex assigns border ownership in natural scenes according to image context. Journal of Vision, 14(10), 588–588.

    Article  Google Scholar 

  • Williford, JR, & von der Heydt, R (2016). Figure-ground organization in visual cortex for natural scenes. eNeuro, 3(6), ENEURO–0127.

    Article  PubMed  PubMed Central  Google Scholar 

  • Yen, SC, & Finkel, LH (1998). Extraction of perceptually salient contours by striate cortical networks. Vision Research, 38(5), 719–741.

    Article  CAS  PubMed  Google Scholar 

  • Zeki, SM (1978). Uniformity and diversity of structure and function in rhesus monkey prestriate visual cortex. The Journal of Physiology, 277(1), 273–290.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Zhang, N, & von der Heydt, R (2010). Analysis of the context integration mechanisms underlying figure–ground organization in the visual cortex. The Journal of Neuroscience, 30(19), 6482–6496. pMC2910339.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Zhaoping, L (2005). Border ownership from intracortical interactions in visual area V2. Neuron, 47, 43–153. pMID 15996554.

    Article  Google Scholar 

  • Zhou, H, Friedman, HS, & von der Heydt, R (2000). Coding of border ownership in monkey visual cortex. Journal of Neuroscience, 20(17), 6594–6611. pMID 10964965.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Zwickel, T, Wachtler, T, & Eckhorn, R (2007). Coding the presence of visual objects in a recurrent neural network of visual cortex. Bio Systems, 89(1), 216–226.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Justin Killebrew for his help in using the computational cluster in order to run the simulations. We would also like to thank Rüdiger von der Heydt for sharing his deep insights on vision with us.

Author information

Authors and Affiliations

  1. Zanvyl Krieger Mind/Brain Institute and Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA

    Brian Hu & Ernst Niebur

  2. Zanvyl Krieger Mind/Brain Institute and Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21218, USA

    Ernst Niebur

Authors
  1. Brian Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ernst Niebur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian Hu.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Action Editor: B. A. Olshausen

This work is supported by the National Institutes of Health under Grants R01EY027544 and R01DA040990.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 3.98 MB)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, B., Niebur, E. A recurrent neural model for proto-object based contour integration and figure-ground segregation. J Comput Neurosci 43, 227–242 (2017). https://doi.org/10.1007/s10827-017-0659-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10827-017-0659-3

Keywords

search

Navigation