Journal of Computational Neuroscience

, Volume 43, Issue 3, pp 227–242 | Cite as

A recurrent neural model for proto-object based contour integration and figure-ground segregation

  • Brian Hu
  • Ernst Niebur


Visual processing of objects makes use of both feedforward and feedback streams of information. However, the nature of feedback signals is largely unknown, as is the identity of the neuronal populations in lower visual areas that receive them. Here, we develop a recurrent neural model to address these questions in the context of contour integration and figure-ground segregation. A key feature of our model is the use of grouping neurons whose activity represents tentative objects (“proto-objects”) based on the integration of local feature information. Grouping neurons receive input from an organized set of local feature neurons, and project modulatory feedback to those same neurons. Additionally, inhibition at both the local feature level and the object representation level biases the interpretation of the visual scene in agreement with principles from Gestalt psychology. Our model explains several sets of neurophysiological results (Zhou et al. Journal of Neuroscience, 20(17), 6594–6611 2000; Qiu et al. Nature Neuroscience, 10(11), 1492–1499 2007; Chen et al. Neuron, 82(3), 682–694 2014), and makes testable predictions about the influence of neuronal feedback and attentional selection on neural responses across different visual areas. Our model also provides a framework for understanding how object-based attention is able to select both objects and the features associated with them.


Recurrent processing Shape perception Feedback Grouping Perceptual organization Contour processing 



We would like to thank Justin Killebrew for his help in using the computational cluster in order to run the simulations. We would also like to thank Rüdiger von der Heydt for sharing his deep insights on vision with us.

Compliance with Ethical Standards

Conflict of interests

The authors declare that they have no conflict of interest.

Supplementary material

10827_2017_659_MOESM1_ESM.pdf (4 mb)
(PDF 3.98 MB)


  1. Ardila, D, Mihalas, S, von der Heydt, R, & Niebur, E (2012). Medial axis generation in a model of perceptual organization. In IEEE CISS-2012 46th Annual conference on information sciences and systems (pp. 1–4). NJ: IEEE, Princeton University.Google Scholar
  2. Baek, K, & Sajda, P (2005). Inferring figure-ground using a recurrent integrate-and-fire neural circuit. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 13(2), 125–130.CrossRefPubMedGoogle Scholar
  3. Bosking, WH, Zhang, Y, Schofield, B, & Fitzpatrick, D (1997). Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. The Journal of Neuroscience, 17(6), 2112–2127.PubMedGoogle Scholar
  4. Brincat, S, & Connor, C (2004). Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nature Neuroscience, 7, 880–886.CrossRefPubMedGoogle Scholar
  5. Chen, M, Yan, Y, Gong, X, Gilbert, CD, Liang, H, & Li, W (2014). Incremental integration of global contours through interplay between visual cortical areas. Neuron, 82(3), 682–694.CrossRefPubMedGoogle Scholar
  6. Cox, MA, Schmid, MC, Peters, AJ, Saunders, RC, Leopold, DA, & Maier, A (2013). Receptive field focus of visual area V4 neurons determines responses to illusory surfaces. Proceedings of the National Academy of Sciences, 110(42), 17,095–17,100.CrossRefGoogle Scholar
  7. Craft, E, Schütze, H, Niebur, E, & von der Heydt, R (2007). A neural model of figure-ground organization. Journal of Neurophysiology, 97(6), 4310–4326. pMID: 17442769.CrossRefPubMedGoogle Scholar
  8. de Oliveira, SC, Thiele, A, & Hoffmann, KP (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. Journal of Neuroscience, 17(23), 9248–9260.PubMedGoogle Scholar
  9. Domijan, D, & Ṡetić, M (2008). A feedback model of figure-ground assignment. Journal of Vision, 8(7), 10–10.CrossRefPubMedGoogle Scholar
  10. Dong, Y, Mihalas, S, Qiu, F, von der Heydt, R, & Niebur, E (2008). Synchrony and the binding problem in macaque visual cortex. Journal of Vision, 8(7), 1–16. pMC2647779.
  11. Duncan, J (1984). Selective attention and the organization of visual information. Journal of Experimental Psychology: General, 113, 501–517.CrossRefGoogle Scholar
  12. Egly, R, Driver, J, & Rafal, R (1994). Shifting visual attention between objects and locations: evidence for normal and parietal lesion subjects. Journal of Experimental Psychology: General, 123, 161–177.CrossRefGoogle Scholar
  13. Epshtein, B, Lifshitz, I, & Ullman, S (2008). Image interpretation by a single bottom-up top-down cycle. Proceedings of the National Academy of Sciences, 105(38), 14,298–14,303.CrossRefGoogle Scholar
  14. Field, DJ, Hayes, A, & Hess, RF (1993). Contour integration by the human visual system: evidence for a local association field. Vision Research, 33(2), 173–193.CrossRefPubMedGoogle Scholar
  15. Gilbert, C, & Wiesel, T (1989). Columnar specificity of intrinsic horizontal and cortico- cortical connections in cat visual cortex. Journal of Neuroscience, 9, 2432–2442.PubMedGoogle Scholar
  16. Girard, P, Hupé, J, & Bullier, J (2001). Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities. Journal of Neurophysiology, 85, 1328–1331.PubMedGoogle Scholar
  17. Gray, C, König, P, Engel, A, & Singer, W (1989). Oscillatory responses in cat visual cortx exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338, 334–337.CrossRefPubMedGoogle Scholar
  18. Green, DM, & Swets, JA. (1966). Signal detection theory and psychophysics. New York: Wiley.Google Scholar
  19. Grossberg, S (1994). 3-D vision and figure-ground separation by visual cortex. Perception & Psychophysics, 55, 48–120.CrossRefGoogle Scholar
  20. Grossberg, S (1997). Cortical dynamics of three-dimensional figure–ground perception of two-dimensional pictures. Psychological Review, 104(3), 618. pMID 9243966.CrossRefPubMedGoogle Scholar
  21. He, ZJ, & Nakayama, K (1995). Visual attention to surfaces in three-dimensional space. Proceedings of the National Academy of Sciences USA, 9(24), 11,155–11,159. pMID 7479956.CrossRefGoogle Scholar
  22. Hegdé, J, & Van Essen, DC (2007). A comparative study of shape representation in macaque visual areas V2 and V4. Cerebral Cortex, 17(5), 1100–1116.CrossRefPubMedGoogle Scholar
  23. Hesse, JK, & Tsao, DY (2016). Consistency of border-ownership cells across artificial stimuli, natural stimuli, and stimuli with ambiguous contours. Journal of Neuroscience, 36(44), 11,338–11,349.CrossRefGoogle Scholar
  24. Ho, MC, & Yeh, SL (2009). Effects of instantaneous object input and past experience on object-based attention. Acta Psychologica, 132(1), 31–39.CrossRefPubMedGoogle Scholar
  25. Hochstein, S, & Ahissar, M (2002). View from the top: hierarchies and reverse hierarchies in the visual system. Neuron, 36(5), 791–804.CrossRefPubMedGoogle Scholar
  26. Hopf, J, Boehler, C, Luck, S, Tsotsos, J, Heinze, H, & Schoenfeld, M (2006). Direct neurophysiological evidence for spatial suppression surrounding the focus of attention in vision. Proceedings of the National Academy of Sciences, 103(4), 1053. pMID 16410356.CrossRefGoogle Scholar
  27. Hu, B, von der Heydt, R, & Niebur, E (2015). A neural model for perceptual organization of 3D surfaces. In IEEE CISS-2015 49th Annual Conference on Information Sciences and Systems. (pp. 1–6). Baltimore: IEEE Information Theory Society.
  28. Hupé, J, James, A, Payne, B, Lomber, S, Girard, P, & Bullier, J (1998). Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature, 394, 784– 787.CrossRefPubMedGoogle Scholar
  29. Intriligator, J, & Cavanagh, P (2001). The spatial resolution of visual attention. Cognitive Psychology, 43, 171–216.CrossRefPubMedGoogle Scholar
  30. Jansen-Amorim, AK, Fiorani, M, & Gattass, R (2012). GABA inactivation of area V4 changes receptive-field properties of V2 neurons in Cebus monkeys. Experimental Neurology, 235(2), 553–562.CrossRefPubMedGoogle Scholar
  31. Jehee, JF, Lamme, VA, & Roelfsema, PR (2007). Boundary assignment in a recurrent network architecture. Vision Research, 47(9), 1153–1165.CrossRefPubMedGoogle Scholar
  32. Kikuchi, M, & Akashi, Y (2001). A model of border-ownership coding in early vision. In Dorffner, G, Bischof, H, & Hornik, K (Eds.) ICANN (Vol. 2001, pp. 1069–1074).Google Scholar
  33. Kimchi, R, Yeshurun, Y, & Cohen-Savransky, A (2007). Automatic, stimulus-driven attentional capture by objecthood. Psychonomic Bulletin and Review, 14(1), 166–172.CrossRefPubMedGoogle Scholar
  34. Koffka, K. (1935). Principles of Gestalt psychology. New York: Harcourt-Brace.Google Scholar
  35. Kreiter, AK, & Singer, W (1992). Oscillatory neuronal responses in the visual cortex of the awake macaque monkey. The European Journal of Neuroscience, 4(4), 369–375.CrossRefPubMedGoogle Scholar
  36. Lamme, VAF (1995). The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience, 15, 1605–1615.PubMedGoogle Scholar
  37. Lamme, VAF, Zipser, K, & Spekreijse, H (1998). Figure-ground activity in primary visual cortex is suppressed by anesthesia. Proceedings of the National Academy of Sciences of the United States of America, 9(6), 3263–3268.CrossRefGoogle Scholar
  38. Layton, OW, Mingolla, E, & Yazdanbakhsh, A (2012). Dynamic coding of border-ownership in visual cortex. Journal of vision, 12(13), 8.CrossRefPubMedGoogle Scholar
  39. Lee, TS, Mumford, D, Romero, R, & Lamme, VAF (1998). The role of the primary visual cortex in higher level vision. Vision Research, 38, 2429–2452.CrossRefPubMedGoogle Scholar
  40. Li, Z (1998). A neural model of contour integration in the primary visual cortex. Neural Computation, 10, 903–940.CrossRefPubMedGoogle Scholar
  41. Li, W, Piëch, V, & Gilbert, CD (2008). Learning to link visual contours. Neuron, 57(3), 442–451.CrossRefPubMedPubMedCentralGoogle Scholar
  42. Martin, AB, & von der Heydt, R (2015). Spike synchrony reveals emergence of proto-objects in visual cortex. The Journal of Neuroscience, 35(17), 6860–6870.CrossRefPubMedPubMedCentralGoogle Scholar
  43. McAdams, CJ, & Maunsell, JHR (1999). Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4. Journal of Neuroscience, 19, 431–441.PubMedGoogle Scholar
  44. Mihalas, S, Dong, Y, von der Heydt, R, & Niebur, E (2011). Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects. Proceedings of the National Academy of Sciences, 108(18), 7583–7588 . pMC3088583.CrossRefGoogle Scholar
  45. Motter, BC (1994). Neural correlates of attentive selection for color or luminance in extrastriate area V4. Journal of Neuroscience, 14, 2178–2189.PubMedGoogle Scholar
  46. Nishimura, H, & Sakai, K (2004). Determination of border-ownership based on the surround context of contrast. Neurocomputing, 58-60, 843–848.CrossRefGoogle Scholar
  47. Nishimura, H, & Sakai, K (2005). The computational model for border-ownership determination consisting of surrounding suppression and facilitation in early vision. Neurocomputing, 65, 77–83.CrossRefGoogle Scholar
  48. Pao, HK, Geiger, D, & Rubin, N (1999). Measuring convexity for figure/ground separation. In 7th international conference on computer vision. Kerkyra.Google Scholar
  49. Pasupathy, A, & Connor, CE (2002). Population coding of shape in area V4. Nature Neuroscience, 5(12), 1332–1338.CrossRefPubMedGoogle Scholar
  50. Piëch, V, Li, W, Reeke, GN, & Gilbert, CD (2013). Network model of top-down influences on local gain and contextual interactions in visual cortex. Proceedings of the National Academy of Sciences, 110(43), E4108–E4117. pMC3808648.CrossRefGoogle Scholar
  51. Polat, U, Mizobe, K, Pettet, MW, Kasamatsu, T, & Norcia, AM (1998). Collinear stimuli regulate visual responses depending on cell’s contrast threshold. Nature, 391, 580–584.CrossRefPubMedGoogle Scholar
  52. Poort, J, Raudies, F, Wannig, A, Lamme, VA, Neumann, H, & Roelfsema, PR (2012). The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex. Neuron, 75(1), 143–156.CrossRefPubMedGoogle Scholar
  53. Qiu, FT, & von der Heydt, R (2005). Figure and ground in the visual cortex: V2 combines stereoscopic cues with Gestalt rules. Neuron, 47, 155–166.CrossRefPubMedPubMedCentralGoogle Scholar
  54. Qiu, FT, & von der Heydt, R (2007). Neural representation of transparent overlay. Nature Neuroscience, 10 (3), 283–284.CrossRefPubMedPubMedCentralGoogle Scholar
  55. Qiu, FT, Sugihara, T, & von der Heydt, R (2007). Figure-ground mechanisms provide structure for selective attention. Nature Neuroscience, 10(11), 1492–1499.CrossRefPubMedPubMedCentralGoogle Scholar
  56. Rensink, RA (2000). The dynamic representation of scenes. Visual Cognition, 7(1/2/3), 17–42.CrossRefGoogle Scholar
  57. Roelfsema, PR (2006). Cortical algorithms for perceptual grouping. Annual Review of Neuroscience, 29, 203–227.CrossRefPubMedGoogle Scholar
  58. Roelfsema, PR, Lamme, VAF, & Spekreijse, H (2004). Synchrony and covariation of firing rates in the primary visual cortex during contour grouping. Nature Neuroscience, 7(9), 982–991. Scholar
  59. Russell, AF, Mihalas, S, von der Heydt, R, Niebur, E, & Etienne-Cummings, R (2014). A model of proto-object based saliency. Vision Research, 94, 1–15.CrossRefPubMedGoogle Scholar
  60. Sajda, P, & Finkel, L (1995). Intermediate-level visual representations and the construction of surface perception. Journal of Cognitive Neuroscience, 7, 267–291.CrossRefPubMedGoogle Scholar
  61. Sakai, K, Nishimura, H, Shimizu, R, & Kondo, K (2012). Consistent and robust determination of border ownership based on asymmetric surrounding contrast. Neural Networks, 33, 257– 274.CrossRefPubMedGoogle Scholar
  62. Scholl, BJ (2001). Objects and attention: the state of the art. Cognition, 80(1-2), 1–46.CrossRefPubMedGoogle Scholar
  63. Schütze, H, Niebur, E, & von der Heydt, R (2003). Modeling cortical mechanisms of border ownership coding. Journals of Vision, 3(9), 114a.CrossRefGoogle Scholar
  64. Singer, W (1999). Neuronal synchrony: a versatile code for the definition of relations? Neuron, 24, 49–65.CrossRefPubMedGoogle Scholar
  65. Stemmler, M, Usher, M, & Niebur, E (1995). Lateral cortical connections may contribute to both contour completion and redundancy reduction in visual processing. Society for Neuroscience Abstracts, 21(1), 510.Google Scholar
  66. Stettler, DD, Das, A, Bennett, J, & Gilbert, CD (2002). Lateral connectivity and contextual interactions in macaque primary visual cortex. Neuron, 36(4), 739–750.CrossRefPubMedGoogle Scholar
  67. Sugihara, T, Qiu, FT, & von der Heydt, R (2011). The speed of context integration in the visual cortex. Journal of Neurophysiology, 106(1), 374–385. pMC3129740.CrossRefPubMedPubMedCentralGoogle Scholar
  68. Sundberg, KA, Mitchell, JF, & Reynolds, JH (2009). Spatial attention modulates center-surround interactions in macaque visual area v4. Neuron, 61, 952–963.CrossRefPubMedPubMedCentralGoogle Scholar
  69. Supèr, H, & Lamme, VA (2007). Altered figure-ground perception in monkeys with an extra-striate lesion. Neuropsychologia, 45(14), 3329–3334.CrossRefPubMedGoogle Scholar
  70. Thiele, A, & Stoner, G (2003). Neuronal synchrony does not correlate with motion coherence in cortical area MT. Nature, 421(6921), 366–370.CrossRefPubMedGoogle Scholar
  71. Treisman, A (1996). The binding problem. Current Opinion in Neurobiology, 6(2), 171–178.CrossRefPubMedGoogle Scholar
  72. Treue, S, & Martinez Trujillo, J.C. (1999). Feature-based attention influences motion processing gain in macaque visual cortex. Nature, 399, 575–579.CrossRefPubMedGoogle Scholar
  73. Treue, S (2001). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24, 295–300.CrossRefPubMedGoogle Scholar
  74. Tschechne, S, & Neumann, H (2014). Hierarchical representation of shapes in visual cortex—from localized features to figural shape segregation. Frontiers in computational neuroscience, 8, 93.CrossRefPubMedPubMedCentralGoogle Scholar
  75. Tsotsos, JK (2011). A computational perspective on visual attention. MIT Press.Google Scholar
  76. Ullman, S (1984). Visual routines. Cognition, 18, 97–159.CrossRefPubMedGoogle Scholar
  77. Ullman, S, Gregory, R, & Atkinson, J (1992). Low-level aspects of segmentation and recognition [and discussion]. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences, 337(1281), 371–379.CrossRefPubMedGoogle Scholar
  78. Ungerleider, LG, Galkin, TW, Desimone, R, & Gattass, R (2007). Cortical connections of area V4 in the macaque. Cerebral Cortex, 18(3), 477–499.CrossRefPubMedGoogle Scholar
  79. von der Heydt, R, Qiu, FT, & He, ZJ (2003). Neural mechanisms in border ownership assignment: motion parallax and gestalt cues. Journal of Vision, 3(9), 666a.CrossRefGoogle Scholar
  80. Wagatsuma, N, von der Heydt, R, & Niebur, E (2016). Spike synchrony generated by modulatory common input through NMDA-type synapses. Journal of Neurophysiology, 116(3), 1418–1433.CrossRefPubMedPubMedCentralGoogle Scholar
  81. Wegener, D, Freiwald, WA, & Kreiter, AK (2004). The influence of sustained selective attention on stimulus selectivity in macaque visual area mt. Journal of Neuroscience, 24(27), 6106–6114.CrossRefPubMedGoogle Scholar
  82. Wertheimer, M (1923). Untersuchungen zur Lehre von der Gestalt II. Psychologische Forschung, 4, 301–350.CrossRefGoogle Scholar
  83. Williford, JR, & von der Heydt, R (2013). Border-ownership coding. Scholarpedia, 8(10), 30,040.CrossRefGoogle Scholar
  84. Williford, JR, & von der Heydt, R (2014). Early visual cortex assigns border ownership in natural scenes according to image context. Journal of Vision, 14(10), 588–588.CrossRefGoogle Scholar
  85. Williford, JR, & von der Heydt, R (2016). Figure-ground organization in visual cortex for natural scenes. eNeuro, 3(6), ENEURO–0127.CrossRefPubMedPubMedCentralGoogle Scholar
  86. Yen, SC, & Finkel, LH (1998). Extraction of perceptually salient contours by striate cortical networks. Vision Research, 38(5), 719–741.CrossRefPubMedGoogle Scholar
  87. Zeki, SM (1978). Uniformity and diversity of structure and function in rhesus monkey prestriate visual cortex. The Journal of Physiology, 277(1), 273–290.CrossRefPubMedPubMedCentralGoogle Scholar
  88. Zhang, N, & von der Heydt, R (2010). Analysis of the context integration mechanisms underlying figure–ground organization in the visual cortex. The Journal of Neuroscience, 30(19), 6482–6496. pMC2910339.CrossRefPubMedPubMedCentralGoogle Scholar
  89. Zhaoping, L (2005). Border ownership from intracortical interactions in visual area V2. Neuron, 47, 43–153. pMID 15996554.CrossRefGoogle Scholar
  90. Zhou, H, Friedman, HS, & von der Heydt, R (2000). Coding of border ownership in monkey visual cortex. Journal of Neuroscience, 20(17), 6594–6611. pMID 10964965.PubMedPubMedCentralGoogle Scholar
  91. Zwickel, T, Wachtler, T, & Eckhorn, R (2007). Coding the presence of visual objects in a recurrent neural network of visual cortex. Bio Systems, 89(1), 216–226.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Zanvyl Krieger Mind/Brain Institute and Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA
  2. 2.Zanvyl Krieger Mind/Brain Institute and Department of NeuroscienceJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations