The Role of Feedback in a Hierarchical Model of Object Perception

  • Salvador Dura-Bernal
  • Thomas Wennekers
  • Susan L. Denham
Part of the Advances in Experimental Medicine and Biology book series (AEMB, volume 718)


We present a model which stems from a well-established model of object recognition, HMAX, and show how this feedforward system can include feedback, using a recently proposed architecture which reconciles biased competition and predictive coding approaches. Simulation results show successful feedforward object recognition, including cases of occluded and illusory images. Recognition is both position and size invariant. The model also provides a functional interpretation of the role of feedback connectivity in accounting for several observed effects such as enhancement, suppression and refinement of activity in lower areas. The model can qualitatively replicate responses in early visual cortex to occluded and illusory contours; and fMRI data showing that high-level object recognition reduces activity in lower areas. A Gestalt-like mechanism based on collinearity, co-orientation and good continuation principles is proposed to explain illusory contour formation which allows the system to adapt a single high-level object prototype to illusory Kanizsa figures of different sizes, shapes and positions. Overall the model provides a biophysiologically plausible interpretation, supported by current experimental evidence, of the interaction between top-down global feedback and bottom-up local evidence in the context of hierarchical object perception.


  1. 1.
    Angelucci, A., Bullier, J.: Reaching beyond the classical receptive field of vi neurons: horizontal or feedback axons? J. Physiol. 97(2–3), 141–154 (2003) Google Scholar
  2. 2.
    Bullier, J.: Integrated model of visual processing. Brains Res. Rev. 36(2–3), 96–107 (2001) CrossRefGoogle Scholar
  3. 3.
    Cadieu, C., Kouh, M., Pasupathy, A., Connor, C.E., Riesenhuber, M., Poggio, T.: A model of v4 shape selectivity and invariance. J. Neurophysiol. 98(3), 1733–1750 (2007) PubMedCrossRefGoogle Scholar
  4. 4.
    Carandini, M., Demb, J.B., Mante, V., Tolhurst, D.J., Dan, Y., Olshausen, B.A., Gallant, J.L., Rust, N.C.: Do we know what the early visual system does? J. Neurosci. 25(46), 10577–10597 (2005) PubMedCrossRefGoogle Scholar
  5. 5.
    Deco, G., Rolls, E.T.: A neurodynamical cortical model of visual attention and invariant object recognition. Vis. Res. 44(6), 621–642 (2004) PubMedCrossRefGoogle Scholar
  6. 6.
    Felleman, D.J., Van Essen, D.C.: Distributed hierarchical processing in primate cerebral cortex. Cereb. Cortex 1(1), 1–47 (1991) PubMedCrossRefGoogle Scholar
  7. 7.
    Gawne, T.J., Martin, J.M.: Responses of primate visual cortical v4 neurons to simultaneously presented stimuli. J. Neurophysiol. 88(3), 1128–1135 (2002) PubMedGoogle Scholar
  8. 8.
    Gilbert, C.D., Sigman, M.: Brain states: Top-down influences in sensory processing. Neuron 54(5), 677–696 (2007) PubMedCrossRefGoogle Scholar
  9. 9.
    Guo, K., Robertson, R.G., Pulgarin, M., Nevado, A., Panzeri, S., Thiele, A., Young, M.P.: Spatio-temporal prediction and inference by v1 neurons. Eur. J. Neurosci. 26(4), 1045–1054 (2007) PubMedCrossRefGoogle Scholar
  10. 10.
    Halko, M.A., Mingolla, E., Somers, D.C.: Multiple mechanisms of illusory contour perception. J. Vis. 8(11), 1–17 (2008) PubMedCrossRefGoogle Scholar
  11. 11.
    Harrison, L.M., Stephan, K.E., Rees, G., Friston, K.J.: Extra-classical receptive field effects measured in striate cortex with fmri. Neuroimage 34(3), 1199–1208 (2007) PubMedCrossRefGoogle Scholar
  12. 12.
    Hochstein, S., Ahissar, M.: View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron 36(5), 791–804 (2002) PubMedCrossRefGoogle Scholar
  13. 13.
    Huang, J.Y., Wang, C., Dreher, B.: The effects of reversible inactivation of postero-temporal visual cortex on neuronal activities in cat’s area 17. Brain Res. 1138, 111–128 (2007) PubMedCrossRefGoogle Scholar
  14. 14.
    Hubel, D.H., Wiesel, T.N.: Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiol. 28, 229–289 (1965) PubMedGoogle Scholar
  15. 15.
    Hung, C.P., Kreiman, G., Poggio, T., Dicarlo, J.J.: Fast readout of object identity from macaque inferior temporal cortex. Science 310(5749), 863–866 (2005) PubMedCrossRefGoogle Scholar
  16. 16.
    Hupe, J.M., James, A.C., Girard, P., Lomber, S.G., Payne, B.R., Bullier, J.: Feedback connections act on the early part of the responses in monkey visual cortex. J. Neurophysiol. 85(1), 134–145 (2001) PubMedGoogle Scholar
  17. 17.
    Keane, B.P., Lu, H., Kellman, P.J.: Classification images reveal spatiotemporal contour interpolation. Vis. Res. 47(28), 3460–3475 (2007) PubMedCrossRefGoogle Scholar
  18. 18.
    Knoblich, U., Bouvrie, J.V., Poggio, T.: Biophysical models of neural computation: max and tuning circuits. In: Zhong, N., Liu, J., Yao, Y., Wu, J.-L., Lu, S., Li, K. (eds.) Web Intelligence Meets Brain Informatics. Lecture Notes in Computer Science, vol. 4845, pp. 164–189. Springer, Beijing (2007) CrossRefGoogle Scholar
  19. 19.
    Kouh, M., Poggio, T.: A canonical neural circuit for cortical nonlinear operations. Neural Comput. 20(6), 1427–1451 (2008) PubMedCrossRefGoogle Scholar
  20. 20.
    Lampl, I., Ferster, D., Poggio, T., Riesenhuber, M.: Intracellular measurements of spatial integration and the max operation in complex cells of the cat primary visual cortex. J. Neurophysiol. 92(5), 2704–2713 (2004) PubMedCrossRefGoogle Scholar
  21. 21.
    Lee, T., Nguyen, M.: Dynamics of subjective contour formation in the early visual cortex. Proc. Natl. Acad. Sci. USA 98(4), 1907–1911 (2001) PubMedCrossRefGoogle Scholar
  22. 22.
    Lee, T.S.: Computations in the early visual cortex. J. Physiol. 97, 121–139 (2003) Google Scholar
  23. 23.
    Maertens, M., Pollmann, S., Hanke, M., Mildner, T., Müller, H.E.: Retinotopic activation in response to subjective contours in primary visual cortex. Frontiers in Human Neuroscience 2(2) (2008). doi: 10.3389/neuro.09.002.2008
  24. 24.
    Murray, M.M., Wylie, G.R., Higgins, B.A., Javitt, D.C., Schroeder, C.E., Foxe, J.J.: The spatiotemporal dynamics of illusory contour processing: Combined high-density electrical mapping, source analysis, and functional magnetic resonance imaging. J. Neurosci. 22(12), 5055–5073 (2002) PubMedGoogle Scholar
  25. 25.
    Murray, S.O., Schrater, P., Kersten, D.: Perceptual grouping and the interactions between visual cortical areas. Neural Netw. 17(5–6), 695–705 (2004) PubMedCrossRefGoogle Scholar
  26. 26.
    Neumann, H., Mingolla, E.: Computational neural models of spatial integration in perceptual grouping. In: Shipley, T.F., Kellman, P.J. (eds.) From Fragments to Objects: Grouping and Segmentation in Vision, pp. 353–400. Elsevier, Amsterdam (2001) CrossRefGoogle Scholar
  27. 27.
    Olshausen, B., Field, D.: How close are we to understanding v1? Neural Comput. 17(8), 1665–1699 (2005) PubMedCrossRefGoogle Scholar
  28. 28.
    Quiroga, Q., Reddy, L., Kreiman, G., Koch, C., Fried, I.: Invariant visual representation by single neurons in the human brain. Nature 435(7045), 1102–1107 (2005) PubMedCrossRefGoogle Scholar
  29. 29.
    Raizada, R.D.S., Grossberg, S.: Towards a theory of the laminar architecture of cerebral cortex: computational clues from the visual system. Cereb. Cortex 13(1), 100–113 (2003) PubMedCrossRefGoogle Scholar
  30. 30.
    Rao, R.P.N., Ballard, D.H.: Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2(1), 79–87 (1999) PubMedCrossRefGoogle Scholar
  31. 31.
    Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nat. Neurosci. 2(11), 1019–1025 (1999) PubMedCrossRefGoogle Scholar
  32. 32.
    Riesenhuber, M., Poggio, T.: Models of object recognition. Nature Neuroscience (2000). doi: 10.1038/81479 PubMedGoogle Scholar
  33. 33.
    Serre, T., Oliva, A., Poggio, T.: A feedforward architecture accounts for rapid categorization. Proc. Natl. Acad. Sci. USA 104(15), 6424–6429 (2007) PubMedCrossRefGoogle Scholar
  34. 34.
    Serre, T., Riesenhuber, M.: Realistic modeling of simple and complex cell tuning in the h max  model, and implications for invariant object recognition in cortex. Massachusetts Institute of Technology, Cambridge, MA. CBCL Paper 239/AI Memo 2004-017 (2004) Google Scholar
  35. 35.
    Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., Poggio, T.: Robust object recognition with cortex-like mechanisms. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 411–426 (2007) PubMedCrossRefGoogle Scholar
  36. 36.
    Sillito, A.M., Cudeiro, J., Jones, H.E.: Always returning: feedback and sensory processing in visual cortex and thalamus. Trends Neurosci. 29(6), 307–316 (2006) PubMedCrossRefGoogle Scholar
  37. 37.
    Spratling, M.: Reconciling predictive coding and biased competition models of cortical function. Front. Comput. Neurosci. 2(4), 1–8 (2008) Google Scholar
  38. 38.
    Stanley, D.A., Rubin, N.: FMRI activation in response to illusory contours and salient regions in the human lateral occipital complex. Neuron 37(2), 323–331 (2003) PubMedCrossRefGoogle Scholar
  39. 39.
    Sterzer, P., Haynes, J.D., Rees, G.: Primary visual cortex activation on the path of apparent motion is mediated by feedback from HMT+/v5. Neuroimage 32(3), 1308–1316 (2006) PubMedCrossRefGoogle Scholar
  40. 40.
    Symes, A., Wennekers, T.: A large-scale model of spatiotemporal patterns of excitation and inhibition evoked by the horizontal network in layer 2/3 of ferret visual cortex. Neural Netw. 2, 1079–1092 (2009). doi: 10.1016/j.neunet.2009.07.017 CrossRefGoogle Scholar
  41. 41.
    Williams, M.A., Baker, C.I., Op de Beeck, H.P., Mok Shim, W., Dang, S., Triantafyllou, C., Kanwisher, N.: Feedback of visual object information to foveal retinotopic cortex. Nat. Neurosci. 11(12), 1439–1445 (2008) PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Salvador Dura-Bernal
    • 1
  • Thomas Wennekers
    • 1
  • Susan L. Denham
    • 1
  1. 1.Centre for Robotics and Neural SystemsUniversity of PlymouthDevonUK

Personalised recommendations