KI - Künstliche Intelligenz

, Volume 29, Issue 1, pp 19–29 | Cite as

Beyond Simple and Complex Neurons: Towards Intermediate-level Representations of Shapes and Objects

  • Antonio Rodríguez-Sánchez
  • Heiko Neumann
  • Justus Piater


Knowledge of the brain has much advanced since the concept of the neuron doctrine developed by Ramón y Cajal (R Trim Histol Norm Patol 1:33–49, 1888). Over the last six decades a wide range of functionalities of neurons in the visual cortex have been identified. These neurons can be hierarchically organized into areas since neurons cluster according to structural properties and related function. The neurons in such areas can be characterized to a first order approximation by their (static) receptive field function, viz their filter characteristic implemented by their connection weights to neighboring cells. This paper aims to provide insights on the steps that computer models in our opinion must pursue in order to develop robust recognition mechanisms that mimic biological processing capabilities beyond the level of cells with classical simple and complex receptive field response properties. We stress the importance of intermediate-level representations to achieve higher-level object abstraction in the context of feature representations, and summarize two current approaches that we consider are advances toward achieving that goal.


Computer modeling Intermediate visual processing Boundary grouping Shape representation Feedback connections junctions 



H. N. has been supported by the Transregional Collaborative Research Centre SFB/TRR 62 “A Companion Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG).


  1. 1.
    Azzopardi G, Petkov N (2012) A corf computational model of a simple cell that relies on lgn input outperforms the gabor function model. Biol Cybernet 106(3):177–189CrossRefGoogle Scholar
  2. 2.
    Azzopardi G, Petkov N (2013) Trainable cosfire filters for keypoint detection and pattern recognition. IEEE Trans Pattern Analy Mach Intell 35(2):490–503CrossRefGoogle Scholar
  3. 3.
    Bastos AM, Usrey WM, Adams RA, Mangun GR, Fries P, Friston KJ (2012) Canonical microcircuits for predictive coding. Neuron 76(4):695–711CrossRefGoogle Scholar
  4. 4.
    Baylis G, Driver J (2001) Shape-coding in IT cells generalizes over contrast and mirror reversal, but not figure-ground reversal. Nat Neurosci 4(9):937–942CrossRefGoogle Scholar
  5. 5.
    Ben-Shahar O, Zucker S (2004) Geometrical computations explain projection patterns of long-range horizontal connections in visual cortex. Neural Comput 16(3):445–476CrossRefzbMATHGoogle Scholar
  6. 6.
    Born RT, Bradley DC (2005) Structure and function of visual area MT. Annu Rev Neurosci 28:157–189CrossRefGoogle Scholar
  7. 7.
    Bosking WH, Zhang Y, Schofield B, Fitzpatrick D (1997) Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J Neurosci 17(6):2112–2127Google Scholar
  8. 8.
    Boynton G, Hegde J (2004) Visual cortex: the continuing puzzle of area V2. Curr Biol 14(13):523–524CrossRefGoogle Scholar
  9. 9.
    Brosch T, Neumann H (2014) Computing with a canonical neural circuits model with pool normalization and modulating feedbackGoogle Scholar
  10. 10.
    Cadieu C, Kouth K, Pasupathy A, Connor C, Riesenhuber M, Poggio T (2007) A model of V4 shape selectivity and invariance. J Neurophysiol 98:1733–1750CrossRefGoogle Scholar
  11. 11.
    Caplovitz GP, Tse PU (2007) V3a processes contour curvature as a trackable feature for the perception of rotational motion. Cereb Cortex 17(5):1179–1189CrossRefGoogle Scholar
  12. 12.
    Carandini M, Ferster D (2000) Membrane potential and firing rate in cat primary visual cortex. J Neurosci 20(1):470–484Google Scholar
  13. 13.
    Carandini M, Heeger DJ (2012) Normalization as a canonical neural computation. Nat R Neurosci 13(1):51–62CrossRefGoogle Scholar
  14. 14.
    Carandini M, Heeger DJ, Movshon JA (1997) Linearity and normalization in simple cells of the macaque primary visual cortex. J Neurosci 17(21):8621–8644Google Scholar
  15. 15.
    Connor C, Brincatt S, Pasupathy A (2007) Transformation of shape information in the ventral pathway. Curr Opin Neurobiol 17(2):140–147CrossRefGoogle Scholar
  16. 16.
    Craft E, Schütze H, Niebur E, von der Heydt R (2007) A neural model of figure-ground organization. J Neurophysiol 97(6):4310–4326CrossRefGoogle Scholar
  17. 17.
    De Pasquale R, Sherman SM (2013) A modulatory effect of the feedback from higher visual. J Neurophysiol 109:2618–2631CrossRefGoogle Scholar
  18. 18.
    Desimone R, Duncan J (1995) Neural mechanisms of selective visual attention. Ann Rev Neurosci 18:193–222CrossRefGoogle Scholar
  19. 19.
    Dobbins A, Zucker S, Cynader M (1987) Endstopped neurons in the visual cortex as a substrate for calculating curvature. Nature 329(6138):438–441CrossRefGoogle Scholar
  20. 20.
    Edelman GM (1993) Neural darwinism: selection and reentrant signaling in higher brain function. Neuron 10(2):115–125CrossRefMathSciNetGoogle Scholar
  21. 21.
    Fukushima K (1980) Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybernet 36(4):193–202CrossRefzbMATHGoogle Scholar
  22. 22.
    Gilbert CD, Li W (2013) Top-down influences on visual processing. Nat Rev Neurosci 14(5):350–363CrossRefGoogle Scholar
  23. 23.
    Girard P, Bullier J (1989) Visual activity in area v2 during reversible inactivation of area 17 in the macaque monkey. J Neurophysiol 62(6):1287–302Google Scholar
  24. 24.
    Grossberg S (1968) Some nonlinear networks capable of learning a spatial pattern of arbitrary complexity. PNAS 2(59):368–372CrossRefGoogle Scholar
  25. 25.
    Grossberg S (1970) Neural pattern discrimination. J Theoret Biol 2(27):291–337CrossRefGoogle Scholar
  26. 26.
    Grossberg S, Mingolla E, Ross WD (1997) Visual brain and visual perception: how does the cortex do perceptual grouping? Trends Neurosci 20(3):106–111CrossRefGoogle Scholar
  27. 27.
    Hansen T, Neumann H (2004) A simple cell model with dominating opponent inhibition for robust image processing. Neural Netw 17(5):647–662CrossRefzbMATHGoogle Scholar
  28. 28.
    Hawken M, Parker A (1987) Spatial properties of neurons in the monkey striate cortex. Proc R Soc Lon Ser B Biol Sci 231:251–288CrossRefGoogle Scholar
  29. 29.
    von der Heydt R, Peterhans E, Baumgartner G (1984) Illusory contours and cortical neuron responses. Science 224(4654):1260–1262CrossRefGoogle Scholar
  30. 30.
    Hinton G, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554CrossRefzbMATHMathSciNetGoogle Scholar
  31. 31.
    Hubel D, Wiesel T (1965) Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J Neurophysiol 28:229–289Google Scholar
  32. 32.
    Hubel D, Wiesel T (1968) Receptive fields and functional architecture of monkey striate cortex. J Physiol 195(1):215–243CrossRefGoogle Scholar
  33. 33.
    Ito M, Komatsu H (2004) Representation of angles embedded within contour stimuli in area V2 of macaque monkeys. J Neurosc 24(13):3313–3324CrossRefGoogle Scholar
  34. 34.
    Jarrett K, Kavukcuoglu K, Ranzato M, LeCun Y (2009) What is the best multi-stage architecture for object recognition? In: Computer vision, 2009 IEEE 12th International Conference on, IEEE, pp 2146–2153Google Scholar
  35. 35.
    Kato H, Bishop P, Orban G (1978) Hypeercomplex and simple/complex cells classifications in cat striate cortex. J Neurophys, pp 1071–1095Google Scholar
  36. 36.
    Kellman PJ, Shipley TF (1991) A theory of visual interpolation in object perception. Cognit Psychol 23(2):141–221CrossRefGoogle Scholar
  37. 37.
    LeCun Y, Bengio Y (1995) Convolutional networks for images, speech, and time series. Handb Brain Theory Neural Netw, 3361Google Scholar
  38. 38.
    Lecun Y, Boser B, Denker J, Henderson D, Howard R, Hubbard W, Jackel L (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551CrossRefGoogle Scholar
  39. 39.
    Logothetis N, Sheinberg D (1996) Visual object recognition. Ann Rev Neurosci 19:577–621CrossRefGoogle Scholar
  40. 40.
    von der Malsburg C, Phillips WA, Singer W (2010) Dynamic coordination in the brain: from neurons to mind. MIT Press, CambridgeCrossRefGoogle Scholar
  41. 41.
    Markov NT, Kennedy H (2013) The importance of being hierarchical. Curr Opin Neurobiol 23(2):187–194CrossRefGoogle Scholar
  42. 42.
    Metzger W (1936) Gesetze des sehens. W. Kramer Frankfurt am Main.Google Scholar
  43. 43.
    Murray SO, Kersten D, Olshausen BA, Schrater P, Woods DL (2002) Shape perception reduces activity in human primary visual cortex. PNAS 99(23):15164–15169CrossRefGoogle Scholar
  44. 44.
    Mutch J, Lowe DG (2008) Object class recognition and localization using sparse features with limited receptive fields. Int J Comput Vis 80(1):45–57CrossRefGoogle Scholar
  45. 45.
    Nakayama K, Shimojo S, Ftamachandran VS (1990) Transparency: relation to depth, subjective contours, luminance, and neon color spreading. Perception 19(4):497–513CrossRefGoogle Scholar
  46. 46.
    Neumann H, Mingolla E (2001) Computational neural models of spatial integration in perceptual grouping. In Fragments to Objects-Segmentation and Grouping in Vision, ch12 130:353–400Google Scholar
  47. 47.
    Neumann H, Sepp W (1999) Recurrent V1–V2 interaction in early visual boundary processing. Biol Cybernet 81(5–6):425–444CrossRefGoogle Scholar
  48. 48.
    Neumann H, Yazdanbakhsh A, Mingolla E (2007) Seeing surfaces: the brain’s vision of the world. Phys Life Rev 4(3):189–222CrossRefGoogle Scholar
  49. 49.
    Parent P, Zucker S (1989) Trace inference, curvature consistency, and curve detection. IEEE Pattern Anal Mach Intell 11(8):823–839CrossRefGoogle Scholar
  50. 50.
    Pasupathy A, Connor C (1999) Responses to contour features in macaque area V4. J Neurophysiol 82(5):2490–2502Google Scholar
  51. 51.
    Pasupathy A, Connor C (2002) Population coding of shape in area V4. Nat Neurosci 5(12):1332–1338CrossRefGoogle Scholar
  52. 52.
    Pessoa L, Thompson E, Noë A (1998) Filling-in is for finding out. Behav Brain Sci 21(06):781–796CrossRefGoogle Scholar
  53. 53.
    Piëch V, Li W, Reeke GN, Gilbert CD (2013) Network model of top-down influences on local gain and contextual interactions in visual cortex. PNAS 110(43):4108–4117CrossRefGoogle Scholar
  54. 54.
    Qiu FT, von Der Heydt R (2005) Figure and ground in the visual cortex: V2 combines stereoscopic cues with gestalt rules. Neuron 47(1):155–166CrossRefGoogle Scholar
  55. 55.
    Ramón y Cajal S (1888) Sobre las fibras nerviosas de la capa molecular del cerebelo. R Trim Histol Norm Patol 1:33–49Google Scholar
  56. 56.
    Rao R, Ballard D (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2(1):79–87CrossRefGoogle Scholar
  57. 57.
    Reynolds JH, Heeger DJ (2009) The normalization model of attention. Neuron 61(2):168–185CrossRefGoogle Scholar
  58. 58.
    Riesenhuber M, Poggio T (1999) Hierarchical models of object recognition in cortex. Nat Neurosci 2(11):1019–1025CrossRefGoogle Scholar
  59. 59.
    Ringach DL, Shapley R (1996) Spatial and temporal properties of illusory contours and amodal boundary completion. Vis Res 36(19):3037–3050CrossRefGoogle Scholar
  60. 60.
    Rodríguez-Sánchez A, Tsotsos J (2011) The importance of intermediate representations for the modeling of 2D shape detection: endstopping and curvature tuned computations. IEEE CVPR pp 4321–4326Google Scholar
  61. 61.
    Rodríguez-Sánchez A, Tsotsos J (2012) The roles of endstopped and curvature tuned computations in a hierarchical representation of 2D shape. PLOS ONE 7(8):1–13CrossRefGoogle Scholar
  62. 62.
    Roelfsema PR (2006) Cortical algorithms for perceptual grouping. Ann Rev Neurosci 29:203–227CrossRefGoogle Scholar
  63. 63.
    Roelfsema PR, Lamme VA, Spekreijse H, Bosch H (2002) Figureground segregation in a recurrent network architecture. J Cogn Neurosci 14(4):525–537CrossRefGoogle Scholar
  64. 64.
    Rubin N (2001a) Figure and ground in the brain. Nat Neurosci 4(9):857–858CrossRefGoogle Scholar
  65. 65.
    Rubin N (2001b) The role of junctions in surface completion and contour matching. Perception 30(3):339–366CrossRefGoogle Scholar
  66. 66.
    Salin PA, Bullier J (1995) Corticocortical connections in the visual-system- structure and function. Physiol Rev 75(1):107–154Google Scholar
  67. 67.
    Serre T, Wolf L, Bileschi S, Riesenhuber M (2007) Robust object recognition with cortex-like mechanisms. IEEE T Pattern Anal Mach Intel 29(3):411–426CrossRefGoogle Scholar
  68. 68.
    Sherman SM, Guillery R (1998) On the actions that one nerve cell can have on another: distinguishing drivers from modulators. PNAS 95(12):7121–7126CrossRefGoogle Scholar
  69. 69.
    Tanaka K (1996) Inferotemporal cortex and object vision. Ann Rev Neurosci 19:109–139CrossRefGoogle Scholar
  70. 70.
    Tanaka K, Saito H, Fukada Y, Moriya M (1991) Coding visual images of objects in the inferotemporal cortex of the macaque monkey. J Neurophysiol 66(1):170–189Google Scholar
  71. 71.
    Thielscher A, Neumann H (2008) Globally consistent depth sorting of overlapping 2d surfaces in a model using local recurrent interactions. Biol Cybernet 98(4):305–337CrossRefzbMATHMathSciNetGoogle Scholar
  72. 72.
    Tschechne S, Neumann H (2014) Hierarchical representation of shapes in visual cortexfrom localized features to figural shape segregation. Front Computat Neurosci 8Google Scholar
  73. 73.
    Tsotsos J, Culhane S, Winky W, Lai Y, Davis N, Nuflo F (1995) Modeling visual attention via selective tuning. Artif Intel 78(1–2):507–545CrossRefGoogle Scholar
  74. 74.
    Ullman S (1995) Sequence seeking and counter streams: a computational model for bidirectional information flow in the visual cortex. Cereb Cortex 5(1):1–11CrossRefGoogle Scholar
  75. 75.
    Wallis G, Rolls E (1997) Invariant face and object recognition in the visual system. Prog Neurobiol 51(2):167–194CrossRefGoogle Scholar
  76. 76.
    Weidenbacher U, Neumann H (2009) Extraction of surface-related features in a recurrent model of V1–V2 interactions. PLOS ONE 4(6):e5909CrossRefGoogle Scholar
  77. 77.
    Williams LR, Jacobs DW (1997) Stochastic completion fields, a neural model of illusory contour shape and salience. Neural Computat 9(4):837–858CrossRefGoogle Scholar
  78. 78.
    Zhou H, Friedman H, von der Heydt R (2000) Coding of border ownership in monkey visual cortex. J Neurosci 20:6594–6611Google Scholar
  79. 79.
    Zipser K, Lamme VA, Schiller PH (1996) Contextual modulation in primary visual cortex. J Neurosci 16(22):7376–7389Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Antonio Rodríguez-Sánchez
    • 1
  • Heiko Neumann
    • 2
  • Justus Piater
    • 1
  1. 1.Institute of Computer ScienceUniversity of InnsbruckInnsbruckAustria
  2. 2.Institute of Neural Information ProcessingUlm UniversityUlmGermany

Personalised recommendations