Models of the Visual Cortex for Object Representation: Learning and Wired Approaches

  • Antonio J. Rodríguez-Sánchez
  • Justus Piater
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8603)


Computational modeling now spans more than three decades. Biologically-plausible models are usually organized into a hierarchy that models the brain in primates after carefully examining neurophysiological and psychophysical studies. Currently, these models extract some values (corners, edges, textures, contours) from images and then apply machine learning algorithms to learn objects or shapes. Are they really that different from classical, non-biologically-inspired, computer vision methods? What facts can we learn from the primate visual system other than the extensively used edge extraction by means of Gabor filters? Should we work more on the representation along this hierarchy before applying a learning strategy? We review the status of computational modeling for object recognition and propose what can be the next challenges to solve.


Computational neuroscience Computer modeling Biological plausibility Machine learning 


  1. 1.
    Ramón y Cajal, S.: Sobre las fibras nerviosas de la capa molecular del cerebelo. Rev. Trim. Histol. Norm. Patol. 1, 33–49 (1888)Google Scholar
  2. 2.
    Ramón y Cajal, S.: The croonian lecture: La fine structure des centres nerveux. Roy. Soc. Lond. Proc. Ser. I 55, 444–468 (1894)CrossRefGoogle Scholar
  3. 3.
    Ramón y Cajal, S.: Variaciones morfologicas, normales y patologicas del reticulo neurofibrilar. Trab. Lab. Investig. Biol. Madrid. 3, 9–15 (1904)Google Scholar
  4. 4.
    Hubel, D., Wiesel, T.: Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148, 574–591 (1959)Google Scholar
  5. 5.
    Hubel, D., Wiesel, T.: Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 195(1), 215–243 (1968)Google Scholar
  6. 6.
    Poggio, T., Serre, T.: Models of visual cortex. Scholarpedia 8(4), 3516 (2013)CrossRefGoogle Scholar
  7. 7.
    Tsotsos, J.K.: Behaviorist intelligence and the scaling problem. Artif. Intell. 75(2), 135–160 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Fukushima, K.: A neural network model for selective attention in visual pattern recognition. Bio. Cybern. 55(1), 5–16 (1986)zbMATHCrossRefGoogle Scholar
  9. 9.
    Barlow, H.: Visual experience and cortical development. Nature 258(5532), 199–204 (1975)CrossRefGoogle Scholar
  10. 10.
    Hubel, D., Wiesel, T.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154 (1962)Google Scholar
  11. 11.
    Hubel, D., Wiesel, T.: Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiol. 28, 229–289 (1965)Google Scholar
  12. 12.
    Grossberg, S.: Some nonlinear networks capable of learning a spatial pattern of arbitrary complexity. PNAS 2(59), 368–372 (1968)CrossRefGoogle Scholar
  13. 13.
    Grossberg, S.: Neural pattern discrimination. J. Theor. Biol. 2(27), 291–337 (1970)CrossRefGoogle Scholar
  14. 14.
    Grossberg, S.: A neural model of attention, reinforcement and discrimination learning. Int. Rev. Neurobiol. 18, 263–327 (1975)CrossRefGoogle Scholar
  15. 15.
    Marr, D.: Vision: A computational investigation into the human representation and processing of visual information. W.H. Freeman, NY (1982)Google Scholar
  16. 16.
    Zucker, S.W.: Computer vision and human perception: an essay on the discovery of constraints. In: Proceedings of the International Conference on Artificial Intelligence, pp. 1102–1116 (1981)Google Scholar
  17. 17.
    Fukushima, K.: Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980)zbMATHCrossRefGoogle Scholar
  18. 18.
    Fukushima, K., Miyake, S., Ito, T.: Neocognitron: a neural network model for a mechanism of visual patter recognition. IEEE Trans. Syst. Man Cybern. 13, 826–834 (1983)CrossRefGoogle Scholar
  19. 19.
    Fukushima, K.: Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Netw. 1, 119–130 (1988)CrossRefGoogle Scholar
  20. 20.
    Crick, F.: Function of the thalamic reticular complex - the searchlight hypothesis. PNAS 81(14), 4586–4590 (1984)CrossRefGoogle Scholar
  21. 21.
    von der Malsburg, C.: Nervous structures with dynamical links. Ber. Bunsenges. Phys. Chem. 89, 703–710 (1985)CrossRefGoogle Scholar
  22. 22.
    Crick, F., Koch, C.: Towards a neurobiological theory of consciousness. 2(263–275), 203 (1990)Google Scholar
  23. 23.
    Anderson, C., Van Essen, D.: Shifter circuits: a computational strategy for dynamic aspects of visual processing. PNAS 84(17), 6297–6301 (1987)CrossRefGoogle Scholar
  24. 24.
    Postma, E., van den Herik, H., Hudson, P.: Dynamic selection through gating lattices. In: IEEE International Joint Conference on Neural Networks, vol. 3, pp. 786–791 (1992)Google Scholar
  25. 25.
    Olshausen, B., Anderson, C., Van Essen, D.: A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information. J. Neurosci. 13(11), 4700–4719 (1993)Google Scholar
  26. 26.
    Heinke, D., Humphreys, G.: Attention, spatial representation, and visual neglect: simulating emergent attention and spatial memory in the selective attention for identification model (SAIM). Psychol. Rev. 110(1), 29–87 (2003)CrossRefGoogle Scholar
  27. 27.
    Orban, G.A.: Higher order visual processing in macaque extrastriate cortex. Psychol. Rev. 88(1), 59–89 (2008)MathSciNetGoogle Scholar
  28. 28.
    Krüger, N., Janssen, P., Kalkan, S., Lappe, M., Leonardis, A., Piater, J., Rodríguez-Sánchez, A., Wiskott, L.: Deep hierarchies in the primate visual cortex: what can we learn for computer vision? IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1847–1871 (2013)CrossRefGoogle Scholar
  29. 29.
    Wallis, G., Rolls, E.: Invariant face and object recognition in the visual system. Prog. Neurobiol. 51(2), 167–194 (1997)CrossRefGoogle Scholar
  30. 30.
    von der Malsburg, C.: Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14(2), 85–100 (1973)CrossRefGoogle Scholar
  31. 31.
    Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nature Neurosci. 2(11), 1019–1025 (1999)CrossRefGoogle Scholar
  32. 32.
    Riesenhuber, M., Poggio, T.: Neural mechanisms of object recognition. Curr. Opin. Neurobiol. 12(2), 162–168 (2002)CrossRefGoogle Scholar
  33. 33.
    Serre, T.: Learning a Dictionary of Shape-Components in Visual Cortex: Comparison with Neurons, Humans and Machines. Ph.D. thesis, Massachusetts Institute of Technology (2006)Google Scholar
  34. 34.
    Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M.: Robust object recognition with cortex-like mechanisms. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 411–426 (2007)CrossRefGoogle Scholar
  35. 35.
    Amit, Y.: A neural network architecture for visual selection. Neural Comput. 12, 1141–1164 (2000)CrossRefGoogle Scholar
  36. 36.
    Suzuki, N., Hashimoto, N., Kashimori, Y., Zheng, M., Kambara, T.: A neural model of predictive recognition in form pathway of visual cortex. BioSystems 76, 33–42 (2004)CrossRefGoogle Scholar
  37. 37.
    Rao, R., Ballard, D.: Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Comput. 9(4), 721–763 (1997)CrossRefGoogle Scholar
  38. 38.
    Rao, R., Ballard, D.: Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neurosci. 2(1), 79–87 (1999)CrossRefGoogle Scholar
  39. 39.
    Fidler, S., Berginc, G., Leonardis, A.: Hierarchical statistical learning of generic parts of object structure. In: IEEE CVPR, pp. 182–189 (2006)Google Scholar
  40. 40.
    Weidenbacher, U., Neumann, H.: Extraction of surface-related features in a recurrent model of V1–V2 interactions. PLOS ONE 4(6), e5909 (2009)CrossRefGoogle Scholar
  41. 41.
    Heitger, F., Rosenthaler, L., von der Heydt, R., Peterhans, E., Kubler, O.: Simulation of neural contour mechanisms: from simple to end-stopped cells. Vis. Res. 32(5), 963–981 (1992)CrossRefGoogle Scholar
  42. 42.
    Murphy, T., Finkel, L.: Shape representation by a network of V4-like cells. Neural Netw. 20, 851–867 (2007)zbMATHCrossRefGoogle Scholar
  43. 43.
    Azzopardi, G., Petkov, N.: Detection of retinal vascular bifurcations by trainable V4-Like filters. In: Real, P., Diaz-Pernil, D., Molina-Abril, H., Berciano, A., Kropatsch, W. (eds.) CAIP 2011, Part I. LNCS, vol. 6854, pp. 451–459. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  44. 44.
    Rodríguez-Sánchez, A., Tsotsos, J.: The importance of intermediate representations for the modeling of 2D shape detection: Endstopping and curvature tuned computations. In: IEEE CVPR, pp. 4321–4326 (2011)Google Scholar
  45. 45.
    Leventhal, A.G., Hirsch, H.V.: Cortical effect of early selective exposure to diagonal lines. Science 190(4217), 902–904 (1975)CrossRefGoogle Scholar
  46. 46.
    Rainer, G., Miller, E.K.: Effects of visual experience on the representation of objects in the prefrontal cortex. Neuron 27(1), 179–189 (2000)CrossRefGoogle Scholar
  47. 47.
    Tommasi, T., Quadrianto, N., Caputo, B., Lampert, C.H.: Beyond dataset bias: multi-task unaligned shared knowledge transfer. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part I. LNCS, vol. 7724, pp. 1–15. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  48. 48.
    Pinto, N., Cox, D., Dicarlo, J.: Why is real-world visual object recognition hard? PLOS Comput. Biol. 4(1), 151–156 (2008)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Wang, G., Zhang, Y., Fei-Fei, L.: Using dependent regions for object categorization in a generative framework. In: IEEE CVPR, pp. 1597–1604 (2006)Google Scholar
  50. 50.
    Grauman, K., Darrell, T.: Pyramid match kernels: Discriminative classification with sets of image features. MIT Technical report CSAIL-TR-2006-20 (2006)Google Scholar
  51. 51.
    Mutch, J., Lowe, D.: Multiclass object recognition with sparse, localized features. IEEE CVPR, pp. 11–18 (2006)Google Scholar
  52. 52.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scenes categories. In: IEEE CVPR, pp. 2169–2178 (2006)Google Scholar
  53. 53.
    Zhang, H., Berg, A., Marie, M., Malik, J.: Svm-knn: Discriminative nearest neighbor classification for visual category recognition. In: IEEE CVPR, pp. 2126–2136 (2006)Google Scholar
  54. 54.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: IEEE CVPR, p. 178 (2004)Google Scholar
  55. 55.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)CrossRefGoogle Scholar
  56. 56.
    Bell, A.J., Sejnowski, T.J.: The of natural scenes are edge filters. Vis. Res. 37(23), 3327–3338 (1997)CrossRefGoogle Scholar
  57. 57.
    Karklin, Y., Lewicki, M.S.: Emergence of complex cell properties by learning to generalize in natural scenes. Nature 457(7225), 83–86 (2008)CrossRefGoogle Scholar
  58. 58.
    Cadieu, C., Kouth, K., Pasupathy, A., Connor, C., Riesenhuber, M., Poggio, T.: A model of V4 shape selectivity and invariance. J. Neurophysiol. 98, 1733–1750 (2007)CrossRefGoogle Scholar
  59. 59.
    Pasupathy, A., Connor, C.: Responses to contour features in macaque area V4. J. Neurophysiol. 82(5), 2490–2502 (1999)Google Scholar
  60. 60.
    Pasupathy, A., Connor, C.: Shape representation in area V4: Position-specific tuning for boundary conformation. J. Neurophysiol. 86(5), 2505–2519 (2001)Google Scholar
  61. 61.
    Pasupathy, A., Connor, C.: Population coding of shape in area V4. Nature Neurosci. 5(12), 1332–1338 (2002)CrossRefGoogle Scholar
  62. 62.
    Rodríguez-Sánchez, A., Tsotsos, J.: The roles of endstopped and curvature tuned computations in a hierarchical representation of 2D shape. PLOS ONE 7(8), 1–13 (2012)CrossRefGoogle Scholar
  63. 63.
    Rodríguez-Sánchez, A.: Intermediate Visual Representations for Attentive Recognition Systems. Ph.D. thesis, York University, Dept. of Computer Science and Engineering (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Antonio J. Rodríguez-Sánchez
    • 1
  • Justus Piater
    • 1
  1. 1.Intelligent and Interactive SystemsUniversity of InnsbruckInnsbruckAustria

Personalised recommendations