Visual Processing in Cortical Architecture from Neuroscience to Neuromorphic Computing

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10087)


Primate cortices are organized into different layers which constitute a compartmental structure on a functional level. We show how composite structural elements form building blocks to define canonical elements for columnar computation in cortex. As a further abstraction, we define a dynamical three-stage model of a cortical column for processing that allows to investigate the dynamic response properties of cortical algorithms, e.g., feedforward signal integration as feature detection filters, lateral feature grouping, and the integration of modulatory (feedback) signals. Using such multi-stage cortical model, we investigate the detection and integration of spatio-temporal motion measured by event-based (frame-less) cameras. We demonstrate how the canonical neural circuit can improve such representations using normalization and feedback and develop key computational elements to map such a model onto neuromorphic hardware (IBM’s TrueNorth chip). This makes a step towards implementing real-time and energy-efficient neuromorphic optical flow detectors based on realistic principles of computation in cortical columns.


Canonical neural circuit Cortical column Motion detection Optical flow Feedback Neuromorphic computing 



This work has been supported in part by a grant from the Transregional Collaborative Research Center “A Companion-Technology for Cognitive Technical Systems” SFB/TRR 62 funded by the German Research Foundation (DFG). The authors also gratefully acknowledge the support via a field test agreement between Ulm University and IBM Research Almaden as well as the support of NVIDIA Corporation with the donation of the Tesla K40 GPU used for this research.


  1. 1.
    Barbas, H., Rempel-Clower, N.: Cortical structure predicts the pattern of corticocortical connections. Cereb. Cortex 7(7), 635–646 (1997)CrossRefGoogle Scholar
  2. 2.
    Born, R.T., Bradley, D.C.: Structure and function of visual area MT. Annu. Rev. Neurosci. 28, 157–189 (2005)CrossRefGoogle Scholar
  3. 3.
    Bosking, W.H., Zhang, Y., Schofield, B., Fitzpatrick, D.: Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J. Neurosci. 17(6), 2112–2127 (1997)Google Scholar
  4. 4.
    Brosch, T., Neumann, H.: Computing with a canonical neural circuits model with pool normalization and modulating feedback. Neural Comput. 26(12), 2735–2789 (2014)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Brosch, T., Neumann, H.: Interaction of feedforward and feedback streams in visual cortex in a firing-rate model of columnar computations. Neural Networks 54, 11–16 (2014)CrossRefzbMATHGoogle Scholar
  6. 6.
    Brosch, T., Neumann, H.: Event-based optical flow on neuromorphic hardware. In: CMVC (2015)Google Scholar
  7. 7.
    Brosch, T., Tschechne, S., Neumann, H.: On event-based optical flow detection. Front. Neurosci. 9(137), 1–15 (2015)Google Scholar
  8. 8.
    Carandini, M., Heeger, D.J.: Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13, 51–62 (2012)CrossRefGoogle Scholar
  9. 9.
    Cassidy, A.S., Merolla, P., Arthur, J.V., Esser, S.K., Jackson, B., Alvarez-Icaza, R., Datta, P., Sawaday, J., Wong, T.M., Feldman, V., Amir, A., Rubin, D.B.D., Akopyan, F., McQuinn, E., Risk, W.P., Modha, D.S.: Cognitive computing building block: a versatile and efficient digital neuron model for neurosynaptic cores. In: IJCNN, pp. 1–10 (2013)Google Scholar
  10. 10.
    De Valois, R.L., Cottaris, N.P., Mahon, L.E., Elfar, S.D., Wilson, J.A.: Spatial and temporal receptive fields of geniculate and cortical cells and directional selectivity. Vision Res. 40(27), 3685–3702 (2000)CrossRefGoogle Scholar
  11. 11.
    DeAngelis, G.C., Ohzawa, I., Freeman, R.D.: Receptive-field dynamics in the central visual pathways. TINS 18(10), 451–458 (1995)Google Scholar
  12. 12.
    Fregnac, Y., Blatow, M., Changeux, J.P., de Felipe, J., Lansner, A., Maass, W., McCormick, D.A., Michel, C.M., Monyer, H., Szathmary, E., Yuste, R.: UPs and DOWNs in cortical computation. In: Grillner, S., Graybiel, A.M. (eds.) The Interface between Neurons and Global Brain Function, pp. 393–433. Dahlem Workshop Report 93, MIT Press (2006)Google Scholar
  13. 13.
    Frégnac, Y., Monier, C., Chavane, F., Baudot, P., Graham, L.: Shunting inhibition a silent step in visual cortical computation. J. Physiol. 97(4–6), 441–451 (2003)Google Scholar
  14. 14.
    Glass, L., Perez, R.: Perception of random dot interference patterns. Nature 246, 360–362 (1973)CrossRefGoogle Scholar
  15. 15.
    Grossberg, S.: How does the cerebral cortex work? Learning, attention, and grouping by the laminar circuits of visual cortex. Spatial Vision 12, 163–185 (1999)CrossRefGoogle Scholar
  16. 16.
    Heeger, D.J.: Normalization of cell responses in cat striate cortex. Visual Neurosci. 9(2), 191–197 (1992)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160(1), 106–154 (1962)CrossRefGoogle Scholar
  18. 18.
    Koch, C.: Biophysics of Computation: Information Processing in Single Neurons. Oxford University Press, New York (1999)Google Scholar
  19. 19.
    Kouh, M., Poggio, T.: A canonical neural circuit for cortical nonlinear operations. Neural Comput. 20(6), 1427–1451 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Krause, M.R., Pack, C.C.: Contextual modulation and stimulus selectivity in extrastriate cortex. Vision Res. 104, 36–46 (2014)CrossRefGoogle Scholar
  21. 21.
    Larkum, M.: A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex. Trends Neurosci. 36(3), 141–151 (2013)CrossRefGoogle Scholar
  22. 22.
    Larkum, M.E., Senn, W., Lüscher, H.R.: Top-down dendritic input increases the gain of layer 5 pyramidal neurons. Cereb. Cortex 14(10), 1059–1070 (2004)CrossRefGoogle Scholar
  23. 23.
    Layher, G., Brosch, T., Neumann, H.: Towards a Mesoscopic-level canonical circuit definition for visual cortical processing. In: CMVC (2015)Google Scholar
  24. 24.
    Lee, C.C., Sherman, S.M.: Modulator property of the intrinsic cortical projection from layer 6 to layer 4. Front. Syst. Neurosci. 3(3), 1–5 (2009)Google Scholar
  25. 25.
    Li, W., Piëch, V., Gilbert, C.D.: Contour saliency in primary visual cortex. Neuron 50(6), 951–962 (2006)CrossRefGoogle Scholar
  26. 26.
    Merolla, P.A., Arthur, J.V., Alvarez-Icaza, R., Cassidy, A.S., Sawada, J., Akopyan, F., Jackson, B.L., Imam, N., Guo, C., Nakamura, Y., Brezzo, B., Vo, I., Esser, S.K., Appuswamy, R., Taba, B., Amir, A., Flickner, M.D., Risk, W.P., Manohar, R., Modha, D.S.: A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197), 668–673 (2014)CrossRefGoogle Scholar
  27. 27.
    Mountcastle, V.B.: The columnar organization of the neocortex. Brain 120(4), 701–722 (1997)CrossRefGoogle Scholar
  28. 28.
    Packer, A.M., Yuste, R.: Dense, unspecific connectivity of neocortical parvalbumin-positive interneurons: a canonical microcircuit for inhibition? J. Neurosci. 31(37), 13260–13271 (2011)CrossRefGoogle Scholar
  29. 29.
    Pfeffer, C.K.: Inhibitory neurons: vip cells hit the brake on inhibition. Curr. Biol. 24(1), R18–20 (2014)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Phillips, W.A., Clark, A., Silverstein, S.M.: On the functions, mechanisms, and malfunctions of intracortial contextual modulation. Neurosci. Biobehav. Rev. 52, 1–20 (2015)CrossRefGoogle Scholar
  31. 31.
    Raudies, F., Neumann, H.: A model of neural mechanisms in monocular transparent motion perception. J. Physiol.-Paris 104(1–2), 7183 (2010)Google Scholar
  32. 32.
    Reynolds, J.H., Heeger, D.J.: The normalization model of attention. Neuron 61, 168–185 (2009)CrossRefGoogle Scholar
  33. 33.
    Roelfsema, P.R.: Cortical algorithms for perceptual grouping. Ann. Rev. Neurosci. 29, 203–227 (2006)CrossRefGoogle Scholar
  34. 34.
    Thielscher, A., Neumann, H.: Neural mechanisms of cortico-cortical interaction in texture boundary detection: a modeling approach. Neuroscience 122, 921–939 (2003)CrossRefGoogle Scholar
  35. 35.
    Tschechne, S., Brosch, T., Sailer, R., von Egloffstein, N., Abdul-Kreem, L.I., Neumann, H.: On event-based motion detection and integration. In: 8th International Conference on Bio-inspired Information and Communications Technologies, BICT, pp. 298–305 (2014)Google Scholar
  36. 36.
    Tschechne, S., Sailer, R., Neumann, H.: Bio-Inspired optic flow from event-based neuromorphic sensor input. In: Gayar, N., Schwenker, F., Suen, C. (eds.) ANNPR 2014. LNCS (LNAI), vol. 8774, pp. 171–182. Springer International Publishing, Cham (2014). doi: 10.1007/978-3-319-11656-3_16 Google Scholar
  37. 37.
    Ullman, S.: Sequence seeking and counter streams: a computational model for bidirectional information flow in the visual cortex. Cereb. Cortex 5(1), 1–11 (1995)CrossRefGoogle Scholar
  38. 38.
    Wagatsuma, N., Potjans, T.C., Diesmann, M., Fukai, T.: Layer-dependent attentional processing by top-down signals in a visual cortical microcircuit model. Front. Comput. Neurosci. 5(31), 1–15 (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Neural Information Processing, Faculty of Engineering, Computer Science, and PsychologyUlm UniversityUlmGermany

Personalised recommendations