Short term memory properties of sensory neural architectures

  • A. M. DubreuilEmail author


A functional role of the cerebral cortex is to form and hold representations of the sensory world for behavioral purposes. This is achieved by a sheet of neurons, organized in modules called cortical columns, that receives inputs in a peculiar manner, with only a few neurons driven by sensory inputs through thalamic projections, and a vast majority of neurons receiving mainly cortical inputs. How should cortical modules be organized, with respect to sensory inputs, in order for the cortex to efficiently hold sensory representations in memory? To address this question we investigate the memory performance of trees of recurrent networks (TRN) that are composed of recurrent networks, modeling cortical columns, connected with each others through a tree-shaped feed-forward backbone of connections, with sensory stimuli injected at the root of the tree. On these sensory architectures two types of short-term memory (STM) mechanisms can be implemented, STM via transient dynamics on the feed-forward tree, and STM via reverberating activity on the recurrent connectivity inside modules. We derive equations describing the dynamics of such networks, which allow us to thoroughly explore the space of possible architectures and quantify their memory performance. By varying the divergence ratio of the tree, we show that serial architectures, where sensory inputs are successively processed in different modules, are better suited to implement STM via transient dynamics, while parallel architectures, where sensory inputs are simultaneously processed by all modules, are better suited to implement STM via reverberating dynamics.


Modular network Short-term memory Cortical hierarchy 



I would like to thank Nicolas Brunel and Haim Sompolinsky for useful discussions throughout the course of this work. I also would like to thank Gaetan Bouchet for his help with the figures.

Compliance with Ethical Standards

Conflict of interests

The authors declare that they have no conflict of interest.

Supplementary material

10827_2019_720_MOESM1_ESM.pdf (357 kb)
(PDF 357 KB)


  1. Amit, D.J. (1989). Modeling brain function. Cambridge University Press.Google Scholar
  2. Amit, D.J., & Brunel, N. (1997). Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral Cortex, 7, 237–252.CrossRefGoogle Scholar
  3. Bosking, W., Zhang, Y., Schofield, B., Fitzpatrick, D. (1997). Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. Journal of Neuroscience, 17, 2112–2127.CrossRefGoogle Scholar
  4. Braitenberg, V., & Schütz, A. (1991). Anatomy of the cortex. Springer.Google Scholar
  5. Brunel, N. (2016). Is cortical connectivity optimized for storing information [quest]? Nature Neuroscience, 19 (5), 749–755.CrossRefGoogle Scholar
  6. Coolen, A., & Viana, L. (1996). Feed-forward chains of recurrent attractor neural networks near saturation. Journal of Physics A: Mathematical and General, 29(24), 7855.CrossRefGoogle Scholar
  7. DeFelipe, J., Conley, M., Jones, E.G. (1986). Long-range focal collateralization of axons arising from corticocortical cells in monkey sensory-motor cortex. Journal of Neuroscience, 6, 3749–3766.CrossRefGoogle Scholar
  8. Dubreuil, A.M., & Brunel, N. (2016). Storing structured sparse memories in a multi-modular cortical network model. Journal of Computational Neuroscience, 40(2), 157–175.CrossRefGoogle Scholar
  9. Evans, M. (1989). Random dilution in a neural network for biased patterns. Journal of Physics A: Mathematical and General, 22(12), 2103.CrossRefGoogle Scholar
  10. Fuster, J.M. (1995). Memory in the cerebral cortex. MIT Press.Google Scholar
  11. Ganguli, S, Huh, D, Sompolinsky, H. (2008). Memory traces in dynamical systems. Proceedings of the National Academy of Sciences of the United States of America, 105, 18970–18975.CrossRefGoogle Scholar
  12. Gilbert, C.D., & Wiesel, T. (1989). Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex. Journal of Neuroscience, 9, 2432–2442.CrossRefGoogle Scholar
  13. Girard, P., & Bullier, J. (1989). Visual activity in area v2 during reversible inactivation of area 17 in the macaque monkey. Journal of neurophysiology, 62(6), 1287–1302.CrossRefGoogle Scholar
  14. Hopfield, J.J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79, 2554–2558.CrossRefGoogle Scholar
  15. Jaeger, H., & Haas, H. (2004). Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science, 304(5667), 78–80.CrossRefGoogle Scholar
  16. Johansson, C., & Lansner, A. (2007). Imposing biological constraints onto an abstract neocortical attractor network model. Neural Computation, 19(7), 1871–1896. Scholar
  17. Klampfl, S., David, S.V., Yin, P., Shamma, S.A., Maass, W. (2012). A quantitative analysis of information about past and present stimuli encoded by spikes of a1 neurons. Journal of Neurophysiology, 108(5), 1366–1380.CrossRefGoogle Scholar
  18. Lim, S., & Goldman, M.S. (2012). Noise tolerance of attractor and feedforward memory models. Neural Computation, 24(2), 332–390.CrossRefGoogle Scholar
  19. Maass, W., Natschläger, T, Markram, H. (2002). Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531–2560.CrossRefGoogle Scholar
  20. Mari, C.F., & Treves, A. (1998). Modeling neocortical areas with a modular neural network. Bio Systems, 48 (1), 47–55.CrossRefGoogle Scholar
  21. Markov, N., Misery, P., Falchier, A., Lamy, C., Vezoli, J., Quilodran, R., Gariel, M., Giroud, P., Ercsey-Ravasz, M., Pilaz, L., et al. (2010). Weight consistency specifies regularities of macaque cortical networks. Cerebral Cortex, 21(6), 1254–1272.CrossRefGoogle Scholar
  22. Nikolić, D, Häusler, S, Singer, W., Maass, W. (2009). Distributed fading memory for stimulus properties in the primary visual cortex. PLoS Biology, 7(12), e1000260.CrossRefGoogle Scholar
  23. O’Kane, D, & Treves, A. (1992). Short-and long-range connections in autoassociative memory. Journal of Physics A: Mathematical and General, 25, 5055.CrossRefGoogle Scholar
  24. Pucak, M.L., Levitt, J.B., Lund, J.S., Lewis, D.A. (1996). Patterns of intrinsic and associational circuitry in monkey prefrontal cortex. Journal of Comparative Neurology, 338, 360–376.Google Scholar
  25. Sejnowski, T.J. (1977). Storing covariance with nonlinearly interacting neurons. Journal of Mathematical Biology, 4, 303–.Google Scholar
  26. Sompolinsky, H., & White, O. (2003). Theory of large recurrent networks: from spikes to behavior. In Methods and models in neurophysics, volume session LXXX: lecture notes of the Les Houches Summer School (p Chapter 8).Google Scholar
  27. Thorpe, S.J., & Fabre-Thorpe, M. (2001). Seeking categories in the brain. Science, 291(5502), 260–263.CrossRefGoogle Scholar
  28. Tkačik, G, Prentice, J.S., Balasubramanian, V., Schneidman, E. (2010). Optimal population coding by noisy spiking neurons. Proceedings of the National Academy of Sciences, 107(32), 14419–14424.CrossRefGoogle Scholar
  29. Tsodyks, M., & Feigel’man, M.V. (1988). The enhanced storage capacity in neural networks with low activity level. Europhysics Letters, 6, 101–105.CrossRefGoogle Scholar
  30. White, O.L., Lee, D.D., Sompolinsky, H. (2004). Short-term memory in orthogonal neural networks. Physical Review Letters, 92(14), 148102.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Group for Neural Theory, Laboratoire de Neurosciences Cognitives Computationnelles, Département d’Etudes Cognitives, Ecole Normale Supérieure, INSERM U960PSL UniversityParisFrance

Personalised recommendations