Skip to main content
SpringerLink
Log in

Fading Memory, Plasticity, and Criticality in Recurrent Networks

  • Chapter
  • First Online:
The Functional Role of Critical Dynamics in Neural Systems

Part of the book series: Springer Series on Bio- and Neurosystems ((SSBN,volume 11))

Abstract

Criticality signatures, in the form of power-law distributed neuronal avalanches, have been widely measured in vitro and provide the foundation for the so-called critical brain hypothesis, which proposes that healthy neural circuits operate near a phase transition state with maximal information processing capabilities. Here, we revisit a recently published analysis on the occurrence of those signatures in the activity of a recurrent neural network model that self-organizes due to biologically inspired plasticity rules. Interestingly, the criticality signatures are input dependent: they transiently break down due to onset of random external input, but do not appear under repeating input sequences during learning tasks. Additionally, we show that an important information processing ability, the fading memory time scale, is improved when criticality signatures appear, potentially facilitating complex computations. Taken together, the results suggest that a combination of plasticity mechanisms that improves the network’s spatio-temporal learning abilities and memory time scale also yields power-law distributed neuronal avalanches under particular input conditions, thus suggesting a link between such abilities and avalanche criticality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alstott, J., Bullmore, E., Plenz, D.: powerlaw: a python package for analysis of heavy-tailed distributions. PloS One 9(1), e85,777 (2014)

    Google Scholar 

  2. de Arcangelis, L., Perrone-Capano, C., Herrmann, H.J.: Self-organized criticality model for brain plasticity. Phys. Rev. Lett. 96(2), 028,107 (2006)

    Google Scholar 

  3. Arviv, O., Goldstein, A., Shriki, O.: Near-critical dynamics in stimulus-evoked activity of the human brain and its relation to spontaneous resting-state activity. J. Neurosci. 35(41), 13927–13942 (2015)

    Article  CAS  Google Scholar 

  4. Bak, P., Tang, C., Wiesenfeld, K.: Self-organized criticality: An explanation of the 1/f noise. Phys. Rev. Lett. 59(4), 381 (1987)

    Article  CAS  Google Scholar 

  5. Bedard, C., Kroeger, H., Destexhe, A.: Does the 1/f frequency scaling of brain signals reflect self-organized critical states? Phys. Rev. Lett. 97(11), 118,102 (2006)

    Google Scholar 

  6. Beggs, J.M., Plenz, D.: Neuronal avalanches in neocortical circuits. J. Neurosci. 23(35), 11167–11177 (2003)

    Article  CAS  Google Scholar 

  7. Beggs, J.M., Timme, N.: Being critical of criticality in the brain. Front. Physiol. 3 (2012)

    Google Scholar 

  8. Bertschinger, N., Natschläger, T.: Real-time computation at the edge of chaos in recurrent neural networks. Neural Comput. 16(7), 1413–1436 (2004)

    Article  Google Scholar 

  9. Bertschinger, N., Rauh, J., Olbrich, E., Jost, J.: Shared information–new insights and problems in decomposing information in complex systems. In: Proceedings of the European Conference on Complex Systems 2012, pp. 251–269. Springer (2013)

    Google Scholar 

  10. Bi, G.Q., Poo, M.M.: Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18(24), 10,464–10,472 (1998)

    Google Scholar 

  11. Boedecker, J., Obst, O., Lizier, J.T., Mayer, N.M., Asada, M.: Information processing in echo state networks at the edge of chaos. Theory Biosci. 131(3), 205–213 (2012)

    Article  Google Scholar 

  12. Chialvo, D.R.: Emergent complex neural dynamics. Nat. Phys. 6(10), 744–750 (2010)

    Article  CAS  Google Scholar 

  13. Clauset, A., Shalizi, C.R., Newman, M.E.: Power-law distributions in empirical data. SIAM Rev. 51(4), 661–703 (2009)

    Article  Google Scholar 

  14. Dahmen, D., Diesmann, M., Helias, M.: Distributions of covariances as a window into the operational regime of neuronal networks (2016). arXiv:1605.04153

  15. Dahmen, D., Grün, S., Diesmann, M., Helias, M.: Two types of criticality in the brain (2017). arXiv:1711.10930

  16. Del Papa, B., Priesemann, V., Triesch, J.: Criticality meets learning: criticality signatures in a self-organizing recurrent neural network. PloS One 12(5), e0178,683 (2017)

    Google Scholar 

  17. Duarte, R., Series, P., Morrison, A.: Self-organized artificial grammar learning in spiking neural networks. In: Proceedings of the 36th Annual Conference of the Cognitive Science Society, pp. 427–432 (2014)

    Google Scholar 

  18. Friedman, N., Ito, S., Brinkman, B.A., Shimono, M., DeVille, R.L., Dahmen, K.A., Beggs, J.M., Butler, T.C.: Universal critical dynamics in high resolution neuronal avalanche data. Phys. Rev. Lett. 108(20), 208,102 (2012)

    Google Scholar 

  19. Gautam, S.H., Hoang, T.T., McClanahan, K., Grady, S.K., Shew, W.L.: Maximizing sensory dynamic range by tuning the cortical state to criticality. PLoS Comput. Biol. 11(12), e1004,576 (2015)

    Google Scholar 

  20. Haldeman, C., Beggs, J.M.: Critical branching captures activity in living neural networks and maximizes the number of metastable states. Phys. Rev. Lett. 94(5), 058,101 (2005)

    Google Scholar 

  21. Hartmann, C., Lazar, A., Nessler, B., Triesch, J.: Where’s the noise? Key features of spontaneous activity and neural variability arise through learning in a deterministic network. PLoS Comput. Biol. 11(12), e1004,640–e1004,640 (2015)

    Google Scholar 

  22. Hesse, J., Gross, T.: Self-organized criticality as a fundamental property of neural systems. Front. Syst. Neurosci. 8, (2014)

    Google Scholar 

  23. Jaeger, H.: Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the “echo state network” approach, vol. 5. GMD-Forschungszentrum Informationstechnik (2002)

    Google Scholar 

  24. Kanders, K., Lorimer, T., Stoop, R.: Avalanche and edge-of-chaos criticality do not necessarily co-occur in neural networks. Chaos Interdiscip. J. Nonlinear Sci. 27(4), 047,408 (2017)

    Google Scholar 

  25. Kanders, K., Lorimer, T., Uwate, Y., Steeb, W.H., Stoop, R.: Robust transformations of firing patterns for neural networks (2017). arXiv:1708.04168

  26. Kinouchi, O., Copelli, M.: Optimal dynamical range of excitable networks at criticality. Nat. Phys. 2(5), 348–351 (2006)

    Article  CAS  Google Scholar 

  27. Lazar, A., Pipa, G., Triesch, J.: Fading memory and time series prediction in recurrent networks with different forms of plasticity. Neural Netw. 20(3), 312–322 (2007)

    Article  Google Scholar 

  28. Lazar, A., Pipa, G., Triesch, J.: Sorn: a self-organizing recurrent neural network. Front. Comput. Neurosci. 3 (2009)

    Google Scholar 

  29. Lazar, A., Pipa, G., Triesch, J.: Emerging bayesian priors in a self-organizing recurrent network. In: Artificial Neural Networks and Machine Learning–ICANN 2011, pp. 127–134. Springer (2011)

    Google Scholar 

  30. Legenstein, R., Maass, W.: Edge of chaos and prediction of computational performance for neural circuit models. Neural Netw. 20(3), 323–334 (2007)

    Article  Google Scholar 

  31. Levina, A., Herrmann, J.M., Geisel, T.: Dynamical synapses causing self-organized criticality in neural networks. Nature Phys. 3(12), 857–860 (2007)

    Article  CAS  Google Scholar 

  32. Levina, A., Priesemann, V.: Subsampling scaling. Nat. Commun. 8, 15,140 (2017)

    Google Scholar 

  33. Lombardi, F., Herrmann, H., Perrone-Capano, C., Plenz, D., De Arcangelis, L.: Balance between excitation and inhibition controls the temporal organization of neuronal avalanches. Phys. Rev. Lett. 108(22), 228,703 (2012)

    Google Scholar 

  34. Lukoševičius, M.: A practical guide to applying echo state networks. In: Neural Networks: Tricks of the Trade, pp. 659–686. Springer (2012)

    Google Scholar 

  35. Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002)

    Article  Google Scholar 

  36. Massobrio, P., de Arcangelis, L., Pasquale, V., Jensen, H.J., Plenz, D.: Criticality as a signature of healthy neural systems. Front. Syst. Neurosci. 9 (2015)

    Google Scholar 

  37. Meisel, C., Gross, T.: Adaptive self-organization in a realistic neural network model. Phys. Rev. E 80(6), 061,917 (2009)

    Google Scholar 

  38. Palva, J.M., Zhigalov, A., Hirvonen, J., Korhonen, O., Linkenkaer-Hansen, K., Palva, S.: Neuronal long-range temporal correlations and avalanche dynamics are correlated with behavioral scaling laws. Proc. Natl. Acad. Sci. 110(9), 3585–3590 (2013)

    Article  CAS  Google Scholar 

  39. Palva, S., Palva, J.M.: Roles of brain criticality and multiscale oscillations in temporal predictions for sensorimotor processing. Trends Neurosci. 41(10), 729–743 (2018)

    Article  CAS  Google Scholar 

  40. Petermann, T., Thiagarajan, T.C., Lebedev, M.A., Nicolelis, M.A., Chialvo, D.R., Plenz, D.: Spontaneous cortical activity in awake monkeys composed of neuronal avalanches. Proc. Natl. Acad. Sci. 106(37), 15921–15926 (2009)

    Article  CAS  Google Scholar 

  41. Plenz, D.: Viewpoint: the critical brain. Physics 6, 47 (2013)

    Article  Google Scholar 

  42. Poil, S.S., Hardstone, R., Mansvelder, H.D., Linkenkaer-Hansen, K.: Critical-state dynamics of avalanches and oscillations jointly emerge from balanced excitation/inhibition in neuronal networks. J. Neurosci. 32(29), 9817–9823 (2012)

    Article  CAS  Google Scholar 

  43. Priesemann, V., Munk, M.H., Wibral, M.: Subsampling effects in neuronal avalanche distributions recorded in vivo. BMC Neurosci. 10(1), 40 (2009)

    Article  Google Scholar 

  44. Priesemann, V., Shriki, O.: Can a time varying external drive give rise to apparent criticality in neural systems? PLoS Comput. Biol. 14(5), e1006,081 (2018)

    Google Scholar 

  45. Priesemann, V., Valderrama, M., Wibral, M., Le Van Quyen, M.: Neuronal avalanches differ from wakefulness to deep sleep–evidence from intracranial depth recordings in humans. PLoS Comput. Biol. 9(3), e1002,985 (2013)

    Google Scholar 

  46. Priesemann, V., Wibral, M., Valderrama, M., Pröpper, R., Le Van Quyen, M., Geisel, T., Triesch, J., Nikolić, D., Munk, M.H.: Spike avalanches in vivo suggest a driven, slightly subcritical brain state. Front. Syst. Neurosci. 8 (2014)

    Google Scholar 

  47. Ribeiro, T.L., Copelli, M., Caixeta, F., Belchior, H., Chialvo, D.R., Nicolelis, M.A., Ribeiro, S.: Spike avalanches exhibit universal dynamics across the sleep-wake cycle. PloS One 5(11), e14,129 (2010)

    Google Scholar 

  48. Schwab, D.J., Nemenman, I., Mehta, P.: Zipf’s law and criticality in multivariate data without fine-tuning. Phys. Rev. Lett. 113(6), 068,102 (2014)

    Google Scholar 

  49. Scott, G., Fagerholm, E.D., Mutoh, H., Leech, R., Sharp, D.J., Shew, W.L., Knöpfel, T.: Voltage imaging of waking mouse cortex reveals emergence of critical neuronal dynamics. J. Neurosci. 34(50), 16611–16620 (2014)

    Article  CAS  Google Scholar 

  50. Shew, W.L., Clawson, W.P., Pobst, J., Karimipanah, Y., Wright, N.C., Wessel, R.: Adaptation to sensory input tunes visual cortex to criticality. Nat. Phys. 11(8), 659–663 (2015)

    Article  CAS  Google Scholar 

  51. Shew, W.L., Plenz, D.: The functional benefits of criticality in the cortex. Neuroscientist 19(1), 88–100 (2013)

    Article  Google Scholar 

  52. Shew, W.L., Yang, H., Petermann, T., Roy, R., Plenz, D.: Neuronal avalanches imply maximum dynamic range in cortical networks at criticality. J. Neurosci. 29(49), 15595–15600 (2009)

    Article  CAS  Google Scholar 

  53. Shew, W.L., Yang, H., Yu, S., Roy, R., Plenz, D.: Information capacity and transmission are maximized in balanced cortical networks with neuronal avalanches. J. Neurosci. 31(1), 55–63 (2011)

    Article  CAS  Google Scholar 

  54. Shriki, O., Alstott, J., Carver, F., Holroyd, T., Henson, R.N., Smith, M.L., Coppola, R., Bullmore, E., Plenz, D.: Neuronal avalanches in the resting meg of the human brain. J. Neurosci. 33(16), 7079–7090 (2013)

    Article  CAS  Google Scholar 

  55. Stepp, N., Plenz, D., Srinivasa, N.: Synaptic plasticity enables adaptive self-tuning critical networks. PLoS Comput. Biol. 11(1) (2015)

    Google Scholar 

  56. Tagliazucchi, E., Balenzuela, P., Fraiman, D., Chialvo, D.R.: Criticality in large-scale brain fmri dynamics unveiled by a novel point process analysis. Front. Physiol. 3(15) (2012). https://doi.org/10.3389/fphys.2012.00015

  57. Tetzlaff, C., Okujeni, S., Egert, U., Wörgötter, F., Butz, M.: Self-organized criticality in developing neuronal networks. PLoS Comput. Biol. 6(12), e1001,013 (2010)

    Google Scholar 

  58. Tomen, N., Rotermund, D., Ernst, U.: Marginally subcritical dynamics explain enhanced stimulus discriminability under attention. Front. Syst. Neurosci. 8, 151 (2014)

    Article  Google Scholar 

  59. Touboul, J., Destexhe, A.: Can power-law scaling and neuronal avalanches arise from stochastic dynamics. PloS One 5(2), e8982 (2010)

    Article  Google Scholar 

  60. Uhlig, M., Levina, A., Geisel, T., Herrmann, J.M.: Critical dynamics in associative memory networks. Front. Comput. Neurosci. 7 (2013)

    Google Scholar 

  61. Wibral, M., Finn, C., Wollstadt, P., Lizier, J.T., Priesemann, V.: Quantifying information modification in developing neural networks via partial information decomposition. Entropy 19(9), 494 (2017)

    Article  Google Scholar 

  62. Wibral, M., Lizier, J.T., Priesemann, V.: Bits from brains for biologically inspired computing. Front. Robot. AI 2, 5 (2015)

    Article  Google Scholar 

  63. Williams, P.L., Beer, R.D.: Nonnegative decomposition of multivariate information (2010). arXiv:1004.2515

  64. Williams-García, R.V., Moore, M., Beggs, J.M., Ortiz, G.: Quasicritical brain dynamics on a nonequilibrium widom line. Phys. Rev. E 90(6), 062,714 (2014)

    Google Scholar 

  65. Wilting, J., Dehning, J., Neto, J.P., Rudelt, L., Wibral, M., Zierenberg, J., Priesemann, V.: Dynamic adaptive computation: tuning network states to task requirements (2018). arXiv:1809.07550

  66. Wilting, J., Priesemann, V.: Inference, validation and predictions about statistics and propagation of cortical spiking in vivo, p. 363085 (2018)

    Google Scholar 

  67. Wilting, J., Priesemann, V.: Inferring collective dynamical states from widely unobserved systems. Nat. Commun. 9(1), 2325 (2018)

    Article  Google Scholar 

  68. Yaghoubi, M., de Graaf, T., Orlandi, J.G., Girotto, F., Colicos, M.A., Davidsen, J.: Neuronal avalanche dynamics indicates different universality classes in neuronal cultures. Sci. Rep. 8(1), 3417 (2018)

    Article  Google Scholar 

  69. Zheng, P., Dimitrakakis, C., Triesch, J.: Network self-organization explains the statistics and dynamics of synaptic connection strengths in cortex. PLoS Comput. Biol. 9(1), e1002,848 (2013)

    Google Scholar 

  70. Zheng, P., Triesch, J.: Robust development of synfire chains from multiple plasticity mechanisms. Front. Comput. Neurosci. 8, 66 (2014)

    Article  Google Scholar 

  71. Zierenberg, J., Wilting, J., Priesemann, V.: Homeostatic plasticity and external input shape neural network dynamics. Phys. Rev. X 8(3), 031,018 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Frankfurt Institute for Advanced Studies, Ruth-Moufang-Str. 1, 60438, Frankfurt, Germany

    Bruno Del Papa & Jochen Triesch

  2. Department of Non-linear Dynamics, Max Planck Institute for Dynamics and Self-Organization, Am Fassberg 17, 37077, Göttingen, Germany

    Viola Priesemann

Authors
  1. Bruno Del Papa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Viola Priesemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jochen Triesch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jochen Triesch .

Editor information

Editors and Affiliations

  1. Institute for Theoretical Physics, University of Bremen, Bremen, Germany

    Nergis Tomen

  2. Institute of Perception, Action and Behaviour, University of Edinburgh, Edinburgh, UK

    J. Michael Herrmann

  3. Institute for Theoretical Physics, University of Bremen, Bremen, Germany

    Udo Ernst

Appendix

Appendix

1.1 Power-Law Fitting

Although power-laws are very common in nature, their characterization is particularly complex and must be carefully evaluated. In particular, many false positives may appear, ranging from inaccurate exponents to wrong distribution fits [13]. Such problems can be better understood and avoided using maximum likelihood estimators. In this study, we followed the procedure of a previous work [16], employing the powerlaw python package [1] in order to estimate the distributions exponents. For all the probability distributions of avalanche durations and sizes shown here, power-laws with cut-offs provided better fits when compared to other single parameter distributions, such as exponential distributions. The cut-off was chosen in a case by case analysis, based on the best power-law fit. For a step-by-step description of power-law fitting via maximum likelihood estimators, see [13].

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Del Papa, B., Priesemann, V., Triesch, J. (2019). Fading Memory, Plasticity, and Criticality in Recurrent Networks. In: Tomen, N., Herrmann, J., Ernst, U. (eds) The Functional Role of Critical Dynamics in Neural Systems . Springer Series on Bio- and Neurosystems, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-20965-0_6

Download citation

Publish with us

Policies and ethics

Search

Navigation