Skip to main content
Book cover

The Functional Role of Critical Dynamics in Neural Systems pp 265–287Cite as

Statistical Models of Neural Activity, Criticality, and Zipf’s Law

Part of the Springer Series on Bio- and Neurosystems book series (SSBN,volume 11)

Abstract

We discuss the connections between the observations of critical dynamics in neuronal networks and the maximum entropy models that are often used as statistical models of neural activity, focusing in particular on the relation between statistical and dynamical criticality. We present examples of systems that are critical in one way, but not in the other, exemplifying thus the difference of the two concepts. We then discuss the emergence of Zipf laws in neural activity, verifying their presence in retinal activity under a number of different conditions. In the second part of the chapter we review connections between statistical criticality and the structure of the parameter space, as described by Fisher information. We note that the model-based signature of criticality, namely the divergence of specific heat, emerges independently of the dataset studied; we suggest this is compatible with previous theoretical findings.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-20965-0_13
  • Chapter length: 23 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   149.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-20965-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   199.99
Price excludes VAT (USA)
Hardcover Book
USD   199.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Aitchison, L., Corradi, N., Latham, P.E.: Zipf’s law arises naturally when there are underlying, unobserved variables. PLOS Comput. Biol. 12(12), 1–32 (2016). https://doi.org/10.1371/journal.pcbi.1005110

    CrossRef  Google Scholar 

  2. Athreya, K.B., Jagers, P.: Classical and Modern Branching Processes. IMA, vol. 84. Springer (1997)

    Google Scholar 

  3. Auerbach, F.: Das Gesetz der Bevölkerungskonzentration. Petermanns Geographische Mitteilungen 59, 74–76 (1913) (Quote translated by J.M.H.)

    Google Scholar 

  4. Barabási, A.L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999)

    CrossRef  Google Scholar 

  5. Beggs, J.M.: The criticality hypothesis: how local cortical networks might optimize information processing. Philos. Trans. R. Soc. Lond. A Math. Phys. Eng. Sci. 366(1864), 329–343 (2008)

    CrossRef  Google Scholar 

  6. Beggs, J.M., Plenz, D.: Neuronal avalanches in neocortical circuits. J. Neurosci. 23(35), 11167–11177 (2003)

    CrossRef  CAS  Google Scholar 

  7. Beggs, J.M., Timme, N.: Being critical of criticality in the brain. Front. Physiol. 3, 163 (2012)

    CrossRef  Google Scholar 

  8. Cristelli, M., Batty, M., Pietronero, L.: There is more than power law in Zipf. Sci. Rep. 2, 812(7) (2012)

    Google Scholar 

  9. Eurich, C.W., Herrmann, J.M., Ernst, U.A.: Finite-size effects of avalanche dynamics. Phys. Rev. E 66(6), 066,137 (2002)

    Google Scholar 

  10. Gabaix, X.: Zipf’s law and the growth of cities. Am. Econ. Rev. 89(2), 129–132 (1999)

    CrossRef  Google Scholar 

  11. Gardella, C., Marre, O., Mora, T.: Blindfold learning of an accurate neural metric. In: Proceedings of the National Academy of Sciences, p. 201718710 (2018)

    Google Scholar 

  12. Gautam, S.H., Hoang, T.T., McClanahan, K., Grady, S.K., Shew, W.L.: Maximizing sensory dynamic range by tuning the cortical state to criticality. PLoS Comput. Biol. 11(12), e1004,576 (2015)

    CrossRef  Google Scholar 

  13. Glauber, R.J.: Time-dependent statistics of the Ising model. J. Math. Phys. 4(2), 294–307 (1963)

    CrossRef  Google Scholar 

  14. Gutenkunst, R.N., Waterfall, J.J., Casey, F.P., Brown, K.S., Myers, C.R., Sethna, J.P.: Universally sloppy parameter sensitivities in systems biology models. PLoS Comput. Biol. 3(10), e189 (2007)

    CrossRef  Google Scholar 

  15. Hahn, G., Ponce-Alvarez, A., Monier, C., Benvenuti, G., Kumar, A., Chavane, F., Deco, G., Frgnac, Y.: Spontaneous cortical activity is transiently poised close to criticality. PLOS Comput. Biol. 13(5), 1–29 (2017). https://doi.org/10.1371/journal.pcbi.1005543

    CrossRef  Google Scholar 

  16. Hennig, M.H., Adams, C., Willshaw, D., Sernagor, E.: Early-stage waves in the retinal network emerge close to a critical state transition between local and global functional connectivity. J. Neurosci. 29(4), 1077–1086 (2009)

    CrossRef  CAS  Google Scholar 

  17. Herzog, R., Escobar, M.J., Cofre, R., Palacios, A.G., Cessac, B.: Dimensionality reduction on spatio-temporal maximum entropy models on spiking networks. Preprint bioRxiv:278606 (2018)

    Google Scholar 

  18. Hilgen, G., Sorbaro, M., Pirmoradian, S., Muthmann, J.O., Kepiro, I.E., Ullo, S., Ramirez, C.J., Encinas, A.P., Maccione, A., Berdondini, L., et al.: Unsupervised spike sorting for large-scale, high-density multielectrode arrays. Cell Rep. 18(10), 2521–2532 (2017)

    CrossRef  CAS  Google Scholar 

  19. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982)

    CrossRef  CAS  Google Scholar 

  20. Ising, E.: Beitrag zur Theorie des Ferromagnetismus. Z. für Phys. 31(1), 253–258 (1925)

    CrossRef  CAS  Google Scholar 

  21. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106(4), 620 (1957)

    CrossRef  Google Scholar 

  22. Jiang, B., Jia, T.: Zipf’s law for all the natural cities in the United States: a geospatial perspective. Int. J. Geogr. Inf. Sci. 25(8), 1269–1281 (2011)

    CrossRef  Google Scholar 

  23. Kinouchi, O., Copelli, M.: Optimal dynamical range of excitable networks at criticality. Nat. Phys. 2, 348–352 (2006)

    CrossRef  CAS  Google Scholar 

  24. Köster, U., Sohl-Dickstein, J., Gray, C.M., Olshausen, B.A.: Modeling higher-order correlations within cortical microcolumns. PLoS Comput. Biol. 10(7), e1003,684 (2014)

    CrossRef  Google Scholar 

  25. Larremore, D.B., Shew, W.L., Restrepo, J.G.: Predicting criticality and dynamic range in complex networks: effects of topology. Phys. Rev. Lett. 106(5), 058,101 (2011)

    Google Scholar 

  26. Li, W.: Random texts exhibit Zipf’s-law-like word frequency distribution. IEEE Trans. Inf. Theory 38(6), 1842–1845 (1992)

    CrossRef  Google Scholar 

  27. Machta, B.B., Chachra, R., Transtrum, M.K., Sethna, J.P.: Parameter space compression underlies emergent theories and predictive models. Science 342(6158), 604–607 (2013)

    CrossRef  CAS  Google Scholar 

  28. Marre, O., El Boustani, S., Frégnac, Y., Destexhe, A.: Prediction of spatiotemporal patterns of neural activity from pairwise correlations. Phys. Rev. Lett. 102(13), 138,101 (2009)

    Google Scholar 

  29. Mastromatteo, I., Marsili, M.: On the criticality of inferred models. J. Stat. Mech. Theory Exp. 2011(10), P10,012 (2011)

    CrossRef  Google Scholar 

  30. Mizuseki, K., Buzsáki, G.: Preconfigured, skewed distribution of firing rates in the hippocampus and entorhinal cortex. Cell Rep. 4(5), 1010–1021 (2013)

    CrossRef  CAS  Google Scholar 

  31. Mora, T., Deny, S., Marre, O.: Dynamical criticality in the collective activity of a population of retinal neurons. Phys. Rev. Lett. 114(7), 078,105 (2015)

    Google Scholar 

  32. Nasser, H., Marre, O., Cessac, B.: Spatio-temporal spike train analysis for large scale networks using the maximum entropy principle and Monte Carlo method. J. Stat. Mech. Theory Exp. 2013(03), P03,006 (2013)

    CrossRef  Google Scholar 

  33. Newman, M.E.: Power laws, Pareto distributions and Zipf’s law. Contemp. Phys. 46(5), 323–351 (2005)

    CrossRef  Google Scholar 

  34. Nishimori, H.: Statistical Physics of Spin Glasses and Information Processing: An Introduction, vol. 111. Clarendon Press (2001)

    Google Scholar 

  35. Nonnenmacher, M., Behrens, C., Berens, P., Bethge, M., Macke, J.H.: Signatures of criticality arise from random subsampling in simple population models. PLoS Comput. Biol. 13(10), e1005,718 (2017)

    CrossRef  Google Scholar 

  36. O’Donnell, C., Gonçalves, J.T., Whiteley, N., Portera-Cailliau, C., Sejnowski, T.J.: The population tracking model: a simple, scalable statistical model for neural population data. Neural Comput. (2016)

    Google Scholar 

  37. Ohiorhenuan, I.E., Mechler, F., Purpura, K.P., Schmid, A.M., Hu, Q., Victor, J.D.: Sparse coding and high-order correlations in fine-scale cortical networks. Nature 466(7306), 617–621 (2010)

    CrossRef  CAS  Google Scholar 

  38. Panas, D., Amin, H., Maccione, A., Muthmann, O., van Rossum, M., Berdondini, L., Hennig, M.H.: Sloppiness in spontaneously active neuronal networks. J. Neurosci. 35(22), 8480–8492 (2015)

    CrossRef  CAS  Google Scholar 

  39. Priesemann, V., Valderrama, M., Wibral, M., Le Van Quyen, M.: Neuronal avalanches differ from wakefulness to deep sleep–evidence from intracranial depth recordings in humans. PLoS Comput. Biol. 9(3), e1002,985 (2013)

    CrossRef  CAS  Google Scholar 

  40. Redner, S.: How popular is your paper? An empirical study of the citation distribution. Eur. Phys. J. B Condens. Matter Complex Syst. 4(2), 131–134 (1998)

    CrossRef  CAS  Google Scholar 

  41. Schneidman, E., Berry, M.J., Segev, R., Bialek, W.: Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440(7087), 1007–1012 (2006)

    CrossRef  CAS  Google Scholar 

  42. Shew, W.L., Plenz, D.: The functional benefits of criticality in the cortex. The Neuroscientist 19(1), 88–100 (2013)

    CrossRef  Google Scholar 

  43. Shew, W.L., Yang, H., Petermann, T., Roy, R., Plenz, D.: Neuronal avalanches imply maximum dynamic range in cortical networks at criticality. J. Neurosci. 29(49), 15595–15600 (2009)

    CrossRef  CAS  Google Scholar 

  44. Shlens, J., Field, G.D., Gauthier, J.L., Grivich, M.I., Petrusca, D., Sher, A., Litke, A.M., Chichilnisky, E.: The structure of multi-neuron firing patterns in primate retina. J. Neurosci. 26(32), 8254–8266 (2006)

    CrossRef  CAS  Google Scholar 

  45. Song, J., Marsili, M., Jo, J.: Emergence and relevance of criticality in deep learning (2017). arXiv preprint arXiv:1710.11324

  46. Tang, A., Jackson, D., Hobbs, J., Chen, W., Smith, J.L., Patel, H., Prieto, A., Petrusca, D., Grivich, M.I., Sher, A., Hottowy, P., Dabrowski, W., Litke, A.M., Beggs, J.M.: A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. J. Neurosci. 28, 505518 (2008)

    CrossRef  Google Scholar 

  47. Tkačik, G., Marre, O., Amodei, D., Schneidman, E., Bialek, W., Berry II, M.J.: Searching for collective behavior in a large network of sensory neurons. PLoS Comput. Biol. 10(1), e1003,408 (2014)

    CrossRef  Google Scholar 

  48. Tkačik, G., Mora, T., Marre, O., Amodei, D., Palmer, S.E., Berry, M.J., Bialek, W.: Thermodynamics and signatures of criticality in a network of neurons. Proc. Natl. Acad. Sci. 112(37), 11508–11513 (2015)

    CrossRef  Google Scholar 

  49. Vázquez-Rodríguez, B., Avena-Koenigsberger, A., Sporns, O., Griffa, A., Hagmann, P., Larralde, H.: Stochastic resonance at criticality in a network model of the human cortex. Sci. Rep. 7(1), 13,020 (2017)

    Google Scholar 

  50. Vitanov, N.K., Ausloos, M.: Test of two hypotheses explaining the size of populations in a system of cities. J. Appl. Stat. 42(12), 2686–2693 (2015)

    CrossRef  Google Scholar 

  51. Yu, S., Yang, H., Nakahara, H., Santos, G.S., Nikolić, D., Plenz, D.: Higher-order interactions characterized in cortical activity. J. Neurosci. 30(48), 17514–17526 (2011)

    CrossRef  Google Scholar 

  52. Zipf, G.K.: Human Behavior and the Principle of Least Effort. Addison-Wesley, Cambridge (1949)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Informatics, University of Edinburgh, 10 Crichton St., Edinburgh, EH8 9AB, UK

    Martino Sorbaro, J. Michael Herrmann & Matthias Hennig

Authors
  1. Martino Sorbaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Michael Herrmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthias Hennig
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martino Sorbaro .

Editor information

Editors and Affiliations

  1. Institute for Theoretical Physics, University of Bremen, Bremen, Germany

    Nergis Tomen

  2. Institute of Perception, Action and Behaviour, University of Edinburgh, Edinburgh, UK

    Dr. J. Michael Herrmann

  3. Institute for Theoretical Physics, University of Bremen, Bremen, Germany

    Dr. Udo Ernst

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Sorbaro, M., Herrmann, J.M., Hennig, M. (2019). Statistical Models of Neural Activity, Criticality, and Zipf’s Law. In: Tomen, N., Herrmann, J., Ernst, U. (eds) The Functional Role of Critical Dynamics in Neural Systems . Springer Series on Bio- and Neurosystems, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-20965-0_13

Download citation