Skip to main content

Going Beyond a Mean-field Model for the Learning Cortex: Second-Order Statistics

Journal of Biological Physics volume 33, pages 213–246 (2007)Cite this article

Abstract

Mean-field models of the cortex have been used successfully to interpret the origin of features on the electroencephalogram under situations such as sleep, anesthesia, and seizures. In a mean-field scheme, dynamic changes in synaptic weights can be considered through fluctuation-based Hebbian learning rules. However, because such implementations deal with population-averaged properties, they are not well suited to memory and learning applications where individual synaptic weights can be important. We demonstrate that, through an extended system of equations, the mean-field models can be developed further to look at higher-order statistics, in particular, the distribution of synaptic weights within a cortical column. This allows us to make some general conclusions on memory through a mean-field scheme. Specifically, we expect large changes in the standard deviation of the distribution of synaptic weights when fluctuation in the mean soma potentials are large, such as during the transitions between the “up” and “down” states of slow-wave sleep. Moreover, a cortex that has low structure in its neuronal connections is most likely to decrease its standard deviation in the weights of excitatory to excitatory synapses, relative to the square of the mean, whereas a cortex with strongly patterned connections is most likely to increase this measure. This suggests that fluctuations are used to condense the coding of strong (presumably useful) memories into fewer, but dynamic, neuron connections, while at the same time removing weaker (less useful) memories.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Wilson, H.R., Cowan, J.D.: Excitatory and inhibitory interactions in localized populations of model neurons. Biophys. J. 12, 1–24 (1972)

    ADS  Article  Google Scholar 

  2. Nunez, P.L.: The brain wave function: a model for the EEG. Math. Biosci. 21, 279–297 (1974)

    MATH  Article  Google Scholar 

  3. Freeman, W.J.: Predictions on neocortical dynamics derived from studies in paleocortex. In: Basar, E., Bullock, T.H. (eds.) Induced Rhythms of the Brain, chap. 9, pp. 183–199. Birkhaeuser, Boston (1992)

    Google Scholar 

  4. Wright, J.J., Liley, D.T.J.: Dynamics of the brain at global and microscopic scales: neural networks and the EEG. Behav. Brain Sci. 19, 285–316 (1996)

    Article  Google Scholar 

  5. Robinson, P.A., Rennie, C.J., Wright, J.J.: Propagation and stability of waves of electrical activity in the cerebral cortex. Phys. Rev. E 56, 826–840 (1997)

    Article  ADS  Google Scholar 

  6. Liley, D.T.J., Cadusch, P.J., Wright, J.J.: A continuum theory of electro-cortical activity. Neurocomputers 26–27, 795–800 (1999)

    Article  Google Scholar 

  7. Rennie, C.J., Wright, J.J., Robinson, P.A.: Mechanisms for cortical electrical activity and emergence of gamma rhythm. J. Theor. Biol. 205, 17–35 (2000)

    Article  Google Scholar 

  8. Steyn-Ross, M.L., Steyn-Ross, D.A., Sleigh, J.W.: Modelling general anaesthesia as a first-order phase transition in the cortex. Prog. Biophys. Mol. Biol. 85, 369–385 (2004)

    Article  Google Scholar 

  9. Hutt, A., Bestehorn, M., Wennekers, T.: Pattern formation in intracortical neuronal fields. Network 14, 351–368 (2003)

    Article  Google Scholar 

  10. Kramer, M.A., Kirsch, H.E., Szeri, A.J.: Pathological pattern formation and epileptic seizures. J. R. Soc. Lond. Interface 2, 113 (2005)

    Article  Google Scholar 

  11. Chizhov, A.V., Graham, L.J., Turbin, A.A.: Simulation of neural population dynamics with a refractory density approach and a conductance-based threshold neuron model. Neurocomputing 70(1–3), 252–262 (2006)

    Article  Google Scholar 

  12. Bazhenov, M., Timofeev, I., Steriade, M., Sejnowski, T.J.: Model of thalamocortical slow-wave sleep oscillations and transitions to activated states. J. Neurosci. 22, 8691–8704 (2002)

    Google Scholar 

  13. Compte, A., Sanchez-Vives, M.V., McCormick, D.A., Wang, X.J.: Cellular and network mechanisms of slow oscillatory activity (<1 Hz) and wave propagations in a cortical network model. J. Neurophysiol. 89, 2707–2725 (2003)

    Article  Google Scholar 

  14. Hill, S., Tononi, G.: Modeling sleep and wakefulness in the thalamocortical system. J. Neurophysiol. 93, 1671–1698 (2005)

    Article  Google Scholar 

  15. Robinson, P.A., Rennie, C.J., Rowe, D.L., O’Connor, S.C., Wright, J.J., Gordon, E., Whitehouse, R.W.: Neurophysical modeling of brain dynamics. Neuropsychopharmacology 28, S74–S79 (2003)

    Article  Google Scholar 

  16. Robinson, P.A., Rennie, C.J., Wright, J.J., Bahramali, H., Gordon, E., Rowe, D.L.: Prediction of electroencephalographic spectra from neurophysiology. Phys. Rev. E 63, 021,903 (2001)

    Google Scholar 

  17. Wilson, M.T., Steyn-Ross, D.A., Sleigh, J.W., Steyn-Ross, M.L., Wilcocks, L.C., Gillies, I.P.: The k-complex and slow oscillation in terms of a mean-field cortical model. J. Comput. Neurosci. 21, 243–257 (2006)

    Article  MathSciNet  Google Scholar 

  18. Bojak, I., Liley, D.T.J.: Modelling the effects of anaesthesia on the electroencephalogram. Phys. Rev. E 71, 41902 (2005)

    Article  ADS  Google Scholar 

  19. Wilson, M.T., Steyn-Ross, M.L., Steyn-Ross, D.A., Sleigh, J.W.: Predictions and simulations of cortical dynamics during natural sleep using a continuum approach. Phys. Rev. E 72, 051910 1–14 (2005)

    Article  ADS  MathSciNet  Google Scholar 

  20. Bienenstock, E.L., Cooper, L.N., Munro, P.W.: Theory for the development of neuron selectivity: orientation specificity and binocular interation in visual cortex. J. Neurosci. 2, 32–48 (1982)

    Google Scholar 

  21. Bienenstock, E., Lehmann, D.: Regulated criticality in the brain? Adv. Complex Systems 1, 361–384 (1998)

    Article  Google Scholar 

  22. Sandberg, A., Tegnér, J., Lansner, A.: A working memory model based on fast Hebbian learning. Netw. Comput. Neural Syst. 14, 789–802 (2003)

    Article  ADS  Google Scholar 

  23. Mongillo, G., Amit, D.J., Brunel, N.: Retrospective and prospective persistent activity induced by Hebbian learning in a recurrent cortical network. Eur. J. Neurosci. 18, 2011–2024 (2003)

    Article  Google Scholar 

  24. Hebb, D.O.: The Organization of Behaviour. Wiley, New York (1949)

    Google Scholar 

  25. Steyn-Ross, M.L., Steyn-Ross, D.A., Sleigh, J.W., Wilson, M.T., Wilcocks, L.C.: A mechanism for learning and memory erasure in a white-noise driven sleeping cortex. Phys. Rev. E 72, 061,910 (2005)

    Article  MathSciNet  Google Scholar 

  26. Stetter, M.: Dynamic functional tuning of nonlinear cortical networks. Phys. Rev. E 73, 031903 (2006)

    Article  ADS  MathSciNet  Google Scholar 

  27. Steyn-Ross, D.A., Steyn-Ross, M.L., Sleigh, J.W., Wilson, M.T., Gillies, I.P., Wright, J.J.: The sleep cycle modelled as a cortical phase transition. J. Biophys. 31, 547–569 (2005)

    Google Scholar 

  28. Tononi, G., Cirelli, C.: Sleep function and synaptic homeostatis. Sleep Med. Rev. 10, 49–62 (2006)

    Article  Google Scholar 

  29. Mountcastle, V.B.: The columnar organization of the neocortex. Brain 120, 701–722 (1997)

    Article  Google Scholar 

  30. Sejnowski, T.J.: Storing covariance with nonlinearly interacting neurons. J. Math. Biol. 4, 303–321 (1977)

    Article  Google Scholar 

  31. Douglas, R.J., Martin, K.A.: Recurrent neuronal circuits in the neocortex. Curr. Biol. 17(13), R496 (2007)

    Article  Google Scholar 

  32. Thomson, A.M., Bannister, A.P.: Interlaminar connections in the neocortex. Cerebral Cortex 13, 5–14 (2003)

    Article  Google Scholar 

  33. Tononi, G., Sporns, O.: Measuring information integration. BMC Neurosci. 4, 31 (2003)

    Article  Google Scholar 

  34. Albert, R., Barabási, A.L.: Statistical mechanics of complex networks. Rev. Mod. Phys. 74(1), 47–97 (2002)

    Article  ADS  Google Scholar 

  35. Kloeden, P.E., Platen, E.: Numerical Solution of Stochastc Differential Equations. Springer, Berlin (1992)

    Google Scholar 

  36. Rudolph, M., Pospischil, M., Timofeev, I., Destexhe, A.: Inhibition determines membrane potential dynamics and controls action potential generation in awake and sleeping cat cortex. J. Neurosci. 27(20), 5280–5290 (2007)

    Article  Google Scholar 

  37. Blumenfeld, B., Preminger, S., Sagi, D.: Dynamics of memory representations in networks with novelty-facilitated synaptic plasticity. Neuron 52, 383–394 (2006)

    Article  Google Scholar 

  38. Hopfield, J.J.: Neural networks and physical systems with emergent computational abilities. Proc. Natl. Acad. Sci. U. S. A. 78, 2554–2558 (1982)

    Article  ADS  MathSciNet  Google Scholar 

  39. Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two state neurons. Proc. Natl. Acad. Sci. U. S. A. 81, 3088–3092 (1984)

    Article  ADS  Google Scholar 

  40. Abraham, W.C., Robins, A.: Memory retention—the synaptic stability versus plasticity dilemma. Trends Neurosci. 28(2), 73–78 (2005)

    Article  Google Scholar 

  41. Horn, D., Levy, N., Ruppin, E.: Memory maintenance via neuronal regulation. Neural Comput. 10, 1–18 (1998)

    Article  Google Scholar 

  42. Pantic, L., Torres, J.J., Kappen, H.J., Gielen, S.C.A.M.: Associate memory with dynamic synapses. Neural Comput. 14, 2903–2923 (2002)

    MATH  Article  Google Scholar 

  43. Steriade, M., Núnez, A., Amzica, F.: A novel slow (<1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. J. Neurosci. 13, 3252–3265 (1993)

    Google Scholar 

  44. Crochet, S., Chauvette, S., Boucetta, S., Timofeev, I.: Modulation of synaptic transmission in neocortex by network activities. Eur. J. Neurosci. 21, 1030–1044 (2005)

    Article  Google Scholar 

  45. Massimini, M., Rosanova, M., Mariotti, M.: EEG slow (∼1 Hz) waves are associated with nonstationarity of thalamo-cortical sensory processing in the sleeping human. J. Neurophysiol. 89, 1205–1213 (2003)

    Article  Google Scholar 

  46. Steriade, M., Timofeev, I., Grenier, F.: Natural waking and sleep states: a view from inside neocortical neurons. J. Neurophysiol. 85, 1969–1985 (2001)

    Google Scholar 

  47. Battaglia, F.P., Sutherland, G.R., McNaughton, B.L.: Hippocampal sharp wave bursts conincide with neocortical “up-state” transitions. Learn. Mem. 11, 697–704 (2004)

    Article  Google Scholar 

  48. Marshall, L., Helgadóttir, H., Mölle, M., Born, J.: Boosting slow oscillations during sleep potentiates memory. Nature 444, 610–613 (2006)

    Article  ADS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering, University of Waikato, Private Bag 3105, Hamilton, 3240, New Zealand

    M. T. Wilson, Moira L. Steyn-Ross & D. A. Steyn-Ross

  2. Waikato Clinical School, Waikato Hospital, University of Auckland, Hamilton, 3204, New Zealand

    J. W. Sleigh

Authors
  1. M. T. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Moira L. Steyn-Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. A. Steyn-Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. W. Sleigh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. T. Wilson.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Wilson, M.T., Steyn-Ross, M.L., Steyn-Ross, D.A. et al. Going Beyond a Mean-field Model for the Learning Cortex: Second-Order Statistics. J Biol Phys 33, 213–246 (2007). https://doi.org/10.1007/s10867-008-9056-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10867-008-9056-5

Keywords

  • Mean-field
  • Cortex
  • Memory
  • Learning
  • Modelling