Journal of Biological Physics

, Volume 33, Issue 3, pp 213–246 | Cite as

Going Beyond a Mean-field Model for the Learning Cortex: Second-Order Statistics

  • M. T. WilsonEmail author
  • Moira L. Steyn-Ross
  • D. A. Steyn-Ross
  • J. W. Sleigh
Research Paper


Mean-field models of the cortex have been used successfully to interpret the origin of features on the electroencephalogram under situations such as sleep, anesthesia, and seizures. In a mean-field scheme, dynamic changes in synaptic weights can be considered through fluctuation-based Hebbian learning rules. However, because such implementations deal with population-averaged properties, they are not well suited to memory and learning applications where individual synaptic weights can be important. We demonstrate that, through an extended system of equations, the mean-field models can be developed further to look at higher-order statistics, in particular, the distribution of synaptic weights within a cortical column. This allows us to make some general conclusions on memory through a mean-field scheme. Specifically, we expect large changes in the standard deviation of the distribution of synaptic weights when fluctuation in the mean soma potentials are large, such as during the transitions between the “up” and “down” states of slow-wave sleep. Moreover, a cortex that has low structure in its neuronal connections is most likely to decrease its standard deviation in the weights of excitatory to excitatory synapses, relative to the square of the mean, whereas a cortex with strongly patterned connections is most likely to increase this measure. This suggests that fluctuations are used to condense the coding of strong (presumably useful) memories into fewer, but dynamic, neuron connections, while at the same time removing weaker (less useful) memories.


Mean-field Cortex Memory Learning Modelling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wilson, H.R., Cowan, J.D.: Excitatory and inhibitory interactions in localized populations of model neurons. Biophys. J. 12, 1–24 (1972)ADSCrossRefGoogle Scholar
  2. 2.
    Nunez, P.L.: The brain wave function: a model for the EEG. Math. Biosci. 21, 279–297 (1974)zbMATHCrossRefGoogle Scholar
  3. 3.
    Freeman, W.J.: Predictions on neocortical dynamics derived from studies in paleocortex. In: Basar, E., Bullock, T.H. (eds.) Induced Rhythms of the Brain, chap. 9, pp. 183–199. Birkhaeuser, Boston (1992)Google Scholar
  4. 4.
    Wright, J.J., Liley, D.T.J.: Dynamics of the brain at global and microscopic scales: neural networks and the EEG. Behav. Brain Sci. 19, 285–316 (1996)CrossRefGoogle Scholar
  5. 5.
    Robinson, P.A., Rennie, C.J., Wright, J.J.: Propagation and stability of waves of electrical activity in the cerebral cortex. Phys. Rev. E 56, 826–840 (1997)CrossRefADSGoogle Scholar
  6. 6.
    Liley, D.T.J., Cadusch, P.J., Wright, J.J.: A continuum theory of electro-cortical activity. Neurocomputers 26–27, 795–800 (1999)CrossRefGoogle Scholar
  7. 7.
    Rennie, C.J., Wright, J.J., Robinson, P.A.: Mechanisms for cortical electrical activity and emergence of gamma rhythm. J. Theor. Biol. 205, 17–35 (2000)CrossRefGoogle Scholar
  8. 8.
    Steyn-Ross, M.L., Steyn-Ross, D.A., Sleigh, J.W.: Modelling general anaesthesia as a first-order phase transition in the cortex. Prog. Biophys. Mol. Biol. 85, 369–385 (2004)CrossRefGoogle Scholar
  9. 9.
    Hutt, A., Bestehorn, M., Wennekers, T.: Pattern formation in intracortical neuronal fields. Network 14, 351–368 (2003)CrossRefGoogle Scholar
  10. 10.
    Kramer, M.A., Kirsch, H.E., Szeri, A.J.: Pathological pattern formation and epileptic seizures. J. R. Soc. Lond. Interface 2, 113 (2005)CrossRefGoogle Scholar
  11. 11.
    Chizhov, A.V., Graham, L.J., Turbin, A.A.: Simulation of neural population dynamics with a refractory density approach and a conductance-based threshold neuron model. Neurocomputing 70(1–3), 252–262 (2006)CrossRefGoogle Scholar
  12. 12.
    Bazhenov, M., Timofeev, I., Steriade, M., Sejnowski, T.J.: Model of thalamocortical slow-wave sleep oscillations and transitions to activated states. J. Neurosci. 22, 8691–8704 (2002)Google Scholar
  13. 13.
    Compte, A., Sanchez-Vives, M.V., McCormick, D.A., Wang, X.J.: Cellular and network mechanisms of slow oscillatory activity (<1 Hz) and wave propagations in a cortical network model. J. Neurophysiol. 89, 2707–2725 (2003)CrossRefGoogle Scholar
  14. 14.
    Hill, S., Tononi, G.: Modeling sleep and wakefulness in the thalamocortical system. J. Neurophysiol. 93, 1671–1698 (2005)CrossRefGoogle Scholar
  15. 15.
    Robinson, P.A., Rennie, C.J., Rowe, D.L., O’Connor, S.C., Wright, J.J., Gordon, E., Whitehouse, R.W.: Neurophysical modeling of brain dynamics. Neuropsychopharmacology 28, S74–S79 (2003)CrossRefGoogle Scholar
  16. 16.
    Robinson, P.A., Rennie, C.J., Wright, J.J., Bahramali, H., Gordon, E., Rowe, D.L.: Prediction of electroencephalographic spectra from neurophysiology. Phys. Rev. E 63, 021,903 (2001)Google Scholar
  17. 17.
    Wilson, M.T., Steyn-Ross, D.A., Sleigh, J.W., Steyn-Ross, M.L., Wilcocks, L.C., Gillies, I.P.: The k-complex and slow oscillation in terms of a mean-field cortical model. J. Comput. Neurosci. 21, 243–257 (2006)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Bojak, I., Liley, D.T.J.: Modelling the effects of anaesthesia on the electroencephalogram. Phys. Rev. E 71, 41902 (2005)CrossRefADSGoogle Scholar
  19. 19.
    Wilson, M.T., Steyn-Ross, M.L., Steyn-Ross, D.A., Sleigh, J.W.: Predictions and simulations of cortical dynamics during natural sleep using a continuum approach. Phys. Rev. E 72, 051910 1–14 (2005)CrossRefADSMathSciNetGoogle Scholar
  20. 20.
    Bienenstock, E.L., Cooper, L.N., Munro, P.W.: Theory for the development of neuron selectivity: orientation specificity and binocular interation in visual cortex. J. Neurosci. 2, 32–48 (1982)Google Scholar
  21. 21.
    Bienenstock, E., Lehmann, D.: Regulated criticality in the brain? Adv. Complex Systems 1, 361–384 (1998)CrossRefGoogle Scholar
  22. 22.
    Sandberg, A., Tegnér, J., Lansner, A.: A working memory model based on fast Hebbian learning. Netw. Comput. Neural Syst. 14, 789–802 (2003)CrossRefADSGoogle Scholar
  23. 23.
    Mongillo, G., Amit, D.J., Brunel, N.: Retrospective and prospective persistent activity induced by Hebbian learning in a recurrent cortical network. Eur. J. Neurosci. 18, 2011–2024 (2003)CrossRefGoogle Scholar
  24. 24.
    Hebb, D.O.: The Organization of Behaviour. Wiley, New York (1949)Google Scholar
  25. 25.
    Steyn-Ross, M.L., Steyn-Ross, D.A., Sleigh, J.W., Wilson, M.T., Wilcocks, L.C.: A mechanism for learning and memory erasure in a white-noise driven sleeping cortex. Phys. Rev. E 72, 061,910 (2005)CrossRefMathSciNetGoogle Scholar
  26. 26.
    Stetter, M.: Dynamic functional tuning of nonlinear cortical networks. Phys. Rev. E 73, 031903 (2006)CrossRefADSMathSciNetGoogle Scholar
  27. 27.
    Steyn-Ross, D.A., Steyn-Ross, M.L., Sleigh, J.W., Wilson, M.T., Gillies, I.P., Wright, J.J.: The sleep cycle modelled as a cortical phase transition. J. Biophys. 31, 547–569 (2005)Google Scholar
  28. 28.
    Tononi, G., Cirelli, C.: Sleep function and synaptic homeostatis. Sleep Med. Rev. 10, 49–62 (2006)CrossRefGoogle Scholar
  29. 29.
    Mountcastle, V.B.: The columnar organization of the neocortex. Brain 120, 701–722 (1997)CrossRefGoogle Scholar
  30. 30.
    Sejnowski, T.J.: Storing covariance with nonlinearly interacting neurons. J. Math. Biol. 4, 303–321 (1977)CrossRefGoogle Scholar
  31. 31.
    Douglas, R.J., Martin, K.A.: Recurrent neuronal circuits in the neocortex. Curr. Biol. 17(13), R496 (2007)CrossRefGoogle Scholar
  32. 32.
    Thomson, A.M., Bannister, A.P.: Interlaminar connections in the neocortex. Cerebral Cortex 13, 5–14 (2003)CrossRefGoogle Scholar
  33. 33.
    Tononi, G., Sporns, O.: Measuring information integration. BMC Neurosci. 4, 31 (2003)CrossRefGoogle Scholar
  34. 34.
    Albert, R., Barabási, A.L.: Statistical mechanics of complex networks. Rev. Mod. Phys. 74(1), 47–97 (2002)CrossRefADSGoogle Scholar
  35. 35.
    Kloeden, P.E., Platen, E.: Numerical Solution of Stochastc Differential Equations. Springer, Berlin (1992)Google Scholar
  36. 36.
    Rudolph, M., Pospischil, M., Timofeev, I., Destexhe, A.: Inhibition determines membrane potential dynamics and controls action potential generation in awake and sleeping cat cortex. J. Neurosci. 27(20), 5280–5290 (2007)CrossRefGoogle Scholar
  37. 37.
    Blumenfeld, B., Preminger, S., Sagi, D.: Dynamics of memory representations in networks with novelty-facilitated synaptic plasticity. Neuron 52, 383–394 (2006)CrossRefGoogle Scholar
  38. 38.
    Hopfield, J.J.: Neural networks and physical systems with emergent computational abilities. Proc. Natl. Acad. Sci. U. S. A. 78, 2554–2558 (1982)CrossRefADSMathSciNetGoogle Scholar
  39. 39.
    Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two state neurons. Proc. Natl. Acad. Sci. U. S. A. 81, 3088–3092 (1984)CrossRefADSGoogle Scholar
  40. 40.
    Abraham, W.C., Robins, A.: Memory retention—the synaptic stability versus plasticity dilemma. Trends Neurosci. 28(2), 73–78 (2005)CrossRefGoogle Scholar
  41. 41.
    Horn, D., Levy, N., Ruppin, E.: Memory maintenance via neuronal regulation. Neural Comput. 10, 1–18 (1998)CrossRefGoogle Scholar
  42. 42.
    Pantic, L., Torres, J.J., Kappen, H.J., Gielen, S.C.A.M.: Associate memory with dynamic synapses. Neural Comput. 14, 2903–2923 (2002)zbMATHCrossRefGoogle Scholar
  43. 43.
    Steriade, M., Núnez, A., Amzica, F.: A novel slow (<1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. J. Neurosci. 13, 3252–3265 (1993)Google Scholar
  44. 44.
    Crochet, S., Chauvette, S., Boucetta, S., Timofeev, I.: Modulation of synaptic transmission in neocortex by network activities. Eur. J. Neurosci. 21, 1030–1044 (2005)CrossRefGoogle Scholar
  45. 45.
    Massimini, M., Rosanova, M., Mariotti, M.: EEG slow (∼1 Hz) waves are associated with nonstationarity of thalamo-cortical sensory processing in the sleeping human. J. Neurophysiol. 89, 1205–1213 (2003)CrossRefGoogle Scholar
  46. 46.
    Steriade, M., Timofeev, I., Grenier, F.: Natural waking and sleep states: a view from inside neocortical neurons. J. Neurophysiol. 85, 1969–1985 (2001)Google Scholar
  47. 47.
    Battaglia, F.P., Sutherland, G.R., McNaughton, B.L.: Hippocampal sharp wave bursts conincide with neocortical “up-state” transitions. Learn. Mem. 11, 697–704 (2004)CrossRefGoogle Scholar
  48. 48.
    Marshall, L., Helgadóttir, H., Mölle, M., Born, J.: Boosting slow oscillations during sleep potentiates memory. Nature 444, 610–613 (2006)CrossRefADSGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  • M. T. Wilson
    • 1
    Email author
  • Moira L. Steyn-Ross
    • 1
  • D. A. Steyn-Ross
    • 1
  • J. W. Sleigh
    • 2
  1. 1.Department of EngineeringUniversity of WaikatoHamiltonNew Zealand
  2. 2.Waikato Clinical School, Waikato HospitalUniversity of AucklandHamiltonNew Zealand

Personalised recommendations