Skip to main content
Log in

Random recurrent neural networks dynamics

  • Published:
The European Physical Journal Special Topics Aims and scope Submit manuscript

Abstract.

This paper is a review dealing with the study of large size random recurrent neural networks. The connection weights are varying according to a probability law and it is possible to predict the network dynamics at a macroscopic scale using an averaging principle. After a first introductory section, the section 2 reviews the various models from the points of view of the single neuron dynamics and of the global network dynamics. A summary of notations is presented, which is quite helpful for the sequel. In section 3, mean-field dynamics is developed. The probability distribution characterizing global dynamics is computed. In section 4, some applications of mean-field theory to the prediction of chaotic regime for Analog Formal Random Recurrent Neural Networks (AFRRNN) are displayed. The case of AFRRNN with an homogeneous population of neurons is studied in section 4.1. Then, a two-population model is studied in section 4.2. The occurrence of a cyclo-stationary chaos is displayed using the results of [16]. In section 5, an insight of the application of mean-field theory to IF networks is given using the results of [9].

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • S. Amari, IEEE Trans. Syst. Man. Cyb. SMC-2, (N5) (1972)

  • S. Amari, K. Yosida, K.I. Kakutani, SIAM J. Appl. Math. 33, 95–126 (1977)

    Article  MATH  MathSciNet  Google Scholar 

  • D.J. Amit, J. Brunel, Netw.: Comput. Neural Syst. 8, 373–404 (1997)

    Article  MATH  Google Scholar 

  • D.J. Amit, J. Brunel, Cerebral Cortex 7, 237–252 (1997)

    Article  Google Scholar 

  • A.M.O. De Almeida, D.J. Thouless, Physics A 11, 983–990 (1978)

    Article  Google Scholar 

  • B. Cessac, Europhys. Lett. 26, 577–582 (1994)

    Google Scholar 

  • G. Ben Arous, A. Guionnet, Probab. Theory Relat. Fields 102, 455–509 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  • N. Brunel, J. Comput. Neurosci. 8, 183–208 (2000)

    Article  MATH  Google Scholar 

  • N. Brunel, V. Hakim, Neural Comput. 11, 1621–1671 (1999)

    Article  Google Scholar 

  • G. Buzsaki, J.J. Chrobak, Curr. Opin. Neurobiol. 5, 504 (1995)

    Article  Google Scholar 

  • B. Cessac, J. Phys. I (France) 5, 409–432 (1995)

    Article  Google Scholar 

  • B. Cessac, B. Doyon, M. Quoy, M. Samuelides, Physics D 74, 24–44 (1994)

    Article  MATH  ADS  MathSciNet  Google Scholar 

  • A. Crisanti, H.J. Sommers, H. Sompolinsky, chaos in neural networks: chaotic solutions (1990)

  • A. Crisanti, H. Sompolinsky, Phys. Rev. A 37, 4865 (1987)

    Article  ADS  MathSciNet  Google Scholar 

  • A. Crisanti, H. Sompolinsky, Phys. Rev. A 36, 4922 (1987)

    Article  ADS  MathSciNet  Google Scholar 

  • E. Daucé, O. Moynot, O. Pinaud, M. Samuelides, Neural Proc. Lett. 14, 115–126 (2001)

    Article  MATH  Google Scholar 

  • E. Daucé, M. Quoy, B. Cessac, B. Doyon, M. Samuelides, Neural Netwo. 11, 521–533 (1998)

    Article  Google Scholar 

  • A. Dembo, O. Zeitouni, Large Deviations Techniques (Jones & Bartlett publishers, 1993)

  • B. Derrida, Y. Pommeau, Europhys. Lett. 1, 45–59 (1986)

    ADS  Google Scholar 

  • P. Dupuis, R.S. Ellis, A Weak Convergence Approach to the Theory of Large Deviations (Wiley, 1997)

  • R.S. Ellis, Entropy, Large Deviations and Statistical Mechanics (Springer Verlag, 1985)

  • W. Gerstner, W.M. Kistler, Spiking Neuron Models, Single Neurons, Populations, Plasticity (Pure and Applied Mathematics Series) (Cambridge University Press, 2002)

  • I.V. Girsanov, Theor. Probab. Appl. 5 (1960), 285–301 (1962); Trans. Teor. Veroyatn. Primen. 5, 314–330 (1960)

    Article  Google Scholar 

  • Gray, J. Comput. Neurosci. 7, 334–338 (1994)

    Google Scholar 

  • A. Skorhokod, I. Guikhman, Introduction à la Théorie des Processus Aléatoires (French) (Éditions Mir, Moscou, 1980)

  • A. Guillot, E. Daucé, Approches Dynamiques de la Cognition Artificielle (Hermès, Paris, 2002)

  • D. Hansel, H. Sompolinsky, J. Comp. Neurosc. 3, 7–34 (1996)

    Article  Google Scholar 

  • J.J. Hopfield, Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)

    Article  ADS  MathSciNet  Google Scholar 

  • O. Kallenberg, Foundations of Modern Probability. Probabilities and its Applications (Springer-Verlag, 2002)

  • N.G. Van Kampen, Stochastic Processes in Physics and Chemistry (North Holland, 1992)

  • L. Molgedey, J. Schuchardt, H.G. Schuster, Phys. Rev. Lett. 69, 3717–3719 (1992)

    Article  ADS  Google Scholar 

  • W.S. McCulloch, W. Pitts, Bull. Math. Biophys. 5, 115–133 (1943)

    Article  MATH  MathSciNet  Google Scholar 

  • O. Moynot, Étude mathématique de la dynamique des réseaux aléatoires récurrents, Ph.D. thesis, Université Paul Sabatier, Toulouse (1999)

  • O. Moynot, M. Samuelides, Prob. Theory Relat. Fields 123, 41–75 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  • R. Ritz, T.J. Sejnowski, Curr. Opin. Neurobiol. 7, 536–546 (1997)

    Article  Google Scholar 

  • W. Singer, C.M. Gray, Annu. Rev. Neurosci. 18, 555 (1995)

    Article  Google Scholar 

  • C.A. Skarda, W.J. Freeman, Chaos and the new science of the brain, in Concepts in Neuroscience 1–2 (World Scientfic Publishing Company, 1990), p. 275–285

  • H. Sompolinsky, A. Crisanti, H.J. Sommers, Phys. Rev. Lett. 61, 259–262 (1988)

    Article  ADS  MathSciNet  Google Scholar 

  • A. Zippelius, H. Sompolinsky, Phys. Rev. B 25, 6860 (1982)

    Article  ADS  Google Scholar 

  • H. Soula, G. Beslon, O. Mazet, Neural Comput. 18, 60–79 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  • A.S. Sznitman, J. Func. Anal. 56, 311–336 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  • B. Cessac, M. Samuelides, Eur. Phys. J. Special Topics 142, 7–88 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Samuelides, M., Cessac, B. Random recurrent neural networks dynamics. Eur. Phys. J. Spec. Top. 142, 89–122 (2007). https://doi.org/10.1140/epjst/e2007-00059-1

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1140/epjst/e2007-00059-1

Keywords

Navigation