Abstract.
This paper is a review dealing with the study of large size random recurrent neural networks. The connection weights are varying according to a probability law and it is possible to predict the network dynamics at a macroscopic scale using an averaging principle. After a first introductory section, the section 2 reviews the various models from the points of view of the single neuron dynamics and of the global network dynamics. A summary of notations is presented, which is quite helpful for the sequel. In section 3, mean-field dynamics is developed. The probability distribution characterizing global dynamics is computed. In section 4, some applications of mean-field theory to the prediction of chaotic regime for Analog Formal Random Recurrent Neural Networks (AFRRNN) are displayed. The case of AFRRNN with an homogeneous population of neurons is studied in section 4.1. Then, a two-population model is studied in section 4.2. The occurrence of a cyclo-stationary chaos is displayed using the results of [16]. In section 5, an insight of the application of mean-field theory to IF networks is given using the results of [9].
Similar content being viewed by others
References
S. Amari, IEEE Trans. Syst. Man. Cyb. SMC-2, (N5) (1972)
S. Amari, K. Yosida, K.I. Kakutani, SIAM J. Appl. Math. 33, 95–126 (1977)
D.J. Amit, J. Brunel, Netw.: Comput. Neural Syst. 8, 373–404 (1997)
D.J. Amit, J. Brunel, Cerebral Cortex 7, 237–252 (1997)
A.M.O. De Almeida, D.J. Thouless, Physics A 11, 983–990 (1978)
B. Cessac, Europhys. Lett. 26, 577–582 (1994)
G. Ben Arous, A. Guionnet, Probab. Theory Relat. Fields 102, 455–509 (1995)
N. Brunel, J. Comput. Neurosci. 8, 183–208 (2000)
N. Brunel, V. Hakim, Neural Comput. 11, 1621–1671 (1999)
G. Buzsaki, J.J. Chrobak, Curr. Opin. Neurobiol. 5, 504 (1995)
B. Cessac, J. Phys. I (France) 5, 409–432 (1995)
B. Cessac, B. Doyon, M. Quoy, M. Samuelides, Physics D 74, 24–44 (1994)
A. Crisanti, H.J. Sommers, H. Sompolinsky, chaos in neural networks: chaotic solutions (1990)
A. Crisanti, H. Sompolinsky, Phys. Rev. A 37, 4865 (1987)
A. Crisanti, H. Sompolinsky, Phys. Rev. A 36, 4922 (1987)
E. Daucé, O. Moynot, O. Pinaud, M. Samuelides, Neural Proc. Lett. 14, 115–126 (2001)
E. Daucé, M. Quoy, B. Cessac, B. Doyon, M. Samuelides, Neural Netwo. 11, 521–533 (1998)
A. Dembo, O. Zeitouni, Large Deviations Techniques (Jones & Bartlett publishers, 1993)
B. Derrida, Y. Pommeau, Europhys. Lett. 1, 45–59 (1986)
P. Dupuis, R.S. Ellis, A Weak Convergence Approach to the Theory of Large Deviations (Wiley, 1997)
R.S. Ellis, Entropy, Large Deviations and Statistical Mechanics (Springer Verlag, 1985)
W. Gerstner, W.M. Kistler, Spiking Neuron Models, Single Neurons, Populations, Plasticity (Pure and Applied Mathematics Series) (Cambridge University Press, 2002)
I.V. Girsanov, Theor. Probab. Appl. 5 (1960), 285–301 (1962); Trans. Teor. Veroyatn. Primen. 5, 314–330 (1960)
Gray, J. Comput. Neurosci. 7, 334–338 (1994)
A. Skorhokod, I. Guikhman, Introduction à la Théorie des Processus Aléatoires (French) (Éditions Mir, Moscou, 1980)
A. Guillot, E. Daucé, Approches Dynamiques de la Cognition Artificielle (Hermès, Paris, 2002)
D. Hansel, H. Sompolinsky, J. Comp. Neurosc. 3, 7–34 (1996)
J.J. Hopfield, Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)
O. Kallenberg, Foundations of Modern Probability. Probabilities and its Applications (Springer-Verlag, 2002)
N.G. Van Kampen, Stochastic Processes in Physics and Chemistry (North Holland, 1992)
L. Molgedey, J. Schuchardt, H.G. Schuster, Phys. Rev. Lett. 69, 3717–3719 (1992)
W.S. McCulloch, W. Pitts, Bull. Math. Biophys. 5, 115–133 (1943)
O. Moynot, Étude mathématique de la dynamique des réseaux aléatoires récurrents, Ph.D. thesis, Université Paul Sabatier, Toulouse (1999)
O. Moynot, M. Samuelides, Prob. Theory Relat. Fields 123, 41–75 (2002)
R. Ritz, T.J. Sejnowski, Curr. Opin. Neurobiol. 7, 536–546 (1997)
W. Singer, C.M. Gray, Annu. Rev. Neurosci. 18, 555 (1995)
C.A. Skarda, W.J. Freeman, Chaos and the new science of the brain, in Concepts in Neuroscience 1–2 (World Scientfic Publishing Company, 1990), p. 275–285
H. Sompolinsky, A. Crisanti, H.J. Sommers, Phys. Rev. Lett. 61, 259–262 (1988)
A. Zippelius, H. Sompolinsky, Phys. Rev. B 25, 6860 (1982)
H. Soula, G. Beslon, O. Mazet, Neural Comput. 18, 60–79 (2006)
A.S. Sznitman, J. Func. Anal. 56, 311–336 (1984)
B. Cessac, M. Samuelides, Eur. Phys. J. Special Topics 142, 7–88 (2007)
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Samuelides, M., Cessac, B. Random recurrent neural networks dynamics. Eur. Phys. J. Spec. Top. 142, 89–122 (2007). https://doi.org/10.1140/epjst/e2007-00059-1
Issue Date:
DOI: https://doi.org/10.1140/epjst/e2007-00059-1