In this paper we continue our investigation of the analogical neural network, by introducing and studying its replica symmetric approximation in the absence of external fields. Bridging the neural network to a bipartite spin-glass, we introduce and apply a new interpolation scheme to its free energy, that naturally extends the interpolation via cavity fields or stochastic perturbations from the usual spin glass case to these models.
While our methods allow the formulation of a fully broken replica symmetry scheme, in this paper we limit ourselves to the replica symmetric case, in order to give the basic essence of our interpolation method. The order parameters in this case are given by the assumed averages of the overlaps for the original spin variables, and for the new Gaussian variables. As a result, we obtain the free energy of the system as a sum rule, which, at least at the replica symmetric level, can be solved exactly, through a self-consistent mini-max variational principle.
The so gained replica symmetric approximation turns out to be exactly correct in the ergodic region, where it coincides with the annealed expression for the free energy, and in the low density limit of stored patterns. Moreover, in the spin glass limit it gives the correct expression for the replica symmetric approximation in this case. We calculate also the entropy density in the low temperature region, where we find that it becomes negative, as expected for this kind of approximation. Interestingly, in contrast with the case where the stored patterns are digital, no phase transition is found in the low temperature limit, as a function of the density of stored patterns.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Amit, D.J.: Modeling Brain Function: The World of Attractor Neural Network. Cambridge University Press, Cambridge (1992)
Amit, D.J., Gutfreund, H., Sompolinsky, H.: Spin glass model of neural networks. Phys. Rev. A 32, 1007–1018 (1985)
Amit, D.J., Gutfreund, H., Sompolinsky, H.: Storing infinite numbers of patterns in a spin glass model of neural networks. Phys. Rev. Lett. 55, 1530–1533 (1985)
Barra, A.: The mean field Ising model trough interpolating techniques. J. Stat. Phys. 132, 12–32 (2008)
Barra, A., Guerra, F.: About the ergodic regime in the analogical Hopfield neural networks: moments of the partition function. J. Math. Phys. 50, 125217 (2008)
Bovier, A., Picco, P.: Mathematical Aspects of Spin Glasses and Neural Networks. Birkhäuser, Basel (1998) and references therein
Bovier, A., van Enter, A.C.D., Niederhauser, B.: Stochastic symmetry-breaking in a Gaussian Hopfield-model. J. Stat. Phys. 95, 181–213 (1999)
Coolen, A.C.C., Kuehn, R., Sollich, P.: Theory of Neural Information Processing Systems. Oxford University Press, Oxford (2005)
Genovese, G., Barra, A.: A certain class of Curie-Weiss models. arXiv:0906.4673 (2009)
Guerra, F.: About the overlap distribution in mean field spin glass models. Int. J. Mod. Phys. B 10, 1675–1684 (1996)
Guerra, F.: Sum rules for the free energy in the mean field spin glass model. In: Mathematical Physics in Mathematics and Physics: Quantum and Operator Algebraic Aspects. Fields Institute Communications, vol. 30 Amer. Math. Soc., Providence (2001)
Guerra, F.: Broken replica symmetry bounds in the mean field spin glass model. Commun. Math. Phys. 233, 1–12 (2003)
Guerra, F.: An introduction to mean field spin glass theory: methods and results. In: Bovier, A., et al. (eds.) Mathematical Statistical Physics, pp. 243–271. Elsevier, Oxford (2006)
Guerra, F., Toninelli, F.L.: The thermodynamic limit in mean field spin glass models. Commun. Math. Phys. 230, 71–79 (2002)
Guerra, F., Toninelli, F.L.: The infinite volume limit in generalized mean field disordered models. Markov Process. Relat. Fields 9, 195–207 (2003)
Hebb, D.O.: Organization of Behaviour. Wiley, New York (1949)
Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79, 2554–2558 (1982)
Pastur, L., Scherbina, M., Tirozzi, B.: The replica symmetric solution of the Hopfield model without replica trick. J. Stat. Phys. 74, 1161–1183 (1994)
Pastur, L., Scherbina, M., Tirozzi, B.: On the replica symmetric equations for the Hopfield model. J. Math. Phys. 40, 3930–3947 (1999)
Talagrand, M.: Rigourous results for the Hopfield model with many patterns. Probab. Theory Relat. Fields 110, 177–276 (1998)
Talagrand, M.: Exponential inequalities and convergence of moments in the replica-symmetric regime of the Hopfield model. Ann. Probab. 38, 1393–1469 (2000)
Talagrand, M.: Spin Glasses: A Challenge for Mathematicians. Cavity and Mean Field Models. Springer, Berlin (2003)
About this article
Cite this article
Barra, A., Genovese, G. & Guerra, F. The Replica Symmetric Approximation of the Analogical Neural Network. J Stat Phys 140, 784–796 (2010). https://doi.org/10.1007/s10955-010-0020-y
- Neural networks
- Replica symmetry