Some theorems concerning the free energy of (Un) constrained stochastic Hopfield neural networks
General stochastic binary Hopfield models are viewed from the angle of statistical mechanics. Both the general unconstrained binary stochastic Hopfield model and a certain constrained one are analyzed yielding explicit expressions of the free energy. Moreover, conditions are given for which some of these free energy expressions are Lyapunov functions of the corresponding differential equations. In mean field approximation, either stochastic model appears to coincide with a specific continuous model. Physically, the models are related to spin and to Potts glass models. By means of an alternative derivation, an expression of a ‘complementary’ free energy is presented. Some surveying computational results are reported and an alternative use of the discussed models in resolving constrained optimization problems is discussed.
KeywordsTransfer Function Partition Function Stationary Point Lyapunov Function Travelling Salesman Problem
Unable to display preview. Download preview PDF.
- 1.J. van den Berg and J. C. Bioch. About the Statistical Mechanics of (Un)Constrained Stochastic Hopfield and ‘Elastic’ Neural Networks. Technical Report EUR-CS-94-08, Comp. Sc. Dept. (AI Section), Faculty of Economics, Erasmus University Rotterdam, 1994.Google Scholar
- 2.J. van den Berg and J. C. Bioch. Constrained Optimization with the Hopfield-Lagrange Model. In Proceedings of the 14th IMACS World Congress, pages 470–473, Atlanta, GA 30332, USA, 1994.Google Scholar
- 3.J. van den Berg and J. C. Bioch. On the (Free) Energy of Hopfield Networks. Submitted to: Workshop on the Theory of Neural Networks: The Statistical Mechanics Perspective, Pohang, Korea, 1995.Google Scholar
- 4.D. E. Van den Bout and T. K. Miller. Improving the Performance of the Hopfield-Tank Neural Network Through Normalization and Annealing. Biological Cybernetics, 62:129–139, 1989.Google Scholar
- 5.J. Hertz, A. Krogh, and R. G. Palmer. Introduction to the Theory of Neural Computation. Addison-Wesley Publishing Company, The Advanced Book Program, 1991.Google Scholar
- 6.J. J. Hopfield. Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences, USA, 79:2554–2558, 1982.Google Scholar
- 7.J. J. Hopfield. Neurons with Graded Responses Have Collective Computational Properties Like Those of Two-State Neurons. Proceedings of the National Academy of Sciences, USA, 81:3088–3092, 1984.Google Scholar
- 8.J. J. Hopfield and D. W. Tank. “Neural” Computation of Decisions in Optimization Problems. Biological Cybernetics, 52:141–152, 1986.Google Scholar
- 9.C. Peterson and B. Söderberg. A New Method for Mapping Optimization Problems onto Neural Networks. International Journal of Neural Systems, 1:3–22, 1989.Google Scholar
- 10.D. Sherrington. Neural Networks: The Spin Glass Approach. In J.G. Taylor, editor, Mathematical Approaches to Neural Networks, pages 261–292. North-Holland, 1993.Google Scholar
- 11.P. D. Simic. Statistical Mechanics as the Underlying Theory of ‘Elastic’ and ‘Neural’ Optimisations. Network, 1:88–103, 1990.Google Scholar
- 12.G. V. Wilson and G. S. Pawley. On the Stabilitity of the Travelling Salesman Problem Algorithm of Hopfield and Tank. Biological Cybernetics, 58:63–70, 1988.Google Scholar