Some theorems concerning the free energy of (Un) constrained stochastic Hopfield neural networks

  • Jan van den Berg
  • Jan C. Bioch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 904)


General stochastic binary Hopfield models are viewed from the angle of statistical mechanics. Both the general unconstrained binary stochastic Hopfield model and a certain constrained one are analyzed yielding explicit expressions of the free energy. Moreover, conditions are given for which some of these free energy expressions are Lyapunov functions of the corresponding differential equations. In mean field approximation, either stochastic model appears to coincide with a specific continuous model. Physically, the models are related to spin and to Potts glass models. By means of an alternative derivation, an expression of a ‘complementary’ free energy is presented. Some surveying computational results are reported and an alternative use of the discussed models in resolving constrained optimization problems is discussed.


Transfer Function Partition Function Stationary Point Lyapunov Function Travelling Salesman Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J. van den Berg and J. C. Bioch. About the Statistical Mechanics of (Un)Constrained Stochastic Hopfield and ‘Elastic’ Neural Networks. Technical Report EUR-CS-94-08, Comp. Sc. Dept. (AI Section), Faculty of Economics, Erasmus University Rotterdam, 1994.Google Scholar
  2. 2.
    J. van den Berg and J. C. Bioch. Constrained Optimization with the Hopfield-Lagrange Model. In Proceedings of the 14th IMACS World Congress, pages 470–473, Atlanta, GA 30332, USA, 1994.Google Scholar
  3. 3.
    J. van den Berg and J. C. Bioch. On the (Free) Energy of Hopfield Networks. Submitted to: Workshop on the Theory of Neural Networks: The Statistical Mechanics Perspective, Pohang, Korea, 1995.Google Scholar
  4. 4.
    D. E. Van den Bout and T. K. Miller. Improving the Performance of the Hopfield-Tank Neural Network Through Normalization and Annealing. Biological Cybernetics, 62:129–139, 1989.Google Scholar
  5. 5.
    J. Hertz, A. Krogh, and R. G. Palmer. Introduction to the Theory of Neural Computation. Addison-Wesley Publishing Company, The Advanced Book Program, 1991.Google Scholar
  6. 6.
    J. J. Hopfield. Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences, USA, 79:2554–2558, 1982.Google Scholar
  7. 7.
    J. J. Hopfield. Neurons with Graded Responses Have Collective Computational Properties Like Those of Two-State Neurons. Proceedings of the National Academy of Sciences, USA, 81:3088–3092, 1984.Google Scholar
  8. 8.
    J. J. Hopfield and D. W. Tank. “Neural” Computation of Decisions in Optimization Problems. Biological Cybernetics, 52:141–152, 1986.Google Scholar
  9. 9.
    C. Peterson and B. Söderberg. A New Method for Mapping Optimization Problems onto Neural Networks. International Journal of Neural Systems, 1:3–22, 1989.Google Scholar
  10. 10.
    D. Sherrington. Neural Networks: The Spin Glass Approach. In J.G. Taylor, editor, Mathematical Approaches to Neural Networks, pages 261–292. North-Holland, 1993.Google Scholar
  11. 11.
    P. D. Simic. Statistical Mechanics as the Underlying Theory of ‘Elastic’ and ‘Neural’ Optimisations. Network, 1:88–103, 1990.Google Scholar
  12. 12.
    G. V. Wilson and G. S. Pawley. On the Stabilitity of the Travelling Salesman Problem Algorithm of Hopfield and Tank. Biological Cybernetics, 58:63–70, 1988.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Jan van den Berg
    • 1
  • Jan C. Bioch
    • 1
  1. 1.Department of Computer ScienceErasmus University RotterdamDR RotterdamThe Netherlands

Personalised recommendations