Entropy Based Associative Model

  • Masahiro Nakagawa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4232)


In this paper, an entropy based associative memory model will be proposed and applied to memory retrievals with an orthogonal learning model to compare with the conventional autoassociative model with a quadratic Lyapunov functionals. In the present approach, the updating dynamics will be constructed on the basis of the entropy minimization strategy which may be reduced asymptotically to the autocorrelation dynamics as a special case. From numerical results, it will be found that the presently proposed novel approach realizes the larger memory capacity in comparison with the autocorrelation model based on dynamics such as associatron according to the higher-order correlation involved in the proposed dynamics.


Computer Vision Administrative Data Learning Model Present Approach Memory Capacity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, J.A.: A simple neural network generating interactive memory. Mathematical Biosciences 14, 197–220 (1972)MATHCrossRefGoogle Scholar
  2. 2.
    Kohonen, T.: Correlation matrix memories. IEEE Transaction on Computers C-21, 353–359 (1972)MATHCrossRefGoogle Scholar
  3. 3.
    Nakano, K.: Associatron-a model of associative memory. IEEE Trans. SMC-2, 381–388 (1972)Google Scholar
  4. 4.
    Amari, S.: Neural Theory of Association and Concept Formation. Biological Cybernetics 26, 175–185 (1977)MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Amit, D.J., Gutfreund, H., Sompolinsky, W.: Storing infinite numbers of patterns in a spin-glass model of neural networks. Physical Review Letters 55, 1530–1533 (1985)CrossRefGoogle Scholar
  6. 6.
    Gardner, E.: Structure of metastable states in the Hopfield Model. Journal of Physics A19, L1047–L1052 (1986)Google Scholar
  7. 7.
    Kohonen, T., Ruohonen, M.: Representation of associated pairs by matrix operators. IEEE Transaction C-22, 701–702 (1973)CrossRefGoogle Scholar
  8. 8.
    Amari, S., Maginu, K.: Statistical Neurodynamics of Associative Memory. Neural Networks 1, 63–73 (1988)CrossRefGoogle Scholar
  9. 9.
    Morita, M.: Neural Networks. Associative Memory with Nonmonotone Dynamics 6, 115–126 (1993)Google Scholar
  10. 10.
    Yanai, H.-F., Amari, S.: Auto-associative Memory with two-stage dynamicsof non-monotonic neurons. IEEE Transactions on Neural Networks 7, 803–815 (1996)CrossRefGoogle Scholar
  11. 11.
    Morita, M.: Associative memory with nonmonotone dynamics. Neural Networks 6, 115–126 (1993)CrossRefGoogle Scholar
  12. 12.
    Kanter, I., Sompolinski, H.: Associative Recall of Memory without Errors. Phys. Rev. A. 35, 380–392 (1987)CrossRefGoogle Scholar
  13. 13.
    Personnaz, L., Guyon, I., Dreyfus, D.: J. Phys. (Paris) Lea. 46, L-359 (1985)Google Scholar
  14. 14.
    Nakagawa, M.: Chaos and Fractals in Engineering, 944 p. World Scientific Inc., Singapore (1999)Google Scholar
  15. 15.
    Nakagawa, M.: PEntropy based Associative Model: EICE Trans. Fundamentals. EA-89(4), 895–901 (2006)Google Scholar
  16. 16.
    Fuchs, A., Haken, H.: Pattern Recognition and Associative Memory as Dynamical Processes in a Synergetic System I. Biological Cybernetics 60, 17–22 (1988)MATHMathSciNetGoogle Scholar
  17. 17.
    Fuchs, A., Haken, H.: Pattern Recognition and Associative Memory as Dynamical Processes in a Synergetic System II. BioIogicoI Cybernetics 60, 107–109 (1988)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Fuchs, A., Haken, H.: Dynamic Patterns in Complex Systems. In: Kelso, J.A.S., Mandell, A.J., Shlesinger, M.F. (eds.) World Scientific, Singapore (1988)Google Scholar
  19. 19.
    Haken, H.: Synergetic Computers and Cognition. Springer, Heidelberg (1991)MATHGoogle Scholar
  20. 20.
    Nakagawa, M.: A study of association model based on synergetics. In: Proceedings of international Joint Conference on Neural Networks 1993, Nagoya, Japan, pp. 2367–2370 (1993)Google Scholar
  21. 21.
    Nakagawa, M.: A synergetic neural network. IEICE Fundamentals E78-A, 412–423 (1995)Google Scholar
  22. 22.
    Nakagawa, M.: A synergetic neural network with crosscorrelation dynamics. lEICE Fundamentals E80-A, 881–893 (1997)Google Scholar
  23. 23.
    Nakagawa, M.: A Circularly Connected Synergetic Neural Networks. IEICE Fundamentals E83-A, 881–893 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Masahiro Nakagawa
    • 1
  1. 1.Nagaoka University of TechnologyNiigataJapan

Personalised recommendations