Hebbian unlearning

  • Stefan Wimbauer
  • J. Leo van Hemmen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 888)


Unlearning, a reverse process to learning according to Hebb's rule, is a local and unsupervised procedure that gives rise to a substantial improvement of the retrieval properties of an associative neural network: (i) an enhancement of both the storage capacity and the domains of attraction, (ii) the possibility to store correlated patterns, and (iii) the capability to distinguish between patterns and non-retrieval states. Three different versions of this type of algorithm are reviewed and the basic properties of these algorithms are investigated. We demonstrate that the same microscopic mechanism underlies all three of them. Furthermore, unlearning is applied succesfully to the storage of temporal sequences of correlated patterns which have been learned in a purely Hebbian way.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    D.O. Hebb, The organization of behavior. Wiley, New York (1949)Google Scholar
  2. [2]
    D. Amit, H. Gutfreund, H. Sompolinsky, Statistical mechanics of neural networks near saturation. Ann. Phys. N.Y. 173, 30–67 (1987)Google Scholar
  3. [3]
    E. Gardner, The space of interactions in neural network models. J. Phys. A: Math. Gen. 21, 257–270 (1988)Google Scholar
  4. [4]
    J.J. Hopfield, D.I. Feinstein, R.G. Palmer, Unlearning has stabilizing effects in collective memories, Nature 304,158–9 (1983)PubMedGoogle Scholar
  5. [5]
    F. Crick, G. Mitchinson The function of dream sleep, Nature 304, 111–114 (1983)PubMedGoogle Scholar
  6. [6]
    A. Plakhov, S. Semenov, Unlearning-type procedures for reaching a perfect storage in neural networks. Moscow, preprint (1992)Google Scholar
  7. [7]
    J.L. van Hemmen, L.B. Ioffe, R. Kühn, M. Vaas, Increasing the efficiency of a neural network through unlearning. Physica A 163, 386–392 (1990)Google Scholar
  8. [8]
    J.L. van Hemmen, N. Klemmer, Unlearning and its relevance to REM sleep: Decorrelating correlated data. In: J.G. Taylor, E.R. Caianiello, R.M.J. Cotterill and J.W. Clark (Eds.) Neural network dynamics. London:Springer, 30–43 (1992)Google Scholar
  9. [9]
    S. Wimbauer, N. Klemmer, J.L. van Hemmen, Universality of unlearning, Neural Networks (to appear)Google Scholar
  10. [10]
    J.J. Hopfield, Neural Networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)PubMedGoogle Scholar
  11. [11]
    W.A. Little, The existence of persistent states in the brain. Math. Biosci. 19, 101–120 (1974)Google Scholar
  12. [12]
    A.V.M. Herz, Z. Li, J.L. van Hemmen, Statistical mechanics of temporal association in neural networks with transmisson delays. Phys. Rev. Lett. 66, 1370–73 (1991)PubMedGoogle Scholar
  13. [13]
    I. Kanter, H. Sompolinsky, Associative recall of memory without errors. Phys. Rev. A 35, 380–392 (1987)Google Scholar
  14. [14]
    W. Kinzel, M. Opper, Dynamics of learning, in: E. Domany, J.L. van Hemmen, K. Schulten (Eds.) Models of neural networks. Springer, Berlin, 149–172 (1991)Google Scholar
  15. [15]
    A.V.M. Herz, B. Sulzer, R. Kühn, J. L. van Hemmen, The Hebb rule: Storing static and dynamic objects in an associative neural network. Europhys. Lett. 7, 663–669 (1988)Google Scholar
  16. [16]
    A.V.M. Herz, B. Sulzer, R. Kühn, J. L. van Hemmen, Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets. Biol. Cybern. 60, 457–467 (1989)PubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Stefan Wimbauer
    • 1
  • J. Leo van Hemmen
    • 1
  1. 1.Physik-DepartmentTechnische Universität MünchenGarching bei MünchenFR Germany

Personalised recommendations