Unlearning, a reverse process to learning according to Hebb's rule, is a local and unsupervised procedure that gives rise to a substantial improvement of the retrieval properties of an associative neural network: (i) an enhancement of both the storage capacity and the domains of attraction, (ii) the possibility to store correlated patterns, and (iii) the capability to distinguish between patterns and non-retrieval states. Three different versions of this type of algorithm are reviewed and the basic properties of these algorithms are investigated. We demonstrate that the same microscopic mechanism underlies all three of them. Furthermore, unlearning is applied succesfully to the storage of temporal sequences of correlated patterns which have been learned in a purely Hebbian way.
Unable to display preview. Download preview PDF.
- D.O. Hebb, The organization of behavior. Wiley, New York (1949)Google Scholar
- D. Amit, H. Gutfreund, H. Sompolinsky, Statistical mechanics of neural networks near saturation. Ann. Phys. N.Y. 173, 30–67 (1987)Google Scholar
- E. Gardner, The space of interactions in neural network models. J. Phys. A: Math. Gen. 21, 257–270 (1988)Google Scholar
- A. Plakhov, S. Semenov, Unlearning-type procedures for reaching a perfect storage in neural networks. Moscow, preprint (1992)Google Scholar
- J.L. van Hemmen, L.B. Ioffe, R. Kühn, M. Vaas, Increasing the efficiency of a neural network through unlearning. Physica A 163, 386–392 (1990)Google Scholar
- J.L. van Hemmen, N. Klemmer, Unlearning and its relevance to REM sleep: Decorrelating correlated data. In: J.G. Taylor, E.R. Caianiello, R.M.J. Cotterill and J.W. Clark (Eds.) Neural network dynamics. London:Springer, 30–43 (1992)Google Scholar
- S. Wimbauer, N. Klemmer, J.L. van Hemmen, Universality of unlearning, Neural Networks (to appear)Google Scholar
- W.A. Little, The existence of persistent states in the brain. Math. Biosci. 19, 101–120 (1974)Google Scholar
- I. Kanter, H. Sompolinsky, Associative recall of memory without errors. Phys. Rev. A 35, 380–392 (1987)Google Scholar
- W. Kinzel, M. Opper, Dynamics of learning, in: E. Domany, J.L. van Hemmen, K. Schulten (Eds.) Models of neural networks. Springer, Berlin, 149–172 (1991)Google Scholar
- A.V.M. Herz, B. Sulzer, R. Kühn, J. L. van Hemmen, The Hebb rule: Storing static and dynamic objects in an associative neural network. Europhys. Lett. 7, 663–669 (1988)Google Scholar