Abstract
How does the brain form a useful representation of its environment? It is shown here that a layer of simple Hebbian units connected by modifiable anti-Hebbian feed-back connections can learn to code a set of patterns in such a way that statistical dependency between the elements of the representation is reduced, while information is preserved. The resulting code is sparse, which is favourable if it is to be used as input to a subsequent supervised associative layer. The operation of the network is demonstrated on two simple problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barlow HB (1961) Possible principles underlying the transformations of sensory messages. In: Rosenblith WA (ed) Sensory communication, MIT Press, Cambridge (Mass) London, pp 217–234
Barlow HB (1972) Single units and sensation: a neuron doctrine for perceptual psychology? Perception 1:371–394
Barlow HB (1985) Cerebral cortex as model builder. In: Rose D, Dobson VG (eds) Models of the visual cortex. Wiley, Chichester, pp 37–46
Barlow HB (1989) Unsupervised learning. Neural Comput 1:295–311
Barlow HB, Földiák P (1989) Adaptation and decorrelation in the cortex. In: Durbin RM, Miall C, Mitchison GJ (eds) The computing neuron, chap 4, Addison-Wesley, Wokingham, pp 54–72
Barlow HB, Kaushal TP, Mitchison GJ (1989) Finding minimum entropy codes. Neural Comput 1:412–423
Földiák P (1989) Adaptive network for optimal linear feature extraction. Proceedings of the IEEE/INNS International Joint Conference on Neural Networks, Washington D. C. June 18–22, 1989, vol. 1. IEEE Press, New York, pp 401–405
Grossberg S (1976) Adaptive pattern classification and universal recoding. I. Parallel development and coding of neural feature detectors. Biol Cybern 23:121–134
Harris CS (1980) Insight or out of sight?: Two examples of perceptual plasticity in the human adult. In: Harris CS (ed) Visual coding and adaptability. Erlbaum, Hillsdale, NJ
Hebb DO (1949) The organization of behaviour. Wiley, New York
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558
Kohonen T (1984) Self-organization and associative memory. Springer, Berlin Heidelberg New York
Malsburg Ch von der (1973) Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14:85–100
Palm G (1980) On associative memory. Biol Cybern 36:19–31
Rubner J, Schulten K (1990) Development of feature detectors by self-organization. Biol Cybern 62:13–199
Rumelhart DE, Zipser D (1985) Feature discovery by competitive learning. Cogn Sci 9:75–112
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In: Rumelhart DE, McClelland J (eds) Parallel distributed processing, vol 1. MIT Press, Cambridge Mass London, pp 318–362
Watanabe S (1960) Information-theoretical aspects of inductive and eductive inference. IBM J Res Dev 4:208–231
Watanabe S (1985) Pattern recognition: human and mechanical. Wiley, New York
Willshaw DJ, Buneman OP, Longuet-Higgins HC (1969) Non-holographic associative memory. Nature 222:960–962
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Földiák, P. Forming sparse representations by local anti-Hebbian learning. Biol. Cybern. 64, 165–170 (1990). https://doi.org/10.1007/BF02331346
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02331346