Abstract
We present a two-layered network of linear neurons that organizes itself as to extract the complete information contained in a set of presented patterns. The weights between layers obey a Hebbian rule. We propose a local anti-Hebbian rule for lateral, hierarchically organized weights within the output layer. This rule forces the activities of the output units to become uncorrelated and the lateral weights to vanish. The weights between layers converge to the eigenvectors of the covariance matrix of input patterns, i.e., the network performs a principal component analysis, yielding all principal components. As a consequence of the proposed learning scheme, the output units become detectors of orthogonal features, similar to ones found in the brain of mammals.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahnelt PK, Kolb H, Pflug R (1987) Identification of a subtype of cone photoreceptor, likely to be blue sensitive, in the human retina. J Comp Neurol 255:18–34
Baldi P, Hornik K (1989) Neural networks and principal component analysis: learning from examples without local minima. Neural Net 2:53–58
Buchsbaum G, Gottschalk A (1983) Trichromacy, opponent colours coding and optimum information transmission in the retina. Proc R Soc London B 220:89–113
Campbell FW, Robson JG (1968) Application of Fourier analysis to the visibility of gratings. J Physiol (London) 197:551–566
Cowan WM (1979) The development of the brain. Sci Am 241:107–117
Daugman JG (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two- dimensional visual cortical filters. J Opt Soc Am A 2:1160–1169
De Valois RL, Abramov I, Jacobs GH (1966) Analysis of response patterns of LGN cells. J Opt Soc Am 56:996–977
Földiák P (1989) Adaptive network for optimal linear feature extraction. Proceedings of the IEEE International Conference on Neural Networks, Washington D. C., June 1989 I:401–405
Hebb DO (1949) The organization of behavior. Wiley, New York
Hopfield JJ, Feinstein DI, Palmer RG (1983) “Unlearning” has a stabilizing effect in collective memories. Nature 304:158–159
Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J Physiol 160:106–154
Hubel DH, Wiesel TN (1979) Brain mechanisms of vision. Sci Am 241:130–144
Jones JP, Palmer LA (1987a) The two-dimensional spatial structure of simple receptive fields in cat striate cortex. J Neurophysiol 58:1187–1211
Jones JP, Stepnoski A, Palmer LA (1987b) The two-dimensional spectral structure of simple receptive fields in cat striate cortex. J Neurophysiol 58:1112–1232
Jones JP, Palmer LA (1987c) An evaluation of the twodimensional Gabor filter model of simple receptive fields in cat striate cortex. J Neurophysiol 58:1133–1258
Judd DB (1949) Response functions for types of vision according to the Müller theory. J Res Nat Bur Standards (Washington DC) 42:1–3
Kohonen T (1982) Self-organization and associative memory. Springer, Berlin Heidelberg New York
Lawley DN, Maxwell AE (1963) Factor analysis as a statistical method. Butterworths, London
Linsker R (1988) Self-organization in a perceptual network. IEEE Computer, March (1988) 105–117
Livingstone MS, Hubel DH (1984) Anatomy and physiology of a color system in the primate visual cortex. J Neurosci 4:309–356
MacKay DM (1981) Strife over visual cortical functions. Nature 289:117–118
Marcelja S (1980) Mathematical description of the responses of simple cortical cells. J Opt Soc Am 70:1297–1300
Oja E (1982) A simplified neuron model as a principal component analyzer. J Math Biol 15:267–272
Pollen DA, Lee JR, Taylor JH (1971) How does the striate cortex begin the reconstruction of the visual world? Science 173:74–77
Rubner J, Tavan P (1989) A self-organizing network for principal component analysis. Europhys Lett (in press)
Rumelhart DE, Zipser D (1985) Feature discovery by competitive learning. Cogn Sci 9:75–112
Schnapf JL, Kraft TW, Baylor DA (1987) Spectral sensitivity of human cone photoreceptors. Nature 325:439–441
Von der Malsburg C (1973) Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14:85–100
Wyszecki G, Stiles WS (1982) Color science. Wiley, New York
Yuille AL, Kammen DM, Cohen DS (1989) Quadrature and the development of orientation selective cortical cells by Hebb rules. Biol Cybern 61:183–194
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Rubner, J., Schulten, K. Development of feature detectors by self-organization. Biol. Cybern. 62, 193–199 (1990). https://doi.org/10.1007/BF00198094
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00198094