Abstract
A hierarchical neural network model with feedback interconnections, which has the function of associative memory and the ability to recognize patterns, is proposed. The model consists of a hierarchical multi-layered network to which efferent connections are added, so as to make positive feedback loops in pairs with afferent connections. The cell-layer at the initial stage of the network is the input layer which receives the stimulus input and at the same time works as an output layer for associative recall. The deepest layer is the output layer for pattern-recognition. Pattern-recognition is performed hierarchically by integrating information by converging afferent paths in the network. For the purpose of associative recall, the integrated information is again distributed to lower-order cells by diverging efferent paths. These two operations progress simultaneously in the network. If a fragment of a training pattern is presented to the network which has completed its self-organization, the entire pattern will gradually be recalled in the initial layer. If a stimulus consisting of a number of training patterns superposed is presented, one pattern gradually becomes predominant in the recalled output after competition between the patterns, and the others disappear. At about the same time when the recalled pattern reaches a steady state in he initial layer, in the deepest layer of the network, a response is elicited from the cell corresponding to the category of the finally-recalled pattern. Once a steady state has been reached, the response of the network is automatically extinguished by inhibitory signals from a steadiness-detecting cell. If the same stimulus is still presented after inhibition, a response for another pattern, formerly suppressed, will now appear, because the cells of the network have adaptation characteristics which makes the same response unlikely to recur. Since inhibition occurs repeatedly, the superposed input patterns are recalled one by one in turn.
Similar content being viewed by others
References
Barchas, J.D., Akil, H., Elliott, G.R., Holman, R.B., Watson, S.J.: Behavioral neurochemistry: neuroregulators and behavioral states. Science 200, 964–973 (1978)
Fukushima, K.: A model of associative memory in the brain. Kybernetik 12, 58–63 (1978)
Fukushima, K.: Cognitron: a self-organizing multilayered neural network. Biol. Cybern. 20, 121–136 (1975)
Fukushima, K.: Neocognitron: a self-organizing neutral network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36, 193–202 (1980)
Fukushima, K.: Cognitron: a self-organizing multilayered neural network model. NHK Technical Monograph, No. 30 (1981)
Fukushima, K: A hierarchical neural network model for associative memory (in Japanese). Pap. Tech. Group MBE82-90, IECE Japan (1983a)
Fukushima, K.: A hierarchical neural network model for associative memory (in Japanese). Trans. IECE Japan (A), J66-A, 1130–1137 (1983b)
Fukushima, K., Miyake, S.: Neocognitron: a new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognition 15, 455–469 (1982)
Hebb, D.O.: Organization of behavior. New York, London, Sydney: Wiley 1949
Hirai, Y.: A learning network resolving multiple match in associative memory. Proc. 6th Intern. Conf. Pattern Recognition, 1049–1052 (1982)
Hubel, D.H., Wiesel, T.N.: Functional architecture of macaque monkey visual cortex. Proc. Roy. Soc. Lond. B. 198, 1–59 (1977)
Kohonen, T.: Correlation matrix memories. IEEE Trans. Computers C-21, 353–359 (1972)
Nakano, K.: Associatron—a model of associative memory. IEEE Trans. Syst. Man Cybern. SMC-2, 380–388 (1972)
Silverstein, J.W.: Asymptotics applied to a neural network. Biol. Cybern. 22, 73–84 (1976)
Tigges, J., Spatz, W.B., Tigges, M.: Reciprocal point-to-point connections between parastriate and striate cortex in the squirrel monkey (Saimiri). J. Comp. Neur. 148, 481–490 (1973)
Tsukahara, N.: The properties of the cerebello-pontine reverberating circuit. Brain Res. 40, 67–71 (1972)
Uesaka, Y., Ozeki, K.: On some properties of associative memory (in Japanese). Trans. IECE Japan (D) 55-D, 323–330 (1972)
Wong-Riley, M.: Columnar cortico-cortical interconnections within the visual system of the squirrel and macaque monkeys. Brain Res. 162, 201–217 (1979)
Westlake, Ph. R.: The possibilities of neural holographic processes within the brain. Kybernetik 7, 129–153 (1970)
Wigström, G.: A model of a neural network with recurrent inhibition. Kybernetik 16, 103–112 (1974)
Willwacher, G.: Storage of a temporal pattern sequence in a network. Biol. Cybern. 43, 115–126 (1982)
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Fukushima, K. A hierarchical neural network model for associative memory. Biol. Cybern. 50, 105–113 (1984). https://doi.org/10.1007/BF00337157
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00337157