Biological Cybernetics

, Volume 40, Issue 3, pp 201–211 | Cite as

Associative search network: A reinforcement learning associative memory

  • Andrew G. Barto
  • Richard S. Sutton
  • Peter S. Brouwer
Article

Abstract

An associative memory system is presented which does not require a “teacher” to provide the desired associations. For each input key it conducts a search for the output pattern which optimizes an external payoff or reinforcement signal. The associative search network (ASN) combines pattern recognition and function optimization capabilities in a simple and effective way. We define the associative search problem, discuss conditions under which the associative search network is capable of solving it, and present results from computer simulations. The synthesis of sensory-motor control surfaces is discussed as an example of the associative search problem.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Albus, J.S.: Mechanisms of planning and problem solving in the brain. Math. Biosci. 45, 247–293 (1979)Google Scholar
  2. Amari, S.: Neural theory of association and concept-formation. Biol. Cybern. 27, 175–185 (1977)Google Scholar
  3. Anderson, J.A., Silverstein, J.W., Ritz, S.A., Jones, R.S.: Distinctive features, categorical perception, and probability learning. Some applications of a neural model. Psychol. Rev. 85, 413–451 (1977)Google Scholar
  4. Cooper, L.N.: A possible organization of animal memory and learning. In: Proceedings of the Nobel Symposium on Collective Properties of Physical Systems. Lundquist, B., Lundquist, S. (eds.) New York: Academic Press 1974Google Scholar
  5. Didday, R.L.: A model of visuomotor mechanisms in the frog optic tectum. Math. Biosci, 30, 169–180 (1976)Google Scholar
  6. Duda, R.O., Hart, P.E.: Pattern classification and scene analysis. New York: Wiley 1973Google Scholar
  7. Freeman, W.J.: Mass action in the nervous system. New York: Academic Press 1975Google Scholar
  8. Grossberg, S.: Adaptive pattern classification and universal recoding. II. Feedback, expectation, olfaction, illusions. Biol. Cybern. 23, 187–202 (1976)Google Scholar
  9. Harth, E., Tzanakou, E.: ALOPEX: a stochastic method for determining visual receptive fields. Vision Res. 14, 1475–1482 (1974)Google Scholar
  10. John, E.R., Schwartz, E.L.: The neurophysiology of information processing and cognition. Annu. Rev. of Psychol. 29, 1–29 (1978)Google Scholar
  11. Kasyap, R.L., Blaydon, C.C., Fu, K.S.: Stochastic approximation. In: Adaptive, learning, and pattern recognition systems: theory and applications. pp. 339–354. Mendel, J.M., Fu, K.S. (eds.). New York: Academic Press 1970Google Scholar
  12. Klopf, A.H.: Brain function and adaptive systems — a heterostatic theory. Air Force Cambridge Research Laboratories research report AFCRL-72-0164, Bedford, MA. (1972) (AD742259). (A summary in: Proceedings of the International Conference on Systems, Man and Cybernetics, IEEE Systems, Man and Cybernetics Society, Dallas, Texas, 1974)Google Scholar
  13. Klopf, A.H.: Goal-seeking systems from goal-sceking components: implications for AI. The Cognition and Brain Theory Newsletter, Vol. III, No. 2 (1979)Google Scholar
  14. Klopf, A.H.: The hedonistic neuron: A theory of memory, learning and intelligence. Washington, D.C.: Hemisphere 1981 (to be published)Google Scholar
  15. Kohonen, T.: Associative memory: a system theoretic approach. Berlin, Heidelberg, New York: Springer 1977Google Scholar
  16. Kohonen, T., Oja, E.: Fast adaptive formation of orthogonalizing filters and associative memory in recurrent networks of neuronlike elements. Biol. Cybern. 21, 85–95 (1976)Google Scholar
  17. Mendel, J.M., McLaren, R.W.: Reinforcement-learning control and pattern recognition systems. In: Adaptive, learning, and pattern recognition systems: theory and applications, pp. 287–317 Mendel, J.M., Fu, K.S. (eds.). New York: Academic Press 1970Google Scholar
  18. Minsky, M.L., Papert, S.: Perceptron: an introduction to computational geometry. Cambridge, MA: MIT Press 1969Google Scholar
  19. Nakano, K.: Associatron — a model of associative memory. IEEE Trans. Syst. Man Cybern. 3, 380–388 (1972)Google Scholar
  20. Narendra, K.S., Thathachar, M.A.L.: Learning automata — a survey. IEEE Trans. Syst. Man Cybern 4, 323–334 (1974)Google Scholar
  21. Nilsson, N.J.: Learning machines. New York: McGraw-Hill 1965Google Scholar
  22. Poggio, T.: On optimal nonlinear associative recall. Biol. Cybern 19, 201–209 (1975)Google Scholar
  23. Rosenblatt, F.: Principles of neurodynamics; perceptrons and the theory of brain mechanisms. Washington: Spartan Press 1962Google Scholar
  24. Sutton, R.S., Barto, A.G.: Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. (in press) (1981)Google Scholar
  25. Tsetlin, M.L.: Automaton theory and modeling of biological systems. New York: Academic Press 1973Google Scholar
  26. Widrow, B., Gupta, N.K., Maitra, S.: Punish/reward: learning with a critic in adaptive threshold systems. IEEE Trans. Syst. Man Cybern. 5, 455–465 (1973)Google Scholar
  27. Wigström, H.: A neuron model with learning capability and its relation to mechanisms of association. Kybernetik 12, 204–215 (1973)Google Scholar
  28. Willshaw, D.J., Buneman, O.P., Longuet-Higgins, H.S.: Nonholographic associative memory. Nature 222, 960–962 (1969)Google Scholar
  29. Wood, C.C.: Variations on a theme by Lashley: lesion experiments on the neural model of Anderson, Silverstein, Ritz, and Jones. Psychol. Rev. 85, 582–591 (1978)Google Scholar

Copyright information

© Springer-Verlag 1981

Authors and Affiliations

  • Andrew G. Barto
    • 1
  • Richard S. Sutton
    • 1
  • Peter S. Brouwer
    • 1
  1. 1.Department of Computer and Information ScienceUniversity of MassachusettsAmherstUSA

Personalised recommendations