Neural Networks and Adaptive Control

  • Andrew G. Barto
  • Vijaykumar Gullapalli
Part of the Research Notes in Neural Computing book series (NEURALCOMPUTING, volume 4)

Abstract

This chapter provides an overview of some adaptive control methods and how artificial neural networks are being used as components of adaptive control systems. It suggests, however, that the adaptive control methods developed by control engineers can be misleading guides to thinking about control in biological systems. Furthermore, it suggests that neural networks, whether artificial or real, might be most effective when used as components of architectures that are not conservative extensions of conventional adaptive control architectures. After a brief discussion of control, several approaches to adaptive control as developed by control engineers are described, followed by presentation of a view of artificial neural networks and their potential roles in control systems. Two examples are described in which artificial neural networks have been applied successfully to difficult control problems, and a model of the cerebellum is discussed in light of conceptual schemes based on engineering control practice.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agre, P. E. (1988). The dynamic structure of everyday life. Technical Report TR 1085, Massachusetts Institute of Technology Artificial Intelligence Laboratory.Google Scholar
  2. Albus, J. S. (1971). A theory of cerebellar function. Mathematical Biosciences, 10, 25–61.CrossRefGoogle Scholar
  3. Arbib, M. A. (1987). Brains, Machines, and Mathematics, Second Edition. New York: Springer-Verlag.Google Scholar
  4. Ashby, W. R. (1960). Design for a Brain. London: Associated Book Publishers.MATHGoogle Scholar
  5. Atkeson, C. G. & Reinkensmeyer, D. J. (1988). Using associative content-addressable memories to control robots. In Proceedings of the IEEE Conference on Decision and Control, pages 792–797.Google Scholar
  6. Barto, A. (1991). Some learning tasks from a control perspective. In Nadel, L. & Stein, D. L. (Eds.), 1990 Lectures in Complex Systems, pages 195–223. Redwood City, CA: Addison-Wesley Publishing Company, The Advanced Book Program.Google Scholar
  7. Barto, A. G. (1985). Learning by statistical cooperation of self-interested neuron-like computing elements. Human Neurobiology, 229–256.Google Scholar
  8. Barto, A. G. (1989). From chemotaxis to cooperativity: Abstract exercises in neuronal learning strategies. In Durbin, R., Maill, R., & Mitchison, G. (Eds.), The Computing Neuron, pages 73–98. Reading, MA: Addison-Wesley.Google Scholar
  9. Barto, A. G. (1990). Connectionist learning for control: An overview. In Miller, T., Sutton, R. S., & Werbos, P. J. (Eds.), Neural Networks for Control, pages 5–58. Cambridge, MA: MIT Press.Google Scholar
  10. Bellman, R. E. (1957). Dynamic Programming. Princeton, NJ: Princeton University Press.MATHGoogle Scholar
  11. Berthier, N., Singh, S., Barto, A., & Houk, J. (1991). Distributed representation of limb motor programs in arrays of adjustable pattern generators. NPB Technical Report 3, Institute for Neuroscience, Northwestern University, Chicago, IL.Google Scholar
  12. Brooks, R. A. (1986). A robust layered central system for a mobile robot. IEEE J. of Robotics and Automation, RA-2, 14–23.Google Scholar
  13. Dean, T. L. & Wellman, M. P. (1991). Planning and Control. San Mateo, CA: Morgan Kaufmann.Google Scholar
  14. Dickmanns, E. D., Mysliwetz, B., & Christians, T. (1990). An integrated spatio-temporal approach to automatic visual guidance of autonomous vehicles. IEEE Transactions on Systems, Man, and Cybernetics, 20, 1273–1284.CrossRefGoogle Scholar
  15. Goodwin, G. C. & Sin, K. S. (1984). Adaptive Filtering Prediction and Control. Englewood Cliffs, N.J.: Prentice-Hall.MATHGoogle Scholar
  16. Gullapalli, V. (1990). A stochastic reinforcement algorithm for learning real-valued functions. Neural Networks, 671–692.Google Scholar
  17. Gullapalli, V. (1992). Reinforcement learning and its application to control. Technical Report COINS Technical Report 92–10, University of Massachusetts, Amherst, MA.Google Scholar
  18. Gullapalli, V., Grupen, R. A., & Barto, A. G. (1992). Learning reactive admittance control. In 1992 IEEE Conference on Robotics and Automation. To appear.Google Scholar
  19. Hollerbach, J. M. (1982). Computers, brains and the control of movement. Trends in Neuroscience, 5, 189–192.CrossRefGoogle Scholar
  20. Houk, J. C. & Barto, A. G. (To appear). Distributed sensorimotor learning. In Stelmach, G. E. & Requin, J. (Eds.), Tutorials in Motor Behavior II. Amsterdam, The Netherlands: Elsevier Science Publishers B. V.Google Scholar
  21. Houk, J. C., Singh, S. P., Fisher, C., k Barto, A. G. (1990). An adaptive network inspired by the anatomy and physiology of the cerebellum. In Miller, T., Sutton, R. S., & Werbos, P. J. (Eds.), Neural Networks for Control, pages 301–348. Cambridge, MA: MIT Press.Google Scholar
  22. Jacobs, R. A., Jordan, M. I., & Barto, A. G. (1991). Task decomposition through competition in a modular connectionist architecture: The what and where vision task. Cognitive Science, 15, 219–250.CrossRefGoogle Scholar
  23. Jordan, M. I. & Rumelhart, D. E. (In press). Forward models: Supervised learning with a distal teacher. Cognitive Science.Google Scholar
  24. Kawato, M. (1990). Computational schemes and neural network models for formation and control of multijoint arm trajectory. In Miller, T., Sutton, R. S., & Werbos, P. J. (Eds.), Neural Networks for Control. Cambridge, MA: MIT Press.Google Scholar
  25. Kawato, M., Furukawa, K., & Suzuki, R. (1987). A hierarchical neural-network model for control and learning of voluntary movement. Biological Cybernetics, 57, 169–185.MATHCrossRefGoogle Scholar
  26. Kohonen, T. (1977). Associative Memory: A System Theoretic Approach. Berlin: Springer- Varlag.Google Scholar
  27. le Cun, Y. (1985). Une procedure d’apprentissage pour reseau a sequil assymetrique [A learning procedure for asymmetric threshold network]. Proceedings of Cognitiva, 85, 599–604.Google Scholar
  28. Marr, D. (1969). A theory of cerebellar cortex. Journal of Physiology (Lond), 202, 437–470.Google Scholar
  29. Miller, T., Sutton, R. S., & Werbos, P. J. (1990). Neural Networks for Control. Cambridge, MA: MIT Press.Google Scholar
  30. Narendra, K. & Thathachar, M. A. L. (1989). Learning Automata: An Introduction. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  31. Parker, D. B. (1985). Learning logic. Technical Report TR-47, Massachusetts Institute of Technology.Google Scholar
  32. Pomerleau, D. A., Gowdy, J., & Thorpe, C. E. (1991). Combining artificial neural network and symbolic processing for autonomous robot guidance. Engineering Applications of Artificial Intelligence, 279–285.Google Scholar
  33. Rohrs, C. E. (1990). Rethinking adaptive control for the 90’s. In Proceedings of the 29th Conference on Decision and Control, pages 3143–3145, Honolulu, Hawaii.Google Scholar
  34. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In Rumelhart, D. E. & McClelland, J. L. (Eds.), Parallel Distributed Processing: Explorations in the Micro structure of Cognition, vol.1: Foundations. Cambridge, MA: Bradford Books/MIT Press.Google Scholar
  35. Stanfill, C. & Waltz, D. (1986). Toward memory-based reasoning. Communications of the ACM, 29, 1213–1228.CrossRefGoogle Scholar
  36. Sutton, R. S., Barto, A. G., & Williams, R. J. (1991). Reinforcement learning is direct adaptive optimal control. In Proceedings of the American Control Conference, pages 2143–2146, Boston, MA.Google Scholar
  37. Tomovic, R. & McGhee, R. B. (1966). A finite state approach to the synthesis of bioengineering control systems. IEEE Trans. HFE, 7, 65–69.Google Scholar
  38. Werbos, P. J. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University.Google Scholar
  39. Wiener, N. (1948). Cybernetics. Cambridge, MA: MIT Press.Google Scholar
  40. Ydstie, B. E. (1986). Bifurcation and complex dynamics in adaptive control systems. In Proceedings of the 25th Conference on Decision and Control, pages 2232–2236, Athens.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1993

Authors and Affiliations

  • Andrew G. Barto
    • 1
  • Vijaykumar Gullapalli
    • 1
  1. 1.Department of Computer ScienceUniversity of MassachusettsAmherstUSA

Personalised recommendations