Advertisement

Machine Learning

, Volume 2, Issue 3, pp 199–228 | Cite as

Classifier systems and the animat problem

  • Stewart W. Wilson
Article

Abstract

This paper characterizes and investigates, from the perspective of machine learning and, particularly, classifier systems, the learning problem faced by animals and autonomous robots (here collectively termed animats). We suggest that, to survive in their environments, animats must in effect learn multiple disjunctive concepts incrementally under payoff (needs-satisfying) feedback. A review of machine learning techniques indicates that most relax at least one of these constraints. In theory, classifier systems satisfy the constraints, but tests have been limited. We show how the standard classifier system model applies to the animat learning problem. Then, in the experimental part of the paper, we specialize the model and test it in a problem environment satisfying the constraints and consisting of a difficult, disjunctive Boolean function drawn from the machine learning literature. Results include: learning the function in significantly fewer trials than a neural-network method; learning under payoff regimes that include both noisy payoff and partial reward for suboptimal performance; demonstration, in a classifier system, of a theoretically predicted property of genetic algorithms: the superiority of crossovers to point mutations; and automatic control of variation (search) rate based on system entropy. We conclude that the results support the classifier system approach to the animat problem, but suggest work aimed at the emergence of behavioral hierarchies of classifiers to offset slower learning rates in larger problems.

Keywords

Classifier systems incremental learning disjunctive concepts payoff animal learning genetic algorithm 

References

  1. Albus, J. S. (1979). Mechanisms of planning and problem solving in the brain. Mathematical Biosciences, 45, 247–293.Google Scholar
  2. Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press.Google Scholar
  3. Anderson, J. R., & Kline, P. J. (1979). A learning system and its psychological implications. Proceedings of the Sixth International Joint Conference on Artificial Intelligence (pp. 16–21). Tokyo, Japan: Morgan Kaufmann.Google Scholar
  4. Baker, J. E. (1985). Adaptive selection methods for genetic algorithms. Proceedings of an International Conference on Genetic Algorithms and Their Applications (pp. 101–111). Pittsburgh, PA.Google Scholar
  5. Barto, A. G. (1985). Learning by statistical cooperation of self-interested neuron-like computing elements. (COINS Technical Report 85–11). Amherst: University of Massachusetts, Department of Computer and Information Science.Google Scholar
  6. Booker, L. (1982). Improving behavior as an adaptation to the task environment. Doctoral dissertation, Department of Computer and Communication Sciences, University of Michigan, Ann Arbor.Google Scholar
  7. Booker, L. (1985). Improving the performance of genetic algorithms in classifier systems. Proceedings of an International Conference on Genetic Algorithms and Their Applications (pp. 80–92). Pittsburgh, PA.Google Scholar
  8. Curtis, H. (1983). Biology. New York: Worth.Google Scholar
  9. Davis, R., & King, J. (1977). An overview of production systems. In E. W., Elcock & D., Michie (Eds.), Machine intelligence (Vol. 8). New York: American Elsevier.Google Scholar
  10. De Jong, K. A. (1975). An analysis of the behavior of a class of genetic adaptive systems. Doctoral dissertation, Department of Computer and Communication Sciences, University of Michigan, Ann Arbor.Google Scholar
  11. Goldberg, D. E. (1983). Computer-aided pipeline operation using genetic algorithms. Doctoral dissertation, Department of Computer and Communication Sciences, University of Michigan, Ann Arbor.Google Scholar
  12. Grefenstette, J. J. (1985) (Ed.). Proceedings of an International Conference on Genetic Algorithms and Their Applications. Pittsburgh, PA.Google Scholar
  13. Holland, J. H. (1975). Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.Google Scholar
  14. Holland, J. H. (1976). Adaptation. In R., Rosen & F. M., Snell (Eds.), Progress in theoretical biology (Vol. 4). New York: Plenum.Google Scholar
  15. Holland, J. H. (1985). Properties of the bucket brigade algorithm. Proceedings of an International Conference on Genetic Algorithms and Their Applications (pp. 1–7). Pittsburgh, PA.Google Scholar
  16. Holland, J. H. (1986). Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In R. S., Michalski, J. G., Carbonell, & T. M., Mitchell (Eds.), Machine learning: An artificial intelligence approach (Vol. 2). Los Altos, CA: Morgan Kaufmann.Google Scholar
  17. Holland, J. H., & Reitman, J. S. (1978). Cognitive systems based on adaptive algorithms. In D. A., Waterman & F., Hayes-Roth (Eds.), Pattern-directed inference systems. New York: Academic Press.Google Scholar
  18. Kirkpatrick, S., Gelatt, C. D.Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.Google Scholar
  19. Laird, J. E., Rosenbloom, P. S., & Newell, A. (1986). Chunking in SOAR: The anatomy of a general learning mechanism. Machine Learning, 1, 11–46.Google Scholar
  20. Michalski, R.S. (1986). Understanding the nature of learning: Issues and research directions. In R. S., Michalski, J. G., Carbonell, & T. M., Mitchell (Eds.), Machine learning: An artificial intelligence approach (Vol. 2). Los Altos, CA: Morgan Kaufmann.Google Scholar
  21. Michalski, R. S., & Larson, J. B. (1978). Selection of most representative training examples and incremental generation of VL 1 hypotheses: The underlying methodology and the description of programs ESEL and AQ11 (Technical Report No. 867). Urbana: University of Illinois, Computer Science Department.Google Scholar
  22. Mitchell, T. M. (1982). Generalization as search. Artificial Intelligence, 18, 203–226.Google Scholar
  23. Quinlan, J. R. (1983). Learning efficient classification procedures and their application to chess end games. In R. S., Michalski, J. G., Carbonell, & T. M., Mitchell (Eds.), Machine learning: An artificial intelligence approach. Los Altos, CA: Morgan KaufmannGoogle Scholar
  24. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985). Learning internal representations by error propagation (ICS Report 8506). San Diego: University of California, Institute for Cognitive Science.Google Scholar
  25. Schaffer, J. D. (1985). Learning multiclass pattern discrimination. Proceedings of an International Conference on Genetic Algorithms and Their Applications (pp. 74–79). Pittsburgh, PA.Google Scholar
  26. Sears, F. W. (1953). Thermodynamics. Reading, MA: Addison-Wesley.Google Scholar
  27. Shannon, C. E. (1948). The mathematical theory of communication. Bell System Technical Journal, 27, 379–423, 623–656.Google Scholar
  28. Smith, S. (1980). A learning system based on genetic algorithms. Doctoral dissertation, Department of Computer Science, University of Pittsburgh, PA.Google Scholar
  29. Waterman, D. A. (1970). Generalization learning techniques for automating the learning of heuristics. Artificial Intelligence, 1, 121–170.Google Scholar
  30. Wilson, S. W. (1985). Knowledge growth in an artificial animal. Proceedings of an International Conference on Genetic Algorithms and Their Applications (pp. 16–23). Pittsburgh, PA.Google Scholar
  31. Wilson, S. W. (in press). Hierarchical credit allocation in a classifier system. In L. D. Davis (Ed.), Genetic algorithms and simulated annealing. London: Pitman.Google Scholar
  32. Winston, P. H. (1975). Learning structural descriptions from examples. In P. H., Winston (Ed.), The psychology of computer vision. New York: McGraw-Hill.Google Scholar

Copyright information

© Kluwer Academic Publishers 1987

Authors and Affiliations

  • Stewart W. Wilson
    • 1
  1. 1.The Rowland Institute for ScienceCambridgeU.S.A.

Personalised recommendations