Machine Learning

, Volume 3, Issue 2–3, pp 225–245 | Cite as

Credit assignment in rule discovery systems based on genetic algorithms

  • John J. Grefenstette
Article

Abstract

In rule discovery systems, learning often proceeds by first assessing the quality of the system's current rules and then modifying rules based on that assessment. This paper addresses the credit assignment problem that arises when long sequences of rules fire between successive external rewards. The focus is on the kinds of rule assessment schemes which have been proposed for rule discovery systems that use genetic algorithms as the primary rule modification strategy. Two distinct approaches to rule learning with genetic algorithms have been previously reported, each approach offering a useful solution to a different level of the credit assignment problem. We describe a system, called RUDI, that exploits both approaches. We present analytic and experimental results that support the hypothesis that multiple levels of credit assignment can improve the performance of rule learning systems based on genetic algorithms.

Keywords

Credit assignment rule discovery genetic algorithms classifier systems bucket brigade algorithm 

References

  1. Booker, L. B. (1982). Intelligent behavior as an adaptation to the task environment. Doctoral dissertation, Department of Computer and Communications Sciences, University of Michigan, Ann Arbor.Google Scholar
  2. De Jong, K. A. (1975). An analysis of the behavior of a class of genetic adaptive systems. Doctoral dissertation, Department of Computer and Communications Sciences, University of Michigan, Ann Arbor.Google Scholar
  3. DeJong K. (1987). On using genetic algorithms to search program spaces. Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms (pp. 210–216). Cambridge, MA: Lawrence Erlbaum.Google Scholar
  4. Fogel L. J., Owens A. J., & Walsh M. J. (1966). Artificial intelligence through simulated evolution. New York: John Wiley.Google Scholar
  5. Goldberg, D. E. (1983). Computer-aided gas pipeline operation using genetic algorithms and rule learning. Doctoral dissertation, Department of Civil Engineering, University of Michigan, Ann Arbor.Google Scholar
  6. Grefenstette J. J. (1986). Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics, 16, 122–128.Google Scholar
  7. Grefenstette J. J. (1987). Incorporating problem specific knowledge into genetic algorithms. In L.Davis (Ed.), Genetic algorithms and simulated annealing. London: Pitman Press.Google Scholar
  8. Holland J. H. (1975). Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.Google Scholar
  9. Holland J. H. (1985). Properties of the bucket brigade algorithm. Proceedings of the First International Conference on Genetic Algorithms and Their Applications (pp. 1–7). Pittsburgh, PA: Lawrence Erlbaum.Google Scholar
  10. Holland J. H. (1986). Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In R. S.Michalski, J. G.Carbonell, & T. M.Mitchell (Eds.), Machine learning: An artificial intelligence approach (Vol. 2). Los Altos, CA: Morgan Kaufmann.Google Scholar
  11. Holland J. H., Holyoak K. J., Nisbett R. E., & Thagard P. R. (1986). Induction: Processes of inference, learning, and discovery. Cambridge, MA: MIT Press.Google Scholar
  12. Holland J. H., & Reitman J. S. (1978). Cognitive systems based on adaptive algorithms. In D. A.Waterman & F.Hayes-Roth (Eds.), Pattern-directed inference systems. New York: Academic Press.Google Scholar
  13. Langley P. (1983). Learning effective search heuristics. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (pp. 419–421). Karlsruhe, West Germany: Morgan Kaufmann.Google Scholar
  14. Minsky, M. (1961). Steps toward artificial intelligence. Proceedings of the Institute of Radio Engineers (pp. 8–30).Google Scholar
  15. Mitchell T. M., Keller R. M., & Kedar-Cabelli S. T. (1986). Explanationbased generalization: A unifying view. Machine Learning, 1, 47–80.Google Scholar
  16. Mitchell T. M., Utgoff P. E., & Banerji R. (1983). Learning by experimentation: Acquiring and refining problem-solving heuristics. In R. S.Michalski, J. G.Carbonell, & T. M.Mitchell (Eds.), Machine learning: An artificial intelligence approach. Los Altos, CA: Morgan Kaufmann.Google Scholar
  17. Riolo R. L. (1986). CFS-C: A package of domain independent subroutines for implementing classifier systems in arbitrary, user-defined environments (Technical Report). Ann Arbor: University of Michigan, Division of Computer Science and Engineering, Logic of Computers Group.Google Scholar
  18. Riolo R. L. (1987). Bucket brigade performance: I. Long sequences of classifiers. Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms (pp. 184–195). Cambridge, MA: Lawrence Erlbaum.Google Scholar
  19. Robertson G. G. (1987). Parallel implementation of genetic algorithms in a classifier system. In L.Davis (Ed.), Genetic algorithms and simulated annealing. London: Pitman Press.Google Scholar
  20. Schaffer J. D., & Grefenstette J. J. (1985). Multi-objective learning via genetic algorithms. Proceedings of the Ninth International Joint Conference on Artificial Intelligence (pp. 593–595). Los Angeles, CA: Morgan Kaufmann.Google Scholar
  21. Sleeman D., Langley P., & Mitchell T. M. (1982). Learning from solution paths: An approach to the credit assignment problem. AI Magazine, 3, 48–52.Google Scholar
  22. Smith, S. F. (1980). A learning system based on genetic adaptive algorithms. Doctoral dissertation, Department of Computer Science, University of Pittsburgh, PA.Google Scholar
  23. Smith S. F. (1983). Flexible learning of problem solving heuristics through adaptive search. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (pp. 422–425). Karlsruhe, West Germany: Morgan Kaufmann.Google Scholar
  24. Sutton R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3, 9–44.Google Scholar
  25. Waterman D. A. (1970). Generalization learning techniques for automating the learning of heuristics. Artificial Intelligence, 1, 121–170.Google Scholar
  26. Westerdale T. H. (1985). The bucket brigade is not genetic. Proceedings of the First International Conference on Genetic Algorithms and Their Applications (pp. 45–59). Pittsburgh, PA: Lawrence Erlbaum.Google Scholar
  27. Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits. 1960 WESTCON Convention Record, Part IV (pp. 96–104).Google Scholar
  28. Wilson S. W. (1985). Knowledge growth in an artificial animal. Proceedings of the First International Conference on Genetic Algorithms and Their Applications (pp. 16–23). Pittsburgh, PA: Lawrence Erlbaum.Google Scholar
  29. Wilson S. W.. (1987a). Classifier systems and the animat problem. Machine Learning, 2, 199–228.Google Scholar
  30. Wilson S. W. (1987b). Hierarchical credit allocation in a classifier system. In L.Davis (Ed.), Genetic algorithms and simulated annealing. London: Pitman Press.Google Scholar
  31. Zhou, H. H. (1987). CSM: A genetic classifier system with memory for learning by analogy. Doctoral dissertation, Department of Computer Science, Vanderbilt University, Nashville, TN.Google Scholar

Copyright information

© Kluwer Academic Publishers 1988

Authors and Affiliations

  • John J. Grefenstette
    • 1
  1. 1.Navy Center for Applied Research in Artificial IntelligenceNaval Research LaboratoryWashington, DCU.S.A.

Personalised recommendations