Soft Computing

, Volume 6, Issue 3–4, pp 162–170 | Cite as

Learning classifier systems from a reinforcement learning perspective

  • P. L. Lanzi


 We analyze learning classifier systems in the light of tabular reinforcement learning. We note that although genetic algorithms are the most distinctive feature of learning classifier systems, it is not clear whether genetic algorithms are important to learning classifiers systems. In fact, there are models which are strongly based on evolutionary computation (e.g., Wilson's XCS) and others which do not exploit evolutionary computation at all (e.g., Stolzmann's ACS). To find some clarifications, we try to develop learning classifier systems “from scratch”, i.e., starting from one of the most known reinforcement learning technique, Q-learning. We first consider thebasics of reinforcement learning: a problem modeled as a Markov decision process and tabular Q-learning. We introduce a formal framework to define a general purpose rule-based representation which we use to implement tabular Q-learning. We formally define generalization within rules and discuss the possible approaches to extend our rule-based Q-learning with generalization capabilities. We suggest that genetic algorithms are probably the most general approach for adding generalization although they might be not the only solution.

Keywords Genetic algorithms, Reinforcement learning, XCS, Q-learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • P. L. Lanzi
    • 1
  1. 1.Dipartimento di Elettronica ed Informazione, Politecnico di Milano, Piazza Leonardo da Vinci n. 32, 1-20133 Milano e-mail: pierluca.lanzi@polimi.itIT

Personalised recommendations