XCS with Adaptive Action Mapping

  • Masaya Nakata
  • Pier Luca Lanzi
  • Keiki Takadama
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7673)

Abstract

The XCS classifier system evolves solutions that represent complete mappings from state-action pairs to expected returns therefore, in every possible situation, XCS can predict the value of all the available actions. Such complete mapping is sometimes considered redundant as most of the applications (like for instance, classification), usually focus only on the best action. In this paper, we introduce an extension of XCS with an adaptive (state-action) mapping mechanism (or XCSAM) that evolves solutions focused actions with the largest returns. While UCS evolves solutions focused on the best available action but can only solve supervised classification problems, our system can solve both supervised and multi-step problems and, in addition, it can adapt the size of the mapping to the problems: Initially, XCSAM starts building a complete mapping and then it slowly tries to focus on the best actions available. If the problem admits only one optimal action in each niche, XCSAM tends to focus on such an action as the evolution proceeds. If more actions with the same return are available, XCSAM tends to evolve a mapping that includes all of them. We applied XCSAM both to supervised problems (the Boolean multiplexer) and to multi-step maze-like problems. Our experimental results show that XCSAM can reach optimal performance but requires smaller populations than XCS as it evolves solutions focused on the best actions available for each subproblem.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bernadó-Mansilla, E., Garrell, J.M.: Accuracy-based learning classifier systems: Models, analysis and applications to classification tasks. Evolutionary Computation 11, 209–238 (2003)CrossRefGoogle Scholar
  2. 2.
    Butz, M.V., Goldberg, D.E., Lanzi, P.L.: Gradient Descent Methods in Learning Classifier Systems: Improving XCS Performance in Multistep Problems. Evolutionary Computation 9(5), 452–473 (2005)CrossRefGoogle Scholar
  3. 3.
    Butz, M.V., Kovacs, T., Lanzi, P.L., Wilson, S.W.: Toward a Theory of Generalization and Learning in XCS. IEEE Transactions on Evolutionary Computation 8(1), 28–46 (2004)CrossRefGoogle Scholar
  4. 4.
    Butz, M.V., Sastry, K., Goldberg, D.E.: Tournament Selection: Stable Fitness Pressure in XCS. In: Cantú-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2724, pp. 1857–1869. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  5. 5.
    Butz, M.V., Wilson, S.W.: An algorithmic description of xcs. Journal of Soft Computing 6(3-4), 144–153 (2002)CrossRefMATHGoogle Scholar
  6. 6.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison Wesley (1989)Google Scholar
  7. 7.
    Holland, J.H.: Escaping Brittleness: The Possibilities of General Purpose Learning Algorithms Applied to Parallel Rule-based system. Machine Learning 2, 593–623 (1986)Google Scholar
  8. 8.
    Kovacs, T.: Evolving optimal populations with XCS classifier systems. Technical Report CSR-96-17 and CSRP-96-17, School of Computer Science, University of Birmingham, Birmingham, U.K. (1996), Available from the technical report archive, ftp://ftp.cs.bham.ac.uk/pub/tech-reports/1996/CSRP-96-17.ps.gz
  9. 9.
    Lanzi, P.L.: An Analysis of Generalization in the XCS Classifier System. Evolutionary Computation Journal 7(2), 125–149 (1999)CrossRefGoogle Scholar
  10. 10.
    Lanzi, P.L.: Learning classifier systems from a reinforcement learning perspective. Soft Computing - A Fusion of Foundations, Methodologies and Applications 6(3), 162–170 (2002)MATHGoogle Scholar
  11. 11.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning – An Introduction. MIT Press (1998)Google Scholar
  12. 12.
    Wilson, S.W.: Classifier Fitness Based on Accuracy. Evolutionary Computation 3(2), 149–175 (1995)CrossRefGoogle Scholar
  13. 13.
    Wilson, S.W.: Classifier Fitness Based on Accuracy. Evolutionary Computation 3(2), 149–175 (1995), http://prediction-dynamics.com/ CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Masaya Nakata
    • 1
  • Pier Luca Lanzi
    • 2
  • Keiki Takadama
    • 1
  1. 1.Department of InformaticsThe University of Electro-CommunicationsTokyoJapan
  2. 2.Dipartimento di Elettronica e InformazionePolitecnico di MilanoMilanoItaly

Personalised recommendations