Evolutionary Intelligence

, 2:141 | Cite as

Sequential problems that test generalization in learning classifier systems

Short Note
  • 91 Downloads

Abstract

We present an approach to build sequential decision making problems which can test the generalization capabilities of classifier systems. The approach can be applied to any sequential problem defined over a binary domain and it generates a new problem with bounded sequential difficulty and bounded generalization difficulty. As an example, we applied the approach to generate two problems with simple sequential structure, huge number of states (more than a million), and many generalizations. These problems are used to compare a classifier system with effective generalization (XCS) and a learner without generalization (Q-learning). The experimental results confirm what was previously found mainly using single-step problems: also in sequential problems with huge state spaces, XCS can generalize effectively by detecting those substructures that are necessary for optimal sequential behavior.

Keywords

Learning classifier systems Genetics-based machine learning Reinforcement learning Generalization XCS 

References

  1. 1.
    Bagnall AJ, Zatuchna ZV (2005) On the classification of maze problems. In: Bull L, Kovacs T (eds) Foundations of learning classifier systems. Studies in fuzziness and soft computing, vol 183. Springer, Berlin, pp 307–316Google Scholar
  2. 2.
    Barry AM (2002) The stability of long action chains in xcs. J Soft Comput 6(3–4):183–199MATHGoogle Scholar
  3. 3.
    Booker LB (1989) Triggered rule discovery in classifier systems. In: Schaffer JD (ed) Proceedings of the 3rd international conference on genetic algorithms (ICGA89). George Mason University/Morgan Kaufmann, Fairfax county/Menlo Park, pp 265–274Google Scholar
  4. 4.
    Bull L (2004) Applications of learning classifier systems. Studies in fuzziness and soft computing. Springer, BerlinGoogle Scholar
  5. 5.
    Bull L, Mansilla EB, Holmes JH (2008) Learning classifier systems in data mining. Studies in computational intelligence. Springer, BerlinGoogle Scholar
  6. 6.
    Butz M, Goldberg DE (2003) Generalized state values in an anticipatory learning classifier system. In: Butz M, Sigaud O, Gérard P (eds) Anticipatory behavior in adaptive learning systems. Lecture notes in computer science, vol 2684. Springer, Berlin, pp 282–301Google Scholar
  7. 7.
    Butz MV (2000) Anticipatory learning classifier systems. Genetic algorithms and evolutionary computation, vol 4. Springer, BerlinGoogle Scholar
  8. 8.
    Butz MV (2006) Rule-based evolutionary online learning systems: a principled approach to LCS analysis and design. Studies in fuzziness and soft computing. Springer, BerlinGoogle Scholar
  9. 9.
    Butz MV, Goldberg DE, Lanzi PL (2005) Gradient descent methods in learning classifier systems: Improving xcs performance in multistep problems. IEEE Trans Evol Comput 9(5):452–473CrossRefGoogle Scholar
  10. 10.
    Butz MV, Kovacs T, Lanzi PL, Wilson SW (2004) Toward a theory of generalization and learning in XCS. IEEE Trans Evol Comput 8(1):28–46CrossRefGoogle Scholar
  11. 11.
    Butz MV, Pelikan M, Llorá X, Goldberg DE (2006) Automated global structure extraction for effective local building block processing in XCS. Evol Comput 14:345–380CrossRefGoogle Scholar
  12. 12.
    Butz MV, Wilson SW (2002) An algorithmic description of XCS. J Soft Comput 6(3–4):144–153MATHGoogle Scholar
  13. 13.
    Cliff D, Ross S (1995) Adding temporary memory to ZCS. Technical report CSRP347, School of Cognitive and Computing Sciences, University of Sussex. ftp://ftp.cogs.susx.ac.uk/pub/reports/csrp/csrp347.ps.Z
  14. 14.
    Drugowitsch J, Barry AM (2005) XCS with eligibility traces. In: Beyer AM, O’Reilly U-M, Arnold DV, Banzhaf W, Blum C, Bonabeau EW, Cantu-Paz E, Dasgupta D, Deb K, Foster JA, de Jong ED, Lipson H, Llora X, Mancoridis S, Pelikan M, Raidl GR, Soule T, Tyrrell AM, Watson J-P, Zitzler E (eds) GECCO 2005: proceedings of the 2005 conference on genetic and evolutionary computation, vol 2. ACM Press, Washington DC, pp 1851–1858Google Scholar
  15. 15.
    Holland JH (1976) Adaptation. In: Rosen R, Snell F (eds) Progress in theoretical biology, vol 4. Academic Press, New York, pp 263–293Google Scholar
  16. 16.
    Holland JH, Reitman JS (1988) Cognitive systems based on adaptive algorithms, 1978. Reprinted In: David B. Fogel (ed) Evolutionary computation. The fossil record. IEEE Press, New York. ISBN: 0-7803-3481-7Google Scholar
  17. 17.
    Lanzi PL (1997) A study on the generalization capabilities of XCS. In: Baeck T (ed) Proceedings of the seventh international conference on genetic algorithms, 19–23 April east lansing (MI). Morgan Kaufmann, San Francisco, pp 418–425Google Scholar
  18. 18.
    Lanzi PL (1999) An analysis of generalization in the XCS classifier system. Evol Comput J 7(2):125–149CrossRefGoogle Scholar
  19. 19.
    Sutton RS, Barto AG (1998) Reinforcement learning—an introduction. MIT Press, CambridgeGoogle Scholar
  20. 20.
    Wilson SW (1994) ZCS: A zeroth level classifier system. Evol Comput 2(1):1–18. http://prediction-dynamics.com/
  21. 21.
    Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175CrossRefGoogle Scholar
  22. 22.
    Wilson SW (1998) Generalization in the XCS classifier system. In: Koza JR, Banzhaf W, Chellapilla K, Dorigo KDM, Fogel DB, Garzon MH, Iba DEGH, Riolo R (eds) Genetic programming 1998: proceedings of the third annual conference. Morgan Kaufmann, Menlo Park, pp 665–674. http://prediction-dynamics.com/
  23. 23.
    Wilson SW (2000) Mining oblique data with XCS. Technical report 2000028, University of Illinois at Urbana-ChampaignGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.University of WürzburgWürzburgGermany
  2. 2.Dipartimento di Elettronica e InformazionePolitecnico di MilanoMilanItaly
  3. 3.Illinois Genetic Algorithm Laboratory (IlliGAL)University of Illinois at Urbana ChampaignUrbanaUSA

Personalised recommendations