# Problem solution sustenance in XCS: Markov chain analysis of niche support distributions and the impact on computational complexity

- 110 Downloads
- 8 Citations

## Abstract

Michigan-style learning classifier systems iteratively evolve a distributed solution to a problem in the form of potentially overlapping subsolutions. Each problem *niche* is covered by subsolutions that are represented by a set of predictive rules, termed classifiers. The genetic algorithm is designed to evolve classifier structures that together cover the whole problem space and represent a complete problem solution. An obvious challenge for such an online evolving, distributed knowledge representation is to continuously sustain all problem subsolutions covering all problem niches, that is, to ensure *niche support*. Effective niche support depends both on the probability of reproduction and on the probability of deletion of classifiers in a niche. In XCS, reproduction is occurrence-based whereas deletion is support-based. In combination, niche support is assured effectively. In this paper we present a Markov chain analysis of the niche support in XCS, which we validate experimentally. Evaluations in diverse Boolean function settings, which require non-overlapping and overlapping solution structures, support the theoretical derivations. We also consider the effects of mutation and crossover on niche support. With respect to computational complexity, the paper shows that XCS is able to maintain (partially overlapping) niches with a computational effort that is linear in the inverse of the niche occurrence frequency.

### Keywords

Learning classifier systems LCS XCS Niching Markov chain analysis Solution sustenance Mutation## Notes

### Acknowledgments

We are grateful to Xavier Llorà, Kei Onishi, Martin Pelikan, Stewart Wilson, Tian-Li Yu, and the whole IlliGAL lab for their help and the useful discussions.

This work was supported by the Air Force Office of Scientific Research, Air Force Materiel Command, USAF, under grant FA9550-06-1-0096, the National Science Foundation under ITR grant DMR-03-25939 (at Materials Computation Center, UIUC), The US Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon. Butz's contribution received additional funding from the European commission contract no. FP6-511931.

The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of any of the organizations mentioned above

### References

- 1.E. Bernadó, X. Llorà, and J. M. Garrell, “XCS and GALE: A comparative study of two learning classifier systems and six other learning algorithms on classification tasks,” in Lanzi et al. [42], pp. 115–132.Google Scholar
- 2.E. Bernadó-Mansilla and J. M. Garrell-Guiu, “Accuracy-based learning classifier systems: Models, analysis, and applications to classification tasks,”
*Evolutionary Computation*, vol. 11, pp. 209–238, 2003.Google Scholar - 3.H.-G. Beyer, U.-M. O’Reily, D. V. Arnold, W. Banzhaf, C. Blum, E. W. Bonabeau, E. Cant-Paz, D. Dasgupta, K. Deb, J. A. Foster, E. D. de Jong, H. Lipson, X. Llora, S. Mancoridis, M. Pelikan, G. R. Raidl, T. Soule, A. M. Tyrrell, J.-P. Watson, and E. Zitzler (eds.).
*GECCO 2005: Genetic and Evolutionary Computation Conference: Volume 2*. Association for Computing Machinery, Inc: New York, NY, 2005.Google Scholar - 4.C. L. Bridges and D. E. Goldberg, “An analysis of reproduction and crossover in a binary-coded genetic algorithm,” in Grefenstette [26], pp. 9–13.Google Scholar
- 5.L. Bull, “Simple Markov models of the genetic algorithm in classifier systems: Accuracy-based fitness,” in Lanzi et al. [41], pp. 21–28. Google Scholar
- 6.L. Bull, “Simple Markov models of the genetic algorithm in classifier systems: Multi-step sasks,” in Lanzi et al. [41], pp. 29–36.Google Scholar
- 7.L. Bull and J. Hurst, “Self-adaptive mutation in ZCS controllers,” in
*Proceedings of the EvoNet Workshops - EvoRob 2000*, S. Cagnoni, R. Poli, G. Smith, D. Corne, M. Oates, E. Hart, P.-L. Lanzi, E. Willem, Y. Li, B. Paechter, and T. C. Fogarty (eds.), Springer-Verlag: Berlin Heidelberg, 2000, pp. 339–346.Google Scholar - 8.L. Bull and J. Hurst, “ZCS Redux,”
*Evolutionary Computation*, vol. 10, pp. 185–205, 2003.Google Scholar - 9.L. Bull, J. Hurst and A. Tomlinson, “Self-adaptive mutation in classifier system controllers,” in
*From Animals to Animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior*, J.-A. Meyer, A. Berthoz, D. Floreano, H. Roitblat, and S. W. Wilson (eds.), MIT Press: Cambridge, MA, 2000, pp. 460–467.Google Scholar - 10.M. V. Butz, “Documentation of XCS + TS C-Code 1.2,” Technical Report 2003023, Illinois Genetic Algorithms Laboratory – University of Illinois at Urbana-Champaign, 2003.Google Scholar
- 11.M. V. Butz, “Kernel-based, ellipsoidal conditions in the real-valued XCS classifier system,” in Beyer et al. [3], pp 1835–1842.Google Scholar
- 12.M. V. Butz, in
*Rule-Based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design*. Studies in Fuzziness and Soft Computing. Springer-Verlag: Berlin Heidelberg, 2006.Google Scholar - 13.M. V. Butz, D. E. Goldberg, and P. L. Lanzi, “Bounding learning time in XCS,” in
*Genetic and Evolutionary Computation Conference - GECCO 2004: Part II*, R. Poli, W. Banzhaf, H.-G. Beyer, E. Burke, P. Darwen, D. Dasgupta, D. Floreano, J. Foster, M. Harman, P. L. Lanzi, L. Spector, A. G. B. Tettamanzi, D. Thierens and A. M. Tyrrell (eds.), Springer-Verlag: Berlin Heidelberg, 2004, pp. 739–750.Google Scholar - 14.M. V. Butz, D. E. Goldberg, P. L. Lanzi, and K. Sastry, “Bounding the population size to ensure niche support in XCS,” Technical Report 2004033, Illinois Genetic Algorithms Laboratory – University of Illinois at Urbana-Champaign, 2004.Google Scholar
- 15.M. V. Butz, D. E. Goldberg, and K. Tharakunnel, “Analysis and improvement of fitness exploitation in XCS: Bounding models, tournament selection, and bilateral accuracy,”
*Evolutionary Computation*, vol. 11, pp. 239–277, 2003.Google Scholar - 16.M. V. Butz, T. Kovacs, P. L. Lanzi, and S. W. Wilson,“Toward a theory of generalization and learning in XCS,”
*IEEE Transactions on Evolutionary Computation*, vol. 8, pp. 28–46, 2004.Google Scholar - 17.M. V. Butz, K. Sastry, and D. E. Goldberg, “Strong, stable, and reliable fitness pressure in XCS due to tournament selection,”
*Genetic Programming and Evolvable Machines*, vol. 6, pp. 53–77, 2004.Google Scholar - 18.M. V. Butz and S. W. Wilson, “An algorithmic description of XCS,”
*Journal of Soft Computing*, vol. 6 pp. 144–153, 2002.Google Scholar - 19.W. J. Conover, in
*Practical Nonparametric Statistics*, 3rd edition, John Wiley & Sons: New York, NY, USA, 1998.Google Scholar - 20.K. A. De Jong, “An analysis of the behavior of a class of genetic adaptive systems,” PhD thesis, University of Michigan, Ann Arbor, 1975. University Microfilms No. 76-9381.Google Scholar
- 21.P. W. Dixon, D. W. Corne, and M. J. Oates, “A preliminary investigation of modified XCS as a generic data mining tool,” in Lanzi et al. [42], pp. 133–150.Google Scholar
- 22.R. A. Fisher.
*Statistical Methods for Research Workers*. Oliver and Boyd, Edinburgh, GB, 1925. Available online at \( {\tt http://psychclassics.yorku.ca/Fisher/Methods/}\).Google Scholar - 23.M. Friendly,
*Visualizing Categorical Data*. SAS Institute: Cary, NC, 2000.Google Scholar - 24.D. Goldberg and P. Segrest, “Finite Markov chain analysis of genetic algorithms,” in Grefenstette [26], pp. 1–8.Google Scholar
- 25.D. E. Goldberg, “Simple genetic algorithms and the minimal, deceptive problem,” in
*Genetic Algorithms and Simulated Annealing*, L. Davis (eds.), Research Notes in Artificial Intelligence, Pitman, London, 1987, pp. 74–88.Google Scholar - 26.J. J. Grefenstette (ed.),
*Proceedings of the Second International Conference on Genetic Algorithms*, Lawrence Erlbaum Associates, Inc: Mahwah, NJ, USA, 1987.Google Scholar - 27.G. Harik, “Finding multiple solutions in problems of bounded difficulty,” in
*Proceedings of the Sixth International Conference on Genetic Algorithms*, L. Eschelman (eds.), Morgan Kaufmann: San Francisco, CA, 1995, pp. 24–31,Google Scholar - 28.G. Harik, E. Cantú-Paz, D. E. Goldberg, and B. L. Miller, “The gambler's ruin problem, genetic algorithms, and the sizing of populations,”
*Evolutionary Computation*, vol. 7, pp. 231–253, 1999.Google Scholar - 29.J. H. Holland,
*Adaptation in Natural and Artificial Systems*, 2nd edition, 1992, University of Michigan Press: Ann Arbor, MI, 1975.Google Scholar - 30.J. H. Holland,
*Hidden Order: How Adaptation Builds Complexity*. Perseus Books: Cambridge, MA, USA, 1995.Google Scholar - 31.J. Horn, “Finite Markov chain analysis of genetic algorithms with niching,” in
*Proceedings of the Fifth International Conference on Genetic Algorithms*, S. Forrest (eds.), Morgan Kaufmann: San Francisco, CA, 1993, pp. 110–117,Google Scholar - 32.J. Horn, D. E. Goldberg, and K. Deb, “Implicit niching in a learning classifier system: Nature's way,“
*Evolutionary Computation*, vol. 2, pp. 37–66, 1994.Google Scholar - 33.J. Hurst and L. Bull, “A Self-Adaptive XCS,” in Lanzi et al. [42], pp. 57–73.Google Scholar
- 34.K. A. D. Jong and W. M. Spears, “Learning concept classification rules using genetic algorithms,” in
*Proceedings of the Twelfth International Conference on Artificial Intelligence IJCAI-91*, J. Mylopoulos and R. Reiter (eds.), vol. 2, Morgan Kaufmann: San Francisco, CA, 1991.Google Scholar - 35.L. Kleinroch,
*Queueing Systems: Theory*, John Wiley & Sons: New York, NY, USA, 1975.Google Scholar - 36.T. Kovacs, “XCS classifier system reliably evolves accurate, complete, and minimal representations for boolean functions,” in
*Soft Computing in Engineering Design and Manufacturing*, P. K. Chawdhry, R. Roy and R. K. Pant (eds.), Springer-Verlag: New York, NY, USA, 1997, pp. 59–68.Google Scholar - 37.T. Kovacs, “Deletion schemes for classifier systems,” in
*Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-99)*, W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela and R. E. Smith (eds.), Morgan Kaufmann: San Francisco, CA, 1999, pp. 329–336.Google Scholar - 38.T. Kovacs,
*Strength or Accuracy: Credit Assignment in Learning Classifier Systems*, Springer-Verlag: Berlin Heidelberg, 2003.Google Scholar - 39.P. L. Lanzi, “The XCS library,” \( {\tt http://xcslib.sourceforge.net, 2002}\).Google Scholar
- 40.P. L. Lanzi, D. Loiacono, S. W. Wilson and D. E. Goldberg, “Extending XCSF beyond linear approximation,” in Beyer et al. [3], pp. 1827–1834.Google Scholar
- 41.P. L. Lanzi, W. Stolzmann, and S. W. Wilson (eds.),
*Advances in Learning Classifier Systems: Third International Workshop, IWLCS 2000: Paris, France, September 2000: Revised Papers*, volume 1996 of LNAI, Springer-Verlag: Berlin Heidelberg, 2001.Google Scholar - 42.P. L. Lanzi, W. Stolzmann, and S. W. Wilson (eds.),
*Advances in Learning Classifier Systems: 4th International Workshop, IWLCS 2001: San Francisco, CA, USA, July 2001: Revised Papers*, volume 2321 of LNAI, Springer-Verlag: Berlin Heidelberg, 2002.Google Scholar - 43.D. G. Luenberger,
*Introduction to Dynamic Systems: Theory, Models, and Applications*, John Wiley & Sons: New York, NY, USA, May 1979.Google Scholar - 44.S. W. Mahfoud, “Crowding and preselection revisited,” in
*Parallel Problem Solving from Nature, 2*, R. Männer and B. Manderick (eds.), Elsevier: Amsterdam, 1992, pp. 27–36.Google Scholar - 45.R. Development Core Team.
*R: A language and environment for statistical computing*. R Foundation for Statistical Computing, Vienna, Austria, 2004. ISBN 3-900051-00-3.Google Scholar - 46.G. Venturini, “Adaptation in dynamic environments through a minimal probability of exploration,” in
*From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior*, D. Cliff, P. Husbands, J.-A. Meyer, and S. W. Wilson (eds.), MIT Press: Cambridge, MA, 1994, pp. 371–381.Google Scholar - 47.S. W. Wilson, “Classifier systems and the animat problem,”
*Machine Learning*, vol. 2, pp. 199–228, 1987.Google Scholar - 48.S. W. Wilson, “ZCS: A zeroth level classifier system,”
*Evolutionary Computation*, vol. 2, pp. 1–18, 1994.Google Scholar - 49.S. W. Wilson, “Classifier fitness based on accuracy,”
*Evolutionary Computation*, vol. 3, pp. 149–175, 1995.Google Scholar - 50.S. W. Wilson, “Generalization in the XCS classifier system,” in
*Genetic Programming 1998: Proceedings of the Third Annual Conference*, J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), Morgan Kaufmann: San Francisco, CA, 1998, pp. 665–674.Google Scholar