Summary
We study the behavior of XCS, a classifier based on genetic algorithms. XCS summarizes the state of the art of the evolutionary learning field and benefits from the long experience and research in the area. We describe the XCS learning mechanisms by which a set of rules describing the class boundaries is evolved. We study XCS’s behavior and its relationship to data complexity. We find that the difficult cases for XCS are those with long boundaries, high class interleaving, and high nonlinearities. Comparison with other classifiers in the complexity space enables identifying domains of competence for XCS as well as domains of poor performance. The study lays the basis to further apply the same methodology to analyze the domains of competence of other classifiers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
D.W. Aha, D. Kibler, M.K. Albert. Instance-based learning algorithms. Machine Learning, 6, 37–66, 1991.
J. Bacardit. Pittsburgh genetic-based machine learning in the data mining era: representations, generalization and run-time. Ph.D. thesis, Enginyeria i Arquitectura La Salle, Ramon Llull University, Barcelona, Spain, 2004.
J. Bacardit, M.V. Butz. Data mining in learning classifier systems: comparing XCS with G Assist. In Seventh International Workshop on Learning Classifier Systems (IWLCS-2004), Seattle, WA, June 26, 2004.
E. Bernadó-Mansilla, J.M. Garrell Guiu. Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evolutionary Computation, 11(3), 209–238, 2003.
E. Bernadó-Mansilla, T.K. Ho. Domain of competence of XCS classifier system in complexity measurement space. IEEE Transactions on Evolutionary Computation, 9(1), 82–104, 2005.
E. Bernadó-Mansilla, X. L. Fàbrega, J.M. Garrell Guiu. XCS and GALE: a comparative study of two learning classifier systems on data mining. In P.L. Lanzi, W. Stolzmann, S.W. Wilson, eds. Advances in Learning Classifier Systems, 4th International Workshop, volume 2321 of Lecture Notes in Computer Science, pages 115–132. New York: Springer, 2002.
C.L. Blake, C.J. Merz. UCI Repository of machine learning databases, [http://www.ics.uci.edu/~mlearn/MLRepository.html]. University of California, Irvine, Department of Information and Computer Sciences, 1998.
L. Breiman. Bagging predictors. Machine Learning, 24, 123–140, 1996.
M.V. Butz, M. Pelikan. Analyzing the Evolutionary Pressures in XCS. In L. Spector, E.D. Goodman, A. Wu, et al., eds. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2001), pages 935–942. San Francisco: Morgan Kaufmann, 2001.
M.V. Butz. Rule-based evolutionary online learning systems: learning bounds, classification, and prediction. Ph.D. thesis, University of Illinois, 2004.
M.V. Butz, S.W. Wilson. An algorithmic description of XCS. In P.L. Lanzi, W. Stolzmann, S.W. Wilson, eds. Advances in Learning Classifier Systems: Proceedings of the Third International Workshop, volume 1996 of Lecture Notes in Artificial Intelligence, pages 253–272. Berlin, Heidelberg: Springer-Verlag, 2001.
E. Cantú-Paz, C. Kamath. Inducing oblique decision trees with evolutionary algorithms. IEEE Transactions on Evolutionary Computation, 7(1), 54–68, 2003.
K.A. De Jong, W.M. Spears, D.F. Gordon. Using genetic algorithms for concept learning. Genetic Algorithms for Machine Learning (John J. Grefenstette ed.), A Special Issue of Machine Learning, 13,2-3, 161–188, 1993.
D.E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. New York, Addison-Wesley, 1989.
D.E. Goldberg. The Design of Innovation. Lessons from and for Competent Genetic Algorithms. Kluwer Academic Publishers, 2002.
T.K. Ho. The random subspace method for constructing decision forests. IEEE Transcations on Pattern Analysis and Machine Intelligence, 20(8), 832–844, 1998.
T.K. Ho, M. Basu. Complexity measures of supervised classification problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(3), 289–300, 2002.
J.H. Holland. Adaptation in Natural and Artificial Systems. Ann Arbor: University of Michigan Press, 1975.
P.L. Lanzi. Extending the representation of classifier conditions. Part I: from binary to messy coding. In W. Banzhaf, J. Daida, A.E. Eiben, et al., eds. Proceedings of the Genetic and Evolutionary Computation Conference, (GECCO-99), pages 337–344. San Francisco: Morgan Kaufmann, 1999.
P.L. Lanzi. Extending the representation of classifier conditions. Part II: from messy coding to S-expressions. In W. Banzhaf, J. Daida, A.E. Eiben, et al., eds. Proceedings of the Genetic and Evolutionary Computation Conference, (GECCO-99), pages 345–352. San Francisco: Morgan Kaufmann, 1999.
X. Llorà, J.M. Garrell Guiu. Co-evolving different knowledge representations with fine-grained parallel learning classifier systems. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO2002), pages 934–941. San Francisco: Morgan Kaufmann, 2002.
S. Murthy, S. Kasif, S. Salzberg. A system for induction of oblique decision trees. Journal of Artificial Intelligence Research, 2(1), 1–32, 1994.
J.D. Schaffer. Combinations of genetic algorithms with neural netwoks or fuzzy systems. In J.M. Zurada, R.J. Marks, J. Marks, II, C.J. Robinson, eds. Computational Intelligence Imitating Life, pages 371–382. New York: IEEE Press, 1994.
F.W. Smith. Pattern classifier design by linear programming. IEEE Transactions on Computers, C-17, 367–372, 1968.
C. Stone, L. Bull. For real! XCS with continuous-valued inputs. Evolutionary Computation, 11(3), 299–336, 2003.
S.W. Wilson. Classifier fitness based on accuracy. Evolutionary Computation, 3(2), 149–175, 1995.
S.W. Wilson. Generalization in the XCS classifier system. In J.R. Koza, W. Banzhaf, K. Chellapilla, et al., eds. Genetic Programming: Proceedings of the Third Annual Conference. San Francisco: Morgan Kaufmann, 1998.
S.W. Wilson. Get real! XCS with continuous-valued inputs. In L. Booker, S. Forrest, M. Mitchell, R. Riolo, eds. Festschrift in Honor of John H. Holland, pages 111–121. Center for the Study of Complex Systems, University of Michigan, 1999.
X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9), 1423–1447, 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer Verlag London Limited
About this chapter
Cite this chapter
Bernadó-Mansilla, E., Kam Ho, T., Orriols, A. (2006). Data Complexity and Evolutionary Learning. In: Basu, M., Ho, T.K. (eds) Data Complexity in Pattern Recognition. Advanced Information and Knowledge Processing. Springer, London. https://doi.org/10.1007/978-1-84628-172-3_6
Download citation
DOI: https://doi.org/10.1007/978-1-84628-172-3_6
Publisher Name: Springer, London
Print ISBN: 978-1-84628-171-6
Online ISBN: 978-1-84628-172-3
eBook Packages: Computer ScienceComputer Science (R0)