Skip to main content
SpringerLink
Log in

A Comparative Study of Several Genetic-Based Supervised Learning Systems

  • Chapter
Learning Classifier Systems in Data Mining

Part of the book series: Studies in Computational Intelligence ((SCI,volume 125))

Summary

This chapter gives insight in the use of Genetic-Based Machine Learning (GBML) for supervised tasks. Five GBML systems which represent different learning methodologies and knowledge representations in the GBML paradigm are selected for the analysis: UCS, GAssist, SLAVE, Fuzzy AdaBoost, and Fuzzy LogitBoost. UCS and GAssist are based on a non-fuzzy representation, while SLAVE, Fuzzy AdaBoost, and Fuzzy LogitBoost use a linguistic fuzzy representation. The models evolved by these five systems are compared in terms of performance and interpretability to the models created by six highly-used non-evolutionary learners. Experimental observations highlight the suitability of GBML systems for classification tasks. Moreover, the analysis points out which systems should be used depending on whether the user prefers to maximize the accuracy or the interpretability of the models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J. Aguilar-Ruiz, J. Riquelme, and M. Toro. Evolutionary Learning of Hierarchical Decision Rules. IEEE Transactions on Systems, Man, and Cybernetics Part B, 33(2):324–331, 2003.

    Article  Google Scholar 

  2. J. Alcalá-Fdez, M.J. del Jesus, J.M. Garrell, F. Herrera, C. Herbás, and L. Sánchez. Proyecto KEEL: Desarrollo de una herramienta para el análisis e implementación de algoritmos de extracción de conocimiento evolutivos. In J.S. Aguilar R. Giráldez, J.C. Riquelme, editor, Tendencias de la Minería de Datos en España, Red Española de Minería de Datos y Aprendizage, pages 413–424, 2004.

    Google Scholar 

  3. J. Bacardit. GAssist Source Code: http://www.asap.cs.nott.ac.uk/jqb/PSP/ GAssist-Java.tar.gz .

  4. J. Bacardit. Pittsburgh Genetic-Based Machine Learning in the Data Mining Era: Representations, generalization and run-Time. PhD thesis, Ramon Llull University, Barcelona, 2004.

    Google Scholar 

  5. T. Bäck. Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, Oxford, 1996.

    MATH  Google Scholar 

  6. E. Bernadó-Mansilla and J.M. Garrell. Accuracy-Based Learning Classifier Systems: Models, Analysis and Applications to Classification Tasks. Evolutionary Computation, 11(3):209–238, 2003.

    Article  Google Scholar 

  7. E. Bernadó-Mansilla, X. Llorà, and J.M. Garrell. XCS and GALE: A Comparative Study of Two Learning Classifier Systems on Data Mining. In Advances in Learning Classifier Systems, volume 2321 of LNAI, pages 115–132. Springer, Berlin Heidelberg New York, 2002.

    Google Scholar 

  8. C.L Blake and C.J. Merz. UCI Repository of ML Databases: http://www.ics.uc.edu/mlearn/MLRepository.html . University of California, 1998.

  9. P. Bonelli and A. Parodi. An efficient classifier system and its experimental comparison with two representative learning methods on three medical domains. In 4th International Conference on Genetic Algorithms, pages 288–295, 1991.

    Google Scholar 

  10. L. Castillo, A. González, and R. Pérez. Including a simplicity criterion in the selection of the best rule in a genetic fuzzy learning algorithm. Fuzzy Sets and Systems, 120:309–321, 2001.

    Article  MATH  MathSciNet  Google Scholar 

  11. O. Cordón, F. Herrera, F. Hoffmann, and L. Magdalena. Genetic Fuzzy Systems: Evolutionary Tuning and Learning of Fuzzy Knowledge Bases, volume 19 of Advances in Fuzzy Systems–Aplications and Theory. World Scientific, Singapore, 2001.

    MATH  Google Scholar 

  12. K.A. de Jong and W. Spears. Learning Concept Classification Rules Using Genetic Algorithms. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 651–656. Sydney, Australia, 1991.

    Google Scholar 

  13. K.A. de Jong, W.M. Spears, and D.F. Gordon. Using Genetic Algorithms for Concept Learning. Genetic Algorithms for Machine Learning, A Special Issue of Machine Learning, 13, 2–3, pages 161–188, 1993.

    Google Scholar 

  14. M.J. del Jesús, F. Hoffmann, L.J. Navascués, and L. Sánchez. Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms. IEEE Transactions on Fuzzy Systems, 12(3):296–308, 2004.

    Article  Google Scholar 

  15. J. Demtilde{s}ar. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 7:1–30, 2006.

    Google Scholar 

  16. P.W. Dixon, D.W. Corne, and M.J. Oates. A Ruleset Reduction Algorithm for the XCSI Learning Classifier System, volume 2661/2003 of Lecture Notes in Computer Science, pages 20–29. Springer, Berlin Heidelberg New York, 2004.

    Google Scholar 

  17. O.J. Dunn. Multiple Comparisons among Means. Journal of the American Statistical Association, 56:52–64, 1961.

    Article  MATH  MathSciNet  Google Scholar 

  18. R.A. Fisher. Statistical Methods and Scientific Inference, 2nd edn. Hafner Publishing Company, New York, 1959.

    Google Scholar 

  19. E. Frank and I.H. Witten. Generating accurate rule sets without global optimization. In Proceedings of the 15th International Conference on Machine Learning, pages 144–151. Morgan Kaufmann, San Francisco, 1998.

    Google Scholar 

  20. A. Freitas. Data Mining and Knowledge Discovery with Evolutionary Algorithms. Springer, Berlin Heidelberg New York, 2002.

    MATH  Google Scholar 

  21. Y. Freund and R.E. Schapire. Experiments with a New Boosting Algorithm. In International Conference on Machine Learning, pages 148–156, 1996.

    Google Scholar 

  22. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 32(2):337–374, 2000.

    Article  MathSciNet  Google Scholar 

  23. M. Friedman. The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance. Journal of the American Statistical Association, 32:675–701, 1937.

    Article  Google Scholar 

  24. M. Friedman. A Comparison of Alternative Tests of Significance for the Problem of m Rankings. Annals of Mathematical Statistics, 11:86–92, 1940.

    Article  MATH  MathSciNet  Google Scholar 

  25. C. Fu and L. Davis. A modified classifier system compaction algorithm. In GECCO’02: Proceedings of the Genetic and Evolutionary Computation Conference, pages 920–925. Morgan Kaufmann, San Francisco, 2002.

    Google Scholar 

  26. D.E. Goldberg. Genetic Algorithms in Search, Optimization & Machine Learning, 1st edn. Addison Wesley, Reading, 1989.

    MATH  Google Scholar 

  27. D.E. Goldberg. The Design of Innovation: Lessons from and for Competent Genetic Algorithms, 1st edn. Kluwer, Boston, 2002.

    MATH  Google Scholar 

  28. A. Gónzalez and R. Pérez. Completeness and Consistency Conditions for Learning Fuzzy Rules. Fuzzy Sets and Systems, 96:37–51, 1998.

    Article  MathSciNet  Google Scholar 

  29. A. Gónzalez and R. Pérez. SLAVE: A Genetic Learning System Based on an Iterative Approach. IEEE Transactions on Fuzzy Systems, 7(2):176–191, 1999.

    Article  Google Scholar 

  30. J.H. Holland. Adaptation in Natural and Artificial Systems. The University of Michigan Press, Michigan, 1975.

    Google Scholar 

  31. J.H Holland. Escaping Brittleness: The possibilities of General-Purpose Learning Algorithms Applied to Parallel Rule-Based Systems. In Michalski Mitchell and Carbonell, editors, Machine Learning, an artificial intelligence approach, volume II of Lecture Notes in Artificial Intelligence, pages 593–623. Morgan Kaufmann, San Francisco, 1986.

    Google Scholar 

  32. C.Z. Janikow. A Knowledge-Intensive Genetic Algorithm for Supervised Learning. Machine Learning, 13(2–3):189–228, 1993.

    Article  Google Scholar 

  33. G.H. John and P. Langley. Estimating Continuous Distributions in Bayesian Classifiers. In 11th Conference on Uncertainty in Artificial Intelligence, pages 338–345. Morgan Kaufmann, San Francisco, 1995.

    Google Scholar 

  34. Z. Liu, A. Liu, C. Wang, and Z. Niu. Evolving neural network using real coded genetic algorithm (GA) for multispectral image classification. Future Generation Computer Systems, 20(7):1119–1129, 2004.

    Article  Google Scholar 

  35. A. Orriols-Puig and E. Bernadó-Mansilla. A Further Look at UCS Classifier System. In GECCO’06: Genetic and Evolutionary Computation Conference Workshop Program, ACM Press, Seattle, 08–12 July 2006.

    Google Scholar 

  36. J. Otero and L. Sánchez. Induction of descriptive fuzzy classifiers with the logitboost algorithm. Soft Computing, 10(9):825–835, 2006.

    Article  Google Scholar 

  37. M. Pelikan. Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithms, volume 170 of Studies in Computational Intelligence. Springer, Berlin Heidelberg New York, 2005.

    MATH  Google Scholar 

  38. M. Pelikan, K. Sastry, and E. Cantú-Paz. Scalable Optimization via Probabilistic Modeling, volume 33 of Studies in Computational Intelligence. Springer, Berlin Heidelberg New York, 2006.

    MATH  Google Scholar 

  39. J. Platt. Fast Training of Support Vector Machines using Sequential Minimal Opt. In Advances in Kernel Methods - Support Vector Lear. MIT Press, 1998.

    Google Scholar 

  40. J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, California, 1995.

    Google Scholar 

  41. J. Rissanen. Modeling by shortest data description. Automatica, vol. 14:465–471, 1978.

    Google Scholar 

  42. R.E. Schapire and Y. Singer. Improved Boosting Algorithms using Confidence-Rated Predictions. Machine Learning, 37(3):297–336, 1999.

    Article  MATH  Google Scholar 

  43. D.J. Sheskin. Handbook of Parametric and Nonparametric Statistical Procedures. Chapman & Hall, Boca Raton, 2000.

    MATH  Google Scholar 

  44. R.S. Sutton and A.G. Barto. Reinforcement learning: An introduction. MIT, Cambridge, 1998.

    Google Scholar 

  45. T.G. Dietterich. Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms. Neural Computation, 10(7):1895–1924, 1998.

    Article  Google Scholar 

  46. V. Vapnik. The Nature of Statistical Learning Theory. Springer, Berlin Heidelberg New York, 1995.

    MATH  Google Scholar 

  47. G. Venturini. SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attributes Based Concepts. In P. B. Brazdil, editor, Machine Learning: ECML-93 - Proceedings of the European Conference on Machine Learning, pages 280–296. Springer, Berlin Heidelberg New York, 1993.

    Google Scholar 

  48. D. Wierstra, F.J. Gómez, and J. Schmidhuber. Modeling Systems with Internal State Using Evolino. In GECCO’05: Proceedings of the 2005 conference on Genetic and evolutionary computation, pages 1795–1802. ACM Press, New York, 2005.

    Google Scholar 

  49. F. Wilcoxon. Individual Comparisons by Ranking Methods. Biometrics, 1:80–83, 1945.

    Article  Google Scholar 

  50. S.W. Wilson. Classifier Fitness Based on Accuracy. Evolutionary Computation, 3(2):149–175, 1995.

    Article  Google Scholar 

  51. S.W. Wilson. Generalization in the XCS Classifier System. In 3rd Annual Conference on Genetic Programming, pages 665–674. Morgan Kaufmann, San Francisco, 1998.

    Google Scholar 

  52. S.W. Wilson. Compact Rulesets from XCSI. In Advances in Learning Classifier Systems, 4th International Workshop, volume 2321 of Lecture Notes in Artificial Intelligence, pages 197–210. Springer, Berlin Heidelberg New York, 2002.

    Google Scholar 

  53. I.H Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco, 2005.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Enginyeria i Arquitectura La Salle, Universitat Ramon Llull, 08022, Barcelona, Spain

    Albert Orriols-Puig & Ester Bernadó-Mansilla

  2. Department of Computer Science and Artificial Intelligence, University of Granada, 18071, Granada, Spain

    Jorge Casillas

Authors
  1. Albert Orriols-Puig
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jorge Casillas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ester Bernadó-Mansilla
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computer Science, University of the West of England, Bristol, BS16 1QY, UK

    Larry Bull

  2. Enginyeria i Arquitectura La Salle, Universitat Ramon Llull, 08022, Barcelona, Spain

    Ester Bernadó-Mansilla

  3. Centre for Clinical Epidemiology and Biostatistics, University of Pennsylvania, Philadelphia, PA, 19104, USA

    John Holmes

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Orriols-Puig, A., Casillas, J., Bernadó-Mansilla, E. (2008). A Comparative Study of Several Genetic-Based Supervised Learning Systems. In: Bull, L., Bernadó-Mansilla, E., Holmes, J. (eds) Learning Classifier Systems in Data Mining. Studies in Computational Intelligence, vol 125. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78979-6_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78979-6_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78978-9

  • Online ISBN: 978-3-540-78979-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation