A Comparative Study of Several Genetic-Based Supervised Learning Systems

  • Albert Orriols-Puig
  • Jorge Casillas
  • Ester Bernadó-Mansilla

Summary

This chapter gives insight in the use of Genetic-Based Machine Learning (GBML) for supervised tasks. Five GBML systems which represent different learning methodologies and knowledge representations in the GBML paradigm are selected for the analysis: UCS, GAssist, SLAVE, Fuzzy AdaBoost, and Fuzzy LogitBoost. UCS and GAssist are based on a non-fuzzy representation, while SLAVE, Fuzzy AdaBoost, and Fuzzy LogitBoost use a linguistic fuzzy representation. The models evolved by these five systems are compared in terms of performance and interpretability to the models created by six highly-used non-evolutionary learners. Experimental observations highlight the suitability of GBML systems for classification tasks. Moreover, the analysis points out which systems should be used depending on whether the user prefers to maximize the accuracy or the interpretability of the models.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    J. Aguilar-Ruiz, J. Riquelme, and M. Toro. Evolutionary Learning of Hierarchical Decision Rules. IEEE Transactions on Systems, Man, and Cybernetics Part B, 33(2):324–331, 2003.CrossRefGoogle Scholar
  2. 2.
    J. Alcalá-Fdez, M.J. del Jesus, J.M. Garrell, F. Herrera, C. Herbás, and L. Sánchez. Proyecto KEEL: Desarrollo de una herramienta para el análisis e implementación de algoritmos de extracción de conocimiento evolutivos. In J.S. Aguilar R. Giráldez, J.C. Riquelme, editor, Tendencias de la Minería de Datos en España, Red Española de Minería de Datos y Aprendizage, pages 413–424, 2004.Google Scholar
  3. 3.
  4. 4.
    J. Bacardit. Pittsburgh Genetic-Based Machine Learning in the Data Mining Era: Representations, generalization and run-Time. PhD thesis, Ramon Llull University, Barcelona, 2004.Google Scholar
  5. 5.
    T. Bäck. Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, Oxford, 1996.MATHGoogle Scholar
  6. 6.
    E. Bernadó-Mansilla and J.M. Garrell. Accuracy-Based Learning Classifier Systems: Models, Analysis and Applications to Classification Tasks. Evolutionary Computation, 11(3):209–238, 2003.CrossRefGoogle Scholar
  7. 7.
    E. Bernadó-Mansilla, X. Llorà, and J.M. Garrell. XCS and GALE: A Comparative Study of Two Learning Classifier Systems on Data Mining. In Advances in Learning Classifier Systems, volume 2321 of LNAI, pages 115–132. Springer, Berlin Heidelberg New York, 2002.Google Scholar
  8. 8.
    C.L Blake and C.J. Merz. UCI Repository of ML Databases: http://www.ics.uc.edu/mlearn/MLRepository.html. University of California, 1998.
  9. 9.
    P. Bonelli and A. Parodi. An efficient classifier system and its experimental comparison with two representative learning methods on three medical domains. In 4th International Conference on Genetic Algorithms, pages 288–295, 1991.Google Scholar
  10. 10.
    L. Castillo, A. González, and R. Pérez. Including a simplicity criterion in the selection of the best rule in a genetic fuzzy learning algorithm. Fuzzy Sets and Systems, 120:309–321, 2001.MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    O. Cordón, F. Herrera, F. Hoffmann, and L. Magdalena. Genetic Fuzzy Systems: Evolutionary Tuning and Learning of Fuzzy Knowledge Bases, volume 19 of Advances in Fuzzy Systems–Aplications and Theory. World Scientific, Singapore, 2001.MATHGoogle Scholar
  12. 12.
    K.A. de Jong and W. Spears. Learning Concept Classification Rules Using Genetic Algorithms. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 651–656. Sydney, Australia, 1991.Google Scholar
  13. 13.
    K.A. de Jong, W.M. Spears, and D.F. Gordon. Using Genetic Algorithms for Concept Learning. Genetic Algorithms for Machine Learning, A Special Issue of Machine Learning, 13, 2–3, pages 161–188, 1993.Google Scholar
  14. 14.
    M.J. del Jesús, F. Hoffmann, L.J. Navascués, and L. Sánchez. Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms. IEEE Transactions on Fuzzy Systems, 12(3):296–308, 2004.CrossRefGoogle Scholar
  15. 15.
    J. Demtilde{s}ar. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 7:1–30, 2006.Google Scholar
  16. 16.
    P.W. Dixon, D.W. Corne, and M.J. Oates. A Ruleset Reduction Algorithm for the XCSI Learning Classifier System, volume 2661/2003 of Lecture Notes in Computer Science, pages 20–29. Springer, Berlin Heidelberg New York, 2004.Google Scholar
  17. 17.
    O.J. Dunn. Multiple Comparisons among Means. Journal of the American Statistical Association, 56:52–64, 1961.MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    R.A. Fisher. Statistical Methods and Scientific Inference, 2nd edn. Hafner Publishing Company, New York, 1959.Google Scholar
  19. 19.
    E. Frank and I.H. Witten. Generating accurate rule sets without global optimization. In Proceedings of the 15th International Conference on Machine Learning, pages 144–151. Morgan Kaufmann, San Francisco, 1998.Google Scholar
  20. 20.
    A. Freitas. Data Mining and Knowledge Discovery with Evolutionary Algorithms. Springer, Berlin Heidelberg New York, 2002.MATHGoogle Scholar
  21. 21.
    Y. Freund and R.E. Schapire. Experiments with a New Boosting Algorithm. In International Conference on Machine Learning, pages 148–156, 1996.Google Scholar
  22. 22.
    J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 32(2):337–374, 2000.CrossRefMathSciNetGoogle Scholar
  23. 23.
    M. Friedman. The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance. Journal of the American Statistical Association, 32:675–701, 1937.CrossRefGoogle Scholar
  24. 24.
    M. Friedman. A Comparison of Alternative Tests of Significance for the Problem of m Rankings. Annals of Mathematical Statistics, 11:86–92, 1940.MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    C. Fu and L. Davis. A modified classifier system compaction algorithm. In GECCO’02: Proceedings of the Genetic and Evolutionary Computation Conference, pages 920–925. Morgan Kaufmann, San Francisco, 2002.Google Scholar
  26. 26.
    D.E. Goldberg. Genetic Algorithms in Search, Optimization & Machine Learning, 1st edn. Addison Wesley, Reading, 1989.MATHGoogle Scholar
  27. 27.
    D.E. Goldberg. The Design of Innovation: Lessons from and for Competent Genetic Algorithms, 1st edn. Kluwer, Boston, 2002.MATHGoogle Scholar
  28. 28.
    A. Gónzalez and R. Pérez. Completeness and Consistency Conditions for Learning Fuzzy Rules. Fuzzy Sets and Systems, 96:37–51, 1998.CrossRefMathSciNetGoogle Scholar
  29. 29.
    A. Gónzalez and R. Pérez. SLAVE: A Genetic Learning System Based on an Iterative Approach. IEEE Transactions on Fuzzy Systems, 7(2):176–191, 1999.CrossRefGoogle Scholar
  30. 30.
    J.H. Holland. Adaptation in Natural and Artificial Systems. The University of Michigan Press, Michigan, 1975.Google Scholar
  31. 31.
    J.H Holland. Escaping Brittleness: The possibilities of General-Purpose Learning Algorithms Applied to Parallel Rule-Based Systems. In Michalski Mitchell and Carbonell, editors, Machine Learning, an artificial intelligence approach, volume II of Lecture Notes in Artificial Intelligence, pages 593–623. Morgan Kaufmann, San Francisco, 1986.Google Scholar
  32. 32.
    C.Z. Janikow. A Knowledge-Intensive Genetic Algorithm for Supervised Learning. Machine Learning, 13(2–3):189–228, 1993.CrossRefGoogle Scholar
  33. 33.
    G.H. John and P. Langley. Estimating Continuous Distributions in Bayesian Classifiers. In 11th Conference on Uncertainty in Artificial Intelligence, pages 338–345. Morgan Kaufmann, San Francisco, 1995.Google Scholar
  34. 34.
    Z. Liu, A. Liu, C. Wang, and Z. Niu. Evolving neural network using real coded genetic algorithm (GA) for multispectral image classification. Future Generation Computer Systems, 20(7):1119–1129, 2004.CrossRefGoogle Scholar
  35. 35.
    A. Orriols-Puig and E. Bernadó-Mansilla. A Further Look at UCS Classifier System. In GECCO’06: Genetic and Evolutionary Computation Conference Workshop Program, ACM Press, Seattle, 08–12 July 2006.Google Scholar
  36. 36.
    J. Otero and L. Sánchez. Induction of descriptive fuzzy classifiers with the logitboost algorithm. Soft Computing, 10(9):825–835, 2006.CrossRefGoogle Scholar
  37. 37.
    M. Pelikan. Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithms, volume 170 of Studies in Computational Intelligence. Springer, Berlin Heidelberg New York, 2005.MATHGoogle Scholar
  38. 38.
    M. Pelikan, K. Sastry, and E. Cantú-Paz. Scalable Optimization via Probabilistic Modeling, volume 33 of Studies in Computational Intelligence. Springer, Berlin Heidelberg New York, 2006.MATHGoogle Scholar
  39. 39.
    J. Platt. Fast Training of Support Vector Machines using Sequential Minimal Opt. In Advances in Kernel Methods - Support Vector Lear. MIT Press, 1998.Google Scholar
  40. 40.
    J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, California, 1995.Google Scholar
  41. 41.
    J. Rissanen. Modeling by shortest data description. Automatica, vol. 14:465–471, 1978.Google Scholar
  42. 42.
    R.E. Schapire and Y. Singer. Improved Boosting Algorithms using Confidence-Rated Predictions. Machine Learning, 37(3):297–336, 1999.MATHCrossRefGoogle Scholar
  43. 43.
    D.J. Sheskin. Handbook of Parametric and Nonparametric Statistical Procedures. Chapman & Hall, Boca Raton, 2000.MATHGoogle Scholar
  44. 44.
    R.S. Sutton and A.G. Barto. Reinforcement learning: An introduction. MIT, Cambridge, 1998.Google Scholar
  45. 45.
    T.G. Dietterich. Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms. Neural Computation, 10(7):1895–1924, 1998.CrossRefGoogle Scholar
  46. 46.
    V. Vapnik. The Nature of Statistical Learning Theory. Springer, Berlin Heidelberg New York, 1995.MATHGoogle Scholar
  47. 47.
    G. Venturini. SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attributes Based Concepts. In P. B. Brazdil, editor, Machine Learning: ECML-93 - Proceedings of the European Conference on Machine Learning, pages 280–296. Springer, Berlin Heidelberg New York, 1993.Google Scholar
  48. 48.
    D. Wierstra, F.J. Gómez, and J. Schmidhuber. Modeling Systems with Internal State Using Evolino. In GECCO’05: Proceedings of the 2005 conference on Genetic and evolutionary computation, pages 1795–1802. ACM Press, New York, 2005.Google Scholar
  49. 49.
    F. Wilcoxon. Individual Comparisons by Ranking Methods. Biometrics, 1:80–83, 1945.CrossRefGoogle Scholar
  50. 50.
    S.W. Wilson. Classifier Fitness Based on Accuracy. Evolutionary Computation, 3(2):149–175, 1995.CrossRefGoogle Scholar
  51. 51.
    S.W. Wilson. Generalization in the XCS Classifier System. In 3rd Annual Conference on Genetic Programming, pages 665–674. Morgan Kaufmann, San Francisco, 1998.Google Scholar
  52. 52.
    S.W. Wilson. Compact Rulesets from XCSI. In Advances in Learning Classifier Systems, 4th International Workshop, volume 2321 of Lecture Notes in Artificial Intelligence, pages 197–210. Springer, Berlin Heidelberg New York, 2002.Google Scholar
  53. 53.
    I.H Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco, 2005.MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Albert Orriols-Puig
    • 1
  • Jorge Casillas
    • 2
  • Ester Bernadó-Mansilla
    • 1
  1. 1.Enginyeria i Arquitectura La SalleUniversitat Ramon LlullBarcelonaSpain
  2. 2.Department of Computer Science and Artificial IntelligenceUniversity of GranadaGranadaSpain

Personalised recommendations