ECML 2006: Machine Learning: ECML 2006 pp 341-352 | Cite as

Automatically Evolving Rule Induction Algorithms

  • Gisele L. Pappa
  • Alex A. Freitas
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4212)

Abstract

Research in the rule induction algorithm field produced many algorithms in the last 30 years. However, these algorithms are usually obtained from a few basic rule induction algorithms that have been often changed to produce better ones. Having these basic algorithms and their components in mind, this work proposes the use of Grammar-based Genetic Programming (GGP) to automatically evolve rule induction algorithms. The proposed GGP is evaluated in extensive computational experiments involving 11 data sets. Overall, the results show that effective rule induction algorithms can be automatically generated using GGP. The automatically evolved rule induction algorithms were shown to be competitive with well-known manually designed ones. The proposed approach of automatically evolving rule induction algorithms can be considered a pioneering one, opening a new kind of research area.

Download to read the full conference paper text

References

  1. 1.
    Clark, P., Boswell, R.: Rule induction with cn2: some recent improvements. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 151–163. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  2. 2.
    Cohen, W.W.: Fast effective rule induction. In: Proc. of the 12th International Conference on Machine Learning (1995)Google Scholar
  3. 3.
    Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)MATHGoogle Scholar
  4. 4.
    Furnkranz, J., Widmer, G.: Incremental reduced error pruning. In: Proc. the 11th Int. Conf. on Machine Learning, New Brunswick, NJ, pp. 70–77 (1994)Google Scholar
  5. 5.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by the means of natural selection. The MIT Press, Massachusetts (1992)MATHGoogle Scholar
  6. 6.
  7. 7.
    Whigham, P.A.: Grammatically-based genetic programming. In: Proc. of the Workshop on GP: From Theory to Real-World Applications (1995)Google Scholar
  8. 8.
    Domingos, P.: Rule induction and instance-based learning: A unified approach. In: Proc. of the 14th International Joint Conference on Artificial Intelligence, pp. 1226–1232 (1995)Google Scholar
  9. 9.
    Lim, T., Loh, W., Shih, Y.: A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms. Machine Learning 40(3), 203–228 (2000)MATHCrossRefGoogle Scholar
  10. 10.
    Pappa, G.L., Freitas, A.A.: Towards a genetic programming algorithm for automatically evolving rule induction algorithms. In: Furnkranz, J. (ed.) Proc. ECML/PKDD-2004 Workshop on Advances in Inductive Learning, pp. 93–108 (2004)Google Scholar
  11. 11.
    Mitchell, T.: Machine Learning. Mc Graw Hill, New York (1997)MATHGoogle Scholar
  12. 12.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (1999)Google Scholar
  13. 13.
    Freitas, A.A.: Data Mining and Knowledge Discovery with Evolutionary Algorithms. Springer, Heidelberg (2002)MATHGoogle Scholar
  14. 14.
    Wong, M.L., Leung, K.S.: Data Mining Using Grammar-Based Genetic Programming and Applications. Kluwer, Dordrecht (2000)MATHGoogle Scholar
  15. 15.
    Furnkranz, J.: Separate-and-conquer rule learning. Artificial Intelligence Review 13(1), 3–54 (1999)CrossRefGoogle Scholar
  16. 16.
    Banzhaf, W., Nordin, P., Keller, R., Francone, F.: GP – An Introduction; On the Automatic Evolution of Computer Programs and its Applications. Morgan Kaufmann, San Francisco (1998)Google Scholar
  17. 17.
    O’Neill, M., Ryan, C.: Grammatical Evolution Evolutionary Automatic Programming in an Arbitrary Language. Morgan Kaufmann, San Francisco (2003)MATHGoogle Scholar
  18. 18.
    Wong, M.L.: An adaptive knowledge-acquisition system using generic genetic programming. Expert Systems with Applications 15(1), 47–58 (1998)CrossRefGoogle Scholar
  19. 19.
    Suyama, A., Negishi, N., Yamaguchi, T.: CAMLET: A platform for automatic composition of inductive learning systems using ontologies. In: Pacific Rim International Conference on Artificial Intelligence, pp. 205–215 (1998)Google Scholar
  20. 20.
    Zhang, J.: Selecting typical instances in instance-based learning. In: Proc. of the 9th Int. Workshop on Machine Learning, pp. 470–479. Morgan Kaufmann, San Francisco (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Gisele L. Pappa
    • 1
  • Alex A. Freitas
    • 1
  1. 1.Computing LaboratoryUniversity of KentCanterbury, KentUK

Personalised recommendations