Advertisement

Mixed Decision Trees: An Evolutionary Approach

  • Marek Krȩtowski
  • Marek Grześ
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4081)

Abstract

In the paper, a new evolutionary algorithm (EA) for mixed tree learning is proposed. In non-terminal nodes of a mixed decision tree different types of tests can be placed, ranging from a typical univariate inequality test up to a multivariate test based on a splitting hyperplane. In contrast to classical top-down methods, our system searches for an optimal tree in a global manner, i.e. it learns a tree structure and tests in one run of the EA. Specialized genetic operators allow for generating new sub-trees, pruning existing ones as well as changing the node type and the tests. The proposed approach was experimentally verified on both artificial and real-life data and preliminary results are promising.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Blake, C., Keogh, E., Merz, C.: UCI repository of machine learning databases. University of California, Dept. of Computer Science, Irvine, CA (1998)Google Scholar
  2. 2.
    Bot, M., Langdon, W.: Application of genetic programming to induction of linear classification trees. In: Poli, R., Banzhaf, W., Langdon, W.B., Miller, J., Nordin, P., Fogarty, T.C. (eds.) EuroGP 2000. LNCS, vol. 1802, pp. 247–258. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  3. 3.
    Breiman, L., Friedman, J., Olshen, R., Stone C.: Classification and Regression Trees. Wadsworth Int. Group (1984).Google Scholar
  4. 4.
    Brodley, C.: Recursive automatic bias selection for classifier construction. Machine Learning 20, 63–94 (1995)Google Scholar
  5. 5.
    Cantu-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Transactions on Evolutionary Computation 7(1), 54–68 (2003)CrossRefGoogle Scholar
  6. 6.
    Chai, B., Huang, T., Zhuang, X., Zhao, Y., Sklansky, J.: Piecewise-linear classifiers using binary tree structure and genetic algorithm. Pattern Recognition 29(11), 1905–1917 (1996)CrossRefGoogle Scholar
  7. 7.
    Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)MathSciNetGoogle Scholar
  8. 8.
    Esposito, F., Malerba, D., Semeraro, G.: A comparative analysis of methods for pruning decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(5), 476–491 (1997)CrossRefGoogle Scholar
  9. 9.
    Freitas, A.: Data Mining and Knowledge Discovery with Evolutionary Algorithms. Springer, Heidelberg (2002)zbMATHGoogle Scholar
  10. 10.
    Koza, J.: Concept formation and decision tree induction using genetic programming paradigm. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 124–128. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  11. 11.
    Krȩtowski, M.: An evolutionary algorithm for oblique decision tree induction. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 432–437. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Krȩtowski, M., Grześ, M.: Global learning of decision trees by an evolutionary algorithm. In: Information Processing and Security Sys., pp. 401–410. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Krȩtowski, M., Grześ, M.: Global induction of oblique decision trees: an evolutionary approach. In: Proc. of IIPWM 2005, pp. 309–318. Springer, Heidelberg (2005)Google Scholar
  14. 14.
    Krȩtowski, M., Grześ, M.: Evolutionary learning of linear trees with embedded feature selection. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, Springer, Heidelberg (2006)Google Scholar
  15. 15.
    Llora, X., Wilson, S.: Mixed decision trees: Minimizing knowledge representation bias in LCS. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 797–809. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer, Heidelberg (1996)zbMATHGoogle Scholar
  17. 17.
    Murthy, S., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. Journal of Artificial Intelligence Research 2, 1–33 (1994)zbMATHGoogle Scholar
  18. 18.
    Murthy, S.: Automatic construction of decision trees from data: A multi-disciplinary survey. Data Mining and Knowledge Discovery 2, 345–389 (1998)CrossRefMathSciNetGoogle Scholar
  19. 19.
    Nikolaev, N., Slavov, V.: Inductive genetic programming with decision trees. Intelligent Data Analysis 2, 31–44 (1998)CrossRefGoogle Scholar
  20. 20.
    Papagelis, A., Kalles, D.: Breeding decision trees using evolutionary techniques. In: Proc. of ICML 2001, pp. 393–400. Morgan Kaufmann, San Francisco (2001)Google Scholar
  21. 21.
    Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Marek Krȩtowski
    • 1
  • Marek Grześ
    • 1
  1. 1.Faculty of Computer ScienceBiałystok Technical UniversityBiałystokPoland

Personalised recommendations