Mixed Decision Trees: An Evolutionary Approach

  • Marek Krȩtowski
  • Marek Grześ
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4081)


In the paper, a new evolutionary algorithm (EA) for mixed tree learning is proposed. In non-terminal nodes of a mixed decision tree different types of tests can be placed, ranging from a typical univariate inequality test up to a multivariate test based on a splitting hyperplane. In contrast to classical top-down methods, our system searches for an optimal tree in a global manner, i.e. it learns a tree structure and tests in one run of the EA. Specialized genetic operators allow for generating new sub-trees, pruning existing ones as well as changing the node type and the tests. The proposed approach was experimentally verified on both artificial and real-life data and preliminary results are promising.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blake, C., Keogh, E., Merz, C.: UCI repository of machine learning databases. University of California, Dept. of Computer Science, Irvine, CA (1998)Google Scholar
  2. 2.
    Bot, M., Langdon, W.: Application of genetic programming to induction of linear classification trees. In: Poli, R., Banzhaf, W., Langdon, W.B., Miller, J., Nordin, P., Fogarty, T.C. (eds.) EuroGP 2000. LNCS, vol. 1802, pp. 247–258. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  3. 3.
    Breiman, L., Friedman, J., Olshen, R., Stone C.: Classification and Regression Trees. Wadsworth Int. Group (1984).Google Scholar
  4. 4.
    Brodley, C.: Recursive automatic bias selection for classifier construction. Machine Learning 20, 63–94 (1995)Google Scholar
  5. 5.
    Cantu-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Transactions on Evolutionary Computation 7(1), 54–68 (2003)CrossRefGoogle Scholar
  6. 6.
    Chai, B., Huang, T., Zhuang, X., Zhao, Y., Sklansky, J.: Piecewise-linear classifiers using binary tree structure and genetic algorithm. Pattern Recognition 29(11), 1905–1917 (1996)CrossRefGoogle Scholar
  7. 7.
    Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)MathSciNetGoogle Scholar
  8. 8.
    Esposito, F., Malerba, D., Semeraro, G.: A comparative analysis of methods for pruning decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(5), 476–491 (1997)CrossRefGoogle Scholar
  9. 9.
    Freitas, A.: Data Mining and Knowledge Discovery with Evolutionary Algorithms. Springer, Heidelberg (2002)zbMATHGoogle Scholar
  10. 10.
    Koza, J.: Concept formation and decision tree induction using genetic programming paradigm. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 124–128. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  11. 11.
    Krȩtowski, M.: An evolutionary algorithm for oblique decision tree induction. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 432–437. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Krȩtowski, M., Grześ, M.: Global learning of decision trees by an evolutionary algorithm. In: Information Processing and Security Sys., pp. 401–410. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Krȩtowski, M., Grześ, M.: Global induction of oblique decision trees: an evolutionary approach. In: Proc. of IIPWM 2005, pp. 309–318. Springer, Heidelberg (2005)Google Scholar
  14. 14.
    Krȩtowski, M., Grześ, M.: Evolutionary learning of linear trees with embedded feature selection. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, Springer, Heidelberg (2006)Google Scholar
  15. 15.
    Llora, X., Wilson, S.: Mixed decision trees: Minimizing knowledge representation bias in LCS. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 797–809. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer, Heidelberg (1996)zbMATHGoogle Scholar
  17. 17.
    Murthy, S., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. Journal of Artificial Intelligence Research 2, 1–33 (1994)zbMATHGoogle Scholar
  18. 18.
    Murthy, S.: Automatic construction of decision trees from data: A multi-disciplinary survey. Data Mining and Knowledge Discovery 2, 345–389 (1998)CrossRefMathSciNetGoogle Scholar
  19. 19.
    Nikolaev, N., Slavov, V.: Inductive genetic programming with decision trees. Intelligent Data Analysis 2, 31–44 (1998)CrossRefGoogle Scholar
  20. 20.
    Papagelis, A., Kalles, D.: Breeding decision trees using evolutionary techniques. In: Proc. of ICML 2001, pp. 393–400. Morgan Kaufmann, San Francisco (2001)Google Scholar
  21. 21.
    Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Marek Krȩtowski
    • 1
  • Marek Grześ
    • 1
  1. 1.Faculty of Computer ScienceBiałystok Technical UniversityBiałystokPoland

Personalised recommendations