Advertisement

Evolutionary Algorithm for Decision Tree Induction

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8838)

Abstract

Decision trees are among the most popular classification algorithms due to their knowledge representation in form of decision rules which are easy for interpretation and analysis. Nonetheless, a majority of decision trees training algorithms base on greedy top-down induction strategy which has the tendency to develop too complex tree structures. Therefore, they are not able to effectively generalise knowledge gathered in learning set. In this paper we propose EVO-Tree hybrid algorithm for decision tree induction. EVO-Tree utilizes evolutionary algorithm based training procedure which processes population of possible tree structures decoded in the form of tree-like chromosomes. Training process aims at minimizing objective functions with two components: misclassification rate and tree size. We test the predictive performance of EVO-Tree using several public UCI data sets, and we compare the results with various state-of-the-art classification algorithms.

Keywords

classification decision tree induction evolutionary algorithms 

References

  1. 1.
    Quinlan, J.: Learning efficient classification procedures and their application to chess end games. Mach. Learn (1983)Google Scholar
  2. 2.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)Google Scholar
  3. 3.
    Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth (1984)Google Scholar
  4. 4.
    Hyafil, L., Rivest, R.L.: Constructing optimal binary decision trees is NP-complete. Inf. Process. Lett. 5, 15–17 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Papagelis, A., Kalles, D.: GATree: Genetically Evolved Decision Trees. In: IEEE Int. Conf. Tools with Artif. Intell., pp. 203–206 (2000)Google Scholar
  6. 6.
    Bramer, M.: Principles of Data Mining. Springer (2007)Google Scholar
  7. 7.
    Rokach, L., Maimon, O.: Data mining with decision trees: theory and applications. World Scientific Publishing Company (2008)Google Scholar
  8. 8.
    Aitkenhead, M.J.: A co-evolving decision tree classification method. Expert Syst. Appl. 34, 18–25 (2008)CrossRefGoogle Scholar
  9. 9.
    Kennedy, H.C., Chinniah, C., Bradbeer, P., Morss, L.: The contruction and evaluation of decision trees: A comparison of evolutionary and concept learning methods. In: Corne, D.W. (ed.) AISB-WS 1997. LNCS, vol. 1305, pp. 147–161. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  10. 10.
    Bandar, Z., Al-Attar, H., McLean, D.: Genetic algorithm based multiple decision tree induction. In: ICONIP 1999. ANZIIS 1999 & ANNES 1999 & ACNN 1999. 6th International Conference on Neural Information Processing. Proceedings (Cat. No.99EX378), pp. 429–434. IEEE (1999)Google Scholar
  11. 11.
    Dumitrescu, D., András, J.: Generalized Decision Trees Built With Evolutionary Techniques. Stud. Informatics Control 14, 15 (2005)zbMATHGoogle Scholar
  12. 12.
    Czajkowski, M., Kretowski, M.: Global induction of oblique model trees: An evolutionary approach. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS (LNAI), vol. 7895, pp. 1–11. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  13. 13.
    Chen, J., Wang, X., Zhai, J.: Pruning Decision Tree Using Genetic Algorithms. In: 2009 International Conference on Artificial Intelligence and Computational Intelligence, pp. 244–248. IEEE (2009)Google Scholar
  14. 14.
    Lomax, S., Vadera, S.: A survey of cost-sensitive decision tree induction algorithms. ACM Comput. Surv. 45, 1–35 (2013)CrossRefzbMATHGoogle Scholar
  15. 15.
    Bratu, C.V., Savin, C., Potolea, R.: A Hybrid Algorithm for Medical Diagnosis. In: EUROCON 2007 - The International Conference on Computer as a Tool, pp. 668–673. IEEE (2007)Google Scholar
  16. 16.
    Basgalupp, M.P., Carvalho, A., Barros, R.C., Freitas, A.: A Survey of Evolutionary Algorithms for Decision-Tree Induction. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 1–10 (2011)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2014

Authors and Affiliations

  1. 1.Department of Systems and Computer NetworksWroclaw University of TechnologyWrocławPoland
  2. 2.IT4InnovationsVSB – Technical University of OstravaOstrava - PorubaCzech Republic

Personalised recommendations