Evolutionary Induction of Cost-Sensitive Decision Trees

  • Marek Krętowski
  • Marek Grześ
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4203)


In the paper, a new method for cost-sensitive learning of decision trees is proposed. Our approach consists in extending the existing evolutionary algorithm (EA) for global induction of decision trees. In contrast to the classical top-down methods, our system searches for the whole tree at the moment. We propose a new fitness function which allows the algorithm to minimize expected cost of classification defined as a sum of misclassification cost and cost of the tests. The remaining components of EA i.e. the representation of solutions and the specialized genetic search operators are not changed. The proposed method is experimentally validated and preliminary results show that the global approach is able to effectively induce cost-sensitive decision trees.


Decision Tree Evolutionary Algorithm Class Label Internal Node Cost Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bradford, J., Kunz, C., Kohavi, R., Brunk, C., Brodley, C.E.: Pruning decision trees with misclassification costs. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 131–136. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  2. 2.
    Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth Int. Group (1984)Google Scholar
  3. 3.
    Domingos, P.: MetaCost: A general method for making classifiers cost-sensitive. In: Proc. of KDD 1999, pp. 155–164. ACM Press, New York (1999)CrossRefGoogle Scholar
  4. 4.
    Knoll, U., Nakhaeizadeh, G., Tausend, B.: Cost-sensitive pruning of decision trees. In: Bergadano, F., De Raedt, L. (eds.) ECML 1994. LNCS, vol. 784, pp. 383–386. Springer, Heidelberg (1994)Google Scholar
  5. 5.
    Krȩtowski, M.: An evolutionary algorithm for oblique decision tree induction. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 432–437. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Krȩtowski, M., Grześ, M.: Global learning of decision trees by an evolutionary algorithm. In: Information Processing and Security Sys., pp. 401–410. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Ling, C., Yang, Q., Wang, J., Zhang, S.: Decision trees with minimal costs. In: Proc. of ICML 2004, Article No. 69, ACM Press, New York (2004)Google Scholar
  8. 8.
    Margineantu, D., Dietterich, T.: Bootstrap methods for the cost-sensitive evaluation of classifiers. In: Proc. of ICML 2000, pp. 583–590. Morgan Kaufmann, San Francisco (2000)Google Scholar
  9. 9.
    Nunez, M.: The use of background knowledge in decision tree induction. Machine Learning 6, 231–250 (1991)Google Scholar
  10. 10.
    Ting, K.: An instance-weighting method to induce cost-sensitive trees. IEEE TKDE 14(3), 659–665 (2002)Google Scholar
  11. 11.
    Turney, P.: Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. J. of Artif. Intel. Res. 2, 369–409 (1995)Google Scholar
  12. 12.
    Turney, P.: Types of cost in inductive concept learning. In: Proc. of ICML 2000 Workshop on Cost-Sensitive Learning, Stanford, CA (2000)Google Scholar
  13. 13.
    Zadrozny, B., Langford, J., Abe, N.: Cost-sensitive learning by cost-proportionate example weighting concept learning. In: Proc. of ICDM 2003, IEEE Press, Los Alamitos (2003)Google Scholar
  14. 14.
    Zhang, S., Qin, Z., Ling, C., Sheng, S.: Missing is usefull: Missing values in cost-sensitive decision trees. IEEE TKDE 17(12), 1689–1693 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Marek Krętowski
    • 1
  • Marek Grześ
    • 1
  1. 1.Faculty of Computer ScienceBiałystok Technical UniversityBiałystokPoland

Personalised recommendations