Evolutionary Induction of Cost-Sensitive Decision Trees
In the paper, a new method for cost-sensitive learning of decision trees is proposed. Our approach consists in extending the existing evolutionary algorithm (EA) for global induction of decision trees. In contrast to the classical top-down methods, our system searches for the whole tree at the moment. We propose a new fitness function which allows the algorithm to minimize expected cost of classification defined as a sum of misclassification cost and cost of the tests. The remaining components of EA i.e. the representation of solutions and the specialized genetic search operators are not changed. The proposed method is experimentally validated and preliminary results show that the global approach is able to effectively induce cost-sensitive decision trees.
KeywordsDecision Tree Evolutionary Algorithm Class Label Internal Node Cost Matrix
Unable to display preview. Download preview PDF.
- 2.Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth Int. Group (1984)Google Scholar
- 4.Knoll, U., Nakhaeizadeh, G., Tausend, B.: Cost-sensitive pruning of decision trees. In: Bergadano, F., De Raedt, L. (eds.) ECML 1994. LNCS, vol. 784, pp. 383–386. Springer, Heidelberg (1994)Google Scholar
- 7.Ling, C., Yang, Q., Wang, J., Zhang, S.: Decision trees with minimal costs. In: Proc. of ICML 2004, Article No. 69, ACM Press, New York (2004)Google Scholar
- 8.Margineantu, D., Dietterich, T.: Bootstrap methods for the cost-sensitive evaluation of classifiers. In: Proc. of ICML 2000, pp. 583–590. Morgan Kaufmann, San Francisco (2000)Google Scholar
- 9.Nunez, M.: The use of background knowledge in decision tree induction. Machine Learning 6, 231–250 (1991)Google Scholar
- 10.Ting, K.: An instance-weighting method to induce cost-sensitive trees. IEEE TKDE 14(3), 659–665 (2002)Google Scholar
- 11.Turney, P.: Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. J. of Artif. Intel. Res. 2, 369–409 (1995)Google Scholar
- 12.Turney, P.: Types of cost in inductive concept learning. In: Proc. of ICML 2000 Workshop on Cost-Sensitive Learning, Stanford, CA (2000)Google Scholar
- 13.Zadrozny, B., Langford, J., Abe, N.: Cost-sensitive learning by cost-proportionate example weighting concept learning. In: Proc. of ICDM 2003, IEEE Press, Los Alamitos (2003)Google Scholar
- 14.Zhang, S., Qin, Z., Ling, C., Sheng, S.: Missing is usefull: Missing values in cost-sensitive decision trees. IEEE TKDE 17(12), 1689–1693 (2005)Google Scholar