Abstract
Most of classification learning methods aim at the reduction of the number of errors. However, in many real-life applications it is misclassification cost, which should be minimized. In the paper we propose a new method for cost-sensitive learning of decision rules from datasets. Our approach consists in modifying the existing system EDRL-MD (Evolutionary Decision Rule Learner with Multivariate Discretization). EDRL-MD learns decision rules using an evolutionary algorithm (EA). We propose a new fitness function, which allows the algorithm to minimize misclassification cost rather than the number of classification errors. The remaining components of EA i.e., the representation of solutions and the genetic search operators are not changed. The performance of our method is compared to that of C5.0 learning system. The results show, that the modified EDRL-MD is able to effectively process datasets with non-equal error costs.
Chapter PDF
Similar content being viewed by others
References
Blake, C., Keogh, E., Merz, C.J.: UCI repository of machine learning databases, [http://www.ics.uci.edu/~mlearn/MLRepository.html]. Irvine, CA: University of California, Dept. of Computer Science (1998).
Bradford, J.P., Kunz, C., Kohavi, R., Brunk, C., Brodley, C.E.: Pruning decision trees with misclassification costs. In Proc. of the Tenth European Conf. on Machine Learning. Springer Verlag (1998) 131–136.
Breiman, L., Friedman, R.A., Olshen, R., Stone, C.J.: Classification and Regression Trees. Wadsworth (1984).
Bruha, I., Quality of decision rules: Definitions and classification schemes for multiple rules. In: Nakhaeizadeh, G., Taylor, C.C., (eds.) Machine Learning and Statistics. The Interface. Wiley-Interscience (1997) 107–131.
De Jong, K., Spears, W.M., Gordon, D.F.: Using genetic algorithm for concept learning. Machine Learning 13 (1993) 168–182.
Domingos, P.: MetaCost: A general method for making classifiers cost-sensitive. In Proc. of Int. Conf. on Knowledge Discovery and Data Mining, KDD’99. ACM Press (1999) 155–164.
Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In Proc. of IJCAI’93. Morgan Kaufmann (1993) 1022–1027.
Fawcett, T., Provost, F.J., Adaptive fraud detection, Data Mining and Knowledge Discovery 1 (1997)
Giordana, A., Neri, F.: Search-intensive concept induction. Evolutionary Computation 3(4) (1995) 375–416.
Janikow, C.: A knowledge intensive genetic algorithm for supervised learning. Machine Learning 13 (1993) 192–228.
Knoll, U., Nakhaeizadeh, G., Tausend, B.: Cost-sensitive pruning of decision trees. In Proc. of the 8th European Conf. on Machine Learning. Springer LNCS 784 (1994) 383–386.
Kwedlo, W., Kretowski, M.: An evolutionary algorithm using multivariate discretization for decision rule induction. In Principles of Data Mining and Knowledge Discovery. 3rd European Conference PKDD’99. Springer LNCS 1704 (1999) 392–397.
Kwedlo, W., Kretowski, M.: Discovery of decision rules from databases: an evolutionary approach. In Principles of Data Mining and Knowledge Discovery. 2nd European Symposium PKDD’98. Springer LNCS 1510 (1998) 370–378.
Margineantu, D.D., Dietterich, T.G.: Bootstrap methods for the cost-sensitive evaluation of classifiers, In Proc. 17th Int. Conf. on Machine Learning ICML’2000, Morgan Kaufmann (2000) 583–590.
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. 3rd edn. Springer (1996).
Michie, D., Spiegelhalter, D.J., Taylor, C.C.: Machine Learning, Neural and Statistical Classification. Ellis Horwood Ltd. (1994) [also available at http://www.amsta.leeds.ac.uk/~charles/statlog/index.html].
Pazzani, M., Merz, C., Murphy, P., Ali, K., Hume, T., Brunk, C.: Reducing misclassification costs. In Proc. of Int. Conf. on Machine Learning, ICML’94. Morgan Kaufmann (1994) 217–225.
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993).
Ting, K.M.: Inducing cost-sensitive trees via instance weighting. In Principles of Data Mining and Knowledge Discovery. 2nd European Symposium PKDD’98. Springer LNCS 1510 (1998) 139–147.
Turney, P.: Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research 2 (1995) 369–409.
Turney, P.: Types of cost in inductive concept learning. In Proc. of ICML’2000 Workshop on Cost-Sensitive Learning. Stanford, CA (2000).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kwedlo, W., Krętowski, M. (2001). An Evolutionary Algorithm for Cost-Sensitive Decision Rule Learning. In: De Raedt, L., Flach, P. (eds) Machine Learning: ECML 2001. ECML 2001. Lecture Notes in Computer Science(), vol 2167. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44795-4_25
Download citation
DOI: https://doi.org/10.1007/3-540-44795-4_25
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42536-6
Online ISBN: 978-3-540-44795-5
eBook Packages: Springer Book Archive