Abstract
In some learning settings, the cost of acquiring features for classification must be paid up front, before the classifier is evaluated. In this paper, we introduce the forensic classification problem and present a new algorithm for building decision trees that maximizes classification accuracy while minimizing total feature costs. By expressing the ID3 decision tree algorithm in an information theoretic context, we derive our algorithm from a well-formulated problem objective. We evaluate our algorithm across several datasets and show that, for a given level of accuracy, our algorithm builds cheaper trees than existing methods. Finally, we apply our algorithm to a real-world system, Clarify. Clarify classifies unknown or unexpected program errors by collecting statistics during program runtime which are then used for decision tree classification after an error has occurred. We demonstrate that if the classifier used by the Clarify system is trained with our algorithm, the computational overhead (equivalently, total feature costs) can decrease by many orders of magnitude with only a slight (<1%) reduction in classification accuracy.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bradford, J., Kunz, C., Kohavi, R., Brunk, C., Brodley, C.: Pruning decision trees with misclassification costs. In: European Conference on Machine Learning (1998)
Brun, Y., Ernst, M.D.: Finding latent code errors via machine learning over program executions. In: ICSE (2004)
Cohen, I., Zhang, S., Goldszmidt, M., Symons, J., Kelly, T., Fox, A.: Capturing, indexing, clustering, and retrieving system history. In: SOSP (2005)
Cover, T.M., Thomas, J.A.: Elements of information theory. Wiley Series in Telecommunications (1991)
Blake, C.L., Newman, D.J., Hettich, S., Merz, C.J.: UCI repository of machine learning databases (1998)
Elkan, C.: The foundations of cost-sensitive learning. In: International joint conference on artifical intelligence (2001)
Ha, J., Ramadan, H., Davis, J., Rossbach, C., Roy, I., Witchel, E.: Navel: Automating software support by classifying program behavior. Technical Report TR-06-11, University of Texas at Austin (2006)
Hangal, S., Lam, M.S.: Tracking down software bugs using automatic anomaly detection. In: ICSE (2002)
Liblit, B., Naik, M., Zheng, A.X., Aiken, A., Jordan, M.I.: Scalable statistical bug isolation. In: PLDI (2005)
Mitchell, T.: Machine Learning. In: WCB. McGraw-Hill, New York (1997)
Norton, S.W.: Generating better decision trees. In: International joint conference on artifical intelligence (1989)
Nunez, M.: The use of background knowledge in decision tree induction. Machine Learning (1991)
Quinlan, R.: C4.5: programs for machine learning. Morgan Kaufmann Publishers, San Francisco (1992)
Turney, P.: Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of artificial intelligence research (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Davis, J.V., Ha, J., Rossbach, C.J., Ramadan, H.E., Witchel, E. (2006). Cost-Sensitive Decision Tree Learning for Forensic Classification. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds) Machine Learning: ECML 2006. ECML 2006. Lecture Notes in Computer Science(), vol 4212. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11871842_60
Download citation
DOI: https://doi.org/10.1007/11871842_60
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45375-8
Online ISBN: 978-3-540-46056-5
eBook Packages: Computer ScienceComputer Science (R0)