Simple Test Strategies for Cost-Sensitive Decision Trees

  • Shengli Sheng
  • Charles X. Ling
  • Qiang Yang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3720)


We study cost-sensitive learning of decision trees that incorporate both test costs and misclassification costs. In particular, we first propose a lazy decision tree learning that minimizes the total cost of tests and misclassifications. Then assuming test examples may contain unknown attributes whose values can be obtained at a cost (the test cost), we design several novel test strategies which attempt to minimize the total cost of tests and misclassifications for each test example. We empirically evaluate our tree-building and various test strategies, and show that they are very effective. Our results can be readily applied to real-world diagnosis tasks, such as medical diagnosis where doctors must try to determine what tests (e.g., blood tests) should be ordered for a patient to minimize the total cost of tests and misclassifications (misdiagnosis). A case study on heart disease is given throughout the paper.


Decision Tree Test Strategy Test Cost Single Batch Misclassification Cost 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases (website). University of California, Irvine (1998)Google Scholar
  2. 2.
    Chai, X., Deng, L., Yang, Q., Ling, C.X.: Test-Cost Sensitive Naïve Bayesian Classification. In: Proceedings of the Fourth IEEE International Conference on Data Mining. IEEE Computer Society Press, Brighton (2004)Google Scholar
  3. 3.
    Special Issue on Learning from Imbalanced Datasets. In: Chawla, N.V., Japkowicz, N., Kolcz, A. (eds.) SIGKDD, vol. 6(1), ACM Press, New York (2004)Google Scholar
  4. 4.
    Domingos, P.: MetaCost: A General Method for Making Classifiers Cost-Sensitive. In: Proceedings of the Fifth International Conference on Knowledge Discovery and Data Mining, pp. 155–164. ACM Press, San Diego (1999)CrossRefGoogle Scholar
  5. 5.
    Elkan, C.: The Foundations of Cost-Sensitive Learning. In: Proceedings of the Seventeenth International Joint Conference of Artificial Intelligence, pp. 973–978. Morgan Kaufmann, Seattle (2001)Google Scholar
  6. 6.
    Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the 13th International Joint Conference on Artificial Intelligence, pp. 1022–1027. Morgan Kaufmann, France (1993)Google Scholar
  7. 7.
    Ting, K.M.: Inducing Cost-Sensitive Trees via Instance Weighting. In: Żytkow, J.M. (ed.) PKDD 1998. LNCS, vol. 1510, pp. 23–26. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  8. 8.
    Ling, C.X., Yang, Q., Wang, J., Zhang, S.: Decision Trees with Minimal Costs. In: Proceedings of the Twenty-First International Conference on Machine Learning. Morgan Kaufmann, Banff (2004)Google Scholar
  9. 9.
    Lizotte, D., Madani, O., Greiner, R.: Budgeted Learning of Naïve-Bayes Classifiers. In: Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann, Acapulco (2003)Google Scholar
  10. 10.
    Quinlan, J.R. (ed.): C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  11. 11.
    Tan, M.: Cost-sensitive learning of classification knowledge and its applications in robotics. Machine Learning Journal 13, 7–33 (1993)Google Scholar
  12. 12.
    Turney, P.D.: Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm. Journal of Artificial Intelligence Research 2, 369–409 (1995)Google Scholar
  13. 13.
    Turney, P.D.: Types of cost in inductive concept learning. In: Proceedings of the Workshop on Cost-Sensitive Learning at the Seventeenth International Conference on Machine Learning, Stanford University, California (2000)Google Scholar
  14. 14.
    Zubek, V.B., Dietterich, T.: Pruning improves heuristic search for cost-sensitive learning. In: Proceedings of the Nineteenth International Conference of Machine Learning, pp. 27–35. Morgan Kaufmann, Sydney (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Shengli Sheng
    • 1
  • Charles X. Ling
    • 1
  • Qiang Yang
    • 2
  1. 1.Department of Computer ScienceThe University of Western OntarioLondonCanada
  2. 2.Department of Computer ScienceHong Kong USTHong Kong

Personalised recommendations