Advertisement

Cost sensitive discretization of numeric attributes

  • Tom Brijs
  • Koen Vanhoof
Communications Session 4. Clustering and Discretization
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1510)

Abstract

Many algorithms in decision tree learning are not designed to handle numeric valued attributes very well. Therefore, discretization of the continuous feature space has to be carried out. In this article we introduce the concept of cost sensitive discretization as a preprocessing step to induction of a classifier and as an elaboration of the error-based discretization method to obtain an optimal multi-interval splitting for each numeric attribute. A transparant description of the method and steps involved in cost sensitive discretization is given. We also evaluate its performance against two other well known methods, i.e. entropy-based discretization and pure error-based discretization on a real life financial dataset. From the algoritmic point of view, we show that an important deficiency from error-based discretization methods can be solved by introducing costs. From the application point of view, we discovered that using a discretization method is recommended. To conclude, we use ROC-curves to illustrate that under particular conditions cost-based discretization may be optimal.

Keywords

Discretization Method Cost Parameter Minority Class Error Cost Misclassification Cost 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Barber C., Dobkin D., and Huhdanpaa H. (1993). The quickhull algorithm for convex hull. Technical Report GCG53, University of Minesota.Google Scholar
  2. 2.
    Catlett J. (1991). On changing continuous attributes into ordered discrete attributes. In Proceedings of the Fifth European Working Session on Learning, 164–178. Berlin: Springer-Verlag.Google Scholar
  3. 3.
    Dougherty J., Kohavi R., and Sahami M. (1995). Supervised and unsupervised discretization of continous features. In Machine Learning: Proceedings of the Twelfth Int. Conference, 194–202. Morgan Kaufmann.Google Scholar
  4. 4.
    Elomaa T., and Rousu J. (1996). Finding Optimal Multi-Splits for Numerical Attributes in Decision Tree Learning. Technical Report NC-TR-96-041, University of Helsinki.Google Scholar
  5. 5.
    Fayyad U., and Irani K. (1992). On the handling of continuous-valued attributes in decision tree generation. In Machine Learning 8. 87–102.Google Scholar
  6. 6.
    Fayyad U., and Irani K. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. In Proceedings of the Thirteenth Int. Joint Conference on Artificial Intelligence, 1022–1027. Morgan Kaufmann.Google Scholar
  7. 7.
    Fulton T., Kasif S., and Salzberg S. (1995). Efficient algorithms for finding multi-way splits for decision trees. In Proceedings of the Twelfth Int. Conference on Machine Learning, 244–251. Morgan Kaufmann.Google Scholar
  8. 8.
    Holte R. (1993). Very simple classification rules perform well on most commonly used datasets. In Machine Learning 11, 63–90.Google Scholar
  9. 9.
    Kerber R. (1992). Chimerge: Discretization of numeric attributes. In Proceedings of the Tenth Nat. Conference on Artificial Intelligence, 123–128. MIT Press.Google Scholar
  10. 10.
    Kohavi R., and Sahami M. (1996). Error-based and Entropy-Based Discretization of Continuous Features. In Proceedings of the Second Int. Conference on Knowledge & Data Mining, 114–119. AAAI Press.Google Scholar
  11. 11.
    Maas W. (1994). Efficient agnostic PAC-learning with simple hypotheses. In Proceedings of the Seventh Annual ACM Conference on Computational Learning Theory, 67–75. ACM Press.Google Scholar
  12. 12.
    Provost F., and Fawcett T. (1997). Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions. In Proceedings of the Third Int. Conference on Knowledge Discovery and Data Mining, 43–48, AAAI Press.Google Scholar
  13. 13.
    Van de Merckt T. (1993). Decision Trees in Numerical Attributes Spaces. In Proceedings of the Thirteenth Int. Joint Conference on Artificial Intelligence, 1016–1021, Morgan Kaufmann.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Tom Brijs
    • 1
  • Koen Vanhoof
    • 2
  1. 1.Faculty of Applied Economic SciencesLimburg University CentreDiepenbeekBelgium
  2. 2.Faculty of Applied Economic SciencesLimburg University CentreDiepenbeekBelgium

Personalised recommendations