A Simple Approach to Ordinal Classification
Machine learning methods for classification problems commonly assume that the class values are unordered. However, in many practical applications the class values do exhibit a natural order—for example, when learning how to grade. The standard approach to ordinal classification converts the class value into a numeric quantity and applies a regression learner to the transformed data, translating the output back into a discrete class value in a post-processing step. A disadvantage of this method is that it can only be applied in conjunction with a regression scheme.
In this paper we present a simple method that enables standard classification algorithms to make use of ordering information in class attributes. By applying it in conjunction with a decision tree learner we show that it outperforms the naive approach, which treats the class values as an unordered set. Compared to special-purpose algorithms for ordinal classification our method has the advantage that it can be applied without any modification to the underlying learning scheme.
KeywordsClass Attribute Ordinal Classification Binary Attribute Ratio Quantity Decision Tree Learner
- 2.E. Frank and I. H. Witten. Making better use of global discretization. In Proceedings of the Sixteenth International Conference on Machine Learning, Bled, Slovenia, 1999. Morgan Kaufmann.Google Scholar
- 3.R. Herbrich, T. Graepel, and K. Obermayer. Regression models for ordinal data: A machine learning approach. Technical report, TU Berlin, 1999.Google Scholar
- 4.R. Kohavi. Wrappers for Performance Enhancement and Oblivious Decision Graphs. PhD thesis, Stanford University, Department of Computer Science, 1995.Google Scholar
- 5.S. Kramer, G. Widmer, B. Pfahringer, and M. DeGroeve. Prediction of ordinal classes using regression trees. Fundamenta Informaticae, 2001.Google Scholar
- 7.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco, 1993.Google Scholar
- 8.L. Torgo. Regression Data Sets. University of Porto, Faculty of Economics, Porto, Portugal, 2001. [http://www.ncc.up.pt/~ltorgo/Regression/DataSets.html].Google Scholar
- 9.I. H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco, 2000.Google Scholar