Encoding Ordinal Features into Binary Features for Text Classification
We propose a method by means of which supervised learning algorithms that only accept binary input can be extended to use ordinal (i.e., integer-valued) input. This is much needed in text classification, since it becomes thus possible to endow these learning devices with term frequency information, rather than just information on the presence/absence of the term in the document. We test two different learners based on “boosting”, and show that the use of our method allows them to obtain effectiveness gains. We also show that one of these boosting methods, once endowed with the representations generated by our method, outperforms an SVM learner with tfidf-weighted input.
Unable to display preview. Download preview PDF.
- 2.McCallum, A.K., Nigam, K.: A comparison of event models for naive Bayes text classification. In: Proceedings of the AAAI Workshop on Learning for Text Categorization, Madison, US, pp. 41–48 (1998)Google Scholar
- 4.Joachims, T.: Making large-scale SVM learning practical. In: Schölkopf, B., Burges, C.J., Smola, A.J. (eds.) Advances in Kernel Methods – Support Vector Learning, pp. 169–184. The MIT Press, Cambridge (1999)Google Scholar
- 5.Dougherty, J., Kohavi, R., Sahami, M.: Supervised and unsupervised discretization of continuous features. In: Proceeding of the 12th International Conference on Machine Learning (ICML 1995), Lake Tahoe, US, pp. 194–202 (1995)Google Scholar
- 6.Kotsiantis, S., Kanellopoulos, D.: Discretization techniques: A recent survey. GESTS International Transactions on Computer Science and Engineering 32(1), 47–58 (2006)Google Scholar