Discriminative Methods for Multi-labeled Classification
In this paper we present methods of enhancing existing discriminative classifiers for multi-labeled predictions. Discriminative methods like support vector machines perform very well for uni-labeled text classification tasks. Multi-labeled classification is a harder task subject to relatively less attention. In the multi-labeled setting, classes are often related to each other or part of a is-a hierarchy. We present a new technique for combining text features and features indicating relationships between classes, which can be used with any discriminative algorithm. We also present two enhancements to the margin of SVMs for building better models in the presence of overlapping classes. We present results of experiments on real world text benchmark datasets. Our new methods beat accuracy of existing methods with statistically significant improvements.
KeywordsSupport Vector Machine Document Vector Discriminative Method Label Dimension Patent Dataset
Unable to display preview. Download preview PDF.
- 1.Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, Springer, Heidelberg (1998)Google Scholar
- 4.Sarawagi, S., Chakrabarti, S., Godbole, S.: Cross training: learning probabilistic mappings between topics. In: Proceedings of the ACM SIGKDD 2003 (2003)Google Scholar
- 5.Godbole, S., Sarawagi, S., Chakrabarti, S.: Scaling multi-class support vector machines using inter-class confusion. In: Proceedings of ACM SIGKDD 2002 (2002)Google Scholar
- 6.Crammer, K., Singer, Y.: A family of additive online algorithms for category ranking. Journal of Machine Learning Research, 1025–1058 (2003)Google Scholar
- 7.Elisseeff, A., Weston, J.: Kernel methods for multi-labelled classification and categorical regression problems. Technical Report, BioWulf Technologies (2001)Google Scholar
- 8.Yu, H., Han, J., Pebl, K.C.-C.: Positive example-based learning for web page classification using SVM. In: Proceedings of ACM SIGKDD 2002 (2002)Google Scholar
- 9.McCallum, A.: Multi-label text classification with a mixture model trained by EM. In: AAAI Workshop on Text Learning 1999 (1999)Google Scholar
- 10.Hofmann, T., Puzicha, J.: Unsupervised learning from dyadic data. Technical Report TR-98-042, Berkeley (1998)Google Scholar