Discrimination-Based Feature Selection for Multinomial Naïve Bayes Text Classification
In this paper we focus on the problem of class discrimination issues to improve performance of text classification, and study a discrimination-based feature selection technique in which the features are selected based on the criterion of enlarging separation among competing classes, referred to as discrimination capability. The proposed approach discards features with small discrimination capability measured by Gaussian divergence, so as to enhance the robustness and the discrimination power of the text classification system. To evaluation its performance, some comparison experiments of multinomial naïve Bayes classifier model are constructed on Newsgroup and Ruters21578 data collection. Experimental results show that on Newsgroup data set divergence measure outperforms MI measure, and has slight better performance than DF measure, and outperforms both measures on Ruters21578 data set. It shows that discrimination-based feature selection method has good contributions to enhance discrimination power of text classification model.
KeywordsFeature Selection Mutual Information Feature Subset Feature Selection Method Text Classification
Unable to display preview. Download preview PDF.
- 1.Lewis, D., Schapire, R., Callan, J., Papka, R.: Training Algorithms for Linear Text Classifiers. In: Proceedings of ACM SIGIR, pp. 298–306 (1996)Google Scholar
- 3.Lewis, D.: A Comparison of Two Learning Algorithms for Text Categorization. In: Symposium on Document Analysis and IR (1994)Google Scholar
- 4.Nigam, K., Lafferty, J., McCallum, A.: Using maximum entropy for text classification. In: IJCAI 1999 Workshop on Machine Learning for Information Filtering, pp. 61–67 (1999)Google Scholar
- 5.McCallum, Nigam, K.: A comparison of event models for naive bayes text classification. In: AAAI 1998 Workshop on Learning for Text Categorization (1998)Google Scholar
- 6.Yiming, Y., Pedersen, J.O.: A comparative study on feature selection in text categorization. In: 14th international conference on machine learning, pp. 412–420 (1997)Google Scholar
- 9.Tol, J.T., Gonzalez, R.C.: Pattern recognition Principles. Addison-Wesley publishing company, Reading (1974)Google Scholar
- 11.Schneider, K.-M.: A new feature selection score for multinomial naïve Bayes text classification based on KL-divergence. In: 42nd Annual meeting of the association for computational linguistics (2004)Google Scholar