A Study on Feature Weighting in Chinese Text Categorization

  • Xue Dejun
  • Sun Maosong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2588)


In Text Categorization (TC) based on Vector Space Model, feature weighting and feature selection are major problems and difficulties. This paper proposes two methods of weighting features by combining the relevant influential factors together. A TC system for Chinese texts is designed in terms of character bigrams as features. Experiments on a document collection of 71,674 texts show that the F1 metric of categorization performance of the system is 85.9%, which is about 5% higher than that of the well-known TF*IDF weighting scheme. Moreover, a multi-step feature selection process is exploited to reduce the dimension of the feature space effectively in the system.


Feature Vector Feature Selection Categorization Performance Information Gain Text Categorization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fabrizio Sebastiani: Machine Learning in Automated Text Categorization. ACM Computing Surveys, Vol. 34(1). ACM Press New York (2002) 1–47.CrossRefGoogle Scholar
  2. 2.
    Salton, G., McGill, M.: Introduction to Modern Information Retrieval. McGraw-Hill Book Company, New York (1983).zbMATHGoogle Scholar
  3. 3.
    Lewis, D.D.: Naïve Bayes at Forty: The Independence Assumption in Information Retrieval. In Proceedings of 10th European Conference on Machine Learning (1998) 4–15.Google Scholar
  4. 4.
    Domingos, P., Pazzani, M.: Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier. In Proceedings of 13 rd International Conference on Machine Learning (1996) 105–112.Google Scholar
  5. 5.
    McCallum, A., Nigam, K.: A Comparison of Event Models for Naïve Bayes Text Classification. In AAAI-98 Workshop on Learning for Text Categorization (1998) 41–48.Google Scholar
  6. 6.
    Wiener, E., Pedersen, J.O., Weigend, A.S.: A Neural Network Approach to Topic Spotting. In Proceedings of 4th Annual Symposium on Document Analysis and Information Retrieval (1995) 317–332.Google Scholar
  7. 7.
    Yang, Y.M.: Expert Network: Effective and Efficient Learning from Human Decisions in Text Categorization and Retrieval. In Proceedings of 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (1994) 11–21.Google Scholar
  8. 8.
    Apte, C., Damerau, F., Weiss, S.M.: Automated Learning of Decision Rules for Text Categorization. ACM Transactions on Information Retrieval, Vol. 12(3). ACM Press New York (1994) 233–251.Google Scholar
  9. 9.
    Theeramunkong T., Lertnattee V.: Improving Centroid-Based Text Classification Using Term-Distribution-Based Weighting System and Clustering. In Proceedings of International Symposium on Communications and Information Technology (2001) 33–36.Google Scholar
  10. 10.
    Joachims, T.: A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization. In Proceedings of 14th of International Conference on Machine Learning (1997) 143–151.Google Scholar
  11. 11.
    Joachims, T: Text Categorization with Support Vector Machines: Learnging with Many Relevant Features. In Proceedings of 10th European Conference on Machine Learning (1998) 137–142.Google Scholar
  12. 12.
    Quinlan, J.: Bagging, Boosting, and C4.5. In Proceedings of 13th National Conference on Artificial Intelligence, AAAI Press/ MIT Press (1996) 163–175.Google Scholar
  13. 13.
    Schapire, R.E., Singer, Y.: BoosTexter: A Boosting-Based System for Text Categorization. Machine Learning, Vol. 39(2/3), (2000) 135–168.zbMATHCrossRefGoogle Scholar
  14. 14.
    Theeramunkong, T., Lertnattee, V.: Multi-Dimensional Text Classification. In Proceedings of 19th International Conference on Computational Linguistics (2002) 1002–1008.Google Scholar
  15. 15.
    Yang Y.M., Pedersen, P.O.: A Comparative Study on Feature Selection in Text Categorization. In Proceedings of 14th International Conference on Machine Learning (1997) 412–420.Google Scholar
  16. 16.
    Ng, H.T., Goh, W.B., Low, K.L.: Feature Selection, Perceptron Learning, and a Usability Case Study for Text Categorization. In Proceedings of 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (1997) 67–73.Google Scholar
  17. 17.
    Xie, C.F., Li, X.: A Sequence-Based Automatic Text Classification Algorithm. Journal of Software, Vol. 13(4), (2002) 783–789.MathSciNetGoogle Scholar
  18. 18.
    Xue, D.J., Sun, M.S.: An Automated Text Categorization System for Chinese Based on the Multinomial Bayesian Model. In Proceedings of Digital Library-IT Opportunities and Challenges in the New Millennium (2002) 131–140.Google Scholar
  19. 19.
    Huang, X.J., Wu, L.D., Hiroyuki, I., Xu, G.W.: Language Independent Text Categorization. Journal of Chinese Information Processing, Vol. 14(6), (2000) 1–7.Google Scholar
  20. 20.
    Lu, S., Li, X.L., Bai, S., Wang, S.: An Improved Approach to Weighting Terms in Text. Journal of Chinese Information Processing, Vol. 14(6), (2000) 8–13.zbMATHGoogle Scholar
  21. 21.
    Gong, X.J., Liu, S.H., Shi, Z.Z.: An Incremental Bayes Classification Model. Chinese J. Computers, Vol. 25(6), (2002) 645–650.MathSciNetGoogle Scholar
  22. 22.
    Nie, J.Y., Brisebois, M., Ren, X.B.: On Chinese Word Segmentation and Word-based Text Retrieval. In Proceedings of International Conference on Chinese Computing (1996) 405–412.Google Scholar
  23. 23.
    Nie, J.Y., Ren, F.J.: Chinese Information Retrieval: Using Characters or Words? Information Processing and Management Vol. 35, (1999) 443–462.CrossRefGoogle Scholar
  24. 24.
    Zhou, S.G., Guan, J.H.: Chinese Documents Classification Based on N-Grams. In Proceedings of 3rd Annual Conference on Intelligent Text Processing and Computational Linguistics (2002) 405–414.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Xue Dejun
    • 1
  • Sun Maosong
    • 1
  1. 1.National Key Laboratory of Intelligent Technology and Systems Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations