Efficient Text Classification Using Term Projection

  • Yabin Zheng
  • Zhiyuan Liu
  • Shaohua Teng
  • Maosong Sun
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5839)

Abstract

In this paper, we propose an efficient text classification method using term projection. Firstly, we use a modified χ2 statistic to project terms into predefined categories, which is more efficient compared to other clustering methods. Afterwards, we utilize the generated clusters as features to represent the documents. The classification is then performed in a rule-based manner or via SVM. Experiment results show that our modified χ2 statistic feature selection method outperforms traditional χ2 statistic especially at lower dimensionalities. And our method is also more efficient than Latent Semantic Analysis (LSA) on homogeneous dataset. Meanwhile, we can reduce the feature dimensionality by three orders of magnitude to save training and testing cost, and maintain comparable accuracy. Moreover, we could use a small training set to gain an approximately 4.3% improvement on heterogeneous dataset as compared to traditional method, which indicates that our method has better generalization capability.

Keywords

Text classification χ2 statistic Term projection Cluster-based classification 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Sebastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys 34(1), 1–47 (2002)CrossRefGoogle Scholar
  2. 2.
    Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Proceedings of the 10th European Conference on Machine Learning, pp. 137–142 (1998)Google Scholar
  3. 3.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification, 2nd edn. Wiley-Interscience, New York (2000)MATHGoogle Scholar
  4. 4.
    Yang, Y.M., Liu, X.: A re-examination of text categorization methods. In: Proceedings of the 22nd Annual International Conference on Research and Development in Information Retrieval, pp. 42–49 (1999)Google Scholar
  5. 5.
    Yang, Y.M., Pedersen, J.O.: A comparative study on feature selection in text categorization. In: Proceedings of 14th International Conference on Machine Learning, pp. 412–420 (1997)Google Scholar
  6. 6.
    Li, J.Y., Sun, M.S., Zhang, X.: A comparison and semi-quantitative analysis of words and character-bigrams as features in Chinese text categorization. In: Proceedings of COLING-ACL 2006, pp. 545–552 (2006)Google Scholar
  7. 7.
    Baker, L.D., McCallum, A.K.: Distributional clustering of words for text classification. In: Proceedings of 21st ACM International Conference on Research and Development in Information Retrieval, pp. 96–103 (1998)Google Scholar
  8. 8.
    Bekkerman, R., El-Yaniv, R., Tishby, N., Winter, Y.: On feature distributional clustering for text categorization. In: Proceedings of 24th ACM International Conference on Research and Development in Information Retrieval, pp. 146–153 (2001)Google Scholar
  9. 9.
    Bekkerman, R., El-Yaniv, R., Tishby, N., Winter, Y.: Distributional word clusters vs. words for text categorization. Journal of Machine Learning Research 3, 1183–1208 (2003)MATHGoogle Scholar
  10. 10.
    Chen, W.L., Chang, X.Z., Wang, H.Z., Zhu, J.B., Yao, T.S.: Automatic word clustering for text categorization using global information. In: First Asia Information Retrieval Symposium, pp. 1–6 (2004)Google Scholar
  11. 11.
    Pereira, F., Tishby, N., Lee, L.: Distributional clustering of English words. In: Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics, pp. 183–190 (1993)Google Scholar
  12. 12.
    Ling, X., Dai, W.Y., Jiang, Y., Xue, G.R., Yang, Q., Yu, Y.: Can Chinese Web Pages be Classified with English Data Source? In: Proceedings of the 17th international conference on World Wide Web (2008)Google Scholar
  13. 13.
    Dai, W.Y., Xue, G.R., Yang, Q., Yu, Y.: Transferring Naive Bayes Classifiers for Text Classification. In: Proceedings of the 22nd AAAI Conference on Artificial Intelligence (2007)Google Scholar
  14. 14.
    Li, J.Y., Sun, M.S.: Scalable term selection for text categorization. In: Proceedings of EMNLP 2007, pp. 774–782 (2007)Google Scholar
  15. 15.
    Forman, G.: An extensive empirical study of feature selection metrics for text classification. Journal of Machine Learning Research 3, 1289–1305 (2003)MATHGoogle Scholar
  16. 16.
    Rennie, J.: 20Newsgroups dataset, http://people.csail.mit.edu/jrennie/20Newsgroups/
  17. 17.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/cjlin/libsvm

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Yabin Zheng
    • 1
  • Zhiyuan Liu
    • 1
  • Shaohua Teng
    • 1
  • Maosong Sun
    • 1
  1. 1.State Key Laboratory on Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations