Feature Selection Strategy in Text Classification

  • Pui Cheong Gabriel Fung
  • Fred Morstatter
  • Huan Liu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6634)

Abstract

Traditionally, the best number of features is determined by the so-called “rule of thumb”, or by using a separate validation dataset. We can neither find any explanation why these lead to the best number nor do we have any formal feature selection model to obtain this number. In this paper, we conduct an in-depth empirical analysis and argue that simply selecting the features with the highest scores may not be the best strategy. A highest scores approach will turn many documents into zero length, so that they cannot contribute to the training process. Accordingly, we formulate the feature selection process as a dual objective optimization problem, and identify the best number of features for each document automatically. Extensive experiments are conducted to verify our claims. The encouraging results indicate our proposed framework is effective.

Keywords

Feature Selection Feature Ranking Text Classification Selection Strategy 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Caropreso, M.F., Matwin, S., Sebastiani, F.: A learner-independent evaluation of the usefulness of statistical phrases for automated text categorization. In: Chin, A.G. (ed.) Text Databases and Document Management: Theory and Practice, pp. 78–102. Idea Group Publishing, USA (2001)Google Scholar
  2. 2.
    Dumais, S.T., Platt, J.C., Hecherman, D., Sahami, M.: Inductive learning algorithms and representations for text categorization. In: Proceedings of the 1998 ACM CIKM International Conference on Information and Knowledge Management (CIKM 1998) (1998)Google Scholar
  3. 3.
    Futr, N., Hartmann, S., Knorz, G., Lustig, G., Schwanter, M., Tzeras, K.: AIR/X – a rule-based multistage indexing system for large subject fields. In: Proceedings of the 3rd International Conference on Intelligent Text and Image Handling (RIAO 1991) (1991)Google Scholar
  4. 4.
    Galavotti, L., Sebastiani, F., Simi, M.: Experiments on the use of feature selection and negative evidence in automated text categorization. In: Borbinha, J.L., Baker, T. (eds.) ECDL 2000. LNCS, vol. 1923, p. 59. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  5. 5.
    Joachims, T.: Text categorization with support vector machines: Learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 491–502. Springer, Heidelberg (1998)Google Scholar
  6. 6.
    Lam, W., Lai, K.Y.: A meta-learning approach for text categorization. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001) (2001)Google Scholar
  7. 7.
    Larkey, L.S., Croft, W.B.: Combining classifiers in text categorization. In: Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1996) (1996)Google Scholar
  8. 8.
    Lewis, D.D.: An evaluation of phrasal and clustered representations on a text categorization task. In: Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1992) (1992)Google Scholar
  9. 9.
    Li, Y.H., Jain, A.K.: Classification of text documents. The Computer Journal 41(8), 537–546 (1998)CrossRefMATHGoogle Scholar
  10. 10.
    Lovins, J.B.: Development of a stemming algorithm. Mechanical Traqnslation and Computational Linguistics 11, 22–31 (1968)Google Scholar
  11. 11.
    McCallum, A., Nigam, K.: A Comparison of Event Models for Naive Bayes Text Classification. In: The 15th National Conference on Artificial Intelligence (AAAI 1998) Workshop on Learning for Text Categorization (1998)Google Scholar
  12. 12.
    Moschitti, A.: A study on optimal parameter tuning for rocchio text classifier. In: Sebastiani, F. (ed.) ECIR 2003. LNCS, vol. 2633, pp. 420–435. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Ng, H.T., Goh, W.B., Low, K.L.: Feature selection, perception learning, and a usability case study for text categorization. In: Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1997) (1997)Google Scholar
  14. 14.
    Ruiz, M.E., Srinivasan, P.: Hierarchical neural networks for text categorization. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1999) (1999)Google Scholar
  15. 15.
    Schütze, H., Hull, D.A., Pedersen, J.O.: A comparison of classifiers and document representations for the routing problem. In: Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1995) (1995)Google Scholar
  16. 16.
    Seabastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys 34(1), 1–47 (2002)CrossRefGoogle Scholar
  17. 17.
    Sebastiani, F., Sperduti, A., Valdambrini, N.: An improved boosting algorithm and its application to automated text categorization. In: Proceedings of the 2000 ACM CIKM International Conference on Information and Knowledge Management (CIKM 2000) (2000)Google Scholar
  18. 18.
    Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1999) (1999)Google Scholar
  19. 19.
    Yang, Y., Pedersen, J.O.: A comparative study on feature selection in text categorization. In: Proceedings of the 14th International Conference on Machine Learning (ICML 1997) (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Pui Cheong Gabriel Fung
    • 1
  • Fred Morstatter
    • 1
  • Huan Liu
    • 1
  1. 1.Arizona State UniversityTempeUnited States of America

Personalised recommendations