A New Inductive Learning Method for Multilabel Text Categorization

  • Yu-Chuan Chang
  • Shyi-Ming Chen
  • Churn-Jung Liau
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4031)


In this paper, we present a new inductive learning method for multilabel text categorization. The proposed method uses a mutual information measure to select terms and constructs document descriptor vectors for each category based on these terms. These document descriptor vectors form a document descriptor matrix. It also uses the document descriptor vectors to construct a document-similarity matrix based on the "cosine similarity measure". It then constructs a term-document relevance matrix by applying the inner product of the document descriptor matrix to the document similarity matrix. The proposed method infers the degree of relevance of the selected terms to construct the category descriptor vector of each category. Then, the relevance score between each category and a testing document is calculated by applying the inner product of its category descriptor vector to the document descriptor vector of the testing document. The maximum relevance score L is then chosen. If the relevance score between a category and the testing document divided by L is not less than a predefined threshold value λ between zero and one, then the document is classified into that category. We also compare the classification accuracy of the proposed method with that of the existing learning methods (i.e., Find Similar, Naïve Bayes, Bayes Nets and Decision Trees) in terms of the break-even point of micro-averaging for categorizing the "Reuters-21578 Aptè split" data set. The proposed method gets a higher average accuracy than the existing methods.


Testing Document Text Categorization Relevance Feedback Relevance Score Inductive Learn 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aptè, C., Damerau, F.J., Weiss, S.M.: Automatic Learning of Decision Rules for Text Categorization. ACM Transactions on Information Systems 1, 233–251 (1997)Google Scholar
  2. 2.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press, New York (1999)Google Scholar
  3. 3.
    Bekkerman, R., Ran, E.Y., Tishby, N., Winter, Y.: Distributional Word Clusters vs. Words for Text Categorization. Journal of Machine Learning Research, 1183–1208 (2003)Google Scholar
  4. 4.
    Caropreso, M.F., Matwin, S., Sebastiani, F.: A Learner-Independent Evaluation of the Usefulness of Statistical Phrases for Automated Text Categorization. In: Chin, A.G. (ed.) Text Databases and Document Management: Theory and Practice, pp. 78–102. Idea Group Publishing, Hershey (2001)Google Scholar
  5. 5.
    Chinkering, D., Heckerman, D., Meek, C.: A Bayesian Approach for Learning Bayesian Networks with Local Structure. In: Proceedings of Thirteen Conference on Uncertainty in Artificial Intelligence, pp. 80–89. Morgan Kaufmann, San Franscisco (1997)Google Scholar
  6. 6.
    Cohen, W.W., Singer, Y.: Context-Sensitive Learning Methods for Text Categorization. ACM Transactions on Information Systems 17, 141–173 (1999)CrossRefGoogle Scholar
  7. 7.
    Dhillon, I.S., Mallela, S., Kumar, R.: A Divisive Information-Theoretic Feature Clustering Algorithm for Text Classification. Journal of Machine Learning Research 3, 1265–1287 (2003)MATHMathSciNetGoogle Scholar
  8. 8.
    Dumais, S.T., Platt, J., Heckerman, D., Sahami, M.: Inductive Learning Algorithms and Representation for Text Categorization. In: Proceedings of CIKM 1998, 7th ACM International Conference on Information and Knowledge Management, Bethesda MD, pp. 148–155 (1998)Google Scholar
  9. 9.
    Fuhr, N., Buckley, C.: A Probabilistic Learning Approach for Document Indexing. ACM Transactions on Information Systems 9, 248–323 (1991)CrossRefGoogle Scholar
  10. 10.
    Fuhr, N., Pfeifer, U.: Probabilistic Information Retrieval as Combination of Abstraction Inductive Learning and Probabilistic Assumptions. ACM Transactions on Information Systems 12, 92–115 (1994)CrossRefGoogle Scholar
  11. 11.
    Hankerson, D., Harris, G.A., Johnson Jr., P.D.: Introduction to Information Theory and Data Compression. CRC Press, Boca Raton (1998)MATHGoogle Scholar
  12. 12.
    Lewis, D.D., Ringuetee, M.: Comparison of Two Learning Algorithms for Text Categorization. In: Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval, pp. 81–93 (1994)Google Scholar
  13. 13.
    Rocchio, J.J.: Relevance Feedback in Information Retrieval. In: Salton, G. (ed.) The SMART Retrieval System: Experiments in Automatic Document Processing, pp. 313–323. Prentice-Hall, New Jersey (1971)Google Scholar
  14. 14.
    Sahami, M.: Learning Limited Dependence Bayesian Classifiers. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pp. 335–338. AAAI Press, Menlo Park (1996)Google Scholar
  15. 15.
    Sebastiani, F.: Machine Learning in Automated Text Categorization. ACM Computing Surveys 34, 1–47 (2002)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Yang, Y.: An Evaluation of Statistical Approaches to Text Categorization. Information Retrieval 1, 69–90 (1999)CrossRefGoogle Scholar
  17. 17.
    Yang, Y., Chute, C.G.: An Example-based Mapping Method for Text Categorization and Retrieval. ACM Transactions on Information Systems 12, 252–277 (1994)CrossRefGoogle Scholar
  18. 18.
    Yang, Y., Liu, X.: A Re-examination of Text Categorization Methods. In: Proceedings of SIGIR 1999 22th ACM International Conference on Research and Development in Information Retrieval, pp. 42–49. Berkeley, California (1999)Google Scholar
  19. 19.
    Yang, Y., Pedersen, J.O.: A Comparative Study on Feature Selection in Text Categorization. In: Proceedings of ICML 1997. 14th International Conference on Machine Learning, Nashville, TN, pp. 412–420 (1997)Google Scholar
  20. 20.
  21. 21.
    Reuters-21578 Aptè split 10 categories data set,

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yu-Chuan Chang
    • 1
  • Shyi-Ming Chen
    • 1
  • Churn-Jung Liau
    • 2
  1. 1.Department of Computer Science and Information EngineeringNational Taiwan University of Science and TechnologyTaipeiTaiwan, R.O.C.
  2. 2.Institute of Information Science, Academia SinicaTaipeiTaiwan, R.O.C.

Personalised recommendations