A Multiclass Classification Framework for Document Categorization

  • Qi Qiang
  • Qinming He
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3872)


With a great amount of textual information are available on the Internet and corporate intranets, it has become a necessary to categorize large documents. As we known, text classification problem is representative multiclass problem. This paper describes a framework, which we call Strong-to-Weak- to-Strong (SWS). It transforms a “strong” learning algorithm to a “weak” algorithm by decreasing its iterative numbers of optimization while preserving its other characteristics like geometric properties and then makes use of the kernel trick for “weak” algorithms to work in high dimensional spaces, finally improves the performances of text classification. We analyzed the particular properties of learning with text and identified why this approach is appropriate for this task. Empirical results show that our approach is competitive with the other methods.


Mahalanobis Distance Binary Classifier Document Categorization Output Code Kernel Trick 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Allwein, E., Schapire, R., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. In: Machine Learning: Proceedings of the SeventeenthInternational Conference (2000)Google Scholar
  2. 2.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  3. 3.
    Crammer, K., Singer, Y.: On the Learnability and Design of Output Codes for Multiclass Problems. In: Proceedings of the Thirteenth Annual Conference on Computational Learning Theory, pp. 35–46 (2000)Google Scholar
  4. 4.
    Lanckriet, R.G., Ghaoui, L.E., Bhattacharyya, C., Jordan, M.I.: A robust minimax approach to classification. Journal of Machine Learning Research 3, 555–582 (2002)CrossRefGoogle Scholar
  5. 5.
    Yang, Y., Pedersen, J.: A comparative study on feature selection in text categorization. In: International Conference on Machine Learning, ICML (1997)Google Scholar
  6. 6.
    Vapnik, V.: The Nature of Statistical Learning Theory. Spinger, New York (1995)zbMATHGoogle Scholar
  7. 7.
    Marshall, A.W., Olkin, I.: Multivariate Chebyshev inequalities. Annals of Mathematical Statistics 31(4), 1001–1014 (1960)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Smola, A.J., Bartlett, P.L., Scholkopf, B., Schuurmans, D.: Advances in large margin classifiers. MIT Press, Cambridge (2000)zbMATHGoogle Scholar
  9. 9.
    Freund, Y.: Boosting a weak learning algorithm by majority. Information and Computation 121(2), 256–285 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Müller, K.-R.: Fisher discriminant analysis with kernels. Neural Networks for Signal Processing IX, 41–48 (1999)Google Scholar
  11. 11.
    Schölkopf, B., Smola, A.J., Müller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)CrossRefGoogle Scholar
  12. 12.
    Aha, D.W., Bankert, R.L.: Cloud classification using error-correcting output codes. In: Artificial Intelligence Applications: Natural Science, Agriculture, and Environmental Science, vol. 11, pp. 13–28 (1997)Google Scholar
  13. 13.
    Hsu, C., Lin, C.A.: Comparison of methods for multiclass support vector machines. Technical report, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, 19 (2001)Google Scholar
  14. 14.
    Platt, J.C., Cristianini, N., Shawe-Taylor, J.: Large margin DAGs for multiclass classifycation. In: Advances in Neural Information Processing Systems, vol. 12, pp. 547–553. MIT Press, Cambridge (2000)Google Scholar
  15. 15.
    Joachims, T.: Text cateforization with support vector machines: Learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398. Springer, Heidelberg (1998)Google Scholar
  16. 16.
    Kudo, T., Matsumoto, Y.: Fast methods for kernel-based text analysis. In: Proceedings of the 41est Annual Meeting of the Association for Computational Linguistics, pp. 24–31 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Qi Qiang
    • 1
  • Qinming He
    • 1
    • 2
  1. 1.College of Computer ScienceZhejiang UniversityHangzhouChina
  2. 2.Ningbo Institute of TechnologyZhejiang UniversityNingboChina

Personalised recommendations