Advertisement

Comparison of Extreme Learning Machine with Support Vector Machine for Text Classification

  • Ying Liu
  • Han Tong Loh
  • Shu Beng Tor
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3533)

Abstract

Extreme Learning Machine, ELM, is a recently available learning algorithm for single layer feedforward neural network. Compared with classical learning algorithms in neural network, e.g. Back Propagation, ELM can achieve better performance with much shorter learning time. In the existing literature, its better performance and comparison with Support Vector Machine, SVM, over regression and general classification problems catch the attention of many researchers. In this paper, the comparison between ELM and SVM over a particular area of classification, i.e. text classification, is conducted. The results of benchmarking experiments with SVM show that for many categories SVM still outperforms ELM. It also suggests that other than accuracy, the indicator combining precision and recall, i.e. F 1 value, is a better performance indicator.

Keywords

Support Vector Machine Feature Selection Extreme Learning Machine Category Support Vector Machine Extreme Learning Machine Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern information retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston (1999)Google Scholar
  2. 2.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1996)zbMATHGoogle Scholar
  3. 3.
    Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)CrossRefGoogle Scholar
  4. 4.
    Cristianini, N., Shawe-Taylor, J.: An introduction to Support Vector Machines: and other kernel-based learning methods. Cambridge University Press, Cambridge (2000)Google Scholar
  5. 5.
    Dumais, S., Chen, H.: Hierarchical classification of Web content. In: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR 2000 (2000)Google Scholar
  6. 6.
    Flach, P.A.: On the state of the art in machine learning: a personal review. Artificial Intelligence 13, 199–222 (2001)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks. In: International Joint Conference on Neural Networks, IJCNN 2004 (2004)Google Scholar
  8. 8.
    Joachims, T.: Text categorization with Support Vector Machines: Learning with many relevant features. In: Machine Learning: ECML 1998, Tenth European Conference on Machine Learning (1998)Google Scholar
  9. 9.
    Joachims, T.: Transductive Inference for Text Classification using Support Vector Machines. In: Proceedings of the 16th International Conference on Machine Learning, ICML (1999)Google Scholar
  10. 10.
    Kasabov, N.: Data mining and knowledge discovery using neural networks (2002)Google Scholar
  11. 11.
    Leopold, E., Kindermann, J.: Text Categorization with Support Vector Machines - How to Represent Texts in Input Space. Machine Learning 46, 423–444 (2002)zbMATHCrossRefGoogle Scholar
  12. 12.
    Liu, Y., Loh, H.T., Tor, S.B.: Building a Document Corpus for Manufacturing Knowledge Retrieval. In: Singapore MIT Alliance Symposium 2004 (2004)Google Scholar
  13. 13.
    Mangasarian, O.L.: Data Mining via Support Vector Machines. In: 20th International Federation for Information Processing (IFIP) TC7 Conference on System Modeling and Optimization (2001)Google Scholar
  14. 14.
    Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)zbMATHGoogle Scholar
  15. 15.
    Mitchell, T.M.: Machine Learning. McGraw-Hill Companies, Inc, New York (1997)zbMATHGoogle Scholar
  16. 16.
    Ng, H.T., Goh, W.B., Low, K.L.: Feature selection, perception learning, and a usability case study for text categorization. In: ACM SIGIR Forum, Proceedings of the 20th annual in-ternational ACM SIGIR conference on Research and development in information retrieval (1997)Google Scholar
  17. 17.
    Ruiz, M.E., Srinivasan, P.: Hierarchical Neural Networks for Text Categorization. In: Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval (1999)Google Scholar
  18. 18.
    Ruiz, M.E., Srinivasan, P.: Hierarchical Text Categorization Using Neural Networks. Information Retrieval 5, 87–118 (2002)zbMATHCrossRefGoogle Scholar
  19. 19.
    Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, 1st edn. MIT Press, Cambridge (2001)Google Scholar
  20. 20.
    Sebastiani, F.: Machine Learning in Automated Text Categorization. ACM Computing Surveys (CSUR) 34, 1–47 (2002)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Sun, A., Lim, E.-P.: Hierarchical Text Classification and Evaluation. In: Proceedings of the 2001 IEEE International Conference on Data Mining, ICDM 2001 (2001)Google Scholar
  22. 22.
    Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (1999)Google Scholar
  23. 23.
    Wiener, E.D., Pedersen, J.O., Weigend, A.S.: A neural network approach to topic spotting. In: Proceedings of SDAIR-95, 4th Annual Symposium on Document Analysis and Information Retrieval (1995)Google Scholar
  24. 24.
    Weigend, A.S., Wiener, E.D., Pedersen, J.O.: Exploiting hierarchy in text categorization. Information Retrieval 1, 193–216 (1999)CrossRefGoogle Scholar
  25. 25.
    Yang, Y.: An evaluation of statistical approaches to text categorization. Information Retrieval 1, 69–90 (1999)CrossRefGoogle Scholar
  26. 26.
    Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Ying Liu
    • 1
  • Han Tong Loh
    • 1
  • Shu Beng Tor
    • 2
  1. 1.Singapore-MIT AllianceNational University of SingaporeSingapore
  2. 2.Singapore-MIT AllianceNanyang Technological UniversitySingapore

Personalised recommendations