Skip to main content
SpringerLink
Log in

Support Vector Machines based Arabic Language Text Classification System: Feature Selection Comparative Study

  • Conference paper
Advances in Computer and Information Sciences and Engineering

Abstract

feature selection (FS) is essential for effective and more accurate text classification (TC) systems. This paper investigates the effectiveness of five commonly used FS methods for our Arabic language TC System. Evaluation used an in-house collected Arabic TC corpus. The experimental results are presented in terms of macro-averaging precision, macro-averaging recall and macro-averaging F1 measure.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. C. Manning, and H. Schütze, “Foundations of Statistical Natural Language Processing”, MIT Press (1999).

    Google Scholar 

  2. F. Sebastiani, “Machine Learning in Automated Text Categorization”, ACM Computing Surveys, Vol. 34, No. 1, 2002, pp.1-47.

    Article  Google Scholar 

  3. A. McCallum, and K. Nigam, “A comparison of event models for naïve Bayes text classification”, AAAI-98 Workshop on Learning for Text Categorization, 1998, pp.41-48.

    Google Scholar 

  4. Y. Yang, and X. Liu, “A re-examination of text categorization methods”, 22$nd$ Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’99), 1999, pp. 42-49.

    Google Scholar 

  5. G. Guo, H. Wang, D. Bell, Y. Bi, and K. Greer, “An kNN Model-based Approach and its Application in Text Categorization”, Proceeding of 5$th$ International Conference on Intelligent Text Processing and Computational Linguistic, CICLing-2004, LNCS 2945, Springer-Verlag, pages, 2004, pp. 559-570.

    Google Scholar 

  6. Y.M. Yang, “Expert network: Effective and efficient learning from human decisions in text categorization and retrieval”, Proceedings of the 17$th$ Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’94), 1994, pp. 13-22.

    Google Scholar 

  7. T. Joachims, “Text categorization with Support Vector Machines: learning with many relevant features”, Proceedings of the European Conference on Machine Learning (ECML’98), Berlin, 1998, pp.137-142, Springer.

    Google Scholar 

  8. D. Lewis, and M. Ringuette, “A comparison of two learning algorithms for text categorization”, The Third Annual Symposium on Document Analysis and Information Retrieval, 1994, pp.81-93.

    Google Scholar 

  9. R. Schapire, and Y. Singer, “BoosTexter: A boosting-based system for text categorization”, Machine Learning, Vol. 39, No.2-3, 2000, pp.135-168.

    Article  MATH  Google Scholar 

  10. S. Gao, W. Wu, C-H. Lee, and T-S. Chua, “A maximal figure-of-merit (MFoM)-learning approach to robust classifier design for text categorization”, ACM Transactions on Information Systems, Vol. 24, No. 2, 2006, pp. 190-218.

    Google Scholar 

  11. J. Zhang, R. Jin, Y.M. Yang, and A. Hauptmann, “A modified logistic regression: an approximation to SVM and its applications in large-scale Text Categorization”, Proceedings of the Twentieth International Conference (ICML 2003), August 21-24, 2003, Washington, DC, USA, pp. 888-895.

    Google Scholar 

  12. F. Ciravegna, et.al., “Flexible Text Classification for Financial Applications: the FACILE System”, Proceedings of PAIS-2000, Prestigious Applications of Intelligent Systems sub-conference of ECAI2000, 2000, pp.696-700.

    Google Scholar 

  13. F. Peng, X. Huang, D. Schuurmans, and S. Wang, “Text Classification in Asian Languages without Word Segmentation”, Proceedings of the Sixth International Workshop on Information Retrieval with Asian Languages (IRAL 2003), Association for Computational Linguistics, July 7, Sapporo, Japan, 2003, pp. 41-48.

    Google Scholar 

  14. J. He, A-H. TAN, and C-L. TAN, “On Machine Learning Methods for Chinese document Categorization”, Applied Intelligence, 2003, pp. 311-322.

    Google Scholar 

  15. A.M. Samir, W. Ata, and N. Darwish, “A New Technique for Automatic Text Categorization for Arabic Documents”, Proceedings of the 5$th$ Conference of the Internet and Information Technology in Modern Organizations, December, Cairo, Egypt, 2005, pp. 13-15.

    Google Scholar 

  16. Sakhr company website: http://www.sakhr.com.

    Google Scholar 

  17. A.M. El-Halees, “Arabic Text Classification Using Maximum Entropy”, The Islamic University Journal, Vol. 15, No. 1, 2007, pp 157-167.

    Google Scholar 

  18. A.M. Mesleh, “CHI Square Feature Extraction Based SVMs Arabic Language Text Categorization System”, Proceedings of the 2$nd$ international Conference on Software and Data Technologies, (Knowledge Engineering), Volume 1, Barcelona, Spain, July, 22—25, 2007, pp. 235-240.

    Google Scholar 

  19. A.M. Mesleh, “CHI Square Feature Extraction Based SVMs Arabic Language Text Categorization System”, Journal of Computer Science, Vol. 3, No. 6, 2007, pp. 430-435.

    Article  Google Scholar 

  20. M. Elkourdi, A. Bensaid, and T. Rachidi, “Automatic Arabic Document Categorization Based on the Naïve Bayes Algorithm”, Proceedings of COLING 20th Workshop on Computational Approaches to Arabic Script-based Languages, Geneva, August 23$rd$-27$th$ , 2004, pp. 51-58.

    Google Scholar 

  21. G. Kanaan, R. Al-Shalabi, and A. AL-Akhras, “KNN Arabic Text Categorization Using IG Feature Selection”, Proceedings of The 4$th$ International Multiconference on Computer Science and Information Technology (CSIT 2006), Vol. 4, Amman, Jordan, April 5-7, 2006, Retrieved March 20, 2007, from http://csit2006.asu.edu.jo/proceedings.

    Google Scholar 

  22. R. Al-Shalabi, G. Kanaan, M. Gharaibeh, “Arabic Text Categorization Using kNN Algorithm”, Proceedings of The 4$th$ International Multiconference on Computer Science and Information Technology (CSIT 2006), Vol. 4, Amman, Jordan, April 5-7, 2006, Retrieved March 20, 2007, from http://csit2006.asu.edu.jo/proceedings.

    Google Scholar 

  23. M. Syiam, Z. Fayed, and M. Habib, “An Intelligent System for Arabic Text Categorization”, International Journal of Intelligent Computing and Information Ssciences, Vol.6, No.1, 2006, pp. 1-19.

    Google Scholar 

  24. H. Sawaf, J. Zaplo, and H. Ney, “Statistical Classification Methods for Arabic News Articles”, Paper presented at the Arabic Natural Language Processing Workshop (ACL2001), Toulouse, France. ( Retrieved from Arabic NLP Workshop at ACL/EACL 2001 website: http://www.elsnet.org/acl2001-arabic.html).

    Google Scholar 

  25. R.M. Duwairi, “A Distance-based Classifier for Arabic Text Categorization”, Proceedings of the 2005 International Conference on Data Mining (DMIN2005), Las Vegas, USA, 2005, pp.187-192.

    Google Scholar 

  26. L. Khreisat, “Arabic Text Classification Using N-Gram Frequency Statistics A Comparative Study”, Proceedings of the 2006 International Conference on Data Mining (DMIN2006). Las Vegas, USA, 2006, pp.78-82.

    Google Scholar 

  27. R.M. Duwairi, “Machine Learning for Arabic Text Categorization”, Journal of American society for Information Science and Technology, Vol. 57, No. 8, 2006, pp.1005-1010.

    Article  Google Scholar 

  28. M. Benkhalifa, A. Mouradi, and H. Bouyakhf, “Integrating WordNet knowledge to supplement training data in semi-supervised agglomerative hierarchical clustering for text categorization”, International Journal of Intelligent Systems, Vol. 16, No. 8, 2001, pp. 929-947..

    Article  MATH  Google Scholar 

  29. Y.M. Yang, and J.O. Pedersen, “A Comparative Study on Feature Selection in Text Categorization”, In J. D. H. Fisher, editor, The 14$th$ International Conference on Machine Learning (ICML’97), Morgan Kaufmann, 1997, pp.412-420.

    Google Scholar 

  30. G. Forman, “An Extensive Empirical Study of Feature Selection Metrics for Text Classification”, Journal of Machine Learning Research, Vol. 3, 2003, pp. 1289-1305.

    Article  MATH  Google Scholar 

  31. G. Salton, A. Wong, and C.S. Yang, “A Vector Space Model for Automatic Indexing”, Communications of the ACM, Vol. 18, No. 11, 1975, pp. 613-620.

    Article  MATH  Google Scholar 

  32. G. Salton, and C. Buckley, “Term weighting approaches in automatic text retrieval”, Information Processing and Management, Vol. 24, No. 5, 1988, pp. 513-523.

    Article  Google Scholar 

  33. T. Hofmann, “Introduction to Machine Learning”, Draft Version 1.1.5, November 10, 2003.

    Google Scholar 

  34. M. Dash, M., K. Choi, P. Scheuermann, and H. Liu, “Feature Selection for Clustering-a Filter Solution”, Proceedings of the second International Conference of Data Mining, 2002, pp. 115-122.

    Google Scholar 

  35. P. Mitra, C.-A. Murthy, and S.-K. Pal, “Unsupervised Feature Selection Using Feature Similarity”. IEEE Transaction of Pattern Analysis and Machine Intelligence, Vol. 24, No. 3, 2002, pp. 301-312.

    Article  Google Scholar 

  36. R. Kohavi, G.H. John, “Wrappers for feature subset selection”, Artificial Intelligence, Vol. 97, No. 1-2, 1997, pp.273-324.

    Article  MATH  Google Scholar 

  37. E. Leopold, and J. Kindermann, “Text Categorization with Support Vector Machines. How to Represent Texts in Input Space?”, Machine Learning, Vol. 46, 2002, pp. 423-444.

    Article  MATH  Google Scholar 

  38. K. Nigam, A.K. Mccallum, S. Thrun, and T. Mitchell, “Text Classification from Labeled and Unlabeled Documents Using EM”, Machine Learning, Vol. 39, 2000, pp. 103—134.

    Article  MATH  Google Scholar 

  39. D. Mladenic, “Feature subset selection in text learning”, Proceedings of European Conference on Machine Learning (ECML), 1998, pp. 95-100.

    Google Scholar 

  40. H. Taira, and M. Haruno, “Feature selection in SVM text categorization”, Proceedings of AAAI-99, 16$th$ Conference of the American Association for Artificial Intelligence (Orlando, US, 1999), 1999, pp. 480-486.

    Google Scholar 

  41. D. Lewis, “Feature Selection and Feature Extraction for Text Categorization”, Proceedings of a workshop on speech and natural language, San Mateo, CA: Morgan Kaufmann, 1992, pp. 212-217.

    Google Scholar 

  42. M. Dash, and H. Liu, “Feature selection for classification”, Intelligent Data Analysis, Vol. 1, No. 3, 1997, pp.131-156.

    Article  Google Scholar 

  43. H. Liu, and L. Yu, “Toward integrating feature selection algorithms for classification and clustering”, IEEE Transaction on Knowledge and Data Engineering, Vol. 17, No. 4, 2005, pp. 491-502.

    Article  Google Scholar 

  44. H. Liu, “Evolving feature selection”, IEEE Intelligent Systems, 2005, pp.64-76.

    Google Scholar 

  45. S. Das, “Filters, wrappers and a boosting-based hybrid for feature selection”, Proceedings of the Eighteenth International Conference on Machine Learning, 2001, pp. 74-81.

    Google Scholar 

  46. A.L Blum, and P. Langley, “Selection of Relevant Features and Examples in machine Learning”, Artificial Intelligence, Vol. 97, No. 1-2, 1997, pp. 245-271.

    Article  MathSciNet  MATH  Google Scholar 

  47. D. Mladenic, and M. Grobelnik, “Feature Selection for Unbalanced Class Distribution and Naïve Bayes”, Proceedings of the Sixteenth International Conference on Machine Learning (ICML), 1999, pp. 258-267.

    Google Scholar 

  48. H.T. Ng, W.B. Goh, and K.L. Low, “Feature Selection, Perceptron Learning, and A usability Case Study for Text Categorization”, Proceedings of SIGIR-97, 20$th$ ACM International Conference on Research and Development in Information Retrieval, Philadelphia, PA, 1997, pp. 67-73.

    Google Scholar 

  49. L. Galavotti, F. Sebastiani, and M. Simi, “Experiments on the Use of Feature Selection and Negative Evidence in Automated Text Categorization”, Proceedings of ECDL-00, 4$th$ European Conference on Research and Advanced Technology for Digital Libraries, Lisbon, Portugal, 2000, pp. 59-68.

    Google Scholar 

  50. V. Vapnik, “The Nature of Statistical Learning Theory”, Springer-Verlag, New York, 1995.

    MATH  Google Scholar 

  51. R. Baeza-Yates, and B. Rieiro-Neto, “Modern Information Retrieval”, Addison-Wesley & ACM Press, 1999.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Faculty of Engineering Technology, Balqa’ Applied University, Amman, Jordan

    Abdelwadood. Moh’d. Mesleh

Authors
  1. Abdelwadood. Moh’d. Mesleh
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Engineering, University of Bridgeport, 221 University Avenue, Bridgeport, CT 06604, USA

    Tarek Sobh

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer Science+Business Media B.V.

About this paper

Cite this paper

Mesleh, A.M. (2008). Support Vector Machines based Arabic Language Text Classification System: Feature Selection Comparative Study. In: Sobh, T. (eds) Advances in Computer and Information Sciences and Engineering. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-8741-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4020-8741-7_3

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-1-4020-8740-0

  • Online ISBN: 978-1-4020-8741-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation