Survey on supervised machine learning techniques for automatic text classification

  • Ammar Ismael KadhimEmail author


Supervised machine learning studies are gaining more significant recently because of the availability of the increasing number of the electronic documents from different resources. Text classification can be defined that the task was automatically categorized a group documents into one or more predefined classes according to their subjects. Thereby, the major objective of text classification is to enable users for extracting information from textual resource and deals with process such as retrieval, classification, and machine learning techniques together in order to classify different pattern. In text classification technique, term weighting methods design suitable weights to the specific terms to enhance the text classification performance. This paper surveys of text classification, process of different term weighing methods and comparison between different classification techniques.


Supervised machine learning Text classification Term weighting Classification techniques 



  1. Agarwal B, Mittal N (2012) Text classification using machine learning methods–a survey. In: Proceedings of the second international conference on soft computing for problem solving (SocProS 2012), December 28–30. Springer, New Delh, pp 701–709Google Scholar
  2. Allahyari M, Pouriyeh S, Assefi M, Safaei S, Trippe ED, Gutierrez JB, Kochut KA (2017) Brief survey of text mining: classification, clustering and extraction techniques. arXiv preprint arXiv:1707.02919
  3. Aytekin Ç (2013) An opinion mining task in Turkish language: a model for assigning opinions in Turkish blogs to the polarities. J Mass Commun 3(3):179–198Google Scholar
  4. Bijalwan V, Kumar V, Kumari P, Pascual J (2014) KNN based machine learning approach for text and document mining. Int J Database Theory Appl 7(1):61–70CrossRefGoogle Scholar
  5. Bindra A (2012) “SocialLDA: scalable topic modeling in social networks”. Dissertation University of WashingtonGoogle Scholar
  6. Burges CJC (1996) Simplified support vector decision rules. In: ICML, Vol. 96, pp 71–77Google Scholar
  7. Canuto S, Salles T, Gonçalves MA, Rocha L, Ramos G, Gonçalves L, Martins W (2014) On efficient meta-level features for effective text classification. In: Proceedings of the 23rd ACM international conference on conference on information and knowledge management. ACM, pp 1709–1718Google Scholar
  8. Chen S (2018) K-nearest neighbor algorithm optimization in text categorization. In: IOP conference series: earth and environmental science. IOP Publishing, Vol. 108, No. 5, p 052074Google Scholar
  9. Chen M, Jin X, Shen D (2011) Short text classification improved by learning multi-granularity topics. In: IJCAI, pp 1776–1781Google Scholar
  10. Chouigui A, Khiroun OB, Elayeb B (2017) ANT Corpus: An Arabic news text collection for textual classification. In: IEEE/ACS 14th international conference on computer systems and applications (AICCSA). IEEE, pp 135–142Google Scholar
  11. Debole F, Sebastiani F (2004) Supervised term weighting for automated text categorization. Text mining and its applications. Springer, Berlin, pp 81–97CrossRefGoogle Scholar
  12. Elmurngi E, Gherbi A (2017) Detecting fake reviews through sentiment analysis using machine learning techniques. In: IARIA/data analytics, pp 65–72Google Scholar
  13. Feng Y, Zhaohui W, Zhou Z (2005) Multi-label text categorization using k-nearest neighbor approach with m-similarity. String Processing and Information Retrieval. Springer, BerlinGoogle Scholar
  14. Fix E, Hodges JL Jr (1951) Discriminatory analysis-nonparametric discrimination: consistency properties. California University, BerkeleyzbMATHGoogle Scholar
  15. HaCohen-Kerner Y, Gross Z, Masa A (2005) Automatic extraction and learning of keyphrases from scientific articles. In: Computational linguistics and intelligent text processing. Springer Berlin, pp 657–669Google Scholar
  16. Han EHS, Karypis G, Kumar V (2001) Text categorization using weight adjusted k-nearest neighbor classification. Springer, Berlin, pp 53–65zbMATHGoogle Scholar
  17. Han J, Pei J, Kamber M (2011) Data mining: concepts and techniques. Elsevier, AmsterdamzbMATHGoogle Scholar
  18. Hao P, Ying D, Longyuan T (2009) Application for web text categorization based on support vector machine. In: International forum on computer science-technology and applications, IFCSTA’09, Vol. 2. IEEE, pp 42–45Google Scholar
  19. Hassan S, Rafi M, Shaikh MS (2011) Comparing SVM and Naive Bayes classifiers for text categorization with wikitology as knowledge enrichment. In: 14th international multitopic conference (INMIC). IEEE, pp 31–34Google Scholar
  20. Hira ZM, Gillies DF (2015) A review of feature selection and feature extraction methods applied on microarray data. Adv Bioinf 2015:198363Google Scholar
  21. Horecki K, Mazurkiewicz J (2015) Natural language processing methods used for automatic prediction mechanism of related phenomenon. In: Artificial intelligence and soft computing. Springer, pp 13–24Google Scholar
  22. Hu J, Li S, Yao Y, Yu L, Yang G, Hu J (2018) Patent keyword extraction algorithm based on distributed representation for patent classification. Entropy 20(2):104CrossRefGoogle Scholar
  23. Huang S, Peng W, Li J, Lee D (2013) Sentiment and topic analysis on social media: a multi-task multi-label classification approach. In: Proceedings of the 5th annual ACM web science conference. ACM, pp 172–181Google Scholar
  24. Ikonomakis M, Kotsiantis S, Tampakas V (2005) Text classification using machine learning techniques. WSEAS Trans Comput 4(8):966–974Google Scholar
  25. Jiang S, Pang G, Wu M, Kuang L (2012) An improved K-nearest-neighbor algorithm for text categorization. Expert Syst Appl 39(1):1503–1509CrossRefGoogle Scholar
  26. Joseph F, Ramakrishnan N (2015) Text categorization using improved K nearest neighbor algorithm. Int J Trends Eng Technol 4:65–68Google Scholar
  27. Jothi CS, Thenmozhi D (2015) Machine learning approach to document classification using concept based features. Int J Comput Appl 118(20):33–36Google Scholar
  28. Kadhim AI, Cheah Y-N, Hieder IA, Ali RA (2017) Improving TF-IDF with singular value decomposition (SVD) for feature extraction on Twitter. In: 3rd international engineering conference on developments in civil and computer engineering applications 2017 (ISSN 2409-6997)Google Scholar
  29. Kamruzzaman SM, Haider F (2010) A hybrid learning algorithm for text classification. arXiv preprint arXiv:1009-4574
  30. Khamar K (2013) Short text classification using kNN based on distance function. In: IJARCCE International Journal of Advanced Research in Computer and Communication Engineering. Government Engineering College, Modasa (ISSN Print: 2319-5940 ISSN Online, pp 2278–1021Google Scholar
  31. Kowsari K, Brown DE, Heidarysafa M, Meimandi KJ, Gerber MS, Barnes LE (2017) Hdltex: hierarchical deep learning for text classification. In: 2017 16th IEEE international conference on machine learning and applications (ICMLA). IEEE, pp 364–371Google Scholar
  32. Kuang Q, Xiaoming X (2011) An improved feature weighting method for text classification. Adv Inf Sci Service Sci 3(7):340–346Google Scholar
  33. Kunchala DR (2015) Applying data mining techniques to social media data for analyzing the student’s learning experience. Ph.D. Dissertation, Texas A&M University-Corpus ChristiGoogle Scholar
  34. Kurada RR, Pavan DKK (2013) Novel text categorization by amalgamation of augmented k-nearest neighborhood classification and k-medoids clustering. arXiv preprint arXiv:1312.2375
  35. Kwok JT-Y (1998) Automated text categorization using support vector machine. In: Proceedings of the international conference on neural information processing (ICONIP 1998)Google Scholar
  36. Kwon O-W, Lee J-H (2003) Text categorization based on k-nearest neighbor approach for web site classification. Inf Process Manag 39(1):25–44CrossRefzbMATHGoogle Scholar
  37. Lai S, Xu L, Liu K, Zhao J (2015) Recurrent convolutional neural networks for text classification. AAAI 333:2267–2273Google Scholar
  38. Lausch A, Schmidt A, Tischendorf L (2015) Data mining and linked open data—new perspectives for data analysis in environmental research. Ecol Model 295:5–17CrossRefGoogle Scholar
  39. Li B, Yu S, Lu Q (2003) An improved k-nearest neighbor algorithm for text categorization. arXiv preprint arXiv:cs/0306099
  40. Marlow C, Naaman M, Boyd D, Davis M (2006) HT06, tagging paper, taxonomy, Flickr, academic article, to read. In: Proceedings of the seventeenth conference on hypertext and hypermedia. ACM, pp 31–40Google Scholar
  41. Masand VH, Mahajan DT, Patil KN, Chinchkhede KD, Jawarkar RD, Hadda TB, Alafeefy AA, Shibi IG (2012) k-NN, quantum mechanical and field similarity based analysis of xanthone derivatives as α-glucosidase inhibitors. Med Chem Res 21(12):4523–4534CrossRefGoogle Scholar
  42. Matsuo Y, Ishizuka M (2004) Keyword extraction from a single document using word co-occurrence statistical information. Int J Artif Intell Tools 13(01):157–169CrossRefGoogle Scholar
  43. Moreno A, Redondo T (2016) Text analytics: the convergence of big data and artificial intelligence. IJIMAI 3(6):57–64CrossRefGoogle Scholar
  44. Mudgal A, Munjal R (2015) Role of support vector machine, fuzzy K-means and Naive Bayes classification in intrusion detection system. Int J Recent and Innov Trends Comput Commun 3:1106–1110CrossRefGoogle Scholar
  45. Pitigala S, Li C (2015) Classification based filtering for personalized information retrieval. In: Proceedings of the international conference on information and knowledge engineering (IKE). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), pp 125–131Google Scholar
  46. Qi X, Davison BD (2009) Web page classification: features and algorithms. ACM Comput Surv (CSUR) 41(2):12CrossRefGoogle Scholar
  47. Rane A, Naik N, Laxminarayana JA (2014) Performance enhancement of K nearest neighbor classification algorithm using 8-bin hashing and feature weighting. In: Proceedings of the 2014 international conference on interdisciplinary advances in applied computing. ACM, p 8Google Scholar
  48. Rennie JDM, Rifkin R (2001) Improving multiclass text classification with the support vector machineGoogle Scholar
  49. Sadiq AT, Abdullah SM (2012) Hybrid intelligent technique for text categorization. In: International conference on advanced computer science applications and technologies (ACSAT). IEEE, pp 238–245Google Scholar
  50. Saeys Y, Inza I, Larrañaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517CrossRefGoogle Scholar
  51. Sahami M, Dumais S, Heckerman D, Horvitz E (1998) A Bayesian approach to filtering junk e-mail. Learn Text Categ 62:98–105Google Scholar
  52. Sebastiani F (2006) Classification of text, automatic. Encycl Lang Linguist 14:457–462CrossRefGoogle Scholar
  53. Sharma D (2012) Stemming algorithms: a comparative study and their analysis. Int J Appl Inf Syst 4(3):7–12Google Scholar
  54. Sharmila V, Vasudevan I, Arasu GT (2014) Pattern based classification for text mining using fuzzy similarity algorithm. J Theor Appl Inf Technol 63(1):92–103Google Scholar
  55. Shathi SP, Hossain MD, Nadim M, Riayadh SGR, Sultana T (2016) Enhancing performance of Naïve Bayes in text classification by introducing an extra weight using less number of training examples. In: International workshop on computational intelligence (IWCI). IEEE, pp 142–147Google Scholar
  56. Sugiyama M, Kawanabe M (2012) Machine learning in non-stationary environments: introduction to covariate shift adaptation. MIT Press, CambridgeCrossRefGoogle Scholar
  57. Suguna N, Thanushkodi K (2010) An improved K-nearest neighbor classification using Genetic Algorithm. Int J Comput Sci Issues 7(2):18–21Google Scholar
  58. Tatu A, Albuquerque G, Eisemann M, Schneidewind J, Theisel H, Magnork M, Keim D (2009) Combining automated analysis and visualization techniques for effective exploration of high-dimensional data. In: IEEE symposium on visual analytics science and technology, 2009, VAST 2009, pp 59–66Google Scholar
  59. Tilve AKS, Jain SN (2017) A survey on machine learning techniques for text classification. Int J Eng Sci Res Technol 6:513–520Google Scholar
  60. Trstenjak B, Mikac S, Donko D (2014) KNN with TF-IDF based framework for text categorization. Proc Eng 69:1356–1364CrossRefGoogle Scholar
  61. Vapnik V (2000) The nature of statistical learning theory. Springer, New YorkCrossRefzbMATHGoogle Scholar
  62. Vogrinčič S, Bosnić Z (2011) Ontology-based multi-label classification of economic articles. Comput Sci Inf Syst 8(1):101–119CrossRefGoogle Scholar
  63. Xu S (2018) Bayesian Naïve Bayes classifiers to text classification. J Inf Sci 44(1):48–59CrossRefGoogle Scholar
  64. Yan Z, Xu C (2010) Combining KNN algorithm and other classifiers. In: 2010 9th IEEE international conference on cognitive informatics (ICCI). IEEE, pp 800–805Google Scholar
  65. Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification. In: Advances in neural information processing systems, pp 649–657Google Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Computer Science, College of MedicineUniversity of BaghdadBaghdadIraq

Personalised recommendations