Abstract
Sentiment analysis is the computational examination of end user’s opinion, attitudes and emotions towards a particular topic or product. Sentiment analysis classifies the message according to their polarity whether it is positive, negative, or neutral. Recently researchers focused on lexical and machine-learning based method for sentiment analysis of social media post. Social media is a micro blogger site in which end users can post their comment in slag language that contains symbols, idioms, misspelled words and sarcastic sentences. Social media data also have curse of dimension problem i.e. high dimension nature of data that required specific pre-processing and feature extraction, which leads to improve classification accuracy. This paper present comprehensive overview of sentiment analysis technique based on recent research and subsequently explores machine learning (SVM, Navies Bayes, Linear Regression and Random Forest) and feature extraction techniques (POS, BOW and HASS tagging) in context of Sentiment analysis over social media data set. Further twitter data-sets are scrutinized and pre-processed with proposed framework,which yield intersecting facts about the capabilities and deficiency of sentiment analysis methods. POS is most suitable feature extraction technique with SVM and Navie Bayes classifier. Whereas Random Forest and linear regression provide the better result with Hass tagging.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abbasi A, Chen H (2007) Affect intensity analysis of dark web forums. In: 2007 IEEE intelligence and security informatics, pp 282–288. https://doi.org/10.1109/ISI.2007.379486
Appel O, Chiclana F, Carter J, Fujita H (2016) A hybrid approach to the sentiment analysis problem at the sentence level. Knowl Based Syst 108(Supplement C):110–124. https://doi.org/10.1016/j.knosys.2016.05.040 (new avenues in knowledge bases for natural language processing)
Bagheri A, Saraee M, de Jong F (2013) Care more about customers: unsupervised domain-independent aspect detection for sentiment analysis of customer reviews. Knowl Based Syst 52(Supplement C):201–213. https://doi.org/10.1016/j.knosys.2013.08.011
Balahur A, Perea-Ortega JM (2015) Sentiment analysis system adaptation for multilingual processing: the case of tweets. Inf Process Manag 51(4):547–556. https://doi.org/10.1016/j.ipm.2014.10.004
Balahur A, Turchi M (2014) Comparative experiments using supervised learning and machine translation for multilingual sentiment analysis. Comput Speech Lang 28(1):56–75. https://doi.org/10.1016/j.csl.2013.03.004
Balahur A, Mihalcea R, Montoyo A (2014) Computational approaches to subjectivity and sentiment analysis: present and envisaged methods and applications. Comput Speech Lang 28(1):1–6. https://doi.org/10.1016/j.csl.2013.09.003
Bravo-Marquez F, Mendoza M, Poblete B (2014) Meta-level sentiment models for big social data analysis. Knowl Based Syst 69(Supplement C):86–99. https://doi.org/10.1016/j.knosys.2014.05.016
Cardie C, Farina C, Bruce T (2006) Using natural language processing to improve erulemaking: Project highlight. In: Proceedings of the 2006 international conference on digital government research. dg.o ’06. Digital Government Society of North America, pp 177–178. https://doi.org/10.1145/1146598.1146651
Di Caro L, Grella M (2013) Sentiment analysis via dependency parsing. Comput Stand Interfaces 35(5):442–453. https://doi.org/10.1016/j.csi.2012.10.005
Carro RM, Ballesteros FJ, Ortigosa A, Guardiola G, Soriano E (2012) Angryemail? emotion-based e-mail tool adaptation. In: Proceedings of the 4th international conference on ambient assisted living and home care. Iwaal’12. Springer, Berlin, pp 399–406. https://doi.org/10.1007/978-3-642-35395-654.978-3-642-35394-9
Cho H, Kim S, Lee J, Lee JS (2014) Data-driven integration of multiple sentiment dictionaries for lexicon-based sentiment classification of product reviews. Knowled Based Syst 71(Supplement C):61–71. https://doi.org/10.1016/j.knosys.2014.06.001
Colace F, Casaburi L, De Santo M, Greco L (2015) Sentiment detection in social networks and in collaborative learning environments. Comput Hum Behav 51(Part B):1061–1067. https://doi.org/10.1016/j.chb.2014.11.090 (computing for human learning, behaviour and collaboration in the social and mobile networks era)
Cruz FL, Troyano JA, Pontes B, Javier Ortega F (2014) Building layered, multilingual sentiment lexicons at synset and lemma levels. Expert Syst Appl 41(13):5984–5994. https://doi.org/10.1016/j.eswa.2014.04.005
Cui Z, Shi X, Chen Y (2016) Sentiment analysis via integrating distributed representations of variable-length word sequence. Neurocomputing 187(Supplement C):126–132. https://doi.org/10.1016/j.neucom.2015.07.129 (recent developments on deep big vision)
da Silva NFF, Hruschka ER, Hruschka ER (2014) Tweet sentiment analysis with classifier ensembles. Decis Support Syst 66(C):170–179. https://doi.org/10.1016/j.dss.2014.07.003
da Silva NFF, Hruschka ER, Hruschka ER (2014) Tweet sentiment analysis with classifier ensembles. Decis Support Syst 66(Supplement C):170–179. https://doi.org/10.1016/j.dss.2014.07.003
da Silva NFF, Coletta LFS, Hruschka ER, Hruschka ER Jr (2016) Using unsupervised information to improve semi-supervised tweet sentiment classification. Inf Sci 355–356(Supplement C):348–365. https://doi.org/10.1016/j.ins.2016.02.002
Das S, Chen M (2001) Yahoo! for amazon: Extracting market sentiment from stock message boards. In: Asia pacific finance association annual conf. (APFA)
Davies M (2018a) Corpus of COCA. https://corpus.byu.edu/coca/. Accessed 1 Jan 2018
Davies M (2018b) Corpus of COHA. https://corpus.byu.edu/coha/. Accessed 1 Jan 2018
Davies M (2018c) Corpus of GloWbE. https://corpus.byu.edu/glowbe/. Accessed 1 Jan 2018
Deng Z-H, Luo K-H, Hong-Liang Y (2014) A study of supervised term weighting scheme for sentiment analysis. Expert Syst Appl 41(7):3506–3513. https://doi.org/10.1016/j.eswa.2013.10.056
Feldman R (2013) Techniques and applications for sentiment analysis. Commun ACM 56(4):82–89. https://doi.org/10.1145/2436256.2436274
Fernndez-Gavilanes M, Tl Lpez, Juncal-Martnez J, Costa-Montenegro E, Gonzlez-Castao FJ (2016) Unsupervised method for sentiment analysis in online texts. Expert Syst Appl 58(Supplement C):57–75. https://doi.org/10.1016/j.eswa.2016.03.031
Garca-Pablos A, Montse C, German R (2018) W2vlda: almost unsupervised system for aspect based sentiment analysis. Expert Syst Appl 91:127–137. https://doi.org/10.1016/j.eswa.2017.08.049
Ghiassi M, Skinner J, Zimbra D (2013) Twitter brand sentiment analysis: a hybrid system using n-gram analysis and dynamic artificial neural network. Expert Syst Appl 40(16):6266–6282. https://doi.org/10.1016/j.eswa.2013.05.057
Go A, Bhayani R, Huang L (2009) Twitter sentiment classification using distant supervision (CS224N–Final Project Report). Stanford University, Stanford, CA. http://www.yuefly.com/Public/Files/2017-03-07/58beb0822faef.pdf
Habernal I, Ptcek T, Steinberger J (2014) Supervised sentiment analysis in czech social media. Inf Process Manag 50(5):693–707. https://doi.org/10.1016/j.ipm.2014.05.001
Haddi E, Liu X, Shi Y (2013) The role of text pre-processing in sentiment analysis. Proc Comput Sci 17(Supplement C):26–32. https://doi.org/10.1016/j.procs.2013.05.005 (first international conference on information technology and quantitative management)
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90
Hogenboom A, Heerschop B, Frasincar F, Kaymak U, de Jong F (2014) Multi-lingual support for lexicon-based sentiment analysis guided by semantics. Decis Support Syst 62(Supplement C):43–53. https://doi.org/10.1016/j.dss.2014.03.004
Huang F, Zhang S, Zhang J, Yu G (2017) Multimodal learning for topic sentiment analysis in microblogging. Neurocomputing 253(Supplement C):144–153. https://doi.org/10.1016/j.neucom.2016.10.086 (learning multimodal data)
Jha V, Savitha R, Deepa Shenoy P, Venugopal KR, Sangaiah AK (2017) A novel sentiment aware dictionary for multi-domain sentiment classification. Comput Electr Eng 1:1. https://doi.org/10.1016/j.compeleceng.2017.10.015
Jin X, Li Y, Mah T, Tong J (2007) Sensitive webpage classification for content advertising. In: Proceedings of the 1st international workshop on data mining and audience intelligence for advertising, Adkdd ’07, PP 28–33. New York, NY, USA. ACM. https://doi.org/10.1145/1348599.1348604.978-1-59593-833-6
Joshi D, Datta R, Fedorovskaya E, Luong QT, Wang JZ, Li J, Luo J (2011) Aesthetics and emotions in images. IEEE Signal Process Mag 28(5):94–115. https://doi.org/10.1109/MSP.2011.941851
Kang D, Park Y (2014) Review-based measurement of customer satisfaction in mobile service: Sentiment analysis and vikor approach. Expert Syst Appl 41(4, Part 1):1041–1050. https://doi.org/10.1016/j.eswa.2013.07.101
Khan FH, Qamar U, Bashir S (2016) ESAP: a decision support framework for enhanced sentiment analysis and polarity classification. Inf Sci 367–368(Supplement C):862–873. https://doi.org/10.1016/j.ins.2016.07.028
Khan FH, Qamar U, Bashir S (2016b) Sentimi: Introducing point-wise mutual information with sentiwordnet to improve sentiment polarity detection. Appl Soft Comput 39:140–153. https://doi.org/10.1016/j.asoc.2015.11.016
Kontopoulos E, Berberidis C, Dergiades T, Bassiliades N (2013) Ontology-based sentiment analysis of twitter posts. Expert Syst Appl 40(10):4065–4074. https://doi.org/10.1016/j.eswa.2013.01.001
Kranjc J, Smailovic J, Podpecan V, Grcar M, Martin N, Nada L (2015) Active learning for sentiment analysis on data streams: methodology and workflow implementation in the clowdflows platform. Inf Process Manag 51(2):187–203. https://doi.org/10.1016/j.ipm.2014.04.001
Lau RYK, Li C, Liao SSY (2014) Social analytics: Learning fuzzy product ontologies for aspect-oriented sentiment analysis. Decis Support Syst 65(Supplement C):80–94. https://doi.org/10.1016/j.dss.2014.05.005 (crowdsourcing and social networks analysis)
Lei J, Rao Y, Li Q, Quan X, Wenyin L (2014) Towards building a social emotion detection system for online news. Future Gen Comput Syst 37(Supplement C):438–448. https://doi.org/10.1016/j.future.2013.09.024 (special section: innovative methods and algorithms for advanced data-intensive computing special section: semantics, intelligent processing and services for big data special section: advances in data-intensive modelling and simulation special section: hybrid intelligence for growing internet and its applications)
Li W, Guo K, Shi Y, Zhu L, Zheng Y (2018) Dwwp: domain-specific new words detection and word propagation system for sentiment analysis in the tourism domain. Knowl Based Syst 146:203–214. https://doi.org/10.1016/j.knosys.2018.02.004
Li X, Xie H, Chen L, Wang J, Deng X (2014) News impact on stock price return via sentiment analysis. Knowl Based Syst 69(Supplement C):14–23. https://doi.org/10.1016/j.knosys.2014.04.022
Liang J, Liu P, Tan J, Bai S (2014) Sentiment classification based on as-lda model. Proc Comput Sci 31(Supplement C):511–516. https://doi.org/10.1016/j.procs.2014.05.296 (2nd international conference on information technology and quantitative management, ITQM 2014)
Liu H, Lieberman H, Selker T (2003) A model of textual affect sensing using real-world knowledge. In: Proceedings of the 8th international conference on intelligent user interfaces. Iui ’03, 125–132. New York, NY, USA. ACM. DOIurl10.1145/604045.604067
Liu SM, Chen J-H (2015) A multi-label classification based approach for sentiment classification. Expert Syst Appl 42(3):1083–1093. https://doi.org/10.1016/j.eswa.2014.08.036
Ma R, Wang K, Qiu T, Sangaiah AK, Lin Dan, Liaqat Hannan Bin (2017) Feature-based compositing memory networks for aspect-based sentiment classification in social internet of things. Future Gen Comput Syst 1:1. https://doi.org/10.1016/j.future.2017.11.036
Mahyoub Fawaz HH, Siddiqui Muazzam A, Dahab Mohamed Y (2014) Building an arabic sentiment lexicon using semi-supervised learning. J King Saud Univ Comput Inf Sci 26(4):417–424. https://doi.org/10.1016/j.jksuci.2014.06.003 (special issue on arabic NLP)
Martn-Valdivia M-T, Martnez-Cmara E, Perea-Ortega J-M, Alfonso Urea-Lpez L (2013) Sentiment polarity detection in spanish reviews combining supervised and unsupervised approaches. Expert Syst Appl 40(10):3934–3942. https://doi.org/10.1016/j.eswa.2012.12.084
Mishne G, Glance NS (2006) Predicting movie sales from blogger sentiment. In: In AAAI spring symposium: computational approaches to analyzing weblogs, pp 155–158. http://dare.uva.nl/personal/search?identifier=4021a1e1-57d8-4943-a667-cae5ec339891
Montejo-Rez A, Daz-Galiano MC, Martnez-Santiago F, Urea-Lpez LA (2014a) Crowd explicit sentiment analysis. Knowl Based Syst 69(Supplement C):134–139. https://doi.org/10.1016/j.knosys.2014.05.007
Montejo-Rez A, Eugenio Martnez-Cmara M, Martn-Valdivia T, Alfonso Urea-Lpez L (2014b) Ranked wordnet graph for sentiment polarity classification in twitter. Comput Speech Lang 28(1):93–107. https://doi.org/10.1016/j.csl.2013.04.001
Nassirtoussi AK, Aghabozorgi S, Wah TY, Ngo DCL (2015) Text mining of news-headlines for forex market prediction: a multi-layer dimension reduction algorithm with semantics and sentiment. Expert Syst Appl 42(1):306–324. https://doi.org/10.1016/j.eswa.2014.08.004
Ortigosa A, Martn JM, Carro RM (2014) Sentiment analysis in facebook and its application to e-learning. Comput Hum Behav 31(Supplement C):527–541. https://doi.org/10.1016/j.chb.2013.05.024
Poria S, Cambria E, Howard N, Huang G-B, Hussain A (2016) Fusing audio, visual and textual clues for sentiment analysis from multimodal content. Neurocomputing 174:50–59. https://doi.org/10.1016/j.neucom.2015.01.095
Ptaszynski M, Rzepka R, Araki K, Momouchi Y (2014) Automatically annotating a five-billion-word corpus of Japanese blogs for sentiment and affect analysis. Comput Speech Lang 28(1):38–55. https://doi.org/10.1016/j.csl.2013.04.010
Rill S, Reinel D, Scheidt J, Zicari RV (2014) Politwi: early detection of emerging political topics on twitter and the impact on concept-level sentiment analysis. Knowl Based Syst 69(Supplement C):24–33. https://doi.org/10.1016/j.knosys.2014.05.008
Rong W, Nie Y, Ouyang Y, Peng B, Xiong Z (2014) Auto-encoder based bagging architecture for sentiment analysis. J Vis Lang Comput 25(6):840–849. https://doi.org/10.1016/j.jvlc.2014.09.005 (distributed multimedia systems DMS2014 part I)
Smailovic J, Grcar M, Lavrac N, Nidaric M (2014) Stream-based active learning for sentiment analysis in the financial domain. Inf Sci 285(Supplement C):181–203. https://doi.org/10.1016/j.ins.2014.04.034 (processing and mining complex data streams)
Stepanov EA, Riccardi G (2011) Detecting general opinions from customer surveys. In: 2011 IEEE 11th international conference on data mining workshops, pp 115–122. https://doi.org/10.1109/ICDMW.2011.63
Vilares D, Gmez-Rodrguez C, Alonso MA (2017) Universal, unsupervised (rule-based), uncovered sentiment analysis. Knowl Based Syst 118:45–55. https://doi.org/10.1016/j.knosys.2016.11.014
Wang G, Zhang Z, Sun J, Yang S, Larson CA (2015) Pos-rs: a random subspace method for sentiment classification based on part-of-speech analysis. Inf Process Manag 51(4):458–479. https://doi.org/10.1016/j.ipm.2014.09.004
Williams L, Bannister C, Arribas-Ayllon M, Preece A, Spasic I (2015) The role of idioms in sentiment analysis. Expert Syst Appl 42(21):7375–7385. https://doi.org/10.1016/j.eswa.2015.05.039
Wu C-E, Tsai RT-H (2014) Using relation selection to improve value propagation in a conceptnet-based sentiment dictionary. Knowl Based Syst 69(Supplement C):100–107. https://doi.org/10.1016/j.knosys.2014.04.043
Xiao Z, Li X, Wang L, Yang Q, Jiayi D, Sangaiah AK (2017) Using convolution control block for chinese sentiment analysis. J Parallel Distrib Comput 1:1. https://doi.org/10.1016/j.jpdc.2017.10.018
Yan G, He W, Shen J, Tang C (2014) A bilingual approach for conducting Chinese and English social media sentiment analysis. Comput Netw 75(Part B):491–503. https://doi.org/10.1016/j.comnet.2014.08.021 (special issue on online social networks)
Yang J, She D, Sun M, Cheng MM, Rosin P, Wang L (2018) Visual sentiment prediction based on automatic discovery of affective regions. IEEE Trans Multimed PP(99):1–1. https://doi.org/10.1109/TMM.2018.2803520
Yu L-C, Wu J-L, Chang P-C, Chu H-S (2013) Using a contextual entropy model to expand emotion words and their intensity for the sentiment classification of stock market news. Knowl Based Syst 41(Supplement C):89–97. https://doi.org/10.1016/j.knosys.2013.01.001
Zhang D, Hua X, Zengcai S, Yunfeng X (2015) Chinese comments sentiment classification based on word2vec and svmperf. Expert Syst Appl 42(4):1857–1863. https://doi.org/10.1016/j.eswa.2014.09.011
Zhang S, Wei Z, Wang Y, Liao T (2018) Sentiment analysis of chinese micro-blog text based on extended sentiment dictionary. Future Gen Comput Syst 81(Supplement C):395–403. https://doi.org/10.1016/j.future.2017.09.048
Zhou F, Jiao JR, Yang XJ, Lei B (2017) Augmenting feature model through customer preference mining by hybrid sentiment analysis. Expert Syst Appl 89(Supplement C):306–317. https://doi.org/10.1016/j.eswa.2017.07.021
Ziegelmayer D, Schrader R (2012) Sentiment polarity classification using statistical data compression models. In: 2012 IEEE 12th international conference on data mining workshops, pp 731–738. 1https://doi.org/10.1109/ICDMW.2012.43
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Singh, N.K., Tomar, D.S. & Sangaiah, A.K. Sentiment analysis: a review and comparative analysis over social media. J Ambient Intell Human Comput 11, 97–117 (2020). https://doi.org/10.1007/s12652-018-0862-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-018-0862-8