A model for sentiment and emotion analysis of unstructured social media text
- 1.5k Downloads
Sentiment analysis has applications in diverse contexts such as in the gathering and analysis of opinions of individuals about various products, issues, social, and political events. Understanding public opinion can help improve decision making. Opinion mining is a way of retrieving information via search engines, blogs, microblogs and social networks. Individual opinions are unique to each person, and Twitter tweets are an invaluable source of this type of data. However, the huge volume and unstructured nature of text/opinion data pose a challenge to analyzing the data efficiently. Accordingly, proficient algorithms/computational strategies are required for mining and condensing tweets as well as finding sentiment bearing words. Most existing computational methods/models/algorithms in the literature for identifying sentiments from such unstructured data rely on machine learning techniques with the bag-of-word approach as their basis. In this work, we use both unsupervised and supervised approaches on various datasets. Unsupervised approach is being used for the automatic identification of sentiment for tweets acquired from Twitter public domain. Different machine learning algorithms such as Multinomial Naive Bayes (MNB), Maximum Entropy and Support Vector Machines are applied for sentiment identification of tweets as well as to examine the effectiveness of various feature combinations. In our experiment on tweets, we achieve an accuracy of 80.68% using the proposed unsupervised approach, in comparison to the lexicon based approach (the latter gives an accuracy of 75.20%). In our experiments, the supervised approach where we combine unigram, bigram and Part-of-Speech as feature is efficient in finding emotion and sentiment of unstructured data. For short message services, using the unigram feature with MNB classifier allows us to achieve an accuracy of 67%.
KeywordsSentiment analysis Bag-of-words Lexicon Laplace smoothing Parts-of-Speech (POS) Machine learning
This research is partially supported by the following projects: (1) Information Security Education & Awareness Project (Phase II), Ministry of Electronics and Information Technology (MeitY), Government of India, and (2) Grant No. ETI/359/2014 by Fund for Improvement of S&T Infrastructure in Universities and Higher Educational Institutions (FIST) Program 2016, Department of Science and Technology, Government of India.
- 1.Bikel, D. M., & Sorensen, J. (2007). If we want your opinion. In: International conference on semantic computing (ICSC 2007) (pp. 493–500). doi: 10.1109/ICSC.2007.81.
- 4.Ding, X., Liu, B., & Yu, P. S. (2008). A holistic lexicon-based approach to opinion mining. In: Proceedings of the 2008 international conference on web search and data mining (pp. 231–240). doi: 10.1145/1341531.1341561.
- 5.Esuli, A., & Sebastiani, F. (2006). Sentiwordnet: A publicly available lexical resource for opinion mining. In: Proceedings of 5th language resources and evaluation (Vol. 6, pp. 417–422).Google Scholar
- 6.Fei, G., Liu, B., Hsu, M., Castellanos, M., & Ghosh, R. (2012). A dictionary-based approach to identifying aspects implied by adjectives for opinion mining. In: Proceedings of 24th international conference on computational linguistics (p. 309).Google Scholar
- 7.Feldman, R., Fresco, M., Goldenberg, J., Netzer, O., & Ungar, L. (2007). Extracting product comparisons from discussion boards. In: Seventh IEEE international conference on data mining (ICDM 2007) (pp. 469–474). doi: 10.1109/ICDM.2007.27.
- 8.Godbole, N., Srinivasaiah, M., & Skiena, S. (2007). Large-scale sentiment analysis for news and blogs. Proceedings of the international conference on weblogs and social media (ICWSM), 7(21), 219–222.Google Scholar
- 9.Hamouda, A., & Rohaim, M. (2011). Reviews classification using sentiwordnet lexicon. In: World congress on computer science and information technology.Google Scholar
- 10.Jindal, N., & Liu, B. (2006). Mining comparative sentences and relations. In: Proceedings of the 21st national conference on artificial intelligence (Vol. 2, pp. 1331–1336).Google Scholar
- 14.Liu, B. (2010). Sentiment analysis and subjectivity. Handbook of Natural Language Processing, 2, 627–666.Google Scholar
- 15.Liu, B. (2011). Opinion mining and sentiment analysis. In: Web data mining: Exploring hyperlinks, contents, and usage data (pp. 459–526). doi: 10.1007/978-3-642-19460-3_11.
- 17.Liu, P., Gulla, J. A., & Zhang, L. (2016). Dynamic topic-based sentiment analysis of large-scale online news. In: Proceedings of the 17th international conference on web information systems engineering (pp. 3–18). doi: 10.1007/978-3-319-48743-4_1.
- 21.Nassirtoussi, A. K., Aghabozorgi, S., Wah, T. Y., & Ngo, D. C. L. (2015). Text mining of news-headlines for forex market prediction: A multi-layer dimension reduction algorithm with semantics and sentiment. Expert Systems with Applications, 42(1), 306–324. doi: 10.1016/j.eswa.2014.08.004.CrossRefGoogle Scholar
- 22.Ohana, B. (2009). Opinion mining with the sentwordnet lexical resource. M.Sc. dissertation, Dublin Institute of Technology.Google Scholar
- 23.Pang, B., & Lee, L. (2004) A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the 42nd annual meeting on association for computational linguistics (p. 271). doi: 10.3115/1218955.1218990.
- 25.Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up?: Sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 conference on empirical methods in natural language processing (Vol. 10, pp. 79–86). doi: 10.3115/1118693.1118704.
- 27.Peng, J., Choo, K. K. R., & Ashman, H. (2016). Astroturfing detection in social media: Using binary n-gram analysis for authorship attribution. In: Proceedings of the 15th IEEE international conference on trust, security and privacy in computing and communications (TrustCom 2016) (pp. 121–1286).Google Scholar
- 30.Pröllochs, N., Feuerriegel, S., & Neumann, D. (2015). Enhancing sentiment analysis of financial news by detecting negation scopes. In: Proceedings of the 48th Hawaii international conference on system sciences (HICSS) (pp. 959–968). doi: 10.1109/HICSS.2015.119.
- 34.Sadegh, M., Ibrahim, R., & Othman, Z. A. (2012). Opinion mining and sentiment analysis: A survey. International Journal of Computers & Technology, 2(3), 171–178.Google Scholar
- 38.Turney, P.D. (2002). Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th annual meeting on association for computational linguistics (pp. 417–424). doi: 10.3115/1073083.1073153.