Advertisement

Comparison of sentiment lexicon development techniques for event prediction

  • Mehmet KayaEmail author
  • Shannon Conley
Original Article

Abstract

What started as a social utility for sharing short bursts of ‘inconsequential information’ has become a powerful information network capable of both tracking and shaping current events. From orchestrating government insurgencies to tracking epidemics, the majority of information shared via Twitter contains semantic relevance to contemporary topic(s), according to recent statistics. And, in consequence, Twitter is considered by researchers as an ideal platform for sentiment analysis. Compared to other online arenas such as forum discussions, blogs, and Facebook postings, Twitter frequently yields a higher degree of sentiment analysis accuracy due to the shortness of each post (140 character limit per Tweet). Various natural language processing techniques have been used to successfully perform sentiment classification on a group of Tweets. However, these techniques analyze text using both English-specific grammar rules and lexicons. Since there are fewer resources or tools in other languages, researchers often attempt to first use machine translation to translate the text into English. Often, translation errors introduce noise that obfuscates the results. In this study, we are analyzing the accuracy of sentiment analysis using an ad hoc and a translated sentiment lexicon in terms of capability of predicting the results of a future occurrence. We collected some 22,000 tweets using Twitter Search and Streaming APIs regarding a highly popular TV Show called “O Ses Türkiye” to predict the winner (Turkish version of globally known voice contest “The Voice of America”). We first performed a frequency-based statistical classification using an English sentiment lexicon translated into Turkish as well as a small ad hoc Turkish sentiment lexicon generated specifically for this study. We also use a k-means clustering technique using the two sentiment lexicons to evaluate the accuracies. Our study concludes that although using a translated sentiment lexicon (or training data for that matter) can also give a rough estimate for the result of a future event successfully, a language-specific ad hoc lexicon yields better granularity with higher discriminative power between negative, positive and neutral tweets. We also show the effect of automatic spell check and stemming in tweets on the predictive and discriminative power of auto-translated sentiment lexicon on a target language.

Keywords

Social media mining Sentiment analysis Sentiment lexicon Multilingual sentiment analysis 

References

  1. Akın AA, Akın MD (2007) Zemberek, an open source NLP framework for Turkic Languages. Structure 10:1–5Google Scholar
  2. Bakhshandeh R, Samadi M, Azimifar Z, Schaeffer J (2011) Degrees of separation in social networks. In: Borrajo D, Likhachev M, Lopez CL (eds) Proceedings of the fourth annual symposium on combinatorial search (SoCS-2011), Castell de Cardona, Barcelona, Spain, July 15–16, 2011, AAAI, Menlo Park, 2011, pp 18–23Google Scholar
  3. Banea C, Mihalcea R, Wiebe J, Hassan S (2008) Multilingual subjectivity analysis using machine translation. In: Proceedings of the conference on empirical methods in natural language processing (EMNLP ‘08). Association for Computational Linguistics, Stroudsburg, PA, USA, pp 127–135Google Scholar
  4. Banea C, Mihalcea R, Wiebe J (2010) Multilingual subjectivity: are more languages better? In: Proceedings of the 23rd international conference on computational linguistics (COLING ‘10). Association for Computational Linguistics, Stroudsburg, PA, USA, pp 28–36Google Scholar
  5. Bar-Haim R, Dinur E, Feldman R, Fresko M, Goldstein G (2011) Identifying and following expert investors in stock microblogs. In: Proceedings of the conference on empirical methods in natural language processing (EMNLP ‘11). Association for Computational Linguistics, Stroudsburg, PA, USA, pp 1310–1319Google Scholar
  6. Bautin M, Vijayarenu L, Skiena S (2008) International sentiment analysis for news and blogs. In: Proceedings of the international conference on weblogs and social media (ICWSM-2008), Seattle, WashingtonGoogle Scholar
  7. Blair-Goldensohn S, Hannan K, McDonald R, Neylon T, Reis GA, Reynar J (2008) Building a sentiment summarizer for local service reviewsGoogle Scholar
  8. Boiy E, Moens M-F (2009) A machine learning approach to sentiment analysis in multilingual web texts. Inf Retr 12(5):526–558CrossRefGoogle Scholar
  9. Bollen J, Mao H, Zeng X (2011) Twitter mood predicts the stock market. J Comput Sci 2(1):1–8 (ISSN 1877-7503) CrossRefGoogle Scholar
  10. Boyd-Graber J, Resnik P (2010) Holistic sentiment analysis across languages: multilingual supervised latent Dirichlet allocation. In: Proceedings of the 2010 conference on empirical methods in natural language processing (EMNLP ‘10). Association for Computational Linguistics, Stroudsburg, PA, USA, pp 45–55Google Scholar
  11. Brooke J, Tofiloski M, Taboada M (2009) Cross-linguistic sentiment analysis: from English to Spanish. In: Proceedings of the 7th international conference on recent advances in natural language processing, pp 50–54, BorovetsGoogle Scholar
  12. Chen Y, Fay S, Wang Q (2011) The role of marketing in social media: how online consumer reviews evolve. J Interact Mark 25(2):85–94Google Scholar
  13. Esuli A, Sebastiani F (2005) Determining the semantic orientation of terms through gloss classification. In: Proceedings of the 14th ACM international conference on information and knowledge management (CIKM ‘05)Google Scholar
  14. Hassan A, Radev D (2010) Identifying text polarity using random walks. In: Proceedings of the 48th annual meeting of the association for computational linguistics (ACL ‘10)Google Scholar
  15. Hong Y, Skiena S (2010) The wisdom of bookies? Sentiment analysis versus the NFL point spread. In: Cohen WW, Gosling S (eds) ‘ICWSM’. The AAAI PressGoogle Scholar
  16. Howard PN, Duffy A (2011) Opening closed regimes, what was the role of social media during the arab spring? Project on Information Technology and Political Islam, pp 1–30Google Scholar
  17. Hu M, Liu B (2004) Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining (KDD ‘04). ACM, New York, NY, USA, pp 168–177Google Scholar
  18. Kaji N, Kitsuregawa M (2006) Automatic construction of polarity-tagged corpus from HTML documents. In: Proceedings of the COLING/ACL on main conference poster sessions (COLING-ACL ‘06)Google Scholar
  19. Kaji N, Kitsuregawa M (2007) Building lexicon for sentiment analysis from massive collection of HTML documents. In: Proc. Of EMNLP’07, pp 1075–1083Google Scholar
  20. Kamps J, Marx M, Mokken RJ, DeRijke M (2004) Using WordNet to measure semantic orientation of adjectives. In: Proceedings of LREC-04, 4th international conference on language resources and evaluation, vol IV, pp 1115–1118, Lisbon, PTGoogle Scholar
  21. Kaplan AM, Haenlein M (2010) Users of the world, unite! The challenges and opportunities of Social Media. Bus Horiz 53(1):59–68CrossRefGoogle Scholar
  22. Kim S-M, Hovy E (2004) Determining the sentiment of opinions. In: Proceedings of the 20th international conference on computational linguistics (COLING ‘04)Google Scholar
  23. Kim S-M, Hovy E (2006) Automatic identification of pro and con reasons in online reviews. In: Proceedings of the COLING/ACL on main conference poster sessions (COLING-ACL ‘06)Google Scholar
  24. Kim J, Li J, Lee J (2010) Evaluating multilanguage-comparability of subjective analysis system. In: Proceedings of annual meeting of the association for computational linguistics (ACL-2010), 2010Google Scholar
  25. https://github.com/MeshkDevs/InvokeTwitterAPIs. Last accessed on 15 Apr 2015
  26. Lin Z, Jin X, Xu X, Wang Y, Tan S, Cheng X (2014) Make it possible: multilingual sentiment analysis without much prior knowledge. In: Proceedings of the 2014 IEEE/WIC/ACM international joint conferences on web intelligence (WI) and intelligent agent technologies (IAT)—Volume 02 (WI-IAT ‘14), vol 2. IEEE Computer Society, Washington, DC, USA, pp 79–86Google Scholar
  27. Liu J, Cao Y, Lin CY, Huang Y, Zhou M (2007) Low-quality product review detection in opinion summarization. In: Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), pp 334–342Google Scholar
  28. Lu Y, Wang F, Maciejewski R (2014) Business Intelligence from social media: a study from the VAST box office challenge. Comput Graph Appl IEEE 34(5):58–69CrossRefGoogle Scholar
  29. Mcglohon M, Glance N, Reiter Z (2010) Aggregating reviews to rank products and merchants. In: Proceedings of fourth international conference on weblogs and social media (ICWSM), AAAIGoogle Scholar
  30. Mihalcea R, Banea C, Wiebe J (2007) Learning multilingual subjective language via cross-lingual projections. In: Proceedings of the 45th annual meeting of the association of computational linguistics, pp 976–983Google Scholar
  31. Miller GA, Beckwith R, Fellbaum CD, Gross D, Miller K (1990) WordNet: an online lexical database. Int J Lexicogr 3(4):235–244CrossRefGoogle Scholar
  32. Mohammad SM, Yang TW (2011) Tracking sentiment in mail: how genders differ on emotional axes. In: Proceedings of the 2nd workshop on computational approaches to subjectivity and sentiment analysis (WASSA ‘11). Association for Computational Linguistics, Stroudsburg, PA, USA, pp 70–79Google Scholar
  33. Mohammad S, Dunne C, Dorr B (2009) Generating high-coverage semantic orientation lexicons from overtly marked words and a thesaurus. In: Proceedings of the 2009 conference on empirical methods in natural language processing: volume 2—volume 2 (EMNLP ‘09), vol 2. Association for Computational Linguistics, Stroudsburg, PA, USA, pp 599–608Google Scholar
  34. O’Connor B, Balasubramanyan R, Routledge BR, Smith NA (2010) From tweets to polls: linking text sentiment to public opinion time series. Fourth international AAAI conference on weblogs and social mediaGoogle Scholar
  35. Oflazar K, Göçmen E, Bozsahin C (1994) An outline of Turkish morphology. Report on Turkish Natural Language Processing Initiative ProjectGoogle Scholar
  36. Pang B, Lee L, Vaithyanathan S (2002) Thumbs up? Sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 conference on empirical methods in natural language processing—volume 10 (EMNLP ‘02)Google Scholar
  37. Porshnev A, Redkin I, Shevchenko A (2013) Machine learning in prediction of stock market indicators based on historical data and data from twitter sentiment analysis. IEEE 13th international conference on data mining workshops (ICDMW), pp 440–444, 7–10 Dec. 2013Google Scholar
  38. Prettenhofer P, Stein B (2010) Cross-language text classification using structural correspondence learning. In: Proceedings of the 48th annual meeting of the association for computational linguistics (ACL ‘10). Association for Computational Linguistics, Stroudsburg, PA, USA, pp 1118–1127Google Scholar
  39. Sadikov E, Parameswaran AG, Venetis P (2009) Blogs as predictors of movie success. Paper presented at the meeting of the ICWSMGoogle Scholar
  40. Sebastiani AF, Esuli A, Sebastiani F (2006) Determining term subjectivity and term orientation for opinion mining. In: Proceedings of the 11th conference of the european chapter of the association for computational linguistics (EACL’06)Google Scholar
  41. Tumasjan A, Sprenger T, Sandner P, Welpe I (2010) Predicting elections with twitter: what 140 characters reveal about political sentiment. International AAAI conference on weblogs and social mediaGoogle Scholar
  42. Wan X (2009) Co-training for cross-lingual sentiment classification. In: Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP: volume 1—volume 1 (ACL ‘09), vol 1. Association for Computational Linguistics, Stroudsburg, PA, USA, pp 235–243Google Scholar
  43. Wiebe JM, Bruce RF, O’Hara TP (1999) Development and use of a gold-standard data set for subjectivity classifications. In: Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics (ACL ‘99)Google Scholar
  44. Williams GK, Anand SS (2009) Predicting the polarity strength of adjectives using WordNet. In: Proceedings of the third international conference on weblogs and social media, ICWSM 2009, San Jose, California, USA, May 17–20, 2009Google Scholar
  45. Zhang W, Skiena S (2010) Trading strategies to exploit blog and news sentiment. International Conference on Weblogs and Social Media—ICWSMGoogle Scholar

Copyright information

© Springer-Verlag Wien 2016

Authors and Affiliations

  1. 1.Department of Electrical and Electronics EngineeringAdiyaman UniversityAdiyamanTurkey
  2. 2.Department of Software EngineeringFirat UniversityElazigTurkey

Personalised recommendations