An evaluation of sentiment analysis for mobile devices

  • Johnnatan Messias
  • João P. Diniz
  • Elias Soares
  • Miller Ferreira
  • Matheus Araújo
  • Lucas Bastos
  • Manoel Miranda
  • Fabrício Benevenuto
Original Article

Abstract

Sentiment analysis has become a key tool to extract knowledge from data containing opinions and sentiments, particularly, data from online social systems. With the increasing use of smartphones to access social media platforms, a new wave of applications that explore sentiment analysis in the mobile environment is beginning to emerge. However, there are various existing sentiment analysis methods and it is unclear which of them are deployable in the mobile environment. In this paper, we provide the first of a kind study in which we compare the performance of 14 sentence-level sentiment analysis methods in the mobile environment. To do that, we adapted these methods to run on Android OS and then, we measure their performance in terms of memory, CPU, and battery consumption. Our findings unveil methods that require almost no adaptations and run relatively fast as well as methods that could not be deployed due to excessive use of memory. We hope our effort provides a guide to developers and researchers interested in exploring sentiment analysis as part of a mobile application and can help new applications to be executed without the dependency of a server-side API. We also share the Android API that implements all the 14 sentiment analysis methods used in this paper.

Keywords

Sentiment analysis Performance evaluation Mobile Android 

Notes

Acknowledgements

This project was supported by grants from Humboldt Foundation, Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), and Fundação de Amparo à Pesquisa do Estado de Minas Gerais (Fapemig).

References

  1. Araújo M, Diniz JP, Bastos L, Soares E, Júnior M, Ferreira M, Ribeiro F, Benevenuto F (2016a) ifeel 2.0: a multilingual benchmarking system for sentence-level sentiment analysis. In: Proceedings of the international AAAI conference on web-blogs and social media. CologneGoogle Scholar
  2. Araújo M, Reis J, Pereira A, Benevenuto F (2016b) An evaluation of machine translation for multilingual sentence-level sentiment analysis. In: Proceedings of the ACM symposium on applied computing (SAC). ACMGoogle Scholar
  3. Bollen J, Mao H, Zeng XJ (2010) Twitter mood predicts the stock market. CoRR abs/1010.3003Google Scholar
  4. Bonnington C (2015) In less than two years, a smartphone could be your only computer. Wired. http://www.wired.com/2015/02/smartphone-only-computer/
  5. Bradley MM, Lang PJ (1999) Affective norms for English words (ANEW): stimuli, instruction manual, and affective ratings. Technical report, Center for Research in Psychophysiology, University of FloridaGoogle Scholar
  6. Burke M, Marlow C, Lento T (2010) Social network activity and social well-being. In: Proceedings of the SIGCHI conference on human factors in computing systemsGoogle Scholar
  7. Cambria E, Speer R, Havasi C, Hussain A (2010) SenticNet: a publicly available semantic resource for opinion mining. In: AAAI fall symposium seriesGoogle Scholar
  8. Canuto S, Gonçalves MA, Benevenuto F (2016) Exploiting new sentiment-based meta-level features for effective sentiment analysis. In: Proceedings of the nineth ACM international conference on web search and data mining. ACMGoogle Scholar
  9. Cha M, Haddadi H, Benevenuto F, Gummadi KP (2010) Measuring user influence in Twitter: the million follower fallacy. In: International AAAI conference on weblogs and social media (ICWSM)Google Scholar
  10. Chambers L, Tromp E, Pechenizkiy M, Gaber MM (2012) Mobile sentiment analysis. In: Proceedings of KES, pp 470–479Google Scholar
  11. Cool-Smileys (2010) List of text emoticons: the ultimate resource. www.cool-smileys.com/text-emoticons
  12. De Choudhury M, Counts S, Horvitz EJ, Hoff A (2014) Characterizing and predicting postpartum depression from shared facebook data. In: Proceedings of the 17th ACM conference on computer supported cooperative work and social computingGoogle Scholar
  13. Esuli A, Sebastiani F (2006) SentiWordNet: a publicly available lexical resource for opinion mining. In: Proceedings on LRECGoogle Scholar
  14. Feldman R (2013) Techniques and applications for sentiment analysis. Commun ACM 56(4):82–89CrossRefGoogle Scholar
  15. Go A, Bhayani R, Huang L (2009) Twitter sentiment classification using distant supervision. CS224N Project Report Stanford 1(12)Google Scholar
  16. Gonçalves P, Araújo M, Benevenuto F, Cha M (2013a) Comparing and combining sentiment analysis methods, Proceedings of the first ACM conference on Online social networks, ACM, pp 27–38Google Scholar
  17. Gonçalves P, Benevenuto F, Cha M (2013b) PANAS-t: a psychometric scale for measuring sentiments on Twitter abs/1308.1857v1Google Scholar
  18. Gonçalves P, Dalip DH, Costa H, Gonçalves MA, Benevenuto F (2016) On the combination of “off-the-shelf” sentiment analysis methods. In: Proceedings of the ACM symposium on applied computing (SAC). ACMGoogle Scholar
  19. Hu M, Liu B (2004) Mining and summarizing customer reviews. In: KDD, pp 168–177Google Scholar
  20. Hutto C, Gilbert E (2014) Vader: a parsimonious rule-based model for sentiment analysis of social media text. In: Eighth international AAAI conference on weblogs and social mediaGoogle Scholar
  21. Levallois C (2013) Umigon: sentiment analysis for tweets based on lexicons and heuristics. In: Proceedings of the international workshop on semantic evaluation, SemEval, p 13Google Scholar
  22. LiKamWa R, Liu Y, Lane ND, Zhong L (2011) Can your smartphone infer your mood? ACM workshop on sensing applications on mobile phones (PhoneSense)Google Scholar
  23. Messias J, Diniz JP, Soares E, Ferreira M, Araujo M, Bastos L, Miranda M, Benevenuto F (2016) Towards sentiment analysis for mobile devices. In: Proceedings of the 2016 IEEE/ACM international conference on advances in social networks analysis and mining, ASONAMGoogle Scholar
  24. Miller GA (1995) Wordnet: a lexical database for english. Commun ACM 38(11):39–41CrossRefGoogle Scholar
  25. Mohammad S (2012) #emotional tweets. In: Proceedings of the sixth international workshop on semantic evaluation (SemEval). Association for Computational LinguisticsGoogle Scholar
  26. Mohammad SM, Kiritchenko S, Zhu X (2013) Nrc-Canada: building the state-of-the-art in sentiment analysis of tweets. In: Proceedings of the seventh international workshop on semantic evaluation exercises (SemEval)Google Scholar
  27. Mohammad S, Turney PD (2013) Crowdsourcing a word-emotion association lexicon. Comput Intell 29(3):436–465MathSciNetCrossRefGoogle Scholar
  28. Nielsen FÅ (2011) A new ANEW: evaluation of a word list for sentiment analysis in microblogs. arXiv preprint arXiv:1103.2903
  29. Ogneva M (2010) How companies can use sentiment analysis to improve their business. MashableGoogle Scholar
  30. Oliveira N, Cortez P, Areal N (2013) On the predictability of stock market behavior using stocktwits sentiment and posting volume. In: Correia L, Reis LP, Cascalho J (eds) EPIA, lecture notes in computer science, vol 8154. Springer, pp 355–365Google Scholar
  31. Osmonitor for android (2015) www.osmonitor.mobi. Accessed 19 Sept 2015
  32. Pappas N, Popescu-Belis A (2013) Sentiment analysis of user comments for one-class collaborative filtering over ted talks. In: Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval. ACM, pp 773–776Google Scholar
  33. Pereira M, Pádua F, Pereira A, Benevenuto F, Dalip D (2016) Fusing audio, textual and visual features for sentiment analysis of news videos. In: Proceedings of the international AAAI conference on web-blogs and social media. CologneGoogle Scholar
  34. Plutchik R (1980) A general psychoevolutionary theory of emotion. Academic press, New York, pp 3–33CrossRefGoogle Scholar
  35. Reis J, Benevenuto F, Vaz de Melo P, Prates R, Kwak H, An J (2015) Breaking the news: first impressions matter on online news. In: Proceedings of the 9th international AAAI conference on web-blogs and social mediaGoogle Scholar
  36. Reis J, Goncalves P, Vaz de Melo P, Prates R, Benevenuto F (2014) Magnet news: you choose the polarity of what you read. In: International AAAI conference on web-blogs and social mediaGoogle Scholar
  37. Ribeiro FN, Araújo M, Gonçalves P, Gonçalves MA, Benevenuto F (2016) Sentibench—a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data ScienceGoogle Scholar
  38. Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng AY, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: 2013 conference on empirical methods in natural language processingGoogle Scholar
  39. Tausczik YR, Pennebaker JW (2010) The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol 29(1):24–54. doi:10.1177/0261927X09351676 CrossRefGoogle Scholar
  40. Thelwall M (2013) Heart and soul: sentiment strength detection in the social web with SentiStrength. http://sentistrength.wlv.ac.uk/documentation/SentiStrengthChapter.pdf
  41. Tumasjan A, Sprenger TO, Sandner PG, Welpe IM (2010) Predicting elections with Twitter: what 140 characters reveal about political sentiment. In: International AAAI conference on weblogs and social media (ICWSM)Google Scholar
  42. Wang H, Can D, Kazemzadeh A, Bar F, Narayanan S (2012) A system for real-time Twitter sentiment analysis of 2012 US Presidential Election Cycle. In: ACL system demonstrationsGoogle Scholar
  43. Watson D, Clark L (1985) Development and validation of brief measures of positive and negative affect: the panas scales. J Personal Soc Psychol 54(1):1063–1070Google Scholar
  44. Wilson T, Hoffmann P, Somasundaran S, Kessler J, Wiebe J, Choi Y, Cardie C, Riloff E, Patwardhan S (2005) Opinionfinder: a system for subjectivity analysis. In: HLT/EMNLP on interactive demonstrations, pp 34–35Google Scholar

Copyright information

© Springer-Verlag Wien 2017

Authors and Affiliations

  1. 1.Universidade Federal de Minas Gerais (UFMG)Belo HorizonteBrazil

Personalised recommendations