Advertisement

Personalized Sentiment Analysis and a Framework with Attention-Based Hawkes Process Model

  • Siwen Guo
  • Sviatlana Höhn
  • Feiyu Xu
  • Christoph Schommer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11352)

Abstract

People use different words when expressing their opinions. Sentiment analysis as a way to automatically detect and categorize people’s opinions in text, needs to reflect this diversity and individuality. One possible approach to analyze such traits is to take a person’s past opinions into consideration. In practice, such a model can suffer from the data sparsity issue, thus it is difficult to develop. In this article, we take texts from social platforms and propose a preliminary model for evaluating the effectiveness of including user information from the past, and offer a solution for the data sparsity. Furthermore, we present a finer-designed, enhanced model that focuses on frequent users and offers to capture the decay of past opinions using various gaps between the creation time of the text. An attention-based Hawkes process on top of a recurrent neural network is applied for this purpose, and the performance of the model is evaluated with Twitter data. With the proposed framework, positive results are shown which opens up new perspectives for future research.

Keywords

Sentiment analysis Hawkes process Personalized model Attention network Recurrent neural networks 

References

  1. 1.
    Meena, A., Prabhakar, T.V.: Sentence level sentiment analysis in the presence of conjuncts using linguistic analysis. In: Amati, G., Carpineto, C., Romano, G. (eds.) ECIR 2007. LNCS, vol. 4425, pp. 573–580. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-71496-5_53CrossRefGoogle Scholar
  2. 2.
    Wiebe, J., Wilson, T., Bell, M.: Identifying collocations for recognizing opinions. In: Proceedings of the ACL Workshop on Collocation: Computational Extraction, Analysis, and Exploitation, pp. 24–31 (2001)Google Scholar
  3. 3.
    Cheng, X., Xu, F.: Fine-grained opinion topic and polarity identification. In: LREC, pp. 2710–2714 (2008)Google Scholar
  4. 4.
    Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., Manandhar, S.: SemEval-2014 task 4: aspect based sentiment analysis. In: Proceedings of SemEval, pp. 27–35 (2014)Google Scholar
  5. 5.
    Liu, B.: Sentiment Analysis: Mining Opinions, Sentiments and Emotions. Cambridge University Press, Cambridge (2015)CrossRefGoogle Scholar
  6. 6.
    Reiter, E., Sripada, S.: Human variation and lexical choice. Comput. Linguist. 28, 545–553 (2002)CrossRefGoogle Scholar
  7. 7.
    Markovikj, D., Gievska, S., Kosinski, M., Stillwell, D.: Mining Facebook data for predictive personality modeling. In: Proceedings of the 7th international AAAI conference on Weblogsand Social Media (ICWSM 2013), Boston, pp. 23–26 (2013)Google Scholar
  8. 8.
    Guo, S., Höhn, S., Xu, F., Schommer, C.: PERSEUS: a personalization framework for sentiment categorization with recurrent neural network. In: International Conference on Agents and Artificial Intelligence, p. 9, 16–18 January 2018, Funchal (2018)Google Scholar
  9. 9.
    Hawkes, A.G.: Spectra of some self-exciting and mutually exciting point processes. Biometrika 58, 83–90 (1971)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Ogata, Y.: Space-time point-process models for earthquake occurrences. Ann. Inst. Stat. Math. 50, 379–402 (1998)CrossRefGoogle Scholar
  11. 11.
    Mohler, G.O., Short, M.B., Brantingham, P.J., Schoenberg, F.P., Tita, G.E.: Self-exciting point process modeling of crime. J. Am. Stat. Assoc. 106, 100–108 (2011)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Bacry, E., Mastromatteo, I., Muzy, J.F.: Hawkes processes in finance. Mark. Microstruct. Liq. 1, 1550005 (2015)CrossRefGoogle Scholar
  13. 13.
    Kobayashi, R., Lambiotte, R.: TiDeH: Time-dependent Hawkes process for predicting retweet dynamics. In: ICWSM, pp. 191–200 (2016)Google Scholar
  14. 14.
    Zhao, Q., Erdogdu, M.A., He, H.Y., Rajaraman, A., Leskovec, J.:Seismic: A self-exciting point process model for predicting tweetpopularity. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1513–1522. ACM (2015)Google Scholar
  15. 15.
    Gilbert, C.H.E.: Vader: a parsimonious rule-based model for sentiment analysis of social media text. In: Eighth International Conference on Weblogs and Social Media (ICWSM 2014) (2014). http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf. Accessed 20 Apr 2016
  16. 16.
    Saif, H., He, Y., Fernandez, M., Alani, H.: Contextual semantics for sentiment analysis of Twitter. Inf. Process. Manag. 52, 5–19 (2016)CrossRefGoogle Scholar
  17. 17.
    Gong, L., Al Boni, M., Wang, H.: Modeling social norms evolution for personalized sentiment classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, (Volume 1: Long Papers), vol. 1, pp. 855–865 (2016)Google Scholar
  18. 18.
    Gong, L., Haines, B., Wang, H.: Clustered model adaption for personalized sentiment analysis. In: Proceedings of the 26th International Conference on World Wide Web, International World Wide Web Conferences Steering Committee, pp. 937–946 (2017)Google Scholar
  19. 19.
    Song, K., Feng, S., Gao, W., Wang, D., Yu, G., Wong, K.F.: Personalized sentiment classification based on latent individuality of microblog users. In: IJCA, pp. 2277–2283 (2015)Google Scholar
  20. 20.
    Chen, T., Xu, R., He, Y., Xia, Y., Wang, X.: Learning user and product distributed representations using a sequence model for sentiment analysis. IEEE Comput. Intell. Mag. 11, 34–44 (2016)CrossRefGoogle Scholar
  21. 21.
    Chen, H., Sun, M., Tu, C., Lin, Y., Liu, Z.: Neural sentiment classification with user and product attention. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1650–1659 (2016)Google Scholar
  22. 22.
    Dou, Z.Y.: Capturing user and product information for document level sentiment analysis with deep memory network. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 521–526 (2017)Google Scholar
  23. 23.
    Tang, D., Qin, B., Liu, T.: Learning semantic representations of users and products for document level sentiment classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, (Volume 1: Long Papers), pp. 1014–1023 (2015)Google Scholar
  24. 24.
    Guo, S., Schommer, C.: Embedding of the personalized sentiment engine PERSEUS in an artificial companion. In: International Conference on Companion Technology, Ulm, 11–13 September 2017. IEEE (2017)Google Scholar
  25. 25.
    Graves, A., Mohamed, A.R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: 2013 ieeeinternational conference on Acoustics, speech and signal processing (ICASSP), pp. 6645–6649. IEEE (2013)Google Scholar
  26. 26.
    Johnson, M., et al.: Google’s multilingual neural machine translation system: enabling zero-shot translation. arXiv preprint arXiv:1611.04558 (2016)
  27. 27.
    Cambria, E., Poria, S., Bajpai, R., Schuller, B.W.: SenticNet 4: a semantic resource for sentiment analysis based on conceptual primitives. In: COLING, pp. 2666–2677 (2016)Google Scholar
  28. 28.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
  29. 29.
    Jia, L., Yu, C., Meng, W.: The effect of negation on sentiment analysis and retrieval effectiveness. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 1827–1830. ACM (2009)Google Scholar
  30. 30.
    Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)Google Scholar
  31. 31.
    Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 6000–6010 (2017)Google Scholar
  32. 32.
    Zaremba, W., Sutskever, I.: Learning to execute. arXiv preprint arXiv:1410.4615 (2014)
  33. 33.
    Laub, P.J., Taimre, T., Pollett, P.K.: Hawkes processes. arXiv preprint arXiv:1507.02822 (2015)
  34. 34.
    Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Surowiecki, J.: The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business. Econ. Soc. Nat. 296 (2004)Google Scholar
  36. 36.
    Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford 1 (2009)Google Scholar
  37. 37.
    Schommer, C., Kampas, D., Bersan, R.: A prospect on how to find the polarity of a financial news by keeping an objective standpoint. In: Proceedings ICAART 2013 (2013)Google Scholar
  38. 38.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
  40. 40.
    Cao, Q., Shen, H., Cen, K., Ouyang, W., Cheng, X.: DeepHawkes: bridging the gap between prediction and understanding of information cascades. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 1149–1158. ACM (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Siwen Guo
    • 1
  • Sviatlana Höhn
    • 1
  • Feiyu Xu
    • 2
  • Christoph Schommer
    • 1
  1. 1.ILIAS Research Lab, CSCUniversity of LuxembourgEsch-sur-AlzetteLuxembourg
  2. 2.AI Lab, LenovoBeijingChina

Personalised recommendations