Fine-Grained Opinion Mining from Mobile App Reviews with Word Embedding Features

  • Mario SängerEmail author
  • Ulf Leser
  • Roman Klinger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10260)


Existing approaches for opinion mining mainly focus on reviews from Amazon, domain-specific review websites or social media. Little efforts have been spent on fine-grained analysis of opinions in review texts from mobile smart phone applications. In this paper, we propose an aspect and subjective phrase extraction model for German reviews from the Google Play store. We analyze the impact of different features, including domain-specific word embeddings. Our best model configuration shows a performance of 0.63 \(F_1\) for aspects and 0.62 \(F_1\) for subjective phrases. Further, we perform cross-domain experiments: A model trained on Amazon reviews and tested on app reviews achieves lower performance (drop by 27% points for aspects and 15% points for subjective phrases). The results indicate that there are strong differences in the way personal opinions on product aspects are expressed in the particular domains.


Sentiment analysis Reviews German App reviews Opinion mining 



We thank Christian Scheible, Peter Adolphs and Steffen Kemmerer for their valuable feedback and fruitful discussions.


  1. 1.
    Al-Rfou, R., Perozzi, B., Skiena, S.: Polyglot: distributed word representations for multilingual NLP. In: Proceedings of the Seventeenth Conference on Computational Natural Language Learning, Sofia, Bulgaria, pp. 183–192. Association for Computational Linguistics, August 2013Google Scholar
  2. 2.
    Blei, D., Ng, A.Y., Jordan, M.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  3. 3.
    Chen, N., Lin, J., Hoi, S.C.H., Xiao, X., Zhang, B.: AR-miner: mining informative reviews for developers from mobile app marketplace. In: Proceedings of the 2014 International Conference on Software Engineering, Hyderabad, India, pp. 767–778 (2014)Google Scholar
  4. 4.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)zbMATHGoogle Scholar
  5. 5.
    Cui, H., Mittal, V., Datar, M.: Comparative experiments on sentiment classification for online product reviews. In: Proceedings of the Eighteenth Conference on Innovative Applications of Artificial Intelligence, Boston, MA, USA, vol. 6, pp. 1265–1270 (2006)Google Scholar
  6. 6.
    Derczynski, L., Maynard, D., Rizzo, G., van Erp, M., Gorrell, G., Troncy, R., Petrak, J., Bontcheva, K.: Analysis of named entity recognition and linking for tweets. Inf. Process. Manag. 51(2), 32–49 (2015)CrossRefGoogle Scholar
  7. 7.
    Faruqui, M., Tsvetkov, Y., Yogatama, D., Dyer, C., Smith, N.: Sparse overcomplete word vector representations. In: Proceedings of Association for Computational Linguistics, Beijing, China (2015)Google Scholar
  8. 8.
    Fu, B., Lin, J., Li, L., Faloutsos, C., Hong, J., Sadeh, N.: Why people hate your app: making sense of user feedback in a mobile app store. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, USA, pp. 1276–1284. Association for Computing Machinery (2013)Google Scholar
  9. 9.
    Gade, T., Pardeshi, N.: A survey on ranking fraud detection using opinion mining for mobile apps. Int. J. Adv. Res. Comput. Commun. Eng. 4(12), 337–339 (2015)Google Scholar
  10. 10.
    Galvis Carreno, L., Winbladh, K.: Analysis of user comments: an approach for software requirements evolution. In: Proceedings of the 2013 International Conference on Software Engineering. pp. 582–591. San Francisco, CA, USA (2013)Google Scholar
  11. 11.
    Gu, X., Kim, S.: What parts of your apps are loved by users? In: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering. pp. 760–770. IEEE, Lincoln, USA (2015)Google Scholar
  12. 12.
    Guo, J., Che, W., Wang, H., Liu, T.: Revisiting embedding features for simple semi-supervised learning. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, pp. 110–120 (2014)Google Scholar
  13. 13.
    Guzman, E., Maalej, W.: How do users like this feature? A fine grained sentiment analysis of app reviews. In: Proceedings of the 22nd International Requirements Engineering Conference, Karlskrona, Sweden, pp. 153–162 (2014)Google Scholar
  14. 14.
    Harman, M., Jia, Y., Zhang, Y.: App store mining and analysis: MSR for app stores. In: Proceedings of the 9th IEEE Working Conference on Mining Software Repositories, Zurich, Switzerland, pp. 108–111 (2012)Google Scholar
  15. 15.
    Hintz, G., Biemann, C.: Delexicalized supervised German lexical substitution. In: Proceedings of GermEval 2015: LexSub, Essen, Germany, pp. 11–16 (2015)Google Scholar
  16. 16.
    Hutto, C.J., Gilbert, E.: Vader: a parsimonious rule-based model for sentiment analysis of social media text. In: Eighth International AAAI Conference on Weblogs and Social Media, Ann Arbor, MI, USA (2014)Google Scholar
  17. 17.
    Iacob, C., Harrison, R.: Retrieving and analyzing mobile apps feature requests from online reviews. In: Proceedings of the 10th IEEE Working Conference on Mining Software Repositories, San Francisco, CA, USA, pp. 41–44 (2013)Google Scholar
  18. 18.
    Jakob, N., Gurevych, I.: Extracting opinion targets in a single-and cross-domain setting with conditional random fields. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA, pp. 1035–1045. Association for Computational Linguistics (2010)Google Scholar
  19. 19.
    Klinger, R., Cimiano, P.: Joint and pipeline probabilistic models for fine-grained sentiment analysis: extracting aspects, subjective phrases and their relations. In: IEEE 13th International Conference on Data Mining Workshops, Dallas, TX, USA, pp. 937–944 (2013)Google Scholar
  20. 20.
    Klinger, R., Cimiano, P.: The usage review corpus for fine grained multi lingual opinion analysis. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation, Reykjavik, Iceland, pp. 2211–2218 (2014)Google Scholar
  21. 21.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning. Morgan Kaufmann, Williamstown, MA, USA (2001)Google Scholar
  22. 22.
    Liang, T.P., Li, X., Yang, C.T., Wang, M.: What in consumer reviews affects the sales of mobile apps: a multifacet sentiment analysis approach. Int. J. Electron. Commer. 20(2), 236–260 (2015)CrossRefGoogle Scholar
  23. 23.
    Liu, B.: Sentiment analysis: mining opinions, sentiments, and emotions. Cambridge University Press (2015)Google Scholar
  24. 24.
    Maalej, W., Nabil, H.: Bug report, feature request, or simply praise? On automatically classifying app reviews. In: Proceedings of the IEEE 23rd International Requirements Engineering Conference, pp. 116–125. IEEE, Karlskrona, Sweden (2015)Google Scholar
  25. 25.
    McCallum, A.: Mallet: a machine learning for language toolkit (2002). Accessed 08 Feb 2017
  26. 26.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of Workshop at International Conference on Learning Representations, Scottsdale, AZ, USA (2013)Google Scholar
  27. 27.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, South Lake Tahoe, NV, USA, pp. 3111–3119 (2013)Google Scholar
  28. 28.
    Pagano, D., Maalej, W.: User feedback in the appstore: an empirical study. In: Proceedings of the 2013 21st IEEE International Requirements Engineering Conference, pp. 125–134. IEEE, Rio de Janeiro (2013)Google Scholar
  29. 29.
    Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  30. 30.
    Sänger, M., Leser, U., Kemmerer, S., Adolphs, P., Klinger, R.: SCARE - the sentiment corpus of app. reviews with fine-grained annotations in german. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). Portorož, Slovenia (2016)Google Scholar
  31. 31.
    Täckström, O., McDonald, R.: Discovering fine-grained sentiment with latent variable structured prediction models. In: Clough, P., Foley, C., Gurrin, C., Jones, G.J.F., Kraaij, W., Lee, H., Mudoch, V. (eds.) ECIR 2011. LNCS, vol. 6611, pp. 368–374. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-20161-5_37 CrossRefGoogle Scholar
  32. 32.
    Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., Qin, B.: Learning sentiment-specific word embedding for twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, MD, USA, pp. 1555–1565 (2014)Google Scholar
  33. 33.
    Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., Kappas, A.: Sentiment strength detection in short informal text. J. Am. Soc. Inf. Sci. Technol. 61(12), 2544–2558 (2010)CrossRefGoogle Scholar
  34. 34.
    Titov, I., McDonald, R.: A joint model of text and aspect ratings for sentiment summarization. In: Proceedings of 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Columbus, OH, USA (2008)Google Scholar
  35. 35.
    Turian, J., Ratinov, L., Bengio, Y.: Word representations: a simple and general method for semi-supervised learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, pp. 384–394 (2010)Google Scholar
  36. 36.
    Vinodhini, G., Chandrasekaran, R.: Sentiment analysis and opinion mining: a survey. Int. J. 2(6), 282–292 (2012)Google Scholar
  37. 37.
    Vu, P.M., Nguyen, T.T., Pham, H.V., Nguyen, T.T.: Mining user opinions in mobile app reviews: a keyword-based approach. In: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering, pp. 749–759. IEEE, Lincoln, NE, USA (2015)Google Scholar
  38. 38.
    Yu, M., Zhao, T., Dong, D., Tian, H., Yu, D.: Compound embedding features for semi-supervised learning. In: Proceedings of Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Atlanta, GA, USA, pp. 563–568 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceHumboldt-Universität zu BerlinBerlinGermany
  2. 2.Institut für Maschinelle SprachverarbeitungUniversität StuttgartStuttgartGermany

Personalised recommendations