What do people think about this monument? Understanding negative reviews via deep learning, clustering and descriptive rules

  • Ana Valdivia
  • Eugenio Martínez-CámaraEmail author
  • Iti Chaturvedi
  • M. Victoria Luzón
  • Erik Cambria
  • Yew-Soon Ong
  • Francisco Herrera
Original Research


Aspect-based sentiment analysis enables the extraction of fine-grained information, as it connects specific aspects that appear in reviews with a polarity. Although we detect that the information from these algorithms is very accurate at local level, it does not contribute to obtain an overall understanding of reviews. To fill this gap, we propose a methodology to portray opinions through the most relevant associations between aspects and polarities. Our methodology combines three off-the-shelf algorithms: (1) deep learning for extracting aspects, (2) clustering for joining together similar aspects, and (3) subgroup discovery for obtaining descriptive rules that summarize the polarity information of set of reviews. Concretely, we aim at depicting negative opinions from three cultural monuments in order to detect those features that need to be improved. Experimental results show that our approach clearly gives an overview of negative aspects, therefore it will be able to attain a better comprehension of opinions.


Sentiment analysis Deep learning Aspect clustering Subgroup discovery 



We would like to thank the reviewers for their thoughtful comments and efforts towards improving our manuscript. This research work was supported by the TIN2017-89517-P project from the Spanish Government. Eugenio Martínez-Cámara was supported by the Juan de la Cierva Formación Programme (FJCI-2016-28353) also from the Spanish Government.


  1. Agrawal R, Mannila H, Srikant R, Toivonen H, Verkamo AI (1996) Fast discovery of association rules. Adv Knowl Discov Data Min 12(1):307–328Google Scholar
  2. Atzmueller M (2015) Subgroup discovery. Wiley Interdiscip Rev Data Min Knowl Discov 5(1):35–49. CrossRefGoogle Scholar
  3. Bay SD, Pazzani MJ (2001) Detecting group differences: mining contrast sets. Data Min Knowl Discov 5(3):213–246CrossRefGoogle Scholar
  4. Cambria E (2016) Affective computing and sentiment analysis. IEEE Intell Syst 31(2):102–107CrossRefGoogle Scholar
  5. Cambria E, Poria S, Hazarika D, Kwok K (2018) SenticNet 5: discovering conceptual primitives for sentiment analysis by means of context embeddings. In: AAAI, pp 1795–1802Google Scholar
  6. Carmona C, del Jesus M, Herrera F (2018) A unifying analysis for the supervised descriptive rule discovery via the weighted relative accuracy. Knowl Based Syst 139:89–100CrossRefGoogle Scholar
  7. Carmona CJ, González P, Del Jesus M, Navío-Acosta M, Jiménez-Trevino L (2011) Evolutionary fuzzy rule extraction for subgroup discovery in a psychiatric emergency department. Soft Comput 15(12):2435–2448CrossRefGoogle Scholar
  8. Carmona CJ, González P, del Jesus MJ, Herrera F (2014) Overview on evolutionary subgroup discovery: analysis of the suitability and potential of the search performed by evolutionary algorithms. Wiley Int Rev Data Min Knowl Discov 4(2):87–103CrossRefGoogle Scholar
  9. Chaturvedi I, Ragusa E, Gastaldo P, Zunino R, Cambria E (2018) Bayesian network based extreme learning machine for subjectivity detection. J Frankl Inst 355(4):1780–1797MathSciNetCrossRefGoogle Scholar
  10. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537zbMATHGoogle Scholar
  11. De Boom C, Van Canneyt S, Demeester T, Dhoedt B (2016) Representation learning for very short texts using weighted word embedding aggregation. Pattern Recognit Lett 80:150–156CrossRefGoogle Scholar
  12. Dong G, Li J (1999) Efficient mining of emerging patterns: discovering trends and differences. In: KDD, ACM, pp 43–52Google Scholar
  13. Fan H, Ramamohanarao K (2003) A bayesian approach to use emerging patterns for classification. In: ADC, Australian Computer Society, Inc., pp 39–48Google Scholar
  14. García-Vico A, Carmona C, Martín D, García-Borroto M, del Jesus M (2017) An overview of emerging pattern mining in supervised descriptive rule discovery: taxonomy, empirical study, trends, and prospects. Wiley Interdiscip Rev Data Min Knowl Discov 8(1):e1231CrossRefGoogle Scholar
  15. Hai Z, Chang K, Kim Jj (2011) Implicit feature identification via co-occurrence association rule mining. In: CICLING, Springer, pp 393–404CrossRefGoogle Scholar
  16. Herrera F, Carmona CJ, González P, Del Jesus MJ (2011) An overview on subgroup discovery: foundations and applications. Knowl Inform Syst 29(3):495–525CrossRefGoogle Scholar
  17. Hu M, Liu B (2004) Mining and summarizing customer reviews. In: KDD, ACM, pp 168–177Google Scholar
  18. Jakob N, Gurevych I (2010) Extracting opinion targets in a single- and cross-domain setting with conditional random fields. In: EMNLP, ACL, pp 1035–1045Google Scholar
  19. Jovanoski V, Lavrač N (2001) Classification rule learning with apriori-c. In: EPIA, Springer, pp 44–51CrossRefGoogle Scholar
  20. Kasper W, Vela M (2011) Sentiment analysis for hotel reviews. Comput Linguist Appl Conf 231527:45–52Google Scholar
  21. Kavšek B, Lavrač N (2006) Apriori-sd: adapting association rule learning to subgroup discovery. Appl Artif Intell 20(7):543–583CrossRefGoogle Scholar
  22. Klösgen W (1996) Explora: A multipattern and multistrategy discovery assistant. In: Advances in knowledge discovery and data mining, American Association for Artificial Intelligence, pp 249–271Google Scholar
  23. Lavrač N, Flach P, Zupan B (1999) Rule evaluation measures: a unifying view. In: ILP, Springer, pp 174–185CrossRefGoogle Scholar
  24. Lavrač N, Kavšek B, Flach P, Todorovski L (2004) Subgroup discovery with cn2-sd. J Mach Learn Res 5:153–188MathSciNetGoogle Scholar
  25. Levy O, Goldberg Y (2014) Dependency-based word embeddings. ACL 2:302–308Google Scholar
  26. Li G, Law R, Rong J, Vu HQ (2010) Incorporating both positive and negative association rules into the analysis of outbound tourism in Hong Kong. J Travel Tour Market 27(8):812–828CrossRefGoogle Scholar
  27. Li G, Law R, Vu HQ, Rong J, Zhao XR (2015) Identifying emerging hotel preferences using emerging pattern mining technique. Tour Manage 46:311–321CrossRefGoogle Scholar
  28. Liu B (2015) Sentiment analysis: mining opinions, sentiments, and emotions. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  29. Lu B, Ott M, Cardie C, Tsou BK (2011) Multi-aspect sentiment analysis with topic models. In: 2011 IEEE 11th international conference on data mining workshops (ICDMW), IEEE, pp 81–88Google Scholar
  30. Marrese-Taylor E, Velásquez JD, Bravo-Marquez F, Matsuo Y (2013) Identifying customer preferences about tourism products using an aspect-based opinion mining approach. Proc Comput Sci 22:182–191CrossRefGoogle Scholar
  31. Mihelčić M, Džeroski S, Lavrač N, Šmuc T (2017) A framework for redescription set construction. Expert Syst Appl 68:196–215CrossRefGoogle Scholar
  32. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:13013781
  33. Nguyen HL, Jung JE (2017) Statistical approach for figurative sentiment analysis on social networking services: a case study on twitter. Multimed Tools Appl 76(6):8901–8914CrossRefGoogle Scholar
  34. Novak PK, Lavrač N, Webb GI (2009) Supervised descriptive rule discovery: a unifying survey of contrast set, emerging pattern and subgroup mining. J Mach Learn Res 10:377–403zbMATHGoogle Scholar
  35. Plutchik R (1984) Emotions: a general psychoevolutionary theory. Approaches Emot 1984:197–219Google Scholar
  36. Poria S, Gelbukh A, Cambria E, Das D, Bandyopadhyay S (2012a) Enriching senticnet polarity scores through semi-supervised fuzzy clustering. In: ICDMW, pp 709–716Google Scholar
  37. Poria S, Gelbukh A, Cambria E, Yang P, Hussain A, Durrani T (2012b) Merging senticnet and wordnet-affect emotion lists for sentiment analysis. ICSP 2:1251–1255Google Scholar
  38. Poria S, Cambria E, Ku LW, Gui C, Gelbukh A (2014) A rule-based approach to aspect extraction from product reviews. SocialNLP 2014:28Google Scholar
  39. Poria S, Cambria E, Gelbukh A (2016) Aspect extraction for opinion mining with a deep convolutional neural network. Knowl Based Syst 108:42–49CrossRefGoogle Scholar
  40. Rajagopal D, Cambria E, Olsher D, Kwok K (2013) A graph-based approach to commonsense concept extraction and semantic similarity detection. In: WWW, pp 565–570Google Scholar
  41. Schouten K, Frasincar F (2016) Survey on aspect-level sentiment analysis. IEEE Trans Knowl Data Eng 28(3):813–830CrossRefGoogle Scholar
  42. Thorndike RL (1953) Who belongs in the family? Psychometrika 18(4):267–276CrossRefGoogle Scholar
  43. Toh Z, Wang W (2014) Dlirec: aspect term extraction and term polarity classification system. In: SemEval, pp 235–240Google Scholar
  44. Valdivia A, Luzón MV, Herrera F (2017) Sentiment analysis in tripadvisor. IEEE Intell Syst 32(4):72–77CrossRefGoogle Scholar
  45. Wrobel S (1997) An algorithm for multi-relational discovery of subgroups. In: European symposium on principles of data mining and knowledge discovery, Springer, pp 78–87Google Scholar
  46. Zhao Y, Qin B, Hu S, Liu T (2010) Generalizing syntactic structures for product attribute candidate extraction. In: HLT-NAACL, Association for Computational Linguistics, pp 377–380Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI)University of GranadaGranadaSpain
  2. 2.School of Computer Science and EngineeringNanyang Technological UniversitySingaporeSingapore

Personalised recommendations