Journal of Intelligent Information Systems

, Volume 39, Issue 2, pp 375–398

Challenges and solutions in the opinion summarization of user-generated content

  • Alexandra Balahur
  • Mijail Kabadjov
  • Josef Steinberger
  • Ralf Steinberger
  • Andrés Montoyo
Article

Abstract

The present is marked by the influence of the Social Web on societies and people worldwide. In this context, users generate large amounts of data, especially containing opinion, which has been proven useful for many real-world applications. In order to extract knowledge from user-generated content, automatic methods must be developed. In this paper, we present different approaches to multi-document summarization of opinion from blogs and reviews. We apply these approaches to: (a) identify positive and negative opinions in blog threads in order to produce a list of arguments in favor and against a given topic and (b) summarize the opinion expressed in reviews. Subsequently, we evaluate the proposed methods on two distinct datasets and analyze the quality of the obtained results, as well as discuss the errors produced. Although much remains to be done, the approaches we propose obtain encouraging results and point to clear directions in which further improvements can be made.

Keywords

Opinion mining Blog threads User-generated content Sentiment analysis Opinion summarization 

References

  1. Akkaya, C., Wiebe, J., & Mihalcea, R. (2009). Subjectivity word sense disambiguation. In Proceedings of the 2009 conference on empirical methods in natural language processing: Volume 1. EMNLP ’09 (Vol. 1. pp. 190–199). Stroudsburg, PA, USA: Association for Computational Linguistics. http://dl.acm.org/citation.cfm?id=1699510.1699535.CrossRefGoogle Scholar
  2. Balahur, A., Boldrini, E., Montoyo, A., & Martínez-Barco, P. (2009a). Cross-topic opinion mining for real-time human-computer interaction. In Proceedings of ICEIS 2009 conference.Google Scholar
  3. Balahur, A., Kabadjov, M., Steinberger, J., Steinberger, R., & Montoyo, A. (2009b). Summarizing opinions in blog threads. In Proceedings of the 23rd pacific asia conference on language, information and computation (PACLIC) (pp. 606–613).Google Scholar
  4. Balahur, A., Lloret, E., Boldrini, E., Montoyo, A., Palomar, M., & Martínez-Barco, P. (2009c). Summarizing threads in blogs using opinion polarity. In Proceeding of the workshop on events in emerging text types at RANLP, Borovetz, Bulgaria.Google Scholar
  5. Balahur, A., Lloret, E., Ferrández, O., Montoyo, A., Palomar, M., & Muñoz, R. (2008). The dlsiuaes team’s participation in the tac 2008 tracks. In Proceedings of the text analysis conference (TAC) 2008. National Institute of Standards and Technology (NIST).Google Scholar
  6. Balahur, A., Steinberger, R., Kabadjov, M., Zavarella, V., van der Goot, E., Halkia, M., et al. (2010). Sentiment analysis in the news. In Proceedings of LREC2010.Google Scholar
  7. Balahur, A., Steinberger, R., van der Goot, E., Pouliquen, B., & Kabadjov, M. (2009). Opinion mining from newspaper quotations. In Proceedings of the workshop on intelligent analysis and processing of web news content at the IEEE/WIC/ACM international conferences on web intelligence and intelligent agent technology (WI-IAT).Google Scholar
  8. Beineke, P., Hastie, T., Manning, C., & Vaithyanathan, S. (2004). An exploration of sentiment summarization. In J. G. Shanahan, J. Wiebe, & Y. Qu (Eds.), Proceedings of the AAAI spring symposium on exploring attitude and affect in text: Theories and applications, Stanford, US. http://nlp.stanford.edu/~manning/papers/rotup.pdf.
  9. Bossard, A., Généreux, M., & Poibeau, T. (2008). Description of the LIPN systems at TAC 2008: Summarizing information and opinions. In Proceedings of the text analysis conference (TAC) 2008. National Institute of Standards and Technology (NIST).Google Scholar
  10. Cerini, S., Compagnoni, V., Demontis, A., Formentelli, M., & Gandini, G. (2007). Micro-WNOp: A gold standard for the evaluation of automatically compiled lexical resources for opinion mining. In A. Sansò (Ed.), Language resources and linguistic theory: Typology, second language acquisition, english linguistics, Franco Angeli, Milano, IT.Google Scholar
  11. Conroy, J., & Schlesinger, S. (2008). Classy at tac 2008 metrics. In Proceedings of the text analysis conference (TAC) 2008. National Institute of Standards and Technology (NIST).Google Scholar
  12. Cruz, F., Troyani, J., Ortega, J., & Enríquez, F. (2008). The Italica system at tac 2008 opinion summarization task. In Proceedings of the text analysis conference (TAC) 2008. National Institute of Standards and Technology (NIST).Google Scholar
  13. Erkan, G., & Radev, D. R. (2004). LexRank: Graph-based centrality as salience in text summarization. Journal of Artificial Intelligence Research (JAIR), 22, 457–479.Google Scholar
  14. Esuli, A., & Sebastiani, F. (2006). SentiWordNet: A publicly available resource for opinion mining. In Proceedings of the 6th international conference on language resources and evaluation, Italy.Google Scholar
  15. Gong, Y., & Liu, X. (2002). Generic text summarization using relevance measure and latent semantic analysis. In Proceedings of ACM SIGIR, New Orleans, US.Google Scholar
  16. He, T., Chen, J., Gui, Z., & Li, F. (2008). CCNU at TAC 2008: Proceeding on using semantic method for automated summarization yield. In Proceedings of the text analysis conference (TAC) 2008. National Institute of Standards and Technology (NIST).Google Scholar
  17. Hovy, E. H. (2005). Automated text summarization. In R. Mitkov (Ed.), The Oxford handbook of computational linguistics (pp. 583–598). Oxford, UK: Oxford University Press.Google Scholar
  18. Kabadjov, M., Balahur, A., & Boldrini, E. (2009). Sentiment intensity: Is it a good summary indicator? In Proceedings of the 4th language and technology conference (LTC) (pp. 380–384).Google Scholar
  19. Kabadjov, M. A., Steinberger, J., Pouliquen, B., Steinberger, R., & Poesio, M. (2009). Multilingual statistical news summarisation: Preliminary experiments with English. In Proceedings of the workshop on intelligent analysis and processing of web news content at the IEEE/WIC/ACM international conferences on web intelligence and intelligent agent technology (WI-IAT).Google Scholar
  20. Kupiec, J., Pedersen, J., & Chen, F. (1995). A trainable document summarizer. In Proceedings of the 18th annual international ACM SIGIR conference on research and development in information retrieval, Seattle, Washington (pp. 68–73).Google Scholar
  21. Lerman, K., Blair-Goldensohn, S., & McDonald, R. (2009). Sentiment summarization: Evaluating and learning user preferences. In Proceedings of the 12th conference of the European chapter of the ACL (EACL 2009) (pp. 514–522). Athens, Greece: Association for Computational Linguistics.Google Scholar
  22. Lerman, K., & McDonald, R. (2009). Contrastive summarization: An experiment with consumer reviews. In Proceedings of human language technologies: The 2009 annual conference of the North American chapter of the association for computational linguistics, companion volume: Short papers. (pp. 113–116). Boulder, Colorado: Association for Computational Linguistics.Google Scholar
  23. Lin, C. Y. (2004). ROUGE: A package for automatic evaluation of summaries. In Proceedings of the workshop on text summarization branches out, Barcelona, Spain.Google Scholar
  24. Lin, C.Y., & Hovy, E. (2003). Automatic evaluation of summaries using n-gram co-occurrence statistics. In Proceedings of HLT-NAACL, Edmonton, Canada.Google Scholar
  25. Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2), 1–135.Google Scholar
  26. Pouliquen, B., Kimler, M., Steinberger, R., Ignat, C., Oellinger, T., Blackler, K., et al. (2006). Geocoding multilingual texts: Recognition, disambiguation and visualisation. In Proceedings of the 5th international conference on language resources and evaluation (LREC 2006), Genoa, Italy (pp. 53–58).Google Scholar
  27. Pouliquen, B., & Steinberger, R. (2009). Automatic construction of multilingual name dictionaries. In C. Goutte, N. Cancedda, M. Dymetman, & G. Foster (Eds.), Learning machine translation. MIT Press, NIPS series.Google Scholar
  28. Riloff, E., Wiebe, J., & Phillips, W. (2005). Exploiting subjectivity classification to improve information extraction. In Proceedings of the 20th conference of the association for the advancement of artificial intelligence (AAAI).Google Scholar
  29. Saggion, H., & Funk, A. (2010). Interpreting SentiWordNet for opinion classification. In Proceedings of LREC 2010.Google Scholar
  30. Saggion, H., Lloret, E., & Palomar, M. (2010). Using text summaries for predicting rating scales. In Proceedings of the 1st workshop on subjectivity and sentiment analysis (WASSA 2010).Google Scholar
  31. Steinberger, J., & Jez̆ek, K. (2004). Text summarization and singular value decomposition. In Proceedings of the 3rd ADVIS conference, Izmir, Turkey.Google Scholar
  32. Steinberger, J., & Jez̆ek, K. (2009). Update summarization based on novel topic distribution. In Proceedings of the 9th ACM DocEng, Munich, Germany.Google Scholar
  33. Steinberger, J., Kabadjov, M., Pouliquen, B., Steinberger, R., & Poesio, M. (2009a). WB-JRC-UT’s participation in TAC 2009: Update summarization and AESOP tasks. In National Institute of Standards and Technology (Ed.), Proceedings of the text analysis conference, Gaithersburg, MD.Google Scholar
  34. Steinberger, J., Lenkova, P., Ebrahim, M., Ehrman, M., Hurriyetoglu, A., Kabadjov, M., et al. (2011). Creating sentiment dictionaries via triangulation. In Proceedings of the 2nd workshop on computational approaches to subjectivity and sentiment analysis (WASSA 2.011) (pp. 28–36). Portland, Oregon: Association for Computational Linguistics. http://www.aclweb.org/anthology/W11-1704.Google Scholar
  35. Steinberger, J., Poesio, M., Kabadjov, M. A., & Jez̆ek, K. (2007). Two uses of anaphora resolution in summarization. Information Processing and Management, 43(6), 1663–1680. Special Issue on Text Summarisation (Donna Harman, ed.).Google Scholar
  36. Steinberger, R., & Pouliquen, B. (2009). Cross-lingual named entity recognition. In Named entities—recognition, classification and use, Benjamins current topics (Vol. 19). John Benjamins Publishing Company. ISBN 978–90–272–8922.Google Scholar
  37. Steinberger, R., Pouliquen, B., & Ignat, C. (2009b). Using language-independent rules to achieve high multilinguality in text mining. In F. Fogelman-Soulié, D. Perrotta, J. Piskorski, & R. Steinberger (Eds.), Mining massive data sets for security. Amsterdam, Holland: IOS-Press.Google Scholar
  38. Steinberger, R., Pouliquen, B., & Van der Goot, E. (2009c). An introduction to the Europe media monitor family of applications. In Information access in a multilingual world—proceedings of the SIGIR 2009 workshop (SIGIR–CLIR 2009) (Vol. 19).Google Scholar
  39. Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1966). The general inquirer: A computer approach to content analysis (Vol. 8). MIT Press. http://mitpress.mit.edu/catalog/item/default.asp?tid=5144&ttype=2.
  40. Stoyanov, V., & Cardie, C. (2006). Toward opinion summarization: Linking the sources. In Proceedings of the COLING-ACL workshop on sentiment and subjectivity in text. Sydney, Australia: Association for Computational Linguistics.Google Scholar
  41. Strapparava, C., & Valitutti, A. (2004). WordNet-affect: An affective extension of wordnet. In Proceedings of the 4th international conference on language resources and evaluation, Lisbon, Portugal (pp. 1083–1086).Google Scholar
  42. Titov, I., & McDonald, R. (2008). A joint model of text and aspect ratings for sentiment summarization. In Proceedings of ACL-08: HLT, Columbus, Ohio (pp. 308–316).Google Scholar
  43. Turney, P. D., & Littman, M. L. (2003). Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems (TOIS), 21(4), 315–346.CrossRefGoogle Scholar
  44. Varma, V., Pingali, P., Katragadda, R., Krisha, S., Ganesh, S., Sarvabhotla, K., et al. (2008). IIT hyderabad at TAC 2008. In Proceedings of the text analysis conference (TAC) 2008. National Institute of Standards and Technology (NIST).Google Scholar
  45. Wiegand, M., Balahur, A., Roth, B., Klakow, D., & Montoyo, A. (2010). A survey on the role of negation in sentiment analysis. In Proceedings of the workshop on negation and speculation in natural language processing (pp. 60–68). NeSp-NLP ’10, Stroudsburg, PA, USA: Association for Computational Linguistics. http://dl.acm.org/citation.cfm?id=1858959.1858970.
  46. Wiegand, M., & Klakow, D. (2010). Convolution kernels for opinion holder extraction. In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics. HLT ’10 (pp. 795–803). Stroudsburg, PA, USA: Association for Computational Linguistics. http://dl.acm.org/citation.cfm?id=1857999.1858120.Google Scholar
  47. Zhuang, L., Jing, F., & Zhu, X. Y. (2006). Movie review mining and summarization. In CIKM ’06: Proceedings of the 15th ACM international conference on Information and knowledge management (pp. 43–50).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Alexandra Balahur
    • 1
  • Mijail Kabadjov
    • 1
  • Josef Steinberger
    • 1
  • Ralf Steinberger
    • 1
  • Andrés Montoyo
    • 2
  1. 1.European Commission Joint Research CentreIspraItaly
  2. 2.DLSIUniversity of AlicanteAlicanteSpain

Personalised recommendations