Human-Computer Interaction

INTERACT 2015: Human-Computer Interaction – INTERACT 2015 pp 249-264 | Cite as

What Users Prefer and Why: A User Study on Effective Presentation Styles of Opinion Summarization

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9297)


Opinion Summarization research addresses how to help people in making appropriate decisions in an effective way. This paper aims to help users in their decision-making by providing them effective opinion presentation styles. We carried out two phases of experiments to systematically compare usefulness of different types of opinion summarization techniques. In the first crowd-sourced study, we recruited 46 turkers to generate high quality summary information. This first phase generated four styles of summaries: Tag Clouds, Aspect Oriented Sentiments, Paragraph Summary and Group Sample. In the follow-up second phase, 34 participants tested the four styles in a card sorting experiment. Each participant was given 32 cards with 8 per presentation styles and completed the task of grouping the cards into five categories in terms of the usefulness of the cards. Results indicated that participants preferred Aspect Oriented Sentiments the most and Tag cloud the least. Implications and hypotheses are discussed.


Text summarization Consumer decision making User studies User interface design 


  1. 1.
    Ando, R., Boguraev, B., Byrd, R., Neff, M.: Multi-document summarization by visualizing topical content. In: Proceedings of the 2000 NAACL-ANLP Workshop on Automatic Summarization, pp. 79–98 (2000)Google Scholar
  2. 2.
    Bateman, S., Gutwin, C., Nacenta, M.: Seeing things in the clouds: the effect of visual features on tag cloud selections. In: Proceedings of the Nineteenth ACM Conference on Hypertext and Hypermedia, pp. 193–202. ACM (2008)Google Scholar
  3. 3.
    Brooks, C., Montanez, N.: Improved annotation of the blogosphere via autotagging and hierarchical clustering. In: Proceedings of the 15th International Conference on World Wide Web, pp. 625–632 (2006)Google Scholar
  4. 4.
    Carenini, G., Ng, R., Pauls, A.: Multi-document summarization of evaluative text. In: Proceedings of the European Chapter of the Association for Computational Linguistics (EACL), pp. 305–312 (2006)Google Scholar
  5. 5.
    Carenini, G., Ng, R., Pauls, A.: Interactive multimedia summaries of evaluative text. In: Proceedings of IUI 2006 of the Association for Computing Machinery, pp. 1–8 (2006)Google Scholar
  6. 6.
    Chen, C., Ibekwe-SanJuan, F., SanJuan, E., Weaver, C.: Visual analysis of conflicting opinions. In: 2006 IEEE Symposium On Visual Analytics Science And Technology, pp. 59–66 (2006)Google Scholar
  7. 7.
    Duan, D., Qian, W., Pan, S., Shi, L., Lin, C.: VISA: a visual sentiment analysis system. In: Proceedings of VINCI 2012, Hangzhou, China (2012)Google Scholar
  8. 8.
    Erkan, G., Radev, D.: Lexrank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. (JAIR) 22, 457–479 (2004)Google Scholar
  9. 9.
    Hearst, M., Pedersen, J.: Reexamining the cluster hypothesis: scatter/gather on retrieval results. In: Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 76–84 (1996)Google Scholar
  10. 10.
    Hovy, E., Lin, C.: Automated text summarization and the summarist system. In: Proceedings of a Workshop on Held at Baltimore, Maryland, 13–15 October 1998, pp. 197–214 (1998)Google Scholar
  11. 11.
    Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 168–177 (2004)Google Scholar
  12. 12.
    Hu, M., Yang, H., Zhou, M.X., Gou, L., Li, Y., Haber, E.: OpinionBlocks: a crowd-powered, self-improving interactive visual analytic system for understanding opinion text. In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds.) INTERACT 2013, Part II. LNCS, vol. 8118, pp. 116–134. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  13. 13.
    Jo, Y., Oh, A.: Aspect and sentiment unification model for online review analysis. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, pp. 815–824 (2011)Google Scholar
  14. 14.
    Ku, L., Liang, Y., Chen, H.: Opinion extraction, summarization and tracking in news and blog corpora. In: Proceedings of AAAI-2006 Spring Symposium on Computational Approaches to Analyzing Weblogs (2006)Google Scholar
  15. 15.
    Lerman, K., Blair-Goldensohn, S., McDonald, R.: Sentiment summarization: evaluating and learning user preferences. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, pp. 514–522 (2009)Google Scholar
  16. 16.
    Lerman, K., McDonald, R.: Contrastive summarization: an experiment with consumer reviews. In: Proceedings of NAACL HLT of the Association for Computational Linguistics, pp. 113–116 (2009)Google Scholar
  17. 17.
    Leuski, A.: Evaluating document clustering for interactive information retrieval. In: Proceedings of the Tenth International Conference on Information and Knowledge Management, pp. 33–40 (2001)Google Scholar
  18. 18.
    Lu, Y., Zhai, C.: Opinion integration through semi-supervised topic modeling. In: Proceedings of the 17th International Conference on World Wide Web, pp. 121–130 (2008)Google Scholar
  19. 19.
    Lu, Y., Zhai, C., Sundaresan, N.: Rated aspect summarization of short comments. In: Proceedings of the 18th international conference on World wide web, pp. 131–140 (2009)Google Scholar
  20. 20.
    Maña-López, M., De Buenaga, M., Gómez-Hidalgo, J.: Multidocument summarization: an added value to clustering in interactive retrieval. ACM Trans. Inf. Syst. (TOIS) 22(2), 215–241 (2004)CrossRefGoogle Scholar
  21. 21.
    Mani, I., House, D., Klein, G., Hirschman, L., Firmin, T., Sundheim, B.: The tipster summac text summarization evaluation. In: Proceedings of the Ninth Conference on European Chapter of the Association for Computational Linguistics, pp. 77–85 (1999)Google Scholar
  22. 22.
    Mei, Q., Ling, X., Wondra, M., Su, H., Zhai, C.: Topic sentiment mixture: modeling facets and opinions in weblogs. In: Proceedings of the 16th International Conference on World Wide Web, pp. 171–180 (2007)Google Scholar
  23. 23.
    Mukherjee, A., Liu, B. Aspect extraction through semi-supervised modeling. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pp. 339–348 (2012)Google Scholar
  24. 24.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retrieval 2(1–2), 1–135 (2008)CrossRefGoogle Scholar
  25. 25.
    Potthast, M., Becker, S.: Opinion summarization of web comments. In: Gurrin, C., He, Y., Kazai, G., Kruschwitz, U., Little, S., Roelleke, T., Rüger, S., van Rijsbergen, K. (eds.) ECIR 2010. LNCS, vol. 5993, pp. 668–669. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  26. 26.
    Qazi, A., Raj, R.G., Tahir, M., Waheed, M., Khan, S.U.R., Abraham, A.: A preliminary investigation of user perception and behavioral intention for different review types: customers and designers perspective. Sci. World J. 2014, 1–8 (2014)MATHGoogle Scholar
  27. 27.
    Radev, D., McKeown, K.: Generating natural language summaries from multiple on-line sources. Comput. Linguist. 24(3), 470–500 (1998)Google Scholar
  28. 28.
    Salton, G., McGill, M.: Introduction to Modern Information Retrieval. McGraw-Hill, Inc., New York (1986)Google Scholar
  29. 29.
    Witten, I., Frank, E., Hall, M.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2011)Google Scholar
  30. 30.
    Wu, M., Fuller, M., Wilkinson, R.: Using clustering and classification approaches in interactive retrieval. Inf. Process. Manage. 37(3), 459–484 (2001)CrossRefGoogle Scholar
  31. 31.
    Yatani, K., Novati, M., Trusty, A., Truong, K.N.: Review spotlight: a user interface for summarizing user-generated reviews using adjective-noun word pairs. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, CHI 2011, pp. 1541–1550 (2011)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  • Xiaojun Yuan
    • 1
  • Ning Sa
    • 1
  • Grace Begany
    • 1
  • Huahai Yang
    • 2
  1. 1.College of Computing and InformationUniversity at Albany, State University of New YorkAlbanyUSA
  2. 2.Juji, Inc.SaratogaUSA

Personalised recommendations