OpinionBlocks: A Crowd-Powered, Self-improving Interactive Visual Analytic System for Understanding Opinion Text

  • Mengdie Hu
  • Huahai Yang
  • Michelle X. Zhou
  • Liang Gou
  • Yunyao Li
  • Eben Haber
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8118)


Millions of people rely on online opinions to make their decisions. To better help people glean insights from massive amounts of opinions, we present the design, implementation, and evaluation of OpinionBlocks, a novel interactive visual text analytic system. Our system offers two unique features. First, it automatically creates a fine-grained, aspect-based visual summary of opinions, which provides users with insights at multiple levels. Second, it solicits and supports user interactions to rectify text-analytic errors, which helps improve the overall system quality. Through two crowd-sourced studies on Amazon Mechanical Turk involving 101 users, OpinionBlocks demonstrates its effectiveness in helping users perform real-world opinion analysis tasks. Moreover, our studies show that the crowd is willing to correct analytic errors, and the corrections help improve user task completion time significantly.


Text analytics text visualization self-improving crowd-sourcing 


  1. 1.
    Sentiment lexicon, (accessed: September 09, 2012)
  2. 2.
    Stanford log-linear part-of-speech tagger, (accessed: September 09, 2012)
  3. 3.
    Amershi, S., Fogarty, J., Weld, D.: Regroup: interactive machine learning for on-demand group creation in social networks. In: CHI, pp. 21–30 (2012)Google Scholar
  4. 4.
    Amershi, S., Lee, B., Kapoor, A., Mahajan, R., Christian, B.: Cuet: human-guided fast and accurate network alarm triage. In: CHI, pp. 157–166 (2011)Google Scholar
  5. 5.
    Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K.: Soylent: a word processor with a crowd inside. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, UIST 2010, pp. 313–322. ACM, New York (2010)CrossRefGoogle Scholar
  6. 6.
    Brody, S., Elhadad, E.: An unsupervised aspect-sentiment model for online reviews. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT 2010 (2010)Google Scholar
  7. 7.
    Carenini, G., Rizoli, L.: A multimedia interface for facilitating comparisons of opinions. In: IUI, pp. 325–334 (2012)Google Scholar
  8. 8.
    Carlier, A., Ravindra, G., Charvillat, V., W., O.: Combining content-based analysis and crowdsourcing to improve user interaction with zoomable video. In: ACM MM, pp. 43–52 (2011)Google Scholar
  9. 9.
    Faridani, S., Bitton, E., Ryokai, K., Goldberg, K.: Opinion space: a scalable tool for browsing online comments. In: CHI 2010, pp. 1175–1184. ACM, New York (2010)Google Scholar
  10. 10.
    Fieller, E., Hartley, H., Pearson, E.: Tests for rank correlation coefficients. I. Biometrika 44(3/4), 470–481 (1957)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Gilbert, E., Karahalios, K.: Understanding deja reviewers. In: Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, CSCW 2010, pp. 225–228. ACM, New York (2010)CrossRefGoogle Scholar
  12. 12.
    Hu, M., Liu, B.: Mining and summarizing customer reviews. In: KDD, pp. 168–177 (2004)Google Scholar
  13. 13.
    Huang, J., Etzioni, O., Zettlemoyer, L., Clark, K., Lee, C.: Revminer: An extractive interface for navigating reviews on a smartphone. In: UIST (2012)Google Scholar
  14. 14.
    Jo, Y., Oh, A.: Aspect and sentiment unification model for online review analysis. In: Proceedings of ACM Conference on Web Search and Data Mining, WSDM 2011 (2011)Google Scholar
  15. 15.
    Lee, Y.E., Benbasat, I.: Interaction design for mobile product recommendation agents: Supporting users’ decisions in retail stores. ACM Trans. Comput.-Hum. Interact. 17(4), 17:1–17:32 (2010)Google Scholar
  16. 16.
    Liu, B., Hu, M., Cheng, J.: Opinion observer: analyzing and comparing opinions on the web. In: WWW, pp. 342–351 (2005)Google Scholar
  17. 17.
    Liu, B., Zhang, L.: A Survey of Opinion Mining and Sentiment Analysis. In: Mining Text Data. Kluwer Academic Publishers (2012)Google Scholar
  18. 18.
    Liu, B.: Sentiment analysis and opinion mining. In: Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers (2012)Google Scholar
  19. 19.
    Moghaddam, S., Ester, M.: ILDA: interdependent LDA model for learning latent aspects and their ratings from online product reviews. In: Proceedings of the Annual ACM SIGIR International Conference on Research and Development in Information Retrieval, SIGIR 2011 (2011)Google Scholar
  20. 20.
    Mukherjee, A., Bing, L.: Aspect Extraction through Semi-Supervised Modeling. In: Proceedings of 50th Anunal Meeting of Association for Computational Linguistics, ACL 2012 (2012)Google Scholar
  21. 21.
    Nelson, P.: Information and consumer behavior. J. of Political Economy 78(2), 311–329 (1970)CrossRefGoogle Scholar
  22. 22.
  23. 23.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval (2008)Google Scholar
  24. 24.
    Park, D., Lee, J., Han, J.: The effect of online consumer reviews on consumer purchasing intention: The moderating role of involvement. Intl. J. of E-Commerce 11(4), 125–148 (2007)CrossRefGoogle Scholar
  25. 25.
    Patel, K.: Lowering the barrier to applying machine learning. In: Adjunct Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, UIST 2010, pp. 355–358. ACM, New York (2010)CrossRefGoogle Scholar
  26. 26.
    Rohrdantz, C., Hao, M.C., Dayal, U., Haug, L.-E., Keim, D.A.: Feature-based visual sentiment analysis of text document streams. ACM Trans. Intell. Syst. Technol. 3(2), 26:1–26:25 (2012)CrossRefGoogle Scholar
  27. 27.
    Schuler, K.K.: VerbNet: A Broad-Coverage, Comprehensive Verb Lexicon. PhD thesis, University of Pennsylvania (2006)Google Scholar
  28. 28.
    Snow, R., O’Connor, B., Jurafsky, D., Ng, A.: Cheap and fast—but is it good?: evaluating non-expertannotations for natural language task. In: Proc. EMNLP 2008, pp. 254–263 (2008)Google Scholar
  29. 29.
    Thet, T., Na, J., Khoo, C.: Aspect-based sentiment analysis of movie reviews on discussion boards. J. Inf. Sci. 36(6), 823–848 (2010)CrossRefGoogle Scholar
  30. 30.
    Wu, Y., Wei, F., Liu, S., Au, N., Cui, W., Zhou, H., Qu, H.: Opinionseer: Interactive visualization of hotel customer feedback. IEEE Trans. Vis. Comput. Graph. 16(6), 1109–1118 (2010)CrossRefGoogle Scholar
  31. 31.
    Yatani, K., Novati, M., Trusty, A., Truong, K.: Review Spotlight: a user interface for summarizing user-generated reviews using adjective-noun word pairs. In: CHI, pp. 1541–1550 (2011)Google Scholar
  32. 32.
    Zhu, F., Zhang, X.: Impract of online consumer reviews on sales: The moderating role of products and consumer. J. of Marketing 74, 133–148 (2010)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2013

Authors and Affiliations

  • Mengdie Hu
    • 1
  • Huahai Yang
    • 2
  • Michelle X. Zhou
    • 2
  • Liang Gou
    • 2
  • Yunyao Li
    • 2
  • Eben Haber
    • 2
  1. 1.Georgia Institute of TechnologyUSA
  2. 2.IBM Almaden Research CenterUSA

Personalised recommendations