Organizing Information on the Web through Agreement-Conflict Relation Classification

  • Junta Mizuno
  • Eric Nichols
  • Yotaro Watanabe
  • Kentaro Inui
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7675)


The vast amount of information on the Web makes it difficult for users to comprehensively survey the various viewpoints on topics of interest. To help users cope with this information overload, we have developed an Information Organization System that applies state-of-the-art technology from Recognizing Textual Entailment to automatically detect Web texts that are relevant to natural language queries and organize them into agreeing and conflicting groups. Users are presented with a bird’s-eye-view visualization of the viewpoints on their queries that makes it easier to gain a deeper understanding of an issue. In this paper, we describe the implementation of our Information Organization System and evaluate our system through empirical analysis of the semantic relation recognition system that classifies texts and through a large-scale usability study. The empirical evaluation and usability study both demonstrate the usefulness of our system. User feedback further shows that by exposing our users to differing viewpoints promotes objective thinking and helps to reduce confirmation bias.


natural language processing information organization recognizing textual entailment agreement and conflict relations 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akamine, S., Kawahara, D., Kato, Y., Nakagawa, T., Inui, K., Kurohashi, S., Kidawara, Y.: WISDOM: A Web Information Credibility Analysis Systematic. In: Proc. of the ACL-IJCNLP 2009 Software Demonstrations, pp. 1–4 (2009)Google Scholar
  2. 2.
    Bentivogli, L., Dagan, I., Dang, H.T., Giampiccolo, D., Magini, B.: The Fifth PASCAL Recognizing Textual Entailment Challenge. In: Proc. of Text Analysis Conference, TAC 2009 (2009)Google Scholar
  3. 3.
    Bond, F., Isahara, H., Fujita, S., Uchimoto, K., Kuribayashi, T., Kanzaki, K.: Enhancing the Japanese WordNet. In: Proc. of ALR 2009, pp. 1–8 (2009)Google Scholar
  4. 4.
    Cox, L.A.T., Popken, D.A.: Overcoming Confirmation Bias in Causal Attribution: A Case Study of Antibiotic Resistance Risks. Risk Analysis an Official Publication of the Society for Risk Analysis 28(5), 1155–1172 (2008)CrossRefGoogle Scholar
  5. 5.
    Dagan, I., Glickman, O., Magnini, B.: The PASCAL Recognising Textual Entailment Challenge. In: Quiñonero-Candela, J., Dagan, I., Magnini, B., d’Alché-Buc, F. (eds.) MLCW 2005. LNCS (LNAI), vol. 3944, pp. 177–190. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    de Marneffe, M.-C., Rafferty, A.N., Manning, C.D.: Finding Contradictions in Text. In: Proc. of ACL-HLT 2008, pp. 1039–1047 (2008)Google Scholar
  7. 7.
    Ennals, R., Trushkowsky, B., Agosta, J.M.: Highlighting Disputed Claims on the Web. In: Proc. of WWW 2010, pp. 341–350 (2010)Google Scholar
  8. 8.
    Fallows, D.: Search engine users: Internet searchers are confident, satisfied and trusting – but they are also unaware and naïve. Pew Internet & American Life Project (2005)Google Scholar
  9. 9.
    Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: LIBLINEAR: A Library for Large Linear Classification. Journal of Machine Learning Research 9, 1871–1874 (2008)zbMATHGoogle Scholar
  10. 10.
    Glickman, O., Dagan, I., Koppel, M.: Web Based Textual Entailment. In: Proc. of the First PASCAL Recognizing Textual Entailment Workshop, pp. 33–36 (2005)Google Scholar
  11. 11.
    Hashimoto, C., Torisawa, K., Kuroda, K., Murata, M., Kazama, J.: Large-Scale Verb Entailment Acquisition from the Web. In: Proc. of EMNLP 2009, pp. 1172–1181 (2009)Google Scholar
  12. 12.
    Hickl, A., Williams, J., Bensley, J., Rink, K.R.B., Shi, Y.: Recognizing Textual Entailment with LCC’s Groundhog System. In: Proc. of the Second PASCAL Challenges Workshop, pp. 80–85 (2005)Google Scholar
  13. 13.
    Higashiyama, M., Inui, K., Matsumoto, Y.: Acquiring noun polarity knowledge using selectional preferences. In: Proc. of NLP, pp. 584–587 (2008) (in Japanese)Google Scholar
  14. 14.
    Iftene, A., Moruz, M.-A.: UAIC Participation at RTE5. In: TAC 2009 (2009)Google Scholar
  15. 15.
    Keselman, A., Browne, A.C., Kaufman, D.R.: Consumer Health Information Seeking as Hypothesis Testing. Journal of the American Medical Informatics Association: JAMIA 15(4), 484–495 (2008)CrossRefGoogle Scholar
  16. 16.
    Kobayashi, N., Inui, K., Matsumoto, Y., Tateishi, K., Fukushima, T.: Collecting Evaluative Expressions for Opinion Extraction. Journal of NLP 12(3), 203–222 (2005) (in Japanese)Google Scholar
  17. 17.
    Kudo, T., Matsumoto, Y.: Japanese Dependency Analysis using Cascaded Chunking. In: Proc of CoNLL 2002, pp. 63–69 (2002)Google Scholar
  18. 18.
    MacCartney, B., Grenager, T., de Marneffe, M.-C., Cer, D., Manning, C.D.: Learning to recognize features of valid textual entailments. In: Proc. of HLT-NAACL 2006, pp. 41–48 (2006)Google Scholar
  19. 19.
    Marsi, E., Krahmer, E.: Classification of Semantic Relations by Humans and Machines. In: Proc. of ACL 2005 Workshop on Empirical Modeling of Semantic Equivalence and Entailment, pp. 1–6 (2005)Google Scholar
  20. 20.
    Matsuyoshi, S., Eguchi, M., Sao, C., Murakami, K., Inui, K., Matsumoto, Y.: Annotating Event Mentions in Text with Modality, Focus, and Source Information. In: Proc. of LREC 2010, pp. 1456–1463 (2010)Google Scholar
  21. 21.
    Matsuyoshi, S., Murakami, K., Matsumoto, Y., Inui, K.: A Database of Relations between Predicate Argument Structures for Recognizing Textual Entailment and Contradiction. In: Proc. of ISUC 2008, pp. 366–373 (2008)Google Scholar
  22. 22.
    Murakami, K., Nichols, E., Inui, K., Mizuno, J., Goto, H., Ohki, M., Matsuyoshi, S., Matsumoto, Y.: Automatic Classification of Semantic Relations between Facts and Opinions. In: Proc. of NLPIX, pp. 21–30 (2010)Google Scholar
  23. 23.
    Okazaki, N., Ichi Tsujii, J.: Simple and Efficient Algorithm for Approximate Dictionary Matching. In: Proc. of Coling, pp. 851–859 (2010)Google Scholar
  24. 24.
    Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? Sentiment Classification using Machine Learning Techniques. In: Proc. of EMNLP 2002, pp. 79–86 (2002)Google Scholar
  25. 25.
    Paul, M., Zhai, C., Girju, R.: Summarizing Contrastive Viewpoints in Opinionated Text. In: Proc. of EMNLP 2010, pp. 66–76 (2010)Google Scholar
  26. 26.
    Seki, Y., Evans, D.K., Ku, L.W.: Overview of Multilingual Opinion Analysis Task at NTCIR-7. In: Proc. of the 7th NTCIR Workshop, pp. 185–203 (2008)Google Scholar
  27. 27.
    Shinzato, K., Shibata, T., Kawahara, D., Hashimoto, C., Kurohashi, S.: TSUBAKI: An Open Search Engine Infrastructure for Developing New Information Access Methodology. In: Proc. of IJCNLP 2008, pp. 89–196 (2008)Google Scholar
  28. 28.
    Sumida, A., Yoshinaga, N., Torisawa, K.: Boosting Precision and Recall of Hyponymy Relation Acquisition from Hierarchical Layouts in Wikipedia. In: Proc. of LREC 2008, pp. 2462–2469 (2008)Google Scholar
  29. 29.
    Szpektor, I., Shnarch, E., Dagan, I.: Instance-based Evaluation of Entailment Rule Acquisition. In: Proc. of ACL 2007, pp. 456–463 (2007)Google Scholar
  30. 30.
    Turney, P.: Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews. In: Proc. of ACL 2002, pp. 417–424 (2002)Google Scholar
  31. 31.
    Jijkoun, V., De Rijke, M.: Recognizing Textual Entailment Using Lexical Similarity. In: Proc. of the First PASCAL Challenges Workshop, pp. 73–76 (2005)Google Scholar
  32. 32.
    Watanabe, Y., Asahara, M., Matsumoto, Y.: A Structured Model for Joint Learning of Argument Roles and Predicate Senses. In: Proc. of the ACL 2010 Conference Short Papers, pp. 98–102 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Junta Mizuno
    • 1
  • Eric Nichols
    • 1
  • Yotaro Watanabe
    • 1
  • Kentaro Inui
    • 1
  1. 1.Tohoku UniversityAobakuJapan

Personalised recommendations