Recognizing Textual Entailment: Is Word Similarity Enough?

  • Valentin Jijkoun
  • Maarten de Rijke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3944)


We describe the system we used at the PASCAL-2005 Recognizing Textual Entailment Challenge. Our method for recognizing entailment is based on calculating “directed” sentence similarity: checking the directed “semantic” word overlap between the text and the hypothesis. We use frequency-based term weighting in combination with two different word similarity measures.

Although one version of the system shows significant improvement over randomly guessing decisions (with an accuracy score of 57.3), we show that this is only due to a subset of the data that can be equally well handled by simple word overlap. Furthermore, we give an in-depth analysis of the system and the data of the challenge.


Reading Comprehension Machine Translation Optimal Threshold Accuracy Score Development Corpus 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press, New York (1999)Google Scholar
  2. Bayer, S., Burger, J., Ferro, L., Henderson, J., Yeh, A.: MITRE’s submission to the EU Pascal RTE challenge. In: Dagan, et al (2005b)Google Scholar
  3. Budanitsky, A., Hirst, G.: Semantic distance in wordnet: An experimental, application-oriented evaluation of five measures. In: Workshop on WordNet and Other Lexical Resources, Second meeting of the North American Chapter of the Association for Computational Linguistics (2001)Google Scholar
  4. Dagan, I., Glickman, O., Magnini, B.: The PASCAL Recognizing Textual Entailment Challenge. In: Dagan, et al (2005b)Google Scholar
  5. Dagan, I., Glickman, O., Magnini, B. (eds.): PASCAL. Proceedings of the first challenge workshop Recognizing Textual Entailment (2005b)Google Scholar
  6. Glickman, O., Dagan, I., Koppel, M.: Web based probabilistic textual entailment. In: Dagan, et al (2005b)Google Scholar
  7. Hirst, G., St-Onge, D.: Lexical chains as representation of context for the detection and correction of malapropisms. In: Christiane, F. (ed.) WordNet: An electronic lexical database. The MIT Press, Cambridge (1998)Google Scholar
  8. Jijkoun, V., de Rijke, M.: Recognizing textual entailment using lexical similarity. In: Dagan, et al (2005b)Google Scholar
  9. Lin, D.: An information-theoretic definition of similarity. In: Proceedings of International Conference on Machine Learning (1998)Google Scholar
  10. Monz, C., de Rijke, M.: Light-weight entailment checking for computational semantics. In: Proceedings of the Workshop on Inference in Computational Semantics, ICoS-3 (2001)Google Scholar
  11. Pascal. Pascal Recognising Textual Entailment Challenge (2005),

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Valentin Jijkoun
    • 1
  • Maarten de Rijke
    • 1
  1. 1.Informatics InstituteUniversity of AmsterdamAmsterdamThe Netherlands

Personalised recommendations