A Syntactic Textual Entailment System Based on Dependency Parser

  • Partha Pakray
  • Alexander Gelbukh
  • Sivaji Bandyopadhyay
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6008)

Abstract

The development of a syntactic textual entailment system that compares the dependency relations in both the text and the hypothesis has been reported. The Stanford Dependency Parser has been run on the 2-way RTE-3 development set and the dependency relations obtained for a text and hypothesis pair has been compared. Some of the important comparisons are: subject-subject comparison, subject-verb comparison, object-verb comparison and cross subject-verb comparison. Corresponding verbs are further compared using the WordNet. Each of the matches is assigned some weight learnt from the development corpus. A threshold has been set on the fraction of matching hypothesis relations based on the development set. The threshold score has been applied on the RTE-4 gold standard test set using the same methods of dependency parsing followed by comparisons. Evaluation scores obtained on the test set show 54.75% precision and 53% recall for YES decisions and 54.45% precision and 56.2% recall for NO decisions.

Keywords

Textual Entailment Dependency parsing Dependency Relations RTE-3 development set RTE-4 gold standard test set 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Dagan, I., Glickman, O., Magnini, B.: The PASCAL Recognising Textual Entailment Challenge. In: Proceedings of the First PASCAL Recognizing Textual Entailment Workshop (2005)Google Scholar
  2. 2.
    Bar-Haim, R., Dagan, I., Dolan, B., Ferro, L., Giampiccolo, D., Magnini, B., Szpektor, I.: The Second PASCAL Recognising Textual Entailment Challenge. In: Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, Venice, Italy (2006)Google Scholar
  3. 3.
    Voorhees, E.M., Harman, D.: Overview of the seventh text retrieval conference. In: Proceedings of the Seventh Text REtrieval Conference (TREC-7). NIST Special Publication (1999)Google Scholar
  4. 4.
    Giampiccolo, D., Dang, H.T., Magnini, B., Dagan, I., Cabrio, E.: The Fourth PASCAL Recognizing Textual Entailment Challenge. In: TAC 2008 Proceedings (2008), http://www.nist.gov/tac/publications/2008/papers.html
  5. 5.
    Vanderwende, L., Coughlin, D., Dolan, B.: What syntax can contribute in entailment task. In: Proceedings of the First PASCAL Recognizing Textual Entailment Workshop, pp. 13–16 (2005)Google Scholar
  6. 6.
    Herrera, J., Peñas, A., Verdejo, F.: Textual Entailment Recognition Based on Dependency Analysis and WordNet. In: Proceedings of the First PASCAL Recognizing Textual Entailment Workshop, pp. 21–24 (2005)Google Scholar
  7. 7.
    Kouylekov, M., Magnini, B.: Tree Edit Distance for Recognizing Textual Entailment: Estimating the Cost of Insertion. In: Proc. of the PASCAL RTE-2 Challenge, pp. 68–73 (2006)Google Scholar
  8. 8.
    Blake, C.: The Role of Sentence Structure in Recognizing Textual Entailment. In: Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 101–106 (2007)Google Scholar
  9. 9.
    Wang, R., Neumann, G.: Recognizing Textual Entailment Using Sentence Similarity based on Dependency Tree Skeletons. In: Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 36–41 (2007)Google Scholar
  10. 10.
    Varma, V., Krishna, S., Garapati, H., Reddy, K., Pingali, P., Ganesh, S., Gopisetty, H., Bysani, P., Katragadda, R., Sarvabhotla, K., Reddy, V.B., Bharadwaj, R.: Recognizing Textual Entailment (RTE) Track. In: Text analysis conference 2008 Proceedings (2008)Google Scholar
  11. 11.
    Klein, D., Manning, C.D.: Accurate unlexicalized parsing. In: ACL 2003, pp. 423–430 (2003)Google Scholar
  12. 12.
    Fellbaum, C.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)MATHGoogle Scholar
  13. 13.

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Partha Pakray
    • 1
  • Alexander Gelbukh
    • 2
  • Sivaji Bandyopadhyay
    • 1
  1. 1.Computer Science and Engineering DepartmentJadavpur UniversityKolkataIndia
  2. 2.Center for Computing ResearchNational Polytechnic InstituteMexico CityMexico

Personalised recommendations