Advertisement

Using Bleu-like Algorithms for the Automatic Recognition of Entailment

  • Diana Pérez
  • Enrique Alfonseca
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3944)

Abstract

The Bleu algorithm has been used in many different fields. Another possible application is the automatic recognition of textual entailment. Bleu works at the lexical level, by comparing a candidate text with several reference texts in order to calculate how close the candidate text is to the references. In this case, the candidate is the text part of the entailment and the hypothesis is the unique reference. The algorithm achieves an accuracy of around 50%. Moreover, in this paper we explore the application of Bleu-like algorithms, finding that they can reach an accuracy of around 56%, which proves its possible use as a baseline for the task of recognizing entailment.

Keywords

Average Precision Word Sense Disambiguation Automatic Recognition Lexical Level Reference Text 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bayer, S., Burger, J., Ferro, L., Henderson, J., Yeh, A.: Mitre’s submissions to the eu pascal rte challenge. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  2. 2.
    Salvo-Braz, R., Girju, R., Punyakanok, V., Roth, D., Sammons, M.: An inference model for semantic entailment in natural language. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  3. 3.
    Raina, R., Haghighi, A., Cox, C., Finkel, J., Michels, J., Toutanova, K., MacCartney, B., Marneffe, M., Manning, C., Ng, A.Y.: Robust textual inference using diverse knowledge sources. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  4. 4.
    Dagan, I., Glickman, O., Magnini, B.: The pascal recognising textual entailment challenge. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  5. 5.
    Glickman, O., Dagan, I., Koppel, M.: Web based probabilistic textual entailment. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  6. 6.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.: Bleu: a method for automatic evaluation of machine translation. Research report, IBM (2001)Google Scholar
  7. 7.
    Perez, D., Alfonseca, E.: Application of the bleu algorithm for recognising textual entailments. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  8. 8.
    Lin, C., Hovy, E.: Automatic evaluation of summaries using n-gram co-occurrence statistics. In: Proceedings of the Human Technology Conference 2003, HLTNAACL- 2003 (2003)Google Scholar
  9. 9.
    Alfonseca, E., Pérez, D.: Automatic assessment of short questions with a bleuinspired algorithm and shallow NLP. In: Vicedo, J.L., Martínez-Barco, P., Muńoz, R., Saiz Noeda, M. (eds.) EsTAL 2004. LNCS (LNAI), vol. 3230, pp. 25–35. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Alfonseca, E.: Wraetlic user guide version 2.0 (2005)Google Scholar
  11. 11.
    Wu, D.: Textual entailment recognition based on inversion transduction grammars. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  12. 12.
    Lesk, M.: Automatic sense disambiguation using machine readable dictionaries. In: Proceedings of the 5th International Conference on Systems Documentation, pp. 24–26 (1986)Google Scholar
  13. 13.
    Ruiz-Casado, M., Alfonseca, E., Castells, P.: Automatic assignment of wikipedia encyclopedic entries to wordnet synsets. In: Szczepaniak, P.S., Kacprzyk, J., Niewiadomski, A. (eds.) AWIC 2005. LNCS (LNAI), vol. 3528, pp. 380–386. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Bos, J., Markert, K.: Combining shallow and deep nlp methods for recognizing textual entailment. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  15. 15.
    Jijkoun, V., Rijke, M.: Recognizing textual entailment using lexical similarity. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  16. 16.
    Akhmatova, E.: Textual entailment resolution via atomic propositions. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  17. 17.
    Herrera, J., Penas, A., Verdejo, F.: Textual entailment recognition based on dependency analysis and wordnet. In: Proceedings of the PASCAL Recognising Textual Entailment workshop, U.K. (2005)Google Scholar
  18. 18.
    Szpektor, I., Tanev, H., Dagan, I., Coppola, B.: Scaling web-based acquisition of entailment relations. In: Lin, D., Wu, D. (eds.) Proceedings of the EMNLP, pp. 41–48. Association for Computational Linguistics (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Diana Pérez
    • 1
  • Enrique Alfonseca
    • 1
  1. 1.Department of Computer ScienceUniversidad Autónoma de MadridMadridSpain

Personalised recommendations