Abstract

Ongoing research work on an Answer Validation System (AV) based on Textual Entailment and Question Answering has been presented. A number of answer validation modules have been developed based on Textual Entailment, Named Entity Recognition, Question-Answer type analysis, Chunk boundary module and Syntactic similarity module. These answer validation modules have been integrated using a voting technique. We combine the question and the answer into the Hypothesis (H) and the Supporting Text as Text (T) to identify the entailment relation as either “VALIDATED” or “REJECTED”. The important features in the lexical Textual Entailment module are: WordNet based unigram match, bi-gram match and skip-gram. In the syntactic similarity module, the important features used are: subject-subject comparison, subject-verb comparison, object-verb comparison and cross subject-verb comparison. The precision, recall and f-score of the integrated AV system on the AVE 2008 English annotated test set have been observed as 0.66, 0.65 and 0.65 respectively that outperforms the best performing system at AVE 2008 in terms of f-score.

Keywords

Answer Validation Exercise (AVE) Textual Entailment (TE) Named Entity (NE) Chunk Boundary Syntactic Similarity Question Type 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Peñas, A., Rodrigo, Á., Sama, V., Verdejo, F.: Overview of the answer validation exercise 2006. In: Peters, C., Clough, P., Gey, F.C., Karlgren, J., Magnini, B., Oard, D.W., de Rijke, M., Stempfhuber, M. (eds.) CLEF 2006. LNCS, vol. 4730, pp. 257–264. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Peñas, A., Rodrigo, Á., Verdejo, F.: Overview of the answer validation exercise 2007. In: Peters, C., Jijkoun, V., Mandl, T., Müller, H., Oard, D.W., Peñas, A., Petras, V., Santos, D. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 237–248. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Rodrigo, Á., Peñas, A., Verdejo, F.: Overview of the answer validation exercise 2008. In: Peters, C., Deselaers, T., Ferro, N., Gonzalo, J., Jones, G.J.F., Kurimo, M., Mandl, T., Peñas, A., Petras, V. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 296–313. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  4. 4.
    Dagan, I., Glickman, O., Magnini, B.: The PASCAL Recognising Textual Entailment Challenge. In: Proceedings of the PASCAL Challenges Workshop on RTE (2005)Google Scholar
  5. 5.
    Rodrigo, Á., Peñas, A., Verdejo, F.: UNED at Answer Validation Exercise 2007. In: Peters, C., Jijkoun, V., Mandl, T., Müller, H., Oard, D.W., Peñas, A., Petras, V., Santos, D. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 404–409. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Wang, R., Neumann, G.: DFKI–LT at AVE 2007: Using Recognizing Textual Entailment as a Core Engine for Answer Validation. In: Peters, C., Jijkoun, V., Mandl, T., Müller, H., Oard, D.W., Peñas, A., Petras, V., Santos, D. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 387–390. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Iftene, A., Balahur-Dobrescu, A.: Answer Validation on English and Romanian Languages. In: Peters, C., Deselaers, T., Ferro, N., Gonzalo, J., Jones, G.J.F., Kurimo, M., Mandl, T., Peñas, A., Petras, V. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 448–451. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Wang, R., Neumann, G.: Information Synthesis for Answer Validation. In: Peters, C., Deselaers, T., Ferro, N., Gonzalo, J., Jones, G.J.F., Kurimo, M., Mandl, T., Peñas, A., Petras, V. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 472–475. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  9. 9.
    Pakray, P., Pal, S., Bandyopadhyay, S., Gelbukh, A.: Automatic Answer Validation System on English Language. In: IEEE ICACTE 2010, vol. 6, pp. 329–333 (2010)Google Scholar
  10. 10.
    Pakray, P., Bandyopadhyay, S., Gelbukh, A.: Lexical based two-way RTE System at RTE-5. System Report. In: TAC Recognizing Textual Entailment Track Notebook (2009)Google Scholar
  11. 11.
    Pakray, P., Gelbukh, A., Bandyopadhyay, S.: A Syntactic Textual Entailment System Based on Dependency Parser. In: Gelbukh, A. (ed.) CICLing 2010. LNCS, vol. 6008, pp. 269–278. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  12. 12.
    Marneffe, M., MacCartney, B., Manning, C.D.: Generating Typed Dependency Parses from Phrase Structure Parses. In: 5th International Conference on Language Resources and Evaluation, LREC (2006)Google Scholar
  13. 13.
    Briscoe, E., Carroll, J., Watson, R.: The Second Release of the RASP System. In: Proceedings of the COLING/ACL 2006 Interactive Presentation Session (2006)Google Scholar
  14. 14.
    Pakray, P., Gelbukh, A., Bandyopadhyay, S.: Answer Validation Using Textual Entailment. In: Gelbukh, A. (ed.) CICLing 2011, Part II. LNCS, vol. 6609, pp. 353–364. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    Pakray, P., Poria, S., Gelbukh, A., Bandyopadhyay, S.: Semantic Textual Entailment using UNL. In: 12th CICLing-2011, Japan (2011) (will be published in IJCLA Journal)Google Scholar
  16. 16.
    Pakray, P., Gelbukh, A., Bandyopadhyay, S.: A Hybrid Textual Entailment System using Lexical and Syntactic Features. In: 9th IEEE ICCI 2010, pp. 291–295 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Partha Pakray
    • 1
  1. 1.Computer Science and Engineering DepartmentJadavpur UniversityKolkataIndia

Personalised recommendations