Coreference Resolution: To What Extent Does It Help NLP Applications?

  • Ruslan Mitkov
  • Richard Evans
  • Constantin Orăsan
  • Iustin Dornescu
  • Miguel Rios
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7499)


This paper describes a study of the impact of coreference resolution on NLP applications. Further to our previous study [1], in which we investigated whether anaphora resolution could be beneficial to NLP applications, we now seek to establish whether a different, but related task—that of coreference resolution, could improve the performance of three NLP applications: text summarisation, recognising textual entailment and text classification. The study discusses experiments in which the aforementioned applications were implemented in two versions, one in which the BART coreference resolution system was integrated and one in which it was not, and then tested in processing input text. The paper discusses the results obtained.


coreference resolution text summarisation recognising textual entailment text classification extrinsic evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mitkov, R., Evans, R., Orăsan, C., Ha, L.A., Pekar, V.: Anaphora Resolution: To What Extent Does It Help NLP Applications? In: Branco, A. (ed.) DAARC 2007. LNCS (LNAI), vol. 4410, pp. 179–190. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Mitkov, R., Evans, R., Orăsan, C.: A New, Fully Automatic Version of Mitkov’s Knowledge-Poor Pronoun Resolution Method. In: Gelbukh, A. (ed.) CICLing 2002. LNCS, vol. 2276, pp. 168–187. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  3. 3.
    Mitkov, R.: Anaphora Resolution. Longman, Cambridge (2002)Google Scholar
  4. 4.
    Sekine, S., Inui, K., Dagan, I., Dolan, B., Giampiccolo, D., Magnini, B. (eds.): Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing. Association for Computational Linguistics, Prague (2007)Google Scholar
  5. 5.
    Ido, R.B.H., Dagan, I., Dolan, B., Ferro, L., Giampiccolo, D., Magnini, B., Szpektor, I.: The second pascal recognising textual entailment challenge (2006)Google Scholar
  6. 6.
    Dagan, I., Glickman, O.: The PASCAL recognising textual entailment challenge. In: Proceedings of the PASCAL Challenges Workshop on Recognising Textual Entailment (2005)Google Scholar
  7. 7.
    Grishman, R., Sundheim, B.: Message understanding conference-6: A brief history. In: Proceedings of the 16th International Conference on Computational Linguistics, COLING 1996, Copenhagen, Denmark (1996)Google Scholar
  8. 8.
    Soon, W.M., Ng, H.T., Lim, D.C.Y.: A machine learning approach to coreference resolution of noun phrases. Computational Linguistics 27, 521–544 (2001)CrossRefGoogle Scholar
  9. 9.
    Ng, V., Cardie, C.: Improving machine learning approaches to coreference resolution. In: Proceedings of ACL 2002. Association for Computational Linguistics (2002)Google Scholar
  10. 10.
    Uryupina, O.: Coreference resolution with and without linguistic knowledge. In: Proceedings of LREC 2006, Genoa, Italy, pp. 893–898 (2006)Google Scholar
  11. 11.
    Cardie, C., Wagstaff, K.: Noun phrase coreference as clustering. In: Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pp. 82–89. Association for Computational Linguistics, College Park (1999)Google Scholar
  12. 12.
    Ng, V.: Supervised noun phrase coreference research: The first fifteen years. In: Proceedings of ACL 2010. Association for Computational Linguistics (2010)Google Scholar
  13. 13.
    Versley, Y., Ponzetto, S.P., Poesio, M., Eidelman, V., Jern, A., Smith, J., Yang, X., Moschitti, A.: Bart: A modular toolkit for coreference resolution. In: Proceedings of LREC 2008 (2008)Google Scholar
  14. 14.
    Yang, X., Su, J., Tan, C.L.: Kernel-based pronoun resolution with structured syntactic knowledge. In: Proceedings of CoLing/ACL 2006. Association for Computational Linguistics (2006)Google Scholar
  15. 15.
    Orăsan, C.: The Influence of Pronominal Anaphora Resolution on Term-based Summarisation. In: Nicolov, N., Angelova, G., Mitkov, R. (eds.) Recent Advances in Natural Language Processing. Current Issues in Linguistic Theory, vol. 309, pp. 291–300. John Benjamins, Amsterdam (2009)Google Scholar
  16. 16.
    Steinberger, J., Kabadjov, M.A., Poesio, M., Sanchez-Graillet, O.: Improving LSA-based summarization with anaphora resolution. In: Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), Vancouver, Canada, pp. 1–8 (2005)Google Scholar
  17. 17.
    Kabadjov, M.: A Comprehensive Evaluation of Anaphora Resolution and Discourse-new Classification. Ph.D. thesis, Department of Computer Science, University of Essex (2007)Google Scholar
  18. 18.
    Andreevskaia, A., Li, Z., Bergler, S.: Can shallow predicate argument structures determine entailment? In: Proceedings of the PASCAL Challenges Workshop on Recognising Textual Entailment (2005)Google Scholar
  19. 19.
    Castillo, J.J.: Textual entailment search task: An initial approach based on coreference resolution. In: International Conference on Intelligent Computing and Cognitive Informatics, pp. 388–391 (2010)Google Scholar
  20. 20.
    Mirkin, S., Dagan, I., Padó, S.: Assessing the role of discourse references in entailment inference. In: ACL, pp. 1209–1219 (2010)Google Scholar
  21. 21.
    Li, Z., Zhou, M.: Use semantic meaning of coreference to improve classification text representation. In: The 2nd IEEE International Conference on Information Management and Engineering (ICIME), pp. 416–420 (2010)Google Scholar
  22. 22.
    Hendrickx, I., Bouma, G., Coppens, F., Daelemans, W., Hoste, V., Kloosterman, G., Mineur, A.M., Vloet, J.V.D., Verschelde, J.L.: A coreference corpus and resolution system for Dutch. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), pp. 144–149 (2008)Google Scholar
  23. 23.
    Hendrickx, I., Hoste, V., Daelemans, W.: Semantic and Syntactic Features for Dutch Coreference Resolution. In: Gelbukh, A. (ed.) CICLing 2008. LNCS, vol. 4919, pp. 351–361. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  24. 24.
    Orăsan, C.: Comparative evaluation of term-weighting methods for automatic summarization. Journal of Quantitative Linguistics 16, 67–95 (2009)CrossRefGoogle Scholar
  25. 25.
    Hasler, L., Orăsan, C., Mitkov, R.: Building better corpora for summarisation. In: Proceedings of Corpus Linguistics 2003, Lancaster, UK, pp. 309–319 (2003)Google Scholar
  26. 26.
    Dagan, I., Dolan, B., Magnini, B., Roth, D.: Recognizing textual entailment: Rational, evaluation and approaches – erratum. Natural Language Engineering 16, 105 (2010)CrossRefGoogle Scholar
  27. 27.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL 2002), Stroudsburg, PA, USA, pp. 311–318 (2002)Google Scholar
  28. 28.
    Banerjee, S., Lavie, A.: METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 65–72. Ann Arbor, Michigan (2005)Google Scholar
  29. 29.
    Rios, M., Aziz, W., Specia, L.: Tine: A metric to assess mt adequacy. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, pp. 116–122. Association for Computational Linguistics, Edinburgh (2011)Google Scholar
  30. 30.
    Tan, C.M., Wang, Y.F., Lee, C.D.: The use of bigrams to enhance text categorization. Inf. Process. Manage. 38, 529–546 (2002)zbMATHCrossRefGoogle Scholar
  31. 31.
    Rogati, M., Yang, Y.: High-performing feature selection for text classification. In: Proceedings of the Eleventh International Conference on Information and Knowledge Management, CIKM 2002, pp. 659–661. ACM, New York (2002)CrossRefGoogle Scholar
  32. 32.
    Debole, F., Sebastiani, F.: An analysis of the relative hardness of Reuters-21578 subsets: Research articles. J. Am. Soc. Inf. Sci. Technol. 56, 584–596 (2005)CrossRefGoogle Scholar
  33. 33.
    Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: Rcv1: A new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004)Google Scholar
  34. 34.
    Bennett, P.N.: Using asymmetric distributions to improve text classifier probability estimates. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, SIGIR 2003, pp. 111–118. ACM, New York (2003)CrossRefGoogle Scholar
  35. 35.
    Bennett, P.N., Dumais, S.T., Horvitz, E.: Probabilistic combination of text classifiers using reliability indicators: models and results. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002), pp. 207–214. ACM, New York (2002)CrossRefGoogle Scholar
  36. 36.
    Nigam, K., McCallum, A.K., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using em. Mach. Learn. 39, 103–134 (2000)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ruslan Mitkov
    • 1
  • Richard Evans
    • 1
  • Constantin Orăsan
    • 1
  • Iustin Dornescu
    • 1
  • Miguel Rios
    • 1
  1. 1.Research Institute in Information and Language ProcessingUniversity of WolverhamptonUnited Kingdom

Personalised recommendations