Language Resources and Evaluation

, Volume 49, Issue 1, pp 77–105

Corpus annotation with paraphrase types: new annotation scheme and inter-annotator agreement measures

  • Marta Vila
  • Manuel Bertran
  • M. Antònia Martí
  • Horacio Rodríguez
Original Paper

DOI: 10.1007/s10579-014-9272-5

Cite this article as:
Vila, M., Bertran, M., Martí, M.A. et al. Lang Resources & Evaluation (2015) 49: 77. doi:10.1007/s10579-014-9272-5
  • 301 Downloads

Abstract

Paraphrase corpora annotated with the types of paraphrases they contain constitute an essential resource for the understanding of the phenomenon of paraphrasing and the improvement of paraphrase-related systems in natural language processing. In this article, a new annotation scheme for paraphrase-type annotation is set out, together with newly created measures for the computation of inter-annotator agreement. Three corpora different in nature and in two languages have been annotated using this infrastructure. The annotation results and the inter-annotator agreement scores for these corpora are proof of the adequacy and robustness of our proposal.

Keywords

Paraphrasing Paraphrase typology Corpus annotation Inter-annotator agreement 

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • Marta Vila
    • 1
  • Manuel Bertran
    • 1
  • M. Antònia Martí
    • 1
  • Horacio Rodríguez
    • 2
  1. 1.CLiCUniversitat de BarcelonaBarcelonaSpain
  2. 2.TALPUniversitat Politècnica de CatalunyaBarcelonaSpain