Language Resources and Evaluation

, Volume 39, Issue 4, pp 267–285

Guidelines for Word Alignment Evaluation and Manual Alignment

  • Patrik Lambert
  • Adrià De Gispert
  • Rafael Banchs
  • José B. Mariño
Article

DOI: 10.1007/s10579-005-4822-5

Cite this article as:
Lambert, P., De Gispert, A., Banchs, R. et al. Lang Resources & Evaluation (2005) 39: 267. doi:10.1007/s10579-005-4822-5

Abstract

The purpose of this paper is to provide guidelines for building a word alignment evaluation scheme. The notion of word alignment quality depends on the application: here we review standard scoring metrics for full text alignment and give explanations on how to use them better. We discuss strategies to build a reference corpus, and show that the ratio between ambiguous and unambiguous links in the reference has a great impact on scores measured with these metrics. In particular, automatically computed alignments with higher precision or higher recall can be favoured depending on the value of this ratio. Finally, we suggest a strategy to build a reference corpus particularly adapted to applications where recall plays a significant role, like in machine translation. The manually aligned corpus we built for the Spanish-English European Parliament corpus is also described. This corpus is freely available.

Keywords

alignment error rate bilingual evaluation gold standard manual alignment parallel corpus precision recall word alignment 

Copyright information

© Springer 2006

Authors and Affiliations

  • Patrik Lambert
    • 1
  • Adrià De Gispert
    • 1
  • Rafael Banchs
    • 1
  • José B. Mariño
    • 1
  1. 1.TALP Research CentreBarcelonaSpain

Personalised recommendations