Machine Translation

, Volume 23, Issue 2, pp 141–155

ATEC: automatic evaluation of machine translation via word choice and word order

Article

DOI: 10.1007/s10590-009-9061-x

Cite this article as:
Wong, B. & Kit, C. Machine Translation (2009) 23: 141. doi:10.1007/s10590-009-9061-x

Abstract

We propose a novel metric ATEC for automatic MT evaluation based on explicit assessment of word choice and word order in an MT output in comparison to its reference translation(s), the two most fundamental factors in the construction of meaning for a sentence. The former is assessed by matching word forms at various linguistic levels, including surface form, stem, sound and sense, and further by weighing the informativeness of each word. The latter is quantified in term of the discordance of word position and word sequence between a translation candidate and its reference. In the evaluations using the MetricsMATR08 data set and the LDC MTC2 and MTC4 corpora, ATEC demonstrates an impressive positive correlation to human judgments at the segment level, highly comparable to the few state-of-the-art evaluation metrics.

Keywords

MT evaluation Evaluation metrics ATEC Word choice Word order 

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  1. 1.Department of Chinese, Translation and LinguisticsCity University of Hong KongKowloonHong Kong