Abstract
Like other cross language tasks, we show that the quality of the translation resource, among other factors, has an effect on retrieval performance. Using data from the ImageCLEF test collection, we investigate the relationship between translation quality and retrieval performance when using Systran, a machine translation (MT) system, as a translation resource. The quality of translation is assessed manually by comparing the original ImageCLEF topics with the output from Systran and rated by assessors based on their semantic content. Quality is also measured using an automatic score derived from the mteval MT evaluation tool, and compared to the manual assessment score. Like other MT tasks, we find that measures based on the automatic score are correlated with the manual assessments for this CLIR task. The results from this short study formed our entry to ImageCLEF 2003.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Flank, S.: Cross-Language Multimedia Information Retrieval. In: Proceedings of Applied Natural Language Processing and the North American Chapter of the Association for Computational Linguistics (ANLP-NAACL 2000) (2000)
Clough, P., Sanderson, M.: The CLEF 2003 cross language image retrieval task. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 581–593. Springer, Heidelberg (2004)
Hutchins, W.J., Somers, H.: An Introduction to Machine Translation. Academic Press, London (1986)
Heisoft: How does Systran work? (2002), http://www.heisoft.de/volltext/systran/dok2/howorke.htm
Systran Ltd: The SYSRAN linguistics platform: A software solution to manage multilingual corporate knowledge (2002), http://www.systransoft.com/Technology/SLP.pdf
Jordan, P.W., Dorr, B.J., Benoit, J.W.: A First-Pass Approach for Evaluating Machine Translation Systems. Machine Translation 8(1-2), 49–58 (1993)
Voorhees, E.M., Harman, D.: Overview of TREC 2001. In: Proceedings of TREC 2001. NIST Special Publication 500-250, NIST (2001)
Peters, C., Braschler, M.: Cross-Language System Evaluation: The CLEF Campaigns. Journal of the American Society for Information Science and Technology 52(12), 1067–1072 (2001)
Nyberg, M., Carbonell, J.: Evaluation Metrics for Knowledge-Based Machine Translation. In: Proceedings of Fifteenth International Conference on Computational Linguistics (COLING 1994) (1994)
Patterson, C.: The Effectiveness of Using Machine Translators to Translate German and Portuguese into English when Searching for Images with English Captions. MSc dissertation for the degree of Masters of Arts in Librarianship, Department of Information Studies, University of Sheffield (2002)
National Institute of Standards and Technology (NIST): Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics (2002), http://www.nist.gov/speech/tests/mt/resources/scoring.htm
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Clough, P., Sanderson, M. (2004). Assessing Translation Quality for Cross Language Image Retrieval. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds) Comparative Evaluation of Multilingual Information Access Systems. CLEF 2003. Lecture Notes in Computer Science, vol 3237. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30222-3_57
Download citation
DOI: https://doi.org/10.1007/978-3-540-30222-3_57
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24017-4
Online ISBN: 978-3-540-30222-3
eBook Packages: Springer Book Archive