Opening Machine Translation Black Box for Cross-Language Information Retrieval
State-of-the-art Statistical Machine Translation (SMT) systems are widely used for query translation in Cross-Language Information Retrieval (CLIR), but usually as a black box. A strong limitation is that only one-best translation is retained. It is known that CLIR can benefit much from using multiple translation alternatives which produces a query expansion effect. It is then desirable to extend the one-best translation output to a richer translation including more translation alternatives. In fact, translation alternatives are available in SMT before the final best output is selected. A natural way is to open the black box of SMT to access the internal search graph in order to select more translation alternatives. In this paper, we consider the translation alternatives included in the N-best translation outputs. Using our approach for CLIR, we report up to 40% improvement in Mean Average Precision over baseline query translation using an SMT system as a black box on TREC-9 English-Chinese CLIR task, and 10% on NTCIR-5 English-Chinese task. This study demonstrates the usefulness of opening the black box of SMT to produce more adequate query translations for CLIR.
KeywordsCross-language Information Retrieval Statistical Machine Translation
Unable to display preview. Download preview PDF.
- 1.Brown, P.F., Della-Pietra, S.A., Della-Pietra, V.J., Mercer, R.L.: The mathematics of Statistical Machine Translation: Parameter estimation. Computational Linguistics 19(2), 263–311 (1993)Google Scholar
- 2.Gao, J., Nie, J.Y., Xun, E., Zhang, J., Zhou, M., Huang, C.: Improving query translation for cross-language information retrieval using statistical models. In: Proceedings of SIGIR 2001, pp. 96–104 (2001)Google Scholar
- 3.Koehn, P., Och, F., Marcu, D.: Statistical Phrase-Based Translation. In: Proceedings of the 2003 Human Language Technology Conference and the North American Chapter of the Association for Computational Linguistics, Edmonton, AB, Canada, pp. 48–54 (2003)Google Scholar
- 5.Liu, Y., Jin, R., Chai, J.Y.: A maximum coference model for dictionary-based cross-language information retrieval. In: Proceedings of SIGIR 2005, pp. 536–543 (2005)Google Scholar
- 7.Nie, J.: Cross-Language Information Retrieval. Morgan & Claypool Publishers (2010)Google Scholar
- 8.Och, F., Ney, H.: Discriminative training and maximum entropy models for Statistical Machine Translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, pp. 295–302 (2002)Google Scholar
- 9.Pirkola, A.: The effects of query structure and dictionary-setups in dictionary-based cross-language information retrieval. In: Proceedings of SIGIR 1998, pp. 55–63 (1998)Google Scholar
- 10.Sakai, T.: Advanced Technologies for Information Access. International Journal of Computer Processing of Oriental Languages 18(2), 95–113 (2005)Google Scholar
- 12.Strohman, T., Metzler, D., Turtle, H., Croft, W.B.: Indri: A language model-based search engine for complex queries. In: Proceedings of the Is, May 2-6, McLean, VA (2005)Google Scholar