Abstract
Data selection has proven its merit for improving Neural Machine Translation (NMT), when applied to authentic data. But the benefit of using synthetic data in NMT training, produced by the popular back-translation technique, raises the question if data selection could also be useful for synthetic data?
In this work we use Infrequent n-gram Recovery (INR) and Feature Decay Algorithms (FDA), two transductive data selection methods to obtain subsets of sentences from synthetic data. These methods ensure that selected sentences share n-grams with the test set so the NMT model can be adapted to translate it.
Performing data selection on back-translated data creates new challenges as the source-side may contain noise originated by the model used in the back-translation. Hence, finding n-grams present in the test set become more difficult. Despite that, in our work we show that adapting a model with a selection of synthetic data is an useful approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Poncelas, A., de Buy Wenniger, G.M., Way, A.: Feature decay algorithms for neural machine translation. In: Proceedings of the 21st Annual Conference of the European Association for Machine Translation, pp. 239–248. Alacant, Spain (2018)
Silva, C.C., Liu, C.H., Poncelas, A., Way, A.: Extracting in-domain training corpora for neural machine translation using data selection methods. In: Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 224–231, Brussels, Belgium (2018)
Luong, M.T., Manning, C.D.: Stanford neural machine translation systems for spoken language domains. In: Proceedings of the International Workshop on Spoken Language Translation, pp. 76–79. Da Nang, Vietnam (2015)
Freitag, M., Al-Onaizan, Y.: Fast domain adaptation for neural machine translation. arXiv preprint arXiv:1612.06897 (2016)
van der Wees, M., Bisazza, A., Monz, C.: Dynamic data selection for neural machine translation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1400–1410. Copenhagen, Denmark (2017)
Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 86–96. Berlin, Germany (2016)
Poncelas, A., de Buy Wenniger, G.M., Way, A.: Data selection with feature decay algorithms using an approximated target side. In: 15th International Workshop on Spoken Language Translation, pp. 173–180. Bruges, Belgium (2018)
Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience (1998)
Parcheta, Z., Sanchis-Trilles, G., Casacuberta, F.: Data selection for NMT using infrequent n-gram recovery. In: Proceedings of the 21st Annual Conference of the European Association for Machine Translation, pp. 219–227. Alacant, Spain (2018)
Gascó, G., Rocha, M.A., Sanchis-Trilles, G., Andrés-Ferrer, J., Casacuberta, F.: Does more data always yield better translations? In: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 152–161. Avignon, France (2012)
Biçici, E., Yuret, D.: Instance selection for machine translation using feature decay algorithms. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, pp. 272–283. Edinburgh, Scotland (2011)
Poncelas, A., Shterionov, D., Way, A., de Buy Wenniger, G.M., Passban, P.: Investigating backtranslation in neural machine translation. In: 21st Annual Conference of the European Association for Machine Translation, pp. 249–258. Alacant, Spain (2018)
Bojar, O., et al.: Findings of the 2015 workshop on statistical machine translation. In: Proceedings of the Tenth Workshop on Statistical Machine Translation, pp. 1–46. Lisboa, Portugal (2015)
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 1715–1725. Berlin, Germany (2016)
Klein, G., Kim, Y., Deng, Y., Senellart, J., Rush, A.M.: OpenNMT: open-source toolkit for neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics-System Demonstrations, pp. 67–72. Vancouver, Canada (2017)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318. Philadelphia, Pennsylvania, USA (2002)
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, pp. 223–231. Cambridge, Massachusetts, USA (2006)
Banerjee, S., Lavie, A.: METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 65–72. Ann Arbor, Michigan (2005)
Yepes, A.J., et al.: Findings of the WMT 2017 biomedical translation shared task. In: Proceedings of the Second Conference on Machine Translation, pp. 234–247 (2017)
Clark, J.H., Dyer, C., Lavie, A., Smith, N.A.: Better hypothesis testing for statistical machine translation: controlling for optimizer instability. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 2, pp. 176–181. Portland, Oregon (2011)
Koehn, P.: Statistical significance tests for machine translation evaluation. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 388–395. Barcelona, Spain (2004)
Poncelas, A., Way, A., Sarasola, K.: The ADAPT system description for the IWSLT 2018 Basque to English translation task. In: 15th International Workshop on Spoken Language Translation, pp. 76–82. Bruges, Belgium (2018)
Chinea-Rios, M., Peris, A., Casacuberta, F.: Adapting neural machine translation with parallel synthetic data. In: Proceedings of the Second Conference on Machine Translation, pp. 138–147. Copenhagen, Denmark (2017)
Poncelas, A., Way, A., Toral, A.: Extending feature decay algorithms using alignment entropy. In: Quesada, J.F., MartÃn Mateos, F.J., López-Soto, T. (eds.) FETLT 2016. LNCS (LNAI), vol. 10341, pp. 170–182. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69365-1_14
Poncelas, A., de Buy Wenniger, G.M., Way, A.: Applying n-gram alignment entropy to improve feature decay algorithms. Prague Bull. Math. Linguist. 108, 245–256 (2017)
Acknowledgements
This research has been supported by the ADAPT Centre for Digital Content Technology which is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.
This work has also received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 713567.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this paper
Cite this paper
Poncelas, A., Wenniger, G.M.d.B., Way, A. (2023). Adaptation of Machine Translation Models with Back-Translated Data Using Transductive Data Selection Methods. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2019. Lecture Notes in Computer Science, vol 13451. Springer, Cham. https://doi.org/10.1007/978-3-031-24337-0_40
Download citation
DOI: https://doi.org/10.1007/978-3-031-24337-0_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-24336-3
Online ISBN: 978-3-031-24337-0
eBook Packages: Computer ScienceComputer Science (R0)