Abstract
Phrase-based machine translation assumes that all words are at the same distance and translates them using feature functions that approximate the probability at different levels. On the other hand, neural machine translation infers a word embedding and translates these word vectors using a neural model. At the moment, both approaches co-exist and are being intensively investigated.
This paper to the best of our knowledge is the first work that both compares and combines these two systems by: using the phrase-based output to solve unknown words in the neural machine translation output; using the neural alignment in the phrase-based system; comparing how the popular strategy of pre-reordering affects both systems; and combining both translation outputs. Improvements are achieved in Catalan-to-Spanish and German-to-English.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sundermeyer, M., Alkhouli, T., Wuebker, J., Ney, H.: Translation modeling with bidirectional recurrent neural networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Doha (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2015)
Bisazza, A., Federico, M.: A survey of word reordering in statistical machine translation: computational models and language phenomena. arXiv preprint arXiv:1502.04938 (2015)
Castaño, M.A., Casacuberta, F.: A connectionist approach to MT. In: Proceedings of the EUROSPEECH Conference (1997)
Chiang, D.: Hierarchical phrase-based translation. Comput. Linguist. 33(2), 201–228 (2007)
Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. In: Proceedings of the 8th Workshop on Syntax, Semantics and Structure in Statistical Translation, Doha (2014)
Collins, M., Koehn, P., Kucerova, I.: Clause restructuring for statistical machine translation. In: Annual Conference of the Association for Computational Lingusitics (ACL 2005), Michigan (2005)
Costa-Jussà, M.R., Fonollosa, J.A.R.: Statistical machine reordering. In: Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, pp. 70–76 (2006)
Costa-Jussà, M., Poch, M., Fonollosa, J., Farrús, M., Marinno, J.: A large Spanish-Catalan parallel corpus release for machine translation. Comput. Inf. 33(4), 907–920 (2014)
Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)
Jean, S., Firat, O., Cho, K., Memisevic, R., Bengio, Y.: Montreal neural machine translation systems for WMT15. In: Proceedings of the 10th Workshop on Statistical Machine Translation, Lisbon (2015)
Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Seattle (2013)
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E.: Moses: open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, pp. 177–180 (2007)
Koehn, P., Och, F., Marcu, D.: Statistical phrase-based translation. In: Proceedings of the 41th Annual Meeting of the Association for Computational Linguistics (2003)
Kumar, S., Byrne, W.J.: Minimum Bayes-risk decoding for statistical machine translation. In: HLT-NAACL, pp. 169–176 (2004)
Luong, T., Sutskever, I., Le, Q.V., Vinyals, O., Zaremba, W.: Addressing the rare word problem in neural machine translation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Beijing, pp. 11–19 (2015)
Och, F.: Minimum error rate training in statistical machine translation. In: Proceedings of the 41th Annual Meeting of the Association for Computational Linguistics, pp. 160–167 (2003)
Quirk, C., Menezes, A., Cherry, C.: Dependency treelet translation: syntactically informed phrasal SMT. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp. 271–279 (2005)
Schwenk, H., Costa-Jussà, M.R., Fonollosa, J.A.R.: Smooth bilingual n-gram translation. In: Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Prague, pp. 430–438 (2007)
Stolcke, A.: SRILM - an extensible language modeling toolkit. In: 7th International Conference on Spoken Language Processing, Denver (2002)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 3104–3112. Curran Associates, Inc. (2014)
Toutanova, K., Suzuki, H., Ruopp, A.: Applying morphology generation models to machine translation. In: Proceedings of the Joint Conference of the Association for Computational Linguistics and Human Language Technology, Columbus, pp. 514–522 (2008)
Acknowledgements
This work has been supported by Spanish Ministerio de Economía y Competitividad, contract TEC2015-69266-P and the Seventh Framework Program of the European Commission through the International Outgoing Fellowship Marie Curie Action (IMTraP-2011-29951).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
R. Costa-jussà, M., Fonollosa, J.A.R. (2018). Combining Phrase and Neural-Based Machine Translation: What Worked and Did Not. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2016. Lecture Notes in Computer Science(), vol 9624. Springer, Cham. https://doi.org/10.1007/978-3-319-75487-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-75487-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-75486-4
Online ISBN: 978-3-319-75487-1
eBook Packages: Computer ScienceComputer Science (R0)