Abstract
Phrase-based lexicalized reordering models have attracted extensive interest in statistical machine translation (SMT) due to their capacity for dealing with swap between consecutive phrases. However, translations between two languages that with significant differences in syntactic structure have made it challenging to generate a semantically and syntactically correct word sequence. In an effort to alleviate this problem, we propose a novel content-based neural reordering model that estimates reordering probabilities based on the words of its surrounding contexts. We first utilize a simple convolutional neural network (CNN) to capture semantic contents conditioned on various sizes of context. And then we employ a softmax layer to predict the reordering orientations and probability distributions. Experimental results show that our model provides statistically obvious improvements for both Chinese-Uyghur (+0.48 on CWMT2015) and Chinese-English (+0.27 on CWMT2013) translation tasks over conventional lexicalized reordering models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based translation. In: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, vol. 1, pp. 48–54. Association for Computational Linguistics (2003)
Och, F.J., Ney, H.: The alignment template approach to statistical machine translation. Comput. Linguist. 30(4), 417–449 (2004)
Och, F.J., Gildea, D., Khudanpur, S., et al.: A smorgasbord of features for statistical machine translation. In: Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004 (2004)
Koehn, P., Hoang, H., Birch, A., et al.: Moses: open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pp. 177–180. Association for Computational Linguistics (2007)
Li, P., Liu, Y., Sun, M., et al.: A neural reordering model for phrase-based translation. In: COLING, pp. 1897–1907 (2014)
Graves, A., Mohamed, A., Hinton, G.: Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6645–6649. IEEE (2013)
Liu, X., Gao, J., He, X., et al.: Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In: HLT-NAACL, pp. 912–921 (2015)
Cho, K., Merrienboer, B.V., Gulcehre, C., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. Comput. Sci. (2014)
Barone, A.V.M., Attardi, G.: Non-projective dependency-based pre-reordering with recurrent neural network for machine translation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Long Papers, vol. 1, pp. 846–856 (2015)
Hadiwinoto, C., Ng, H.T.: A dependency-based neural reordering model for statistical machine translation. In: AAAI, pp. 109–115 (2017)
Schwenk, H., Dchelotte, D., Gauvain, J.L.: Continuous space language models for statistical machine translation. In: COLING/ACL on Main Conference Poster Sessions, pp. 723–730. Association for Computational Linguistics (2006)
Xiong, D., Liu, Q., Lin, S.: Maximum entropy based phrase reordering model for statistical machine translation. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pp. 521–528. Association for Computational Linguistics (2006)
Socher, R., Pennington, J., Huang, E.H., et al.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 151–161. Association for Computational Linguistics (2011)
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Bojanowski, P., Grave, E., Joulin, A., et al.: Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606 (2016)
Hensman, P., Masko, D.: The impact of imbalanced training data for convolutional neural networks. Degree Project in Computer Science, KTH Royal Institute of Technology (2015)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation (1985)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012)
Loper, E., Bird, S.: NLTK: the natural language toolkit. In: ACL 2002 Workshop on Effective TOOLS and Methodologies for Teaching Natural Language Processing and Computational Linguistics, pp. 63–70. Association for Computational Linguistics (2002)
Och, F.J., Ney, H.: GIZA++: training of statistical translation models (2000)
Stolcke, A.: SRILM-an extensible language modeling toolkit. In: Interspeech 2002, vol. 2002 (2002)
Papineni, K., Roukos, S., Ward, T., et al.: BLEU: a method for automatic evaluation of machine translation. In: Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)
Acknowledgements
This research is supported by the West Light Foundation of Chinese Academy of Sciences (No. YBXM-2014-04), the Important Project on Science and Technology of Xinjiang Province (No. 2016A03007-3), the Xinjiang Province Open Project of Key Laboratory (No. 2015KL031) and the Natural Science Foundation of Xinjiang Province (No. 2015211B034).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Pan, Y., Li, X., Yang, Y., Mi, C., Dong, R., Zeng, W. (2017). A Content-Based Neural Reordering Model for Statistical Machine Translation. In: Wong, D., Xiong, D. (eds) Machine Translation. CWMT 2017. Communications in Computer and Information Science, vol 787. Springer, Singapore. https://doi.org/10.1007/978-981-10-7134-8_11
Download citation
DOI: https://doi.org/10.1007/978-981-10-7134-8_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-7133-1
Online ISBN: 978-981-10-7134-8
eBook Packages: Computer ScienceComputer Science (R0)