Abstract
Neural machine translation (NMT) has shown very promising results for some resourceful languages like En-Fr and En-De. The success partly relies on the availability of large scale and high quality parallel corpora. We research on how to adapt NMT to very low-resource Mongolian-Chinese machine translation by introducing attention mechanism, sub-words translation, monolingual data and a NMT correction model. We proposed a sub-words model to address the out-of-vocabulary (OOV) problem in attention-based NMT model. Monolingual data help alleviate the low-resource problem. Besides, we explore a Chinese NMT correction model to enhance the translation performance. The experiments show that the adapted Mongolian-Chinese attention-based NMT machine translation obtains an improvement of 1.70 BLEU points over the phrased-based statistical machine translation baseline and 3.86 BLEU points over normal NMT baseline on an open training set.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kalchbrenner, N., Kalchbrenner, N.: Recurrent convolutional neural networks for discourse compositionality. In: Proceedings of ACL 2013 (2013)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: NIPS 2014 (2014)
Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of EMNLP 2014 (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of ACL – IJCNLP 2015, vol. 1 (2015). Long Papers
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with sub-word units. In: Proceeding of CoRR 2015 (2015)
Jean, S., Cho, K., Memisevic, R., Bengio, Y.: On using very large target vocabulary for neural machine translation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, vol. 2 (2015). Short Papers
Sutskever, I., Le, Q.V., Vinyals, O., Zaremba, W.: Addressing the rare word problem in neural machine translation. In: Proceedings of ACL – IJCNLP 2015, vol. 1, 11–19, 127–133 pages (2015). Long Papers
Wu, J., Hou, H., Monghjaya, F.B., Xie, C.: Introduction of traditional Mongolian-Chinese machine translation. In: Proceeding of International Conference on Electrical, Automation and Mechanical Engineering, EAME 2015, Phuket, pp. 357–360 (2015)
Gülçehre, Ç., Firat, O., Xu, K., Cho, K., Barrault, L., Lin, H., Bougares, F., Schwenk, H., Bengio, Y.: On using monolingual corpora in neural machine translation. In: Proceedings of CoRR 2015 (2015)
Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based translation. In: Proceedings of HLT-NAACL (2003)
Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. In: Proceeding of ACL 2016 (2016)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. Sig. Process. IEEE Trans. 45(11), 2673–2681 (1997)
Huang, C., Zhao, H.: Chinese word segmentation: a decade review. J. Chin. Inf. Process. 21(3), 8–20 (2007)
Wu, J., Hou, H., Bao, F., Jiang, Y.: Template-based model for BiRNN Mongolian – Chinese machine translation. In: Proceedings of TAAI 2015 (2015)
Chung, J., Cho, K., Bengio, Y.: A character-level decoder without explicit segmentation for neural machine translation. In: Proceedings of ACL 2016 (2016)
Costajussà , M.R., Fonollosa, J.A.R.: Character-based neural machine translation. In: Proceedings of ACL 2016 (2016)
Yu, M., Hou, H.: Researching of mongolian word segmentation system based on dictionary, rules and language model. Inner Mongolian University (2011)
Lambert, P., Schwenk, H., Servan, C., Abdul-Rauf, S.: Investigations on translation model adaptation using monolingual data. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, Edinburgh, Scotland, pp. 284–293 (2011)
Yuan, Z., Felice, M.: Constrained grammatical error correction using statistical machine translation. In: Proceeding of CoNLL 2013 (2013)
Wang, Y., Wang, L., Wong, D.F., Chao, L.S., Zeng, X., Lu, Y.: Factored statistical machine translation for grammatical error correction. In: Proceedings of the Eighteenth Conference on Computational Natural Language Learning 2014 (2014)
Zhao, H., Lv, Y., Ben, G., Huang, Y., Liu, Q.: Summary on CWMT2011 MT translation evaluation. J. Chin. Inf. Process. 26(1), 22–30 (2011)
Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based translation. In: Proceeding of NAACL 2003 (2003)
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E.: Moses: open source toolkit for statistical machine translation. In: ACL 2007 Demonstration Session (2007)
Och, F.J., Ney, H.: A systematic comparison of various statistical alignment models. Comput. Linguist. 29(1), 19–51 (2003)
Och, F.J.: Minimum error rate training in statistical machine translation. In: Proceedings of ACL 2003, pp. 440–447 (2003)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No. 61362028).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Wu, J., Hou, H., Shen, Z., Du, J., Li, J. (2016). Adapting Attention-Based Neural Network to Low-Resource Mongolian-Chinese Machine Translation. In: Lin, CY., Xue, N., Zhao, D., Huang, X., Feng, Y. (eds) Natural Language Understanding and Intelligent Applications. ICCPOL NLPCC 2016 2016. Lecture Notes in Computer Science(), vol 10102. Springer, Cham. https://doi.org/10.1007/978-3-319-50496-4_39
Download citation
DOI: https://doi.org/10.1007/978-3-319-50496-4_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50495-7
Online ISBN: 978-3-319-50496-4
eBook Packages: Computer ScienceComputer Science (R0)