Abstract
The unknown words in neural machine translation (NMT) may undermine the integrity of sentence structure, increase ambiguity and have adverse effect on the translation. In order to solve this problem, we propose a method of processing unknown words in NMT based on integrating syntactic structure and semantic concept. Firstly, the semantic concept network is used to construct the set of in-vocabulary synonyms corresponding to the unknown words. Secondly, a semantic similarity calculation method based on the syntactic structure and semantic concept is proposed. The best substitute is selected from the set of in-vocabulary synonyms by calculating the semantic similarity between the unknown words and their candidate substitutes. English-Chinese translation experiments demonstrate that this method can maintain the semantic integrity of the source language sentences. Meanwhile, in performance, our proposed method can obtain an improvement by 2.9 BLEU points when compared with the conventional NMT method, and the method can also achieve an improvement by 0.95 BLEU points when compared with the traditional method of positioning the UNK character based on word alignment information.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. Adv. Neural. Inf. Process. Syst. 4, 3104–3112 (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Luong, M.T., Sutskever, I., Le, Q.V., et al.: Addressing the rare word problem in neural machine translation. Bull. Univ. Agricu. Sci. Vet. Med. Cluj-Napoca. Vet. Med. 27(2), 82–86 (2014)
Gulcehre, C., Ahn, S., Nallapati, R., et al.: Pointing the unknown words. CoRR abs/1603.08148 (2016)
Jean, S., Cho, K., Memisevic, R., et al.: On using very large target vocabulary for neural machine translation. CoRR abs/1412.2007 (2014)
Li, X., Zhang, J., Zong, C.: Towards zero unknown word in neural machine translation. In: International Joint Conference on Artificial Intelligence, pp. 2852–2858. AAAI Press (2016)
Costa-Jussà , M.R., Fonollosa, J.A.R.: Character-based neural machine translation. CoRR abs/1603.00810 (2016)
Ling, W., Trancoso, I., Dyer, C., et al.: Character-based neural machine translation. arXiv preprint arXiv:1511.04586 (2015)
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015)
Luong, M.T., Manning, C.D.: Achieving open vocabulary neural machine translation with hybrid word-character models. CoRR abs/1604.00788 (2016)
Chung, J., Cho, K., Bengio, Y.: A character-level decoder without explicit segmentation for neural machine translation. arXiv preprint arXiv:1603.06147 (2016)
Papineni, K., Roukos, S., Ward, T., et al.: BLEU: a method for automatic evaluation of machine translation. In: Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2007)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. CoRR abs/1212.5701 (2012)
Srivastava, N., Hinton, G., Krizhevsky, A., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Miller, G.A.: WordNet: a lexical database for english. Commun. ACM 38(11), 39–41 (1995)
Och, F.J., Ney, H.: A systematic comparison of various statistical alignment models. Comput. Linguist. 29(1), 19–51 (2003)
Collins, M., Koehn, P.: Clause restructuring for statistical machine translation. In: Meeting on Association for Computational Linguistics, pp. 531–540. Association for Computational Linguistics (2005)
Socher, R., Bauer, J., Manning, C.D., et al.: Parsing with compositional vector grammars. In: Meeting of the Association for Computational Linguistics, pp. 455–465 (2013)
Acknowledgments
The research work has been supported by the National Nature Science Foundation of China (Contract 61370130, 61473294 and 61502149), and Beijing Natural Science Foundation under Grant No. 4172047, and the Fundamental Research Funds for the Central Universities (2015JBM033), and the International Science and Technology Cooperation Program of China under grant No. 2014DFA11350.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Miao, G., Xu, J., Li, Y., Li, S., Chen, Y. (2017). An Unknown Word Processing Method in NMT by Integrating Syntactic Structure and Semantic Concept. In: Wong, D., Xiong, D. (eds) Machine Translation. CWMT 2017. Communications in Computer and Information Science, vol 787. Springer, Singapore. https://doi.org/10.1007/978-981-10-7134-8_5
Download citation
DOI: https://doi.org/10.1007/978-981-10-7134-8_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-7133-1
Online ISBN: 978-981-10-7134-8
eBook Packages: Computer ScienceComputer Science (R0)