The Solution of the Problem of Unknown Words Under Neural Machine Translation of the Kazakh Language
- 218 Downloads
The paper proposes a solution to the problem of unknown words for neural machine translation. The proposed solution is shown by the example of a neural machine translation of a Kazakh-English language pair. The novelty of the proposed technology for solving the problem of unknown words in the neural machine translation of the Kazakh language is the proposed algorithm for searching of unknown words in the vocabulary of a trained model of neural machine translation using the dictionary of synonyms of the Kazakh language. A dictionary of synonyms is used to search for words that are similar in meaning to the unknown words, which was defined. Moreover, the found synonyms are checked for the presence in the vocabulary of a trained model of neural machine translation. After that, a new translation of the edited sentence of the source language is performed. The base of words-synonyms of the Kazakh language is collected. The total number of synonymous words collected is 1995. Software solutions to the unknown word problem have been developed in the python programming language. The proposed technology solution to the problem of unknown words for neural machine translation was tested on the two source parallel Kazakh-English corpus (KAZNU Kazakh-English parallel corpus and WMT19 Kazakh-English parallel corpus) in both variants: baseline and with using of the proposed technology.
KeywordsNeural machine translation Unknown words Kazakh language
This work was carried out under grant No. AP05131415 “Development and research of the neural machine translation system of Kazakh language” and grant No. AP05132950 “Development of an information-analytical search system of data in the Kazakh language”, funded by the Ministry of Education and Science of the Republic of Kazakhstan for 2018–2020.
- 1.Luong, M.T., Sutskever, I., Le, Q.V., Vinyals, O., Zaremba, W.: Addressing the rare word problem in neural machine translation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 11–19 (2015)Google Scholar
- 2.Hermann, K.M., et al.: Teaching machines to read and comprehend. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, pp. 1693–1701 (2015)Google Scholar
- 3.Generating sequences with recurrent neural networks. https://arxiv.org/pdf/1308.0850.pdf. Accessed 27 Aug 2019
- 4.Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 1715–1725 (2016)Google Scholar
- 5.Marton, Y., Callison-Burch, Ch., Resnik, Ph.: Improved statistical machine translation using monolingually-derived paraphrases, In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language, pp. 381–390 (2009)Google Scholar
- 6.Zhang, J., Zhai, F., Zong, Ch.: Handling unknown words in statistical machine translation from a new perspective. In: Proceedings of the First CCF Conference Natural Language Processing and Chinese Computing, pp. 176–187 (2012)Google Scholar
- 8.Gulcehre, C., Ahn, S., Nallapati, R., Zhou, B., Bengio, Y.: Pointing the unknown words. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 140–149 (2016)Google Scholar
- 9.Li, X., Zhang, J., Zong, C.: Towards zero unknown word in neural machine translation. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 2852–2858. AAAI Press (2016)Google Scholar
- 11.Arthur, Ph., Neubig, G., Nakamura, S.: Incorporating discrete translation lexicons into neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1557–1567. Association for Computational Linguistics (2016)Google Scholar
- 12.Tukeyev, U.: Automaton models of the morphology analysis and the completeness of the endings of the Kazakh language. In: Proceedings of the International Conference Turkic Languages Processing, TURKLANG 2015, pp. 91–100 (2015)Google Scholar
- 13.Tukeyev, U., Sundetova, A., Abduali, B., Akhmadiyeva, Z., Zhanbussunov, N.: Inferring of the morphological chunk transfer rules on the base of complete set of Kazakh endings. In: Nguyen, N.-T., Manolopoulos, Y., Iliadis, L., Trawiński, B. (eds.) ICCCI 2016. LNCS (LNAI), vol. 9876, pp. 563–574. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45246-3_54CrossRefGoogle Scholar