Abstract
Machine reading comprehension (MRC) has attracted considerable attention in NLP. However, due to the singularity of the word vector space, MRC models cannot be used for multiple languages. Developing a separate training model for each language would be time consuming. In addition, a supervised machine reading comprehension model for multiple languages would require many training samples and expensive parallel corpora. Therefore, this paper adopts cross-lingual word embedding for cross-lingual MRC for multiple languages. The bilingual word-embedding model discards the dependence on the parallel corpus to train the shared word vector using adversarial learning. In addition, the Procrustes method and cross-domain similarity local scaling are introduced in confrontation training to fine-tune the transition matrix so that the representations of the bilingual word vectors in the shared word vector space overlap as much as possible to achieve better performance. The final experimental results show that the orthogonal Procrustes method and local scaling of cross-domain similarity enhance the training effect of cross-lingual word vectors. Compared with monolingual MRC models, the proposed machine reading comprehension model, which uses cross-lingual word vectors, works effectively.
This work was supported by the National Natural Science Foundation of China (NSFC) under Grant No. 61702443, No. 61762091 and No. 61966038.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hermann, K.M., et al.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems, pp. 1693–1701 (2015)
Kadlec, R., Schmid, M., Bajgar, O., Kleindienst, J.: Text understanding with the attention sum reader network. arXiv preprint arXiv:1603.01547 (2016)
Cui, Y., Liu, T., Chen, Z., Wang, S., Hu, G.: Consensus attention-based neural networks for Chinese reading comprehension. arXiv preprint arXiv:1607.02250 (2016)
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)
Seo, M., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603 (2016)
Wang, S., Jiang, J.: Machine comprehension using match-LSTM and answer pointer. arXiv preprint arXiv:1608.07905 (2016)
Qu, Z., Cao, B., Wang, X., Li, F., Xu, P., Zhang, L.: Feedback LSTM network based on attention for image description generator. CMC-Comput. Mater. Con. 59(2), 575–589 (2019)
Qiu, J., et al.: Dependency-based local attention approach to neural machine translation. Comput. Mater. Con. 58(2), 547–562 (2019)
Hong, X., Zheng, X., Xia, J., Wei, L., Xue, W.: Cross-lingual non-ferrous metals related news recognition method based on CNN with a limited bi-lingual dictionary. Comput. Mater. Con. 58(2), 379–389 (2019)
Joulin, A., Bojanowski, P., Mikolov, T., Jégou, H., Grave, E.: Loss in translation: learning bilingual word mapping with a retrieval criterion. arXiv preprint arXiv:1804.07745 (2018)
Mikolov, T., Le, Q.V., Sutskever, I.: Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168 (2013)
Hardoon, D.R., Szedmak, S., Shawe-Taylor, J.: Canonical correlation analysis: an overview with application to learning methods. Neural Comput. 16(12), 2639–2664 (2004)
Lample, G., Conneau, A., Denoyer, L., Ranzato, M.: Unsupervised machine translation using monolingual corpora only. arXiv preprint arXiv:1711.00043 (2017)
Zhang, J.: Sparse orthogonal procrustes problem based regression for face recognition with pose variations. Comput. Sci. 44(2), 302–305 (2017)
Lample, G., Conneau, A., Denoyer, L., Jégou, H., et al.: Word translation without parallel data (2018)
Zhang, M., Liu, Y., Luan, H., Sun, M.: Adversarial training for unsupervised bilingual lexicon induction. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1959–1970 (2017)
Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)
Schönemann, P.H.: A generalized solution of the orthogonal procrustes problem. Psychometrika 31(1), 1–10 (1966). https://doi.org/10.1007/BF02289451
Cui, Y., et al.: A span-extraction dataset for Chinese machine reading comprehension, pp. 5886–5891, November 2019. https://www.aclweb.org/anthology/D19-1600
Cui, Y., Che, W., Liu, T., Qin, B., Wang, S., Hu, G.: Cross-lingual machine reading comprehension, pp. 1586–1595, November 2019. https://www.aclweb.org/anthology/D19-1169
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yi, Z., Jin, W., Zhang, X. (2020). Research on Cross-lingual Machine Reading Comprehension Technology Based on Non-parallel Corpus. In: Sun, X., Wang, J., Bertino, E. (eds) Artificial Intelligence and Security. ICAIS 2020. Communications in Computer and Information Science, vol 1252. Springer, Singapore. https://doi.org/10.1007/978-981-15-8083-3_36
Download citation
DOI: https://doi.org/10.1007/978-981-15-8083-3_36
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-8082-6
Online ISBN: 978-981-15-8083-3
eBook Packages: Computer ScienceComputer Science (R0)