Abstract
Translation quality estimation aims at evaluating the machine translation output without references. State-of-the-art quality estimation methods based on neural networks have certain capability of implicitly learning the syntactic information from sentence-aligned parallel corpus. However, they still fail to capture the deep structural syntactic details of the sentences. This paper proposes a method that explicitly incorporates source syntax in neural quality estimation. Specifically, the parse trees of source sentences are linearized, and the sequence labels are combined with the source sequence through hierarchical encoding to obtain a more complete and deeper source encoding vector. The hidden relationships between the source syntactic structure and the translation quality are modeled to discover the syntactic errors in the translation. Experimental results on WMT17 quality estimation datasets show that the sentence-level Pearson correlation score and the word-level F1–mult score can both be improved by the syntactic knowledge.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Specia, L., Shah, K., De Souza, J.G.C., et al.: QuEst-A translation quality estimation framework. In: Proceedings of ACL, pp. 79–84 (2013)
Shah, K., Cohn, T., Specia, L.: A bayesian non-linear method for feature selection in machine translation quality estimation. Mach. Transl. 29(2), 101–125 (2015)
González-Rubio, J., Navarro-Cerdán, J.R., Casacuberta, F.: Dimensionality reduction methods for machine translation quality estimation. Mach. Transl. 27(3–4), 281–301 (2013)
Han, A.L.F., Lu, Y., Wong, D.F., et al.: Quality estimation for machine translation using the joint method of evaluation criteria and statistical modeling. In: Proceedings of WMT, pp. 365–372 (2013)
Kreutzer, J., Schamoni, S., Riezler, S.: Quality estimation from ScraTCH(QUETCH): deep learning for word-level translation quality estimation. In: Proceedings of ACL, pp. 316–322 (2015)
Patel, R.N., Sasikumar, M.: Translation quality estimation using recurrent neural network. In: Proceedings of ACL, pp. 819–824 (2016)
Kim, H., Lee, J.H., Na, S.H.: Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation. In: Proceedings of WMT, pp. 562–568 (2017)
Wang, J., Fan, K., Li, B., et al.: Alibaba submission for WMT18 quality estimation task. In: Proceedings of WMT, pp. 809–815 (2018)
Li, M., Xiang, Q., Chen, Z., et al.: A unified neural network for quality estimation of machine translation. IEICE Trans. Inf. Syst. 101(9), 2417–2421 (2018)
Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2015 workshop on statistical machine translation. In: Proceedings of WMT, pp. 1–46 (2015)
Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2016 conference on machine translation. In: Proceedings of WMT, pp. 131–198 (2016)
Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2017 conference on machine translation (WMT 2017). In: Proceedings of WMT, pp. 169–214 (2017)
Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2018 conference on machine translation (WMT 2018). In: Proceedings of WMT, pp. 272–303 (2018)
Hardmeier, C., Nivre, J., Tiedemann, J.: Tree kernels for machine translation quality estimation. In: Proceedings of ACL, pp. 109–113 (2012)
Rubino, R., Foster, J., Wagner, J., et al.: DCU-Symantec submission for the WMT 2012 quality estimation task. In: Proceedings of ACL, pp. 138–144 (2012)
Specia, L., Giménez, J.: Combining confidence estimation and reference-based metrics for segment-level MT evaluation. In: Proceedings of AMTA (2010)
Kaljahi, R., Foster, J., Roturier, J., et al.: Quality estimation of English-French machine translation: a detailed study of the role of syntax. In: Proceedings of COLING, pp. 2052–2063 (2014)
Kozlova, A., Shmatova, M., Frolov, A.: YSDA participation in the WMT 2016 quality estimation shared task. In: Proceedings of WMT, pp. 793–799 (2016)
Martins, A.F.T., Junczys-Dowmunt, M., Kepler, F.N., et al.: Pushing the limits of translation quality estimation. TACL 5, 205–218 (2017)
Eriguchi, A., Hashimoto, K., Tsuruoka, Y.: Tree-to-sequence attentional neural machine translation. In: Proceedings of ACL, pp. 823–833 (2016)
Chen, H., Huang, S., Chiang, D., et al.: Improved neural machine translation with a syntax-aware encoder and decoder. In: Proceedings of ACL, pp. 1936–1947 (2017)
Currey, A., Heafield, K.: Multi-source syntactic neural machine translation. In: Proceedings of EMNLP, pp. 2961–2966 (2018)
Li, J., Xiong, D., Tu, Z., et al.: Modeling source syntax for neural machine translation. In: Proceedings of ACL, pp. 688–697 (2017)
Shi, X., Padhi, I., Knight, K.: Does string-based neural MT learn source syntax? In: Proceedings of EMNLP, pp. 1526–1534 (2016)
Linzen, T., Dupoux, E., Goldberg, Y.: Assessing the ability of LSTMs to learn syntax-sensitive dependencies. TACL 4, 521–535 (2016)
Graves, A.: Supervised Sequence Labelling with Recurrent Neural Networks. Studies in Computational Intelligence, vol. 385. Springer, Berlin (2008)
Hokamp, C.: Ensembling factored neural machine translation models for automatic post-editing and quality estimation. In: Proceedings of WMT, pp. 647–654 (2017)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR 2015 (2015)
Zoph, B., Knight, K.: Multi-source neural translation. In: Proceedings of NAACL, pp. 647–654 (2016)
Acknowledgements
This work is supported by the Humanities and Social Sciences Foundation for the Youth Scholars of Ministry of Education of China (19YJC740107).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ye, N., Wang, Y., Cai, D. (2019). Incorporating Syntactic Knowledge in Neural Quality Estimation for Machine Translation. In: Huang, S., Knight, K. (eds) Machine Translation. CCMT 2019. Communications in Computer and Information Science, vol 1104. Springer, Singapore. https://doi.org/10.1007/978-981-15-1721-1_3
Download citation
DOI: https://doi.org/10.1007/978-981-15-1721-1_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1720-4
Online ISBN: 978-981-15-1721-1
eBook Packages: Computer ScienceComputer Science (R0)