Advertisement

Incorporating Syntactic Knowledge in Neural Quality Estimation for Machine Translation

  • Na YeEmail author
  • Yuanyuan Wang
  • Dongfeng Cai
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1104)

Abstract

Translation quality estimation aims at evaluating the machine translation output without references. State-of-the-art quality estimation methods based on neural networks have certain capability of implicitly learning the syntactic information from sentence-aligned parallel corpus. However, they still fail to capture the deep structural syntactic details of the sentences. This paper proposes a method that explicitly incorporates source syntax in neural quality estimation. Specifically, the parse trees of source sentences are linearized, and the sequence labels are combined with the source sequence through hierarchical encoding to obtain a more complete and deeper source encoding vector. The hidden relationships between the source syntactic structure and the translation quality are modeled to discover the syntactic errors in the translation. Experimental results on WMT17 quality estimation datasets show that the sentence-level Pearson correlation score and the word-level F1–mult score can both be improved by the syntactic knowledge.

Keywords

Quality estimation Neural networks Syntactic representation Parse tree Hierarchical encoding 

Notes

Acknowledgements

This work is supported by the Humanities and Social Sciences Foundation for the Youth Scholars of Ministry of Education of China (19YJC740107).

References

  1. 1.
    Specia, L., Shah, K., De Souza, J.G.C., et al.: QuEst-A translation quality estimation framework. In: Proceedings of ACL, pp. 79–84 (2013)Google Scholar
  2. 2.
    Shah, K., Cohn, T., Specia, L.: A bayesian non-linear method for feature selection in machine translation quality estimation. Mach. Transl. 29(2), 101–125 (2015)CrossRefGoogle Scholar
  3. 3.
    González-Rubio, J., Navarro-Cerdán, J.R., Casacuberta, F.: Dimensionality reduction methods for machine translation quality estimation. Mach. Transl. 27(3–4), 281–301 (2013)CrossRefGoogle Scholar
  4. 4.
    Han, A.L.F., Lu, Y., Wong, D.F., et al.: Quality estimation for machine translation using the joint method of evaluation criteria and statistical modeling. In: Proceedings of WMT, pp. 365–372 (2013)Google Scholar
  5. 5.
    Kreutzer, J., Schamoni, S., Riezler, S.: Quality estimation from ScraTCH(QUETCH): deep learning for word-level translation quality estimation. In: Proceedings of ACL, pp. 316–322 (2015)Google Scholar
  6. 6.
    Patel, R.N., Sasikumar, M.: Translation quality estimation using recurrent neural network. In: Proceedings of ACL, pp. 819–824 (2016)Google Scholar
  7. 7.
    Kim, H., Lee, J.H., Na, S.H.: Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation. In: Proceedings of WMT, pp. 562–568 (2017)Google Scholar
  8. 8.
    Wang, J., Fan, K., Li, B., et al.: Alibaba submission for WMT18 quality estimation task. In: Proceedings of WMT, pp. 809–815 (2018)Google Scholar
  9. 9.
    Li, M., Xiang, Q., Chen, Z., et al.: A unified neural network for quality estimation of machine translation. IEICE Trans. Inf. Syst. 101(9), 2417–2421 (2018)CrossRefGoogle Scholar
  10. 10.
    Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2015 workshop on statistical machine translation. In: Proceedings of WMT, pp. 1–46 (2015)Google Scholar
  11. 11.
    Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2016 conference on machine translation. In: Proceedings of WMT, pp. 131–198 (2016)Google Scholar
  12. 12.
    Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2017 conference on machine translation (WMT 2017). In: Proceedings of WMT, pp. 169–214 (2017)Google Scholar
  13. 13.
    Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2018 conference on machine translation (WMT 2018). In: Proceedings of WMT, pp. 272–303 (2018)Google Scholar
  14. 14.
    Hardmeier, C., Nivre, J., Tiedemann, J.: Tree kernels for machine translation quality estimation. In: Proceedings of ACL, pp. 109–113 (2012)Google Scholar
  15. 15.
    Rubino, R., Foster, J., Wagner, J., et al.: DCU-Symantec submission for the WMT 2012 quality estimation task. In: Proceedings of ACL, pp. 138–144 (2012)Google Scholar
  16. 16.
    Specia, L., Giménez, J.: Combining confidence estimation and reference-based metrics for segment-level MT evaluation. In: Proceedings of AMTA (2010)Google Scholar
  17. 17.
    Kaljahi, R., Foster, J., Roturier, J., et al.: Quality estimation of English-French machine translation: a detailed study of the role of syntax. In: Proceedings of COLING, pp. 2052–2063 (2014)Google Scholar
  18. 18.
    Kozlova, A., Shmatova, M., Frolov, A.: YSDA participation in the WMT 2016 quality estimation shared task. In: Proceedings of WMT, pp. 793–799 (2016)Google Scholar
  19. 19.
    Martins, A.F.T., Junczys-Dowmunt, M., Kepler, F.N., et al.: Pushing the limits of translation quality estimation. TACL 5, 205–218 (2017)Google Scholar
  20. 20.
    Eriguchi, A., Hashimoto, K., Tsuruoka, Y.: Tree-to-sequence attentional neural machine translation. In: Proceedings of ACL, pp. 823–833 (2016)Google Scholar
  21. 21.
    Chen, H., Huang, S., Chiang, D., et al.: Improved neural machine translation with a syntax-aware encoder and decoder. In: Proceedings of ACL, pp. 1936–1947 (2017)Google Scholar
  22. 22.
    Currey, A., Heafield, K.: Multi-source syntactic neural machine translation. In: Proceedings of EMNLP, pp. 2961–2966 (2018)Google Scholar
  23. 23.
    Li, J., Xiong, D., Tu, Z., et al.: Modeling source syntax for neural machine translation. In: Proceedings of ACL, pp. 688–697 (2017)Google Scholar
  24. 24.
    Shi, X., Padhi, I., Knight, K.: Does string-based neural MT learn source syntax? In: Proceedings of EMNLP, pp. 1526–1534 (2016)Google Scholar
  25. 25.
    Linzen, T., Dupoux, E., Goldberg, Y.: Assessing the ability of LSTMs to learn syntax-sensitive dependencies. TACL 4, 521–535 (2016)Google Scholar
  26. 26.
    Graves, A.: Supervised Sequence Labelling with Recurrent Neural Networks. Studies in Computational Intelligence, vol. 385. Springer, Berlin (2008)zbMATHGoogle Scholar
  27. 27.
    Hokamp, C.: Ensembling factored neural machine translation models for automatic post-editing and quality estimation. In: Proceedings of WMT, pp. 647–654 (2017)Google Scholar
  28. 28.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR 2015 (2015)Google Scholar
  29. 29.
    Zoph, B., Knight, K.: Multi-source neural translation. In: Proceedings of NAACL, pp. 647–654 (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Human-Computer Intelligence Research CenterShenyang Aerospace UniversityShenyangChina

Personalised recommendations