Skip to main content
SpringerLink
Log in
Book cover

China Conference on Machine Translation

CCMT 2019: Machine Translation pp 23–34Cite as

Incorporating Syntactic Knowledge in Neural Quality Estimation for Machine Translation

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1104))

Abstract

Translation quality estimation aims at evaluating the machine translation output without references. State-of-the-art quality estimation methods based on neural networks have certain capability of implicitly learning the syntactic information from sentence-aligned parallel corpus. However, they still fail to capture the deep structural syntactic details of the sentences. This paper proposes a method that explicitly incorporates source syntax in neural quality estimation. Specifically, the parse trees of source sentences are linearized, and the sequence labels are combined with the source sequence through hierarchical encoding to obtain a more complete and deeper source encoding vector. The hidden relationships between the source syntactic structure and the translation quality are modeled to discover the syntactic errors in the translation. Experimental results on WMT17 quality estimation datasets show that the sentence-level Pearson correlation score and the word-level F1–mult score can both be improved by the syntactic knowledge.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/slavpetrov/berkeleyparser.

  2. 2.

    https://nlp.stanford.edu/software/lex-parser.shtml.

References

  1. Specia, L., Shah, K., De Souza, J.G.C., et al.: QuEst-A translation quality estimation framework. In: Proceedings of ACL, pp. 79–84 (2013)

    Google Scholar 

  2. Shah, K., Cohn, T., Specia, L.: A bayesian non-linear method for feature selection in machine translation quality estimation. Mach. Transl. 29(2), 101–125 (2015)

    Article  Google Scholar 

  3. González-Rubio, J., Navarro-Cerdán, J.R., Casacuberta, F.: Dimensionality reduction methods for machine translation quality estimation. Mach. Transl. 27(3–4), 281–301 (2013)

    Article  Google Scholar 

  4. Han, A.L.F., Lu, Y., Wong, D.F., et al.: Quality estimation for machine translation using the joint method of evaluation criteria and statistical modeling. In: Proceedings of WMT, pp. 365–372 (2013)

    Google Scholar 

  5. Kreutzer, J., Schamoni, S., Riezler, S.: Quality estimation from ScraTCH(QUETCH): deep learning for word-level translation quality estimation. In: Proceedings of ACL, pp. 316–322 (2015)

    Google Scholar 

  6. Patel, R.N., Sasikumar, M.: Translation quality estimation using recurrent neural network. In: Proceedings of ACL, pp. 819–824 (2016)

    Google Scholar 

  7. Kim, H., Lee, J.H., Na, S.H.: Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation. In: Proceedings of WMT, pp. 562–568 (2017)

    Google Scholar 

  8. Wang, J., Fan, K., Li, B., et al.: Alibaba submission for WMT18 quality estimation task. In: Proceedings of WMT, pp. 809–815 (2018)

    Google Scholar 

  9. Li, M., Xiang, Q., Chen, Z., et al.: A unified neural network for quality estimation of machine translation. IEICE Trans. Inf. Syst. 101(9), 2417–2421 (2018)

    Article  Google Scholar 

  10. Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2015 workshop on statistical machine translation. In: Proceedings of WMT, pp. 1–46 (2015)

    Google Scholar 

  11. Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2016 conference on machine translation. In: Proceedings of WMT, pp. 131–198 (2016)

    Google Scholar 

  12. Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2017 conference on machine translation (WMT 2017). In: Proceedings of WMT, pp. 169–214 (2017)

    Google Scholar 

  13. Bojar, O., Chatterjee, R., Federmann, C., et al.: Findings of the 2018 conference on machine translation (WMT 2018). In: Proceedings of WMT, pp. 272–303 (2018)

    Google Scholar 

  14. Hardmeier, C., Nivre, J., Tiedemann, J.: Tree kernels for machine translation quality estimation. In: Proceedings of ACL, pp. 109–113 (2012)

    Google Scholar 

  15. Rubino, R., Foster, J., Wagner, J., et al.: DCU-Symantec submission for the WMT 2012 quality estimation task. In: Proceedings of ACL, pp. 138–144 (2012)

    Google Scholar 

  16. Specia, L., Giménez, J.: Combining confidence estimation and reference-based metrics for segment-level MT evaluation. In: Proceedings of AMTA (2010)

    Google Scholar 

  17. Kaljahi, R., Foster, J., Roturier, J., et al.: Quality estimation of English-French machine translation: a detailed study of the role of syntax. In: Proceedings of COLING, pp. 2052–2063 (2014)

    Google Scholar 

  18. Kozlova, A., Shmatova, M., Frolov, A.: YSDA participation in the WMT 2016 quality estimation shared task. In: Proceedings of WMT, pp. 793–799 (2016)

    Google Scholar 

  19. Martins, A.F.T., Junczys-Dowmunt, M., Kepler, F.N., et al.: Pushing the limits of translation quality estimation. TACL 5, 205–218 (2017)

    Google Scholar 

  20. Eriguchi, A., Hashimoto, K., Tsuruoka, Y.: Tree-to-sequence attentional neural machine translation. In: Proceedings of ACL, pp. 823–833 (2016)

    Google Scholar 

  21. Chen, H., Huang, S., Chiang, D., et al.: Improved neural machine translation with a syntax-aware encoder and decoder. In: Proceedings of ACL, pp. 1936–1947 (2017)

    Google Scholar 

  22. Currey, A., Heafield, K.: Multi-source syntactic neural machine translation. In: Proceedings of EMNLP, pp. 2961–2966 (2018)

    Google Scholar 

  23. Li, J., Xiong, D., Tu, Z., et al.: Modeling source syntax for neural machine translation. In: Proceedings of ACL, pp. 688–697 (2017)

    Google Scholar 

  24. Shi, X., Padhi, I., Knight, K.: Does string-based neural MT learn source syntax? In: Proceedings of EMNLP, pp. 1526–1534 (2016)

    Google Scholar 

  25. Linzen, T., Dupoux, E., Goldberg, Y.: Assessing the ability of LSTMs to learn syntax-sensitive dependencies. TACL 4, 521–535 (2016)

    Google Scholar 

  26. Graves, A.: Supervised Sequence Labelling with Recurrent Neural Networks. Studies in Computational Intelligence, vol. 385. Springer, Berlin (2008)

    MATH  Google Scholar 

  27. Hokamp, C.: Ensembling factored neural machine translation models for automatic post-editing and quality estimation. In: Proceedings of WMT, pp. 647–654 (2017)

    Google Scholar 

  28. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR 2015 (2015)

    Google Scholar 

  29. Zoph, B., Knight, K.: Multi-source neural translation. In: Proceedings of NAACL, pp. 647–654 (2016)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the Humanities and Social Sciences Foundation for the Youth Scholars of Ministry of Education of China (19YJC740107).

Author information

Authors and Affiliations

  1. Human-Computer Intelligence Research Center, Shenyang Aerospace University, Shenyang, 110136, China

    Na Ye, Yuanyuan Wang & Dongfeng Cai

Authors
  1. Na Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanyuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongfeng Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Na Ye .

Editor information

Editors and Affiliations

  1. Nanjing University, Nanjing, China

    Shujian Huang

  2. Didi Labs, University of Southern California, Marina Del Rey, CA, USA

    Kevin Knight

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ye, N., Wang, Y., Cai, D. (2019). Incorporating Syntactic Knowledge in Neural Quality Estimation for Machine Translation. In: Huang, S., Knight, K. (eds) Machine Translation. CCMT 2019. Communications in Computer and Information Science, vol 1104. Springer, Singapore. https://doi.org/10.1007/978-981-15-1721-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-1721-1_3

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-1720-4

  • Online ISBN: 978-981-15-1721-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation