Advertisement

NiuTrans Submission for CCMT19 Quality Estimation Task

  • Ziyang Wang
  • Hui Liu
  • Hexuan Chen
  • Kai Feng
  • Zeyang Wang
  • Bei Li
  • Chen Xu
  • Tong XiaoEmail author
  • Jingbo Zhu
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1104)

Abstract

This paper describes our system submitted for the CCMT 2019 Quality Estimation (QE) Task, including sentence-level and word-level. We propose a new method based on predictor-estimator architecture [7] in this task. For the predictor, we adopt Transformer-DLCL [17] (dynamic linear combination of previous layers) as our feature extracting models. In order to obtain the information of translations in both directions, we use right-to-left and left-to-right two models, concatenate two feature vectors as whole quality feature vectors. For the estimator, we use a multi-layer bi-directional GRU to predict HTER scores or OK/BAD labels for different tasks. We pre-train the predictor according to machine translation (MT) method with bilingual data from WMT2019 EN-ZH task, and then jointly train predictor and estimator with the QE task data. We also construct 50K pseudo data in different methods in respond to the data scarcity. The final system integrates multiple single models to generate results.

Keywords

Quality estimation Deep Transformer Bi-GRU 

Notes

Acknowledgments

This work was supported in part by the National Science Foundation of China (Nos. 61876035, 61732005 and 61432013), the National Key R&D Program of China (No. 2019QY1801) and the Opening Project of Beijing Key Laboratory of Internet Culture and Digital Dissemination Research. We also thank the reviewers for their insightful comments.

References

  1. 1.
    Bapna, A., Chen, M.X., Firat, O., Cao, Y., Wu, Y.: Training deeper neural machine translation models with transparent attention. arXiv preprint arXiv:1808.07561 (2018)
  2. 2.
    Blatz, J., et al.: Confidence estimation for machine translation. In: Coling 2004: Proceedings of the 20th International Conference on Computational Linguistics (2004)Google Scholar
  3. 3.
    Douglas, S.P., Craig, C.S.: Collaborative and iterative translation: an alternative approach to back translation. J. Int. Mark. 15(1), 30–43 (2007)CrossRefGoogle Scholar
  4. 4.
    Edunov, S., Ott, M., Auli, M., Grangier, D.: Understanding back-translation at scale. arXiv preprint arXiv:1808.09381 (2018)
  5. 5.
    Fan, K., Li, B., Zhou, F., Wang, J.: “Bilingual expert” can find translation errors, July 2018Google Scholar
  6. 6.
    Junczys-Dowmunt, M., Grundkiewicz, R.: Log-linear combinations of monolingual and bilingual neural machine translation models for automatic post-editing. In: WMT (2016)Google Scholar
  7. 7.
    Kim, H., Jung, H.Y., Kwon, H., Lee, J.H., Na, S.H.: Predictor-estimator: neural quality estimation based on target word prediction for machine translation. ACM Trans. Asian Low-Resour. Lang. Inf. Process. (TALLIP) 17(1), 3 (2017)Google Scholar
  8. 8.
    Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016)
  9. 9.
    Niehues, J., Herrmann, T., Vogel, S., Waibel, A.: Wider context by using bilingual language models in machine translation. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, pp. 198–206. Association for Computational Linguistics (2011)Google Scholar
  10. 10.
    Ott, M., et al.: fairseq: A fast, extensible toolkit for sequence modeling. In: Proceedings of NAACL-HLT 2019: Demonstrations (2019)Google Scholar
  11. 11.
    Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709 (2015)
  12. 12.
    Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015)
  13. 13.
    Shaw, P., Uszkoreit, J., Vaswani, A.: Self-attention with relative position representations. arXiv preprint arXiv:1803.02155 (2018)
  14. 14.
    Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of Association for Machine Translation in the Americas, vol. 200 (2006)Google Scholar
  15. 15.
    Specia, L., Paetzold, G., Scarton, C.: Multi-level translation quality prediction with quest++. In: Proceedings of ACL-IJCNLP 2015 System Demonstrations, pp. 115–120 (2015)Google Scholar
  16. 16.
    Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 6000–6010 (2017)Google Scholar
  17. 17.
    Wang, Q., et al.: Learning deep transformer models for machine translation. arXiv preprint arXiv:1906.01787 (2019)
  18. 18.
    Xiao, T., Zhu, J., Zhang, H., Li, Q.: Niutrans: an open source toolkit for phrase-based and syntax-based machine translation. In: Proceedings of the ACL 2012 System Demonstrations ACL 2012, pp. 19–24. Association for Computational Linguistics, Stroudsburg, PA, USA (2012). http://dl.acm.org/citation.cfm?id=2390470.2390474

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Ziyang Wang
    • 1
  • Hui Liu
    • 1
  • Hexuan Chen
    • 1
  • Kai Feng
    • 1
  • Zeyang Wang
    • 1
  • Bei Li
    • 1
  • Chen Xu
    • 1
  • Tong Xiao
    • 1
    • 2
    Email author
  • Jingbo Zhu
    • 1
    • 2
  1. 1.NLP LabNortheastern UniversityShenyangChina
  2. 2.NiuTrans Co., Ltd.ShenyangChina

Personalised recommendations