Advertisement

Machine Translation

, Volume 27, Issue 3–4, pp 257–280 | Cite as

Weakly supervised approaches for quality estimation

  • Erwan MoreauEmail author
  • Carl Vogel
Article

Abstract

Currently, quality estimation (QE) is mostly addressed using supervised learning approaches. In this paper we show that unsupervised and weakly supervised approaches (using a small training set) perform almost as well as supervised ones, for a significantly lower cost. More generally, we study the various possible definitions, parameters, evaluation methods and approaches for QE, in order to show that there are multiple possible configurations for this task.

Keywords

Quality estimation Evaluation Unsupervised learning  Weakly supervised learning 

Notes

Acknowledgments

We are grateful to the organizers of the WMT12 Shared Task on QE for not only organizing the task but also collecting and releasing the sets of features of all participants afterwards. We also thank the reviewers for their valuable comments. This research is supported by Science Foundation Ireland (Grant 07/CE/I1142) as part of the Centre for Next Generation Localisation (www.cngl.ie) funding at Trinity College, University of Dublin. Most calculations were performed on the Lonsdale cluster maintained by the Trinity Centre for High Performance Computing. This cluster was funded through Grants from Science Foundation Ireland. The graphics in this paper were created with R Core Team (2012), using the ggplot2 library (Wickham 2009).

References

  1. Callison-Burch C, Koehn P, Monz C, Post M, Soricut R, Specia L (2012) Findings of the 2012 workshop on statistical machine translation. In: Proceedings of the seventh workshop on statistical machine translation. Association for Computational Linguistics, MontréalGoogle Scholar
  2. Gamon M, Aue A, Smets M (2005) Sentence-level MT evaluation without reference translations: beyond language modeling. In: Proceedings of EAMT, pp 103–111Google Scholar
  3. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten I (2009) The weka data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10–18CrossRefGoogle Scholar
  4. He Y, Ma Y, van Genabith J, Way A (2010) Bridging SMT and TM with translation recommendation. In: Proceedings of the 48th annual meeting of the association for computational linguistics. Association for Computational Linguistics, Montréal, pp 622–630Google Scholar
  5. Koehn P (2005) Europarl: a parallel corpus for statistical machine translation. In: MT summit, vol 5Google Scholar
  6. Quinlan J (1992) Learning with continuous classes. In: Proceedings of the 5th Australian joint conference on artificial intelligence, Singapore, pp 343–348Google Scholar
  7. R Core Team (2012) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. ISBN 3-900051-07-0Google Scholar
  8. Schütze C (1996) The empirical base of linguistics: grammaticality judgements and linguistic methodology. University of Chicago Press, ChicagoGoogle Scholar
  9. Schütze C (2005) Thinking about what we are asking speakers to do. In: Kepsar S, Reis M (eds) Linguistic evidence: empirical, theoretical and computational perspectives. Studies in generative grammar, vol 85. Mouton De Gruyter, Berlin, pp 457–485Google Scholar
  10. Shevade S, Keerthi S, Bhattacharyya C, Murthy K (2000) Improvements to the SMO algorithm for SVM regression. IEEE Trans Neural Netw 11(5):1188–1193CrossRefGoogle Scholar
  11. Smola A, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222MathSciNetCrossRefGoogle Scholar
  12. Soricut R, Echihabi A (2010) Trustrank: inducing trust in automatic translations via ranking. In: Proceedings of the 48th annual meeting of the association for computational linguistics. Association for Computational Linguistics, Montréal, pp 612–621Google Scholar
  13. Soricut R, Bach N, Wang Z (2012) The SDL language weaver systems in the WMT12 quality estimation shared task. In: Proceedings of the seventh workshop on statistical machine translation. Association for Computational Linguistics, Montréal, pp 145–151. http://www.aclweb.org/anthology/W12-3118
  14. Specia L, Turchi M, Cancedda N, Dymetman M, Cristianini N (2009) Estimating the sentence-level quality of machine translation systems. In: Proceedings of the 13th conference of the European association for machine translation, pp 28–35Google Scholar
  15. Specia L, Hajlaoui N, Hallett C, Aziz W (2011) Predicting machine translation adequacy. In: Machine translation summit XIII, XiamenGoogle Scholar
  16. Stolcke A et al (2002) SRILM-an extensible language modeling toolkit. In: Proceedings of the international conference on spoken language processing, vol 2, pp 901–904Google Scholar
  17. Wang Y, Witten I (1996) Induction of model trees for predicting continuous classes. Working paper series. University of Waikato, HamiltonGoogle Scholar
  18. Wickham H (2009) Ggplot2: elegant graphics for data analysis. Springer, New YorkCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.CNGL and Computational Linguistics GroupCentre for Computing and Language Studies, School of Computer Science and Statistics, Trinity College DublinDublin 2Ireland
  2. 2.Computational Linguistics GroupCentre for Computing and Language Studies, School of Computer Science and Statistics, Trinity College DublinDublin 2Ireland

Personalised recommendations