Abstract
In machine-learning applications, data selection is of crucial importance if good runtime performance is to be achieved. Feature Decay Algorithms (FDA) have demonstrated excellent performance in a number of tasks. While the decay function is at the heart of the success of FDA, its parameters are initialised with the same weights. In this paper, we investigate the effect on Machine Translation of assigning more appropriate weights to words using word-alignment entropy. In experiments on German to English, we show the effect of calculating these weights using two popular alignment methods, GIZA++ and FastAlign, using both automatic and human evaluations. We demonstrate that our novel FDA model is a promising research direction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Note that we were unable to calculate statistical significance for the CHRF metric. Note too that prior to tuning, statistically significant improvements were seen for GIZA++ over FastAlign for BLEU, NIST, TER and METEOR.
- 2.
In experiments before tuning (excluded here for reasons of space), the METEOR and CHRF scores of the output of the system executed with GIZA++ did outperform the baseline system before tuning.
- 3.
Why is it the case that better scores are more likely with the METEOR evaluation metric? This measure evaluates a hypothesis against a reference calculating sentence-level similarity scores. In so doing it searches for all the possible matches of the words between the two sentences. The words can match (i) if they are the same, (ii) if they have the same stem, (iii) if they are synonyms, or (iv) if they are found as a match in a paraphrase table. Therefore, this metric takes into consideration not only the n-grams, but also the semantic of the words. As the human evaluation shows, many semantically equivalent translations are output by our GIZA++ system, which are penalised by most automatic metrics, but not by METEOR.
References
Koehn, P., Och, F., Marcu, D.: Statistical phrase-based translation. In: Proceedings of Conference combining Human Language Technology Conference series and the North American Chapter of the Association for Computational Linguistics Conference serie. (edmonton, Canada), pp. 48–54 (2003)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th ACL, Philadelphia, PA, USA, pp. 311–318 (2002)
Koehn, P.: Europarl: a parallel corpus for statistical machine translation. In: MT Summit X Conference Proceedings: The Tenth Machine Translation Summit, Phuket, Thailand, pp. 79–86 (2005)
Ozdowska, S., Way, A.: Optimal bilingual data for French-English PB-SMT. In: Proceedings of the 13th Annual Meeting of the European Association for Machine Translation, Barcelona, Spain, pp. 96–103 (2009)
Biçici, E., Yuret, D.: Instance selection for machine translation using feature decay algorithms. In: Proceedings of the Sixth Workshop on Statistical Machine Translatio, Edinburgh, Scotland, pp. 272–283 (2011)
Dyer, C., Chahuneau, V., Smith, N.: A simple, fast, and effective reparameterization of IBM model 2. In: Proceedings of Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, Atlanta, Georgia, USA, pp. 644–648 (2013)
Och, F., Ney, H.: A systematic comparison of various statistical alignment models. Comput. Linguist. 29(1), 19–51 (2003)
Shterionov, D., Du, J., Palminteri, M.A., Casanellas, L., ODowd, T., Way, A.: Improving KantanMT training efficiency with FastAlign. In: Proceedings of AMTA 2016, Austin, TX, pp. 222–231 (2016)
Eetemadi, S., Lewis, W., Toutanova, K., Radha, H.: Survey of data-selection methods in statistical machine translation. Mach. Transl. 29(3–4), 189–223 (2015)
Biçici, E., Liu, Q., Way, A.: ParFDA for fast deployment of accurate statistical machine translation systems, benchmarks, and statistics. In: Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisbon, Portugal, pp. 74–78 (2015)
Brown, P.F., Della Pietra, S.A., Della Pietra, V.J., Mercer, R.L.: The mathematics of statistical machine translation: parameter estimation. Comput. Linguist. 19, 263–311 (1993)
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E.: Moses: open source toolkit for SMT. In: Proceedings of 45th Annual Meeting of the ACL on Interactive Poster & Demonstration Sessions, (Prague, Czech Republic), pp. 177–180 (2007)
Och, F.: Minimum error rate training in statistical machine translation. In: ACL-2003: 41st Annual Meeting of the Association for Computational Linguistics, Proceedings, Sapporo, Japan, pp. 160–167 (2003)
Clark, J.H., Dyer, C., Lavie, A., Smith, N.A.: Better hypothesis testing for statistical machine translation: controlling for optimizer instability. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 2, Short Papers, Portland, Oregon, pp. 176–181 (2011)
Heafield, K.: KenLM: faster and smaller language model queries. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, Edinburgh, Scotland, pp. 187–197 (2011)
Doddington, G.: Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In: Proceedings of the Second International Conference on Human Language Technology Research, San Diego, CA, pp. 138–145 (2002)
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, Cambridge, Massachusetts, USA, pp. 223–231 (2006)
Banerjee, S., Lavie, A.: Meteor: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, Michigan, pp. 65–72 (2005)
Popovic, M.: chrF: character n-gram F-score for automatic MT evaluation. In: Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisbon, Portugal, pp. 392–395 (2015)
Koehn, P.: Statistical significance tests for machine translation evaluation. In: Proceedings of EMNLP 2004, Barcelona, Spain, pp. 388–395 (2004)
He, Y., Way, A.: Improving the objective function in minimum error rate training. In: Proceedings of the Twelfth Machine Translation Summit, Ottawa, Canada, pp. 238–245 (2009)
Acknowledgements
This research is supported by the ADAPT Centre for Digital Content Technology, funded under the SFI Research Centres Programme (Grant 13/RC/2106) and co-funded under the European Regional Development Fund, and the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement PIAP-GA-2012-324414 (AbuMaTran).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Poncelas, A., Way, A., Toral, A. (2017). Extending Feature Decay Algorithms Using Alignment Entropy. In: Quesada, J., Martín Mateos , FJ., López Soto, T. (eds) Future and Emerging Trends in Language Technology. Machine Learning and Big Data. FETLT 2016. Lecture Notes in Computer Science(), vol 10341. Springer, Cham. https://doi.org/10.1007/978-3-319-69365-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-69365-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69364-4
Online ISBN: 978-3-319-69365-1
eBook Packages: Computer ScienceComputer Science (R0)