Extending Feature Decay Algorithms Using Alignment Entropy

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10341)


In machine-learning applications, data selection is of crucial importance if good runtime performance is to be achieved. Feature Decay Algorithms (FDA) have demonstrated excellent performance in a number of tasks. While the decay function is at the heart of the success of FDA, its parameters are initialised with the same weights. In this paper, we investigate the effect on Machine Translation of assigning more appropriate weights to words using word-alignment entropy. In experiments on German to English, we show the effect of calculating these weights using two popular alignment methods, GIZA++ and FastAlign, using both automatic and human evaluations. We demonstrate that our novel FDA model is a promising research direction.


Data selection Machine translation Mathematical foundations 



This research is supported by the ADAPT Centre for Digital Content Technology, funded under the SFI Research Centres Programme (Grant 13/RC/2106) and co-funded under the European Regional Development Fund, and the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement PIAP-GA-2012-324414 (AbuMaTran).


  1. 1.
    Koehn, P., Och, F., Marcu, D.: Statistical phrase-based translation. In: Proceedings of Conference combining Human Language Technology Conference series and the North American Chapter of the Association for Computational Linguistics Conference serie. (edmonton, Canada), pp. 48–54 (2003)Google Scholar
  2. 2.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th ACL, Philadelphia, PA, USA, pp. 311–318 (2002)Google Scholar
  3. 3.
    Koehn, P.: Europarl: a parallel corpus for statistical machine translation. In: MT Summit X Conference Proceedings: The Tenth Machine Translation Summit, Phuket, Thailand, pp. 79–86 (2005)Google Scholar
  4. 4.
    Ozdowska, S., Way, A.: Optimal bilingual data for French-English PB-SMT. In: Proceedings of the 13th Annual Meeting of the European Association for Machine Translation, Barcelona, Spain, pp. 96–103 (2009)Google Scholar
  5. 5.
    Biçici, E., Yuret, D.: Instance selection for machine translation using feature decay algorithms. In: Proceedings of the Sixth Workshop on Statistical Machine Translatio, Edinburgh, Scotland, pp. 272–283 (2011)Google Scholar
  6. 6.
    Dyer, C., Chahuneau, V., Smith, N.: A simple, fast, and effective reparameterization of IBM model 2. In: Proceedings of Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, Atlanta, Georgia, USA, pp. 644–648 (2013)Google Scholar
  7. 7.
    Och, F., Ney, H.: A systematic comparison of various statistical alignment models. Comput. Linguist. 29(1), 19–51 (2003)CrossRefzbMATHGoogle Scholar
  8. 8.
    Shterionov, D., Du, J., Palminteri, M.A., Casanellas, L., ODowd, T., Way, A.: Improving KantanMT training efficiency with FastAlign. In: Proceedings of AMTA 2016, Austin, TX, pp. 222–231 (2016)Google Scholar
  9. 9.
    Eetemadi, S., Lewis, W., Toutanova, K., Radha, H.: Survey of data-selection methods in statistical machine translation. Mach. Transl. 29(3–4), 189–223 (2015)CrossRefGoogle Scholar
  10. 10.
    Biçici, E., Liu, Q., Way, A.: ParFDA for fast deployment of accurate statistical machine translation systems, benchmarks, and statistics. In: Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisbon, Portugal, pp. 74–78 (2015)Google Scholar
  11. 11.
    Brown, P.F., Della Pietra, S.A., Della Pietra, V.J., Mercer, R.L.: The mathematics of statistical machine translation: parameter estimation. Comput. Linguist. 19, 263–311 (1993)Google Scholar
  12. 12.
    Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E.: Moses: open source toolkit for SMT. In: Proceedings of 45th Annual Meeting of the ACL on Interactive Poster & Demonstration Sessions, (Prague, Czech Republic), pp. 177–180 (2007)Google Scholar
  13. 13.
    Och, F.: Minimum error rate training in statistical machine translation. In: ACL-2003: 41st Annual Meeting of the Association for Computational Linguistics, Proceedings, Sapporo, Japan, pp. 160–167 (2003)Google Scholar
  14. 14.
    Clark, J.H., Dyer, C., Lavie, A., Smith, N.A.: Better hypothesis testing for statistical machine translation: controlling for optimizer instability. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 2, Short Papers, Portland, Oregon, pp. 176–181 (2011)Google Scholar
  15. 15.
    Heafield, K.: KenLM: faster and smaller language model queries. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, Edinburgh, Scotland, pp. 187–197 (2011)Google Scholar
  16. 16.
    Doddington, G.: Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In: Proceedings of the Second International Conference on Human Language Technology Research, San Diego, CA, pp. 138–145 (2002)Google Scholar
  17. 17.
    Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, Cambridge, Massachusetts, USA, pp. 223–231 (2006)Google Scholar
  18. 18.
    Banerjee, S., Lavie, A.: Meteor: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, Michigan, pp. 65–72 (2005)Google Scholar
  19. 19.
    Popovic, M.: chrF: character n-gram F-score for automatic MT evaluation. In: Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisbon, Portugal, pp. 392–395 (2015)Google Scholar
  20. 20.
    Koehn, P.: Statistical significance tests for machine translation evaluation. In: Proceedings of EMNLP 2004, Barcelona, Spain, pp. 388–395 (2004)Google Scholar
  21. 21.
    He, Y., Way, A.: Improving the objective function in minimum error rate training. In: Proceedings of the Twelfth Machine Translation Summit, Ottawa, Canada, pp. 238–245 (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.ADAPT Centre, School of ComputingDublin City UniversityDublinIreland

Personalised recommendations