Crowdsourcing and Translation Quality: Novel Approaches in the Language Industry and Translation Studies

  • Miguel A. Jiménez-CrespoEmail author
Part of the Machine Translation: Technologies and Applications book series (MATRA, volume 1)


Crowdsourcing involves the outsourcing of processes previously conducted by professionals in structured ways to communities and crowds using innovative workflows in order to achieve the best possible results. This chapter deals with the way in which the notion of quality has been impacted by the crowdsourcing revolution in translation. After defining the scope of what crowdsourcing is in translational contexts, it delves into the impact of crowdsourcing in terms of how the industry and translation studies conceptualise and implement quality. The main issues reviewed will be the consolidation of process-based approaches to guarantee quality, the expansion of the fitness for purpose model, and the distribution of responsibility to different agents that participate in the translation event. The chapter ends with an exploration of novel practices and workflows to guarantee quality inspired both by professional approaches and by MT research in existing crowdsourcing initiatives.


Translation quality assessment Principles to practice Community translation Fitness for purpose Translation process Translation workflows Translation studies 


  1. Aikawa T, Yamamoto K, Isahara H (2012) The impact of crowdsourcing post-editing with the collaborative translation framework. In: Proceedings of JapTAL 2012: advances in natural language processing 7614, pp 1–10. Springer, Berlin. Available via: Accessed 1 Oct 2016Google Scholar
  2. Allen J (2003) Post-editing. In: Somers H (ed) Computers and translation: a translators guide. John Benjamins, Amsterdam, pp 297–317CrossRefGoogle Scholar
  3. Ambati V, Vogel S, Carbonell J (2012) Collaborative workflow for crowdsourcing translation. In: Proceedings of ACM 2012 conference on computer supported cooperative work, pp 1191–1194. Available via: Accessed 1 Oct 2016
  4. Anastasiou D, Gupta R (2011) Comparison of crowdsourcing translation with machine translation. J Inf Sci 37(6):637–659CrossRefGoogle Scholar
  5. Arjona Reina L, Robles G, González-Barahona JM (2013) A preliminary analysis of localization in free software: how translations are performed. In: Petrinja E, Succi G, El Ioini N, Sillitti N (eds), Open source software: quality verification, pp 153–167, Springer, Heidelberg. CrossRefGoogle Scholar
  6. Bowker L, Buitrago J (2016) Investigating the usefulness of machine translation to newcomers in the public library. Trans Interpret 10(2):165–186Google Scholar
  7. Brabham D (2013) Crowdsourcing. MIT Press, Cambridge, MAGoogle Scholar
  8. Camara L (2015) Motivation for collaboration in TED open translation. Int J Web-Based Commun 11(2):210–229CrossRefGoogle Scholar
  9. Cao Y (2015) Crowdsourcing translation in contemporary China: an inspiring perspective of translation in the Web 2.0 age. Meta 60:316CrossRefGoogle Scholar
  10. Carson-Berndsen J, Somers H, Vogel C (2009) Integrated language technology as a part of next-generation localization. Localis Focus 81:53–66Google Scholar
  11. Chan S-W (ed) (2014) Routledge encyclopedia of translation technology. Routledge, LondonGoogle Scholar
  12. De Wille T, Exton C, Schäler R (2015) Multi-language communities, technology and perceived quality. Int Rep Socio-Inform 12:25–33Google Scholar
  13. DePalma DA (2015) CSOFT swipes left for translation, right for the source to mobilize translation. Common Sense Advisory Publications. Available iva: Accessed Oct 10 2016
  14. DePalma DA, Kelly N (2011) Project management for crowdsourced translation: how user-translated content projects work in real life. In: Dunne K, Dunne E (eds) Translation and localization project management: the art of the possible. John Benjamins, Amsterdam, pp 379–408CrossRefGoogle Scholar
  15. Deriemaeker J (2014) Power of the crowd: assessing crowd translation quality of tourist literature. Dissertation, Universiteit GhentGoogle Scholar
  16. Desilets A, van de Meer J (2011) Co-creating a repository of best-practices for collaborative translation. Linguistica Antverpiensia 10:11–27Google Scholar
  17. Doherty S (2016) The impact of translation technologies on the process and product of translation. Int J Commun Stud 9:1–23Google Scholar
  18. Dombek M (2014) A study into the motivations of internet users contributing to translation crowdsourcing: the case of Polish Facebook user-translators. Dissertation, Dublin City UniversityGoogle Scholar
  19. Drugan J (2013) Quality in professional translation. Bloomsbury, LondonGoogle Scholar
  20. Dunne K, Dunne E (eds) (2011) Translation and localization project management: the art of the possible. John Benjamins, AmsterdamGoogle Scholar
  21. Ehara Y, Baba Y, Utiyama M, Sumita E (2016) Assessing translation ability through vocabulary ability assessment. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence (IJCAI-16). Available via: Accessed 10 Oct 2016
  22. Estellés E, González F (2012) Towards an integrated crowdsourcing definition. J Inf Sci 38(2):189–200CrossRefGoogle Scholar
  23. Estellés E, Navarro-Giner R, González-Ladrón-de-Guevara F (2015) Crowdsourcing: definition and typology. In: Garrigos S, Gil P, Estellés M (eds) Advances in crowdsourcing. Springer, Heidelberg, pp 33–48CrossRefGoogle Scholar
  24. Exton C, Wasala A, Buckley J, Schäler R (2009) Micro crowdsourcing: a new model for software localization. Localis Focus 8(1):81–89Google Scholar
  25. Fernandez Costales A (2011) Facing the challenges of the global era. Paper presented at Tralogy I, Le Centre national de la recherche scientifique, Paris, 3–4 March 2011. Available via: Accessed 10 Dec 2017
  26. Filip D, Ó Conchúir E (2011) An argument for business process management in localisation. Localis Focus 10:4–17Google Scholar
  27. Gao M, Xu W, Callison-Burch C (2015) Cost optimization in crowdsourcing translation: low cost translations made even cheaper. In: Proceedings of the 2015 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pp 705–713Google Scholar
  28. Garcia I (2010) Is machine translation ready yet? Target 22(1):7–21CrossRefGoogle Scholar
  29. Garcia I (2015) Cloud marketplaces: procurement of translators in the age of social media. JoSTrans 23:18–38. Available via: Accessed 2 Oct 2016Google Scholar
  30. Görög A (2014a) Quantification and comparative evaluation of quality: the TAUS dynamic quality framework. Revista Tradumática 12:443–454. Available via:ática/article/view/n12-gorog2/pdf. Accessed 10 Oct 2016Google Scholar
  31. Görög A (2014b) Translation and quality: editorial. Revista Tradumática 12:388–391. Available via:ática/article/view/n12-gorog/pdf_2. Accessed 10 October 2016Google Scholar
  32. Goto S, Lin D, Ishida T (2014) Crowdsourcing for evaluating Machine Translation quality. In: Proceedings of LREC 2014, Reykjavík, pp 3456–3463Google Scholar
  33. Gouadec D (2007) Translation as a profession. John Benjamins, AmsterdamCrossRefGoogle Scholar
  34. Gouadec D (2010) Quality in translation. In: Gambier Y, van Doorslaer L (eds) Handbook of translation studies, p 270–275. John Benjamins, AmsterdamGoogle Scholar
  35. House J (1997) Translation quality assessment: a model revisited. Gunter Narr, TübingenGoogle Scholar
  36. House J (2014) Translation quality assessment: past and present. Routledge, LondonCrossRefGoogle Scholar
  37. Howe J (2008) Crowdsourcing: why the power of the crowd is driving the future of business. Crown Publishing Group, New YorkGoogle Scholar
  38. Hu C, Resnik P, Kronrod Y, Eidelman V, Buzek O, Bederson BB (2011) The value of monolingual crowdsourcing in a real-world translation scenario: simulation using Haitian creole emergency SMS messages. In: Proceedings of the sixth workshop on Statistical Machine Translation, Edinburgh, pp 399–404Google Scholar
  39. Jääskeläinen R (2016) Quality and translation process research. In: Muñoz Martín R (ed) Reembedding translation process research. John Benjamins, Amsterdam, pp 89–106CrossRefGoogle Scholar
  40. Jiménez-Crespo MA (2013) Crowdsourcing, corpus use, and the search for translation naturalness: a comparable corpus study of Facebook and non-translated social networking sites. Trans Interpret 8:23–49Google Scholar
  41. Jiménez-Crespo MA (2015) Translation quality, use and dissemination in an internet era: using single-translation and multi-translation parallel corpora to research translation quality on the web. JoSTrans 23:39–63Google Scholar
  42. Jiménez-Crespo MA (2016) Testing explicitation in translation: triangulating corpus and experimental studies. Across Lang Cult 16(2):257–283CrossRefGoogle Scholar
  43. Jiménez-Crespo MA (2017a) Crowdsourcing and collaborative translations: expanding the limits of translation studies. John Benjamins, AmsterdamCrossRefGoogle Scholar
  44. Jiménez-Crespo MA (2017b) How much would you like to pay? Reframing and expanding the notion of translation quality through crowdsourcing and volunteer approaches. Perspectives 25(3):478–491CrossRefGoogle Scholar
  45. Jiménez-Crespo MA (2017c) Mobile apps and translation crowdsourcing: The next frontier in the evolution of translation. Revista Tradumática 14:75–84. Available via:ática/article/view/167/pdf_31. Accessed 10 December 2017Google Scholar
  46. Kelly D (2005) A handbook for translator trainers. St Jerome, ManchesterGoogle Scholar
  47. Klaus C (2014) Translationsqualität und crowdsourced translation: Untertitlung und ihre Bewertung–am Beispiel des audiovisuellen Mediums TEDTalk. Frank & Timme GmbH, BerlinGoogle Scholar
  48. Koby GS, Fields P, Hague D, Lommel A, Melby A (2014) Defining translation quality. Revista Tradumàtica 12:413–420. Available via:ática/Tradumática_a2014n12/Tradumática_a2014n12p413.pdf. Accessed 10 Dec 2017Google Scholar
  49. Koponen M, Salmi L (2015) On the correctness of machine translation: A machine translation post-editing task. JoSTrans 23:118–136. Available via: Accessed 30 October 2016Google Scholar
  50. Lommel A, Burchardt A, Uszkoreit H (2014) Multidimensional Quality Metrics MQM: a framework for declaring and describing translation quality metrics. Revista Tradumática 12:455–463. Available via:ática/article/view/n12-lommel-uzskoreit-burchardt/pdf. Accessed 2 Oct 2016Google Scholar
  51. Martínez Melis N, Hurtado Albir A (2001) Assessment in translation studies: research needs. Meta 46(2):272–287CrossRefGoogle Scholar
  52. McDonough-Dolmaya J (2012) Analyzing the crowdsourcing model and its impact on public perceptions of translation. Translator 18(2):167–191CrossRefGoogle Scholar
  53. Mesipuu M (2012) Translation crowdsourcing and user-translator motivation at Facebook and Skype. Trans Space 1:33–53CrossRefGoogle Scholar
  54. Michalak K (2015) Online localization of Zooniverse citizen science projects - on the use of translation platforms as tools for translator education. Teach English Technol 3:61–72. Available via: Accessed 4 Oct 2016Google Scholar
  55. Mitchell L (2015) Community post-editing of machine-translated user-generated content. Dissertation, Dublin City UniversityGoogle Scholar
  56. Mitchell L, O’Brien S, Roturier J (2014) Quality evaluation in community post-editing. Mach Transl 28(3):237–262CrossRefGoogle Scholar
  57. Morera-Mesa A (2014) Crowdsourced translation practices from the process flow perspective. Dissertation, University of LimerickGoogle Scholar
  58. Morera-Mesa A, Aouad L, Collins JJ (2012) Assessing support for community workflows in localisation. Bus Process Manag Workshop Ser Lecture Note Bus Inf Process 99:195–206Google Scholar
  59. Morera-Mesa A, Collins JJ, Filip D (2014) Selected crowdsourced translation practices. In: Proceedings of translating and the computer 35, London, 28–29 November 2013. Available via: Accessed 2 Oct 2016
  60. Muzii L (2013) Is quality under pressure? Or is translation? Paper presented at TMT conference 2013, The Hague, 27 September 2013Google Scholar
  61. Nida E (1964) Towards a science of translation. Brill, LeidenGoogle Scholar
  62. Nida E, Taber CR (1969) The theory and practice of translation. Brill, LeidenGoogle Scholar
  63. Nord C (1997) Functionalist approaches explained. St. Jerome, ManchesterGoogle Scholar
  64. O’Brien S (2012) Towards a dynamic quality evaluation model for translation. JoSTrans 17:55–77. Available via: Accessed 4 Oct 2016Google Scholar
  65. O’Brien S, Schäler R (2010) Next generation translation and localization: users are taking charge. In: Proceedings from translating and the computer 32, London, 18–19 November 2010. Available via: Accessed 10 Oct 2016
  66. O’Hagan M (2013) The impact of new technologies on translation studies: a technological turn? In: Millán-Varela C, Bartrina F (eds) Routledge handbook of translation studies. Routledge, London, pp 503–518Google Scholar
  67. Olohan M (2014) Why do you translate? Motivation to volunteer and TED translation. Perspect Stud Translatol 7(1):17–33CrossRefGoogle Scholar
  68. Orrego-Carmona D (2015) The reception of non-professional subtitling. Dissertation, University Rovira i VirgiliGoogle Scholar
  69. Papineni K, Roukos S, Ward T, Zhu W (2002) BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting on Association for Computational Linguistics, Philadelphia, pp 311–318Google Scholar
  70. Persaud A, O’Brien S (2017) Quality and acceptance of crowdsourced translation of web content. Int J Technol Hum Interact 13(1):100–115CrossRefGoogle Scholar
  71. Pym A (2012) On translator ethics: principles for mediation between cultures. John Benjamins, AmsterdamCrossRefGoogle Scholar
  72. Ray R, Kelly N (2011) Crowdsourced translation: best practices for implementation. Common Sense Advisory, BostonGoogle Scholar
  73. Raymond ES (2001) The cathedral and the bazaar. O’Reilly and Associates, SebastopolGoogle Scholar
  74. Risku H, Rogl R, Pein-Weber C (2016) Mutual dependencies: centrality in translation networks. JoSTrans 25:232–253. Available via: Accessed 30 Oct 2016Google Scholar
  75. Shimohata S, Kitamura M, Sukehiro T, Murata T (2001) Collaborative translation environment on the web. In: Proceedings from MT Summit VIII, Santiago de Compostela, pp 331–334Google Scholar
  76. Siddique H (2011) Mob rule: Iceland crowdsources its next constitution. The Guardian, Thursday 9 June 2011. Available via: Accessed 10 Oct 2016
  77. Suojanen T, Koskinen K, Tuominen T (2015) User-centred translation. Routledge, LondonGoogle Scholar
  78. Tatsumi M, Aikawa T, Yamamoto K, Isahara H (2012) How good is crowd post-editing: its potential and limitations. In: Proceedings of the tenth biennial conference of the Association for Machine Translation in the Americas, San Diego, 28 October – 1 November 2012Google Scholar
  79. TAUS (2010) Post-editing guidelines. Available via: Accessed 10 Oct 2016
  80. TAUS (2014) Community evaluation best practices. Accessed 4 May 2018
  81. Utiyama M, Isahara H (2003) Reliable measures for aligning Japanese-English news articles and sentences. In: Proceedings of the 41st annual meeting of the Association for Computational Linguistics, pp 72–79Google Scholar
  82. Valli P (2015) Disrupt me not. Keynotes 2015 A review of the TAUS October Events, San Francisco, pp 46–54. Available via: Accessed 4 Mar 2016
  83. Volk M, Harder S (2007) Evaluating MT with translations or translators: what is the difference? In: Proceedings of MT Summit XI, Copenhagen, pp 499–506Google Scholar
  84. Wright SE (2006) Language industry standards. In: Dunne K (ed) Perspectives on localization. John Benjamins, Amsterdam, pp 241–278CrossRefGoogle Scholar
  85. Yan J, Song Y, Li CT, Zhang M, Hu X (2015) Opportunities or risks to reduce labor in crowdsourcing translation? Characterizing cost versus quality via a pagerank-hits hybrid model. In: Proceedings of the twenty-fourth international joint conference on artificial intelligence, Buenos Aires, pp 1025–1032Google Scholar
  86. Zaidan OF, Callison-Burch C (2011) Crowdsourcing translation: professional quality from non-professionals. In: Proceedings of the 49th annual meeting of the Association of Computational Linguistics, Portland, 19–24 June 2011, pp 1120–1129Google Scholar
  87. Zbib R, Markiewicz G, Matsoukas S, Schwartz R, Makhoul J (2013) Systematic comparison of professional and crowdsourced reference translations for machine translation. In: Proceedings of the 2013 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, 9–14 June 2013, pp 612–616Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Arts and SciencesRutgers UniversityNew BrunswickUSA
  2. 2.Rutgers, The State University of New JerseyNew BrunswickUSA

Personalised recommendations