Skip to main content

Crowdsourcing and Translation Quality: Novel Approaches in the Language Industry and Translation Studies

  • Chapter
  • First Online:
Translation Quality Assessment

Part of the book series: Machine Translation: Technologies and Applications ((MATRA,volume 1))

Abstract

Crowdsourcing involves the outsourcing of processes previously conducted by professionals in structured ways to communities and crowds using innovative workflows in order to achieve the best possible results. This chapter deals with the way in which the notion of quality has been impacted by the crowdsourcing revolution in translation. After defining the scope of what crowdsourcing is in translational contexts, it delves into the impact of crowdsourcing in terms of how the industry and translation studies conceptualise and implement quality. The main issues reviewed will be the consolidation of process-based approaches to guarantee quality, the expansion of the fitness for purpose model, and the distribution of responsibility to different agents that participate in the translation event. The chapter ends with an exploration of novel practices and workflows to guarantee quality inspired both by professional approaches and by MT research in existing crowdsourcing initiatives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.innocentive.com/

  2. 2.

    https://www.galaxyzoo.org/

  3. 3.

    https://unbabel.com/

  4. 4.

    https://www.stepes.com/

  5. 5.

    These are Read-Only Memory files, commonly used for emulation of outdated or incompatible software such as video or arcade games.

  6. 6.

    According to the working definition of quality for MQM, “a quality translation demonstrates the accuracy and fluency required for the audience and purpose and complies with all other specifications negotiated between the requester and provider, taking into account end-user needs” (Koby et al. 2014).

  7. 7.

    https://www.kiva.org/

  8. 8.

    https://www.ted.com/participate/translate

  9. 9.

    https://gengo.com/

  10. 10.

    https://www.getlocalization.com/

  11. 11.

    http://www.speaklike.com/

  12. 12.

    The notion of adequacy here is understood in functionalist terms (Nord 1997) to accept that translations can be more or less adequate for the purposes intended, and not the common use in MT to indicate that the translation is more or less coherent with the meaning of the source text (Papineni et al. 2002), also often used in recent standardisation efforts (i.e. Görög 2014a).

  13. 13.

    Defined as a rough translation “to get some essential information about what is in the text and for a user to define whether to translate it in full or not to serve some specific purposes” (Chan 2014).

  14. 14.

    https://www.onehourtranslation.com

  15. 15.

    https://www.facebook.com/TranslateFacebookTeam/

  16. 16.

    https://translatorswithoutborders.org/

  17. 17.

    https://translations.launchpad.net/

  18. 18.

    http://www.cucumis.org/translation_1_w/

  19. 19.

    http://www.linqapp.com/

  20. 20.

    https://www.transifex.com/

  21. 21.

    https://ackuna.com/

  22. 22.

    https://trommons.org/

  23. 23.

    http://ecom.trans-aid.jp/

  24. 24.

    https://www.symantec.com/

  25. 25.

    https://translate.google.com/

  26. 26.

    https://www.bing.com/translator

  27. 27.

    http://translate.google.com/manager/website/?hl=en

  28. 28.

    More recently Omniscien Technologies (see https://omniscien.com/)

  29. 29.

    https://www.crowdflower.com/

  30. 30.

    https://www.wikipedia.org/

  31. 31.

    https://amara.org/en/

References

  • Aikawa T, Yamamoto K, Isahara H (2012) The impact of crowdsourcing post-editing with the collaborative translation framework. In: Proceedings of JapTAL 2012: advances in natural language processing 7614, pp 1–10. Springer, Berlin. Available via: http://research.microsoft.com/pubs/172592/JapTal2012.pdf. Accessed 1 Oct 2016

    Google Scholar 

  • Allen J (2003) Post-editing. In: Somers H (ed) Computers and translation: a translators guide. John Benjamins, Amsterdam, pp 297–317

    Chapter  Google Scholar 

  • Ambati V, Vogel S, Carbonell J (2012) Collaborative workflow for crowdsourcing translation. In: Proceedings of ACM 2012 conference on computer supported cooperative work, pp 1191–1194. Available via: https://www.cs.cmu.edu/~jgc/CollaborativeWorkflowforCrowdsourcingTranslationACMCoCSCW2012.pdf. Accessed 1 Oct 2016

  • Anastasiou D, Gupta R (2011) Comparison of crowdsourcing translation with machine translation. J Inf Sci 37(6):637–659

    Article  Google Scholar 

  • Arjona Reina L, Robles G, González-Barahona JM (2013) A preliminary analysis of localization in free software: how translations are performed. In: Petrinja E, Succi G, El Ioini N, Sillitti N (eds), Open source software: quality verification, pp 153–167, Springer, Heidelberg. https://doi.org/10.1007/978-3-642-38928-3_11

    Chapter  Google Scholar 

  • Bowker L, Buitrago J (2016) Investigating the usefulness of machine translation to newcomers in the public library. Trans Interpret 10(2):165–186

    Google Scholar 

  • Brabham D (2013) Crowdsourcing. MIT Press, Cambridge, MA

    Google Scholar 

  • Camara L (2015) Motivation for collaboration in TED open translation. Int J Web-Based Commun 11(2):210–229

    Article  Google Scholar 

  • Cao Y (2015) Crowdsourcing translation in contemporary China: an inspiring perspective of translation in the Web 2.0 age. Meta 60:316

    Article  Google Scholar 

  • Carson-Berndsen J, Somers H, Vogel C (2009) Integrated language technology as a part of next-generation localization. Localis Focus 81:53–66

    Google Scholar 

  • Chan S-W (ed) (2014) Routledge encyclopedia of translation technology. Routledge, London

    Google Scholar 

  • De Wille T, Exton C, Schäler R (2015) Multi-language communities, technology and perceived quality. Int Rep Socio-Inform 12:25–33

    Google Scholar 

  • DePalma DA (2015) CSOFT swipes left for translation, right for the source to mobilize translation. Common Sense Advisory Publications. Available iva: http://www.commonsenseadvisory.com/Default.aspx?Contenttype=ArticleDetAD&tabID=63&Aid=36417&moduleId=390. Accessed Oct 10 2016

  • DePalma DA, Kelly N (2011) Project management for crowdsourced translation: how user-translated content projects work in real life. In: Dunne K, Dunne E (eds) Translation and localization project management: the art of the possible. John Benjamins, Amsterdam, pp 379–408

    Chapter  Google Scholar 

  • Deriemaeker J (2014) Power of the crowd: assessing crowd translation quality of tourist literature. Dissertation, Universiteit Ghent

    Google Scholar 

  • Desilets A, van de Meer J (2011) Co-creating a repository of best-practices for collaborative translation. Linguistica Antverpiensia 10:11–27

    Google Scholar 

  • Doherty S (2016) The impact of translation technologies on the process and product of translation. Int J Commun Stud 9:1–23

    Google Scholar 

  • Dombek M (2014) A study into the motivations of internet users contributing to translation crowdsourcing: the case of Polish Facebook user-translators. Dissertation, Dublin City University

    Google Scholar 

  • Drugan J (2013) Quality in professional translation. Bloomsbury, London

    Google Scholar 

  • Dunne K, Dunne E (eds) (2011) Translation and localization project management: the art of the possible. John Benjamins, Amsterdam

    Google Scholar 

  • Ehara Y, Baba Y, Utiyama M, Sumita E (2016) Assessing translation ability through vocabulary ability assessment. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence (IJCAI-16). Available via: https://pdfs.semanticscholar.org/47ea/223867dbe59f73f4f8ed1f429cfdb7abd68e.pdf. Accessed 10 Oct 2016

  • Estellés E, González F (2012) Towards an integrated crowdsourcing definition. J Inf Sci 38(2):189–200

    Article  Google Scholar 

  • Estellés E, Navarro-Giner R, González-Ladrón-de-Guevara F (2015) Crowdsourcing: definition and typology. In: Garrigos S, Gil P, Estellés M (eds) Advances in crowdsourcing. Springer, Heidelberg, pp 33–48

    Chapter  Google Scholar 

  • Exton C, Wasala A, Buckley J, Schäler R (2009) Micro crowdsourcing: a new model for software localization. Localis Focus 8(1):81–89

    Google Scholar 

  • Fernandez Costales A (2011) Facing the challenges of the global era. Paper presented at Tralogy I, Le Centre national de la recherche scientifique, Paris, 3–4 March 2011. Available via: http://lodel.irevues.inist.fr/tralogy/index.php?id=120. Accessed 10 Dec 2017

  • Filip D, Ó Conchúir E (2011) An argument for business process management in localisation. Localis Focus 10:4–17

    Google Scholar 

  • Gao M, Xu W, Callison-Burch C (2015) Cost optimization in crowdsourcing translation: low cost translations made even cheaper. In: Proceedings of the 2015 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pp 705–713

    Google Scholar 

  • Garcia I (2010) Is machine translation ready yet? Target 22(1):7–21

    Article  Google Scholar 

  • Garcia I (2015) Cloud marketplaces: procurement of translators in the age of social media. JoSTrans 23:18–38. Available via: http://www.jostrans.org/issue23/art_garcia.pdf. Accessed 2 Oct 2016

    Google Scholar 

  • Görög A (2014a) Quantification and comparative evaluation of quality: the TAUS dynamic quality framework. Revista Tradumática 12:443–454. Available via: http://revistes.uab.cat/Tradumática/article/view/n12-gorog2/pdf. Accessed 10 Oct 2016

    Google Scholar 

  • Görög A (2014b) Translation and quality: editorial. Revista Tradumática 12:388–391. Available via: http://revistes.uab.cat/Tradumática/article/view/n12-gorog/pdf_2. Accessed 10 October 2016

    Google Scholar 

  • Goto S, Lin D, Ishida T (2014) Crowdsourcing for evaluating Machine Translation quality. In: Proceedings of LREC 2014, Reykjavík, pp 3456–3463

    Google Scholar 

  • Gouadec D (2007) Translation as a profession. John Benjamins, Amsterdam

    Book  Google Scholar 

  • Gouadec D (2010) Quality in translation. In: Gambier Y, van Doorslaer L (eds) Handbook of translation studies, p 270–275. John Benjamins, Amsterdam

    Google Scholar 

  • House J (1997) Translation quality assessment: a model revisited. Gunter Narr, Tübingen

    Google Scholar 

  • House J (2014) Translation quality assessment: past and present. Routledge, London

    Book  Google Scholar 

  • Howe J (2008) Crowdsourcing: why the power of the crowd is driving the future of business. Crown Publishing Group, New York

    Google Scholar 

  • Hu C, Resnik P, Kronrod Y, Eidelman V, Buzek O, Bederson BB (2011) The value of monolingual crowdsourcing in a real-world translation scenario: simulation using Haitian creole emergency SMS messages. In: Proceedings of the sixth workshop on Statistical Machine Translation, Edinburgh, pp 399–404

    Google Scholar 

  • Jääskeläinen R (2016) Quality and translation process research. In: Muñoz Martín R (ed) Reembedding translation process research. John Benjamins, Amsterdam, pp 89–106

    Chapter  Google Scholar 

  • Jiménez-Crespo MA (2013) Crowdsourcing, corpus use, and the search for translation naturalness: a comparable corpus study of Facebook and non-translated social networking sites. Trans Interpret 8:23–49

    Google Scholar 

  • Jiménez-Crespo MA (2015) Translation quality, use and dissemination in an internet era: using single-translation and multi-translation parallel corpora to research translation quality on the web. JoSTrans 23:39–63

    Google Scholar 

  • Jiménez-Crespo MA (2016) Testing explicitation in translation: triangulating corpus and experimental studies. Across Lang Cult 16(2):257–283

    Article  Google Scholar 

  • Jiménez-Crespo MA (2017a) Crowdsourcing and collaborative translations: expanding the limits of translation studies. John Benjamins, Amsterdam

    Book  Google Scholar 

  • Jiménez-Crespo MA (2017b) How much would you like to pay? Reframing and expanding the notion of translation quality through crowdsourcing and volunteer approaches. Perspectives 25(3):478–491

    Article  Google Scholar 

  • Jiménez-Crespo MA (2017c) Mobile apps and translation crowdsourcing: The next frontier in the evolution of translation. Revista Tradumática 14:75–84. Available via: http://revistes.uab.cat/Tradumática/article/view/167/pdf_31. Accessed 10 December 2017

    Google Scholar 

  • Kelly D (2005) A handbook for translator trainers. St Jerome, Manchester

    Google Scholar 

  • Klaus C (2014) Translationsqualität und crowdsourced translation: Untertitlung und ihre Bewertung–am Beispiel des audiovisuellen Mediums TEDTalk. Frank & Timme GmbH, Berlin

    Google Scholar 

  • Koby GS, Fields P, Hague D, Lommel A, Melby A (2014) Defining translation quality. Revista Tradumàtica 12:413–420. Available via: https://ddd.uab.cat/pub/Tradumática/Tradumática_a2014n12/Tradumática_a2014n12p413.pdf. Accessed 10 Dec 2017

    Google Scholar 

  • Koponen M, Salmi L (2015) On the correctness of machine translation: A machine translation post-editing task. JoSTrans 23:118–136. Available via: http://www.jostrans.org/issue23/art_koponen.pdf. Accessed 30 October 2016

    Google Scholar 

  • Lommel A, Burchardt A, Uszkoreit H (2014) Multidimensional Quality Metrics MQM: a framework for declaring and describing translation quality metrics. Revista Tradumática 12:455–463. Available via: http://revistes.uab.cat/Tradumática/article/view/n12-lommel-uzskoreit-burchardt/pdf. Accessed 2 Oct 2016

    Google Scholar 

  • Martínez Melis N, Hurtado Albir A (2001) Assessment in translation studies: research needs. Meta 46(2):272–287

    Article  Google Scholar 

  • McDonough-Dolmaya J (2012) Analyzing the crowdsourcing model and its impact on public perceptions of translation. Translator 18(2):167–191

    Article  Google Scholar 

  • Mesipuu M (2012) Translation crowdsourcing and user-translator motivation at Facebook and Skype. Trans Space 1:33–53

    Article  Google Scholar 

  • Michalak K (2015) Online localization of Zooniverse citizen science projects - on the use of translation platforms as tools for translator education. Teach English Technol 3:61–72. Available via: http://yadda.icm.edu.pl/yadda/element/bwmeta1.element.desklight-32a1582e-2f8d-412c-80c6-d6d03086424d. Accessed 4 Oct 2016

    Google Scholar 

  • Mitchell L (2015) Community post-editing of machine-translated user-generated content. Dissertation, Dublin City University

    Google Scholar 

  • Mitchell L, O’Brien S, Roturier J (2014) Quality evaluation in community post-editing. Mach Transl 28(3):237–262

    Article  Google Scholar 

  • Morera-Mesa A (2014) Crowdsourced translation practices from the process flow perspective. Dissertation, University of Limerick

    Google Scholar 

  • Morera-Mesa A, Aouad L, Collins JJ (2012) Assessing support for community workflows in localisation. Bus Process Manag Workshop Ser Lecture Note Bus Inf Process 99:195–206

    Google Scholar 

  • Morera-Mesa A, Collins JJ, Filip D (2014) Selected crowdsourced translation practices. In: Proceedings of translating and the computer 35, London, 28–29 November 2013. Available via: http://www.mt-archive.info/10/Aslib-2013-Morera-Mesa.pdf. Accessed 2 Oct 2016

  • Muzii L (2013) Is quality under pressure? Or is translation? Paper presented at TMT conference 2013, The Hague, 27 September 2013

    Google Scholar 

  • Nida E (1964) Towards a science of translation. Brill, Leiden

    Google Scholar 

  • Nida E, Taber CR (1969) The theory and practice of translation. Brill, Leiden

    Google Scholar 

  • Nord C (1997) Functionalist approaches explained. St. Jerome, Manchester

    Google Scholar 

  • O’Brien S (2012) Towards a dynamic quality evaluation model for translation. JoSTrans 17:55–77. Available via: http://www.jostrans.org/issue17/art_obrien.pdf. Accessed 4 Oct 2016

    Google Scholar 

  • O’Brien S, Schäler R (2010) Next generation translation and localization: users are taking charge. In: Proceedings from translating and the computer 32, London, 18–19 November 2010. Available via: http://doras.dcu.ie/16695/1/Paper_6.pdf. Accessed 10 Oct 2016

  • O’Hagan M (2013) The impact of new technologies on translation studies: a technological turn? In: Millán-Varela C, Bartrina F (eds) Routledge handbook of translation studies. Routledge, London, pp 503–518

    Google Scholar 

  • Olohan M (2014) Why do you translate? Motivation to volunteer and TED translation. Perspect Stud Translatol 7(1):17–33

    Article  Google Scholar 

  • Orrego-Carmona D (2015) The reception of non-professional subtitling. Dissertation, University Rovira i Virgili

    Google Scholar 

  • Papineni K, Roukos S, Ward T, Zhu W (2002) BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting on Association for Computational Linguistics, Philadelphia, pp 311–318

    Google Scholar 

  • Persaud A, O’Brien S (2017) Quality and acceptance of crowdsourced translation of web content. Int J Technol Hum Interact 13(1):100–115

    Article  Google Scholar 

  • Pym A (2012) On translator ethics: principles for mediation between cultures. John Benjamins, Amsterdam

    Book  Google Scholar 

  • Ray R, Kelly N (2011) Crowdsourced translation: best practices for implementation. Common Sense Advisory, Boston

    Google Scholar 

  • Raymond ES (2001) The cathedral and the bazaar. O’Reilly and Associates, Sebastopol

    Google Scholar 

  • Risku H, Rogl R, Pein-Weber C (2016) Mutual dependencies: centrality in translation networks. JoSTrans 25:232–253. Available via: http://www.jostrans.org/issue25/art_risku.pdf. Accessed 30 Oct 2016

    Google Scholar 

  • Shimohata S, Kitamura M, Sukehiro T, Murata T (2001) Collaborative translation environment on the web. In: Proceedings from MT Summit VIII, Santiago de Compostela, pp 331–334

    Google Scholar 

  • Siddique H (2011) Mob rule: Iceland crowdsources its next constitution. The Guardian, Thursday 9 June 2011. Available via: http://www.theguardian.com/world/2011/jun/09/iceland-crowdsourcing-constitution-facebook. Accessed 10 Oct 2016

  • Suojanen T, Koskinen K, Tuominen T (2015) User-centred translation. Routledge, London

    Google Scholar 

  • Tatsumi M, Aikawa T, Yamamoto K, Isahara H (2012) How good is crowd post-editing: its potential and limitations. In: Proceedings of the tenth biennial conference of the Association for Machine Translation in the Americas, San Diego, 28 October – 1 November 2012

    Google Scholar 

  • TAUS (2010) Post-editing guidelines. Available via: https://www.taus.net/academy/best-practices/postedit-best-practices/machine-translation-post-editing-guidelines. Accessed 10 Oct 2016

  • TAUS (2014) Community evaluation best practices. https://www.taus.net/academy/best-practices/evaluate-best-practices/community-evaluation-best-practices. Accessed 4 May 2018

  • Utiyama M, Isahara H (2003) Reliable measures for aligning Japanese-English news articles and sentences. In: Proceedings of the 41st annual meeting of the Association for Computational Linguistics, pp 72–79

    Google Scholar 

  • Valli P (2015) Disrupt me not. Keynotes 2015 A review of the TAUS October Events, San Francisco, pp 46–54. Available via: https://www.taus.net/blog/disrupt-me-not. Accessed 4 Mar 2016

  • Volk M, Harder S (2007) Evaluating MT with translations or translators: what is the difference? In: Proceedings of MT Summit XI, Copenhagen, pp 499–506

    Google Scholar 

  • Wright SE (2006) Language industry standards. In: Dunne K (ed) Perspectives on localization. John Benjamins, Amsterdam, pp 241–278

    Chapter  Google Scholar 

  • Yan J, Song Y, Li CT, Zhang M, Hu X (2015) Opportunities or risks to reduce labor in crowdsourcing translation? Characterizing cost versus quality via a pagerank-hits hybrid model. In: Proceedings of the twenty-fourth international joint conference on artificial intelligence, Buenos Aires, pp 1025–1032

    Google Scholar 

  • Zaidan OF, Callison-Burch C (2011) Crowdsourcing translation: professional quality from non-professionals. In: Proceedings of the 49th annual meeting of the Association of Computational Linguistics, Portland, 19–24 June 2011, pp 1120–1129

    Google Scholar 

  • Zbib R, Markiewicz G, Matsoukas S, Schwartz R, Makhoul J (2013) Systematic comparison of professional and crowdsourced reference translations for machine translation. In: Proceedings of the 2013 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, 9–14 June 2013, pp 612–616

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Miguel A. Jiménez-Crespo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jiménez-Crespo, M.A. (2018). Crowdsourcing and Translation Quality: Novel Approaches in the Language Industry and Translation Studies. In: Moorkens, J., Castilho, S., Gaspari, F., Doherty, S. (eds) Translation Quality Assessment. Machine Translation: Technologies and Applications, vol 1. Springer, Cham. https://doi.org/10.1007/978-3-319-91241-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91241-7_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91240-0

  • Online ISBN: 978-3-319-91241-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics