Skip to main content
SpringerLink
Log in

Turking in a Global Labour Market

  • Published:
Computer Supported Cooperative Work (CSCW) Aims and scope Submit manuscript

Abstract

This paper examines how working in the global labour market of Amazon Mechanical Turk (AMT) impacts upon and is understood by two different groups of workers. To do this we report on two qualitative studies; one of US and another of Indian crowdworkers (or ‘Turkers’) that we analysed from an ethnomethodological orientation. Our data is naturalistic and comes from a variety of sources—interviews, observations and forum posts—where Turkers describe their work, and their understandings of that work and of the transnational market they work in. We compare and contrast their situations, their reflections on the marketplace and its global reach, and we take a look at their understandings of one another. Our focus is on ‘the work to making turking work’ (Martin et al. 2014). That is, the work that turkers do to organise and make sense of their work as they operate in the AMT marketplace, such that they can do so effectively. AMT is a technologically mediated marketplace—that is the distribution, completion and payment of work is done online, almost completely through the AMT platform. The design of the platform has important consequences for how Turkers experience and understand the market (including its global or transnational nature). We discuss how our findings relate to a variety of CSCW issues and provide an initial examination of how they relate to globalisation both as a mundanely experienced phenomenon and as a topic of academic interest. We finish the article by drawing on our own experiences in research and design to look at how technology can be used to intervene in a market like this to try to address imbalances in power and agency between employers and workers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4

Similar content being viewed by others

Notes

  1. See Felsteiner (2011) for a detailed examination of wages and the legal issues.

  2. Ipeirotis offers more up-to-date and dynamic demographic and statistical data with MTurk Tracker: http://www.mturk-tracker.com/#/general

  3. “The current federal minimum wage is $7.25 per hour.” Source: http://poverty.ucdavis.edu/faq/what-are-annual-earnings-full-time-minimum-wage-worker

  4. http://turkernation.com/showthread.php?21305-A-Paper-on-Turker-Nation-Oh-My!-(Now-with-the-link-to-the-full-paper!)

  5. http://www.census.gov/content/dam/Census/library/publications/2014/demo/p60-249.pdf - page 43, $12,119 for a single person under 65 years of age.

  6. Suddenly Amazon made electronic payments available to Indian Turkers in March 2015, in a move that we welcome, finally redressing this asymmetry: http://www.mturkgrind.com/threads/mturk-earnings-withdrawal-transfer-options.27510/

  7. One of the clearest benefits of the Masters qualification is that Turkers do not have to complete captchas on many tasks that usually require them – this saves quite a lot of time over repetitive bulk HITs.

  8. http://turkrequesters.blogspot.fr/2013/01/the-reasons-why-amazon-mechanical-turk.html

  9. Although at the time of writing we have heard of considerable rumours about the widespread closing of Indian accounts. Unfortunately the lack of transparency at Amazon means it is impossible at this stage to verify this.

  10. MMOs provide significant commercial opportunities (selling virtual currency and items). It has been reported that in e.g. India and China people work to accumulate these under duress so they can be sold to other gamers to profit ‘ringleaders’. Aleppo suggests something similar may be happening with AMT.

  11. It was reported to us that a steady flow of unreasonable requests (i.e. find me good HITs, get my suspension lifted) accompanied by some abuse had led to a blanket ban [personal communication from Turker Nation admin]. It seems that this was a product of misunderstanding on some Indian Turkers’ part (that the forum had an influential relationship with AMT, that US workers did not need the money so much).

  12. Source: State University of New York, Levin Institute: http://www.globalization101.org/what-is-globalisation

  13. http://turkernation.com/showthread.php?8027-Must-read-for-turkers!-Guideline-for-requester-pay

  14. http://wiki.wearedynamo.org/

References

  • Ahmed, Syed Ishtiaque, Nicola Bidwell, Himanshu Zade, Srihari Muralidhar, Anupama Dhareshwar, Baneen Karachiwala, Cedrick Tandong, Jacki O’Neill (2016). Peer-to-peer in the workplace: A view from the road. To appear in C. Lampe and K. Inkpen (Eds): Proceedings of CHI 2016, San Jose, CA, USA, May 7–12, 2016. New York: ACM Press.

  • Bederson, Benjamin. B. and Alexander. J. Quinn (2011). Web workers unite! Addressing challenges of online laborers. In B. Begole and W. Kellog (Eds): Proceedings of CHI ‘11 Extended Abstracts. Vancouver, Canada 7–12 May 2011. New York: ACM press, 97–106.

  • Bjørn, Pernille and Nina Boulus-Rødje (2015). The multiple intersecting sites of design in CSCW research. Computer Supported Cooperative Work (CSCW): An International Journal, vol. 24, no. 3, 2015, pp. 319–351.

  • Blomberg, Jeanette and Helen Karasti (2013). Reflections on 25 years or ethnography in CSCW. Computer Supported Cooperative Work (CSCW): An International Journal, vol. 22, nos. 4–6, 2013, pp. 373–423.

  • Boden, Deirdre. (1994). The Business of Talk: organisations in action. Cambridge: Polity Press.

    Google Scholar 

  • Boden, Alexander, Bernhard Nett, and Volker Wolf (2009). Trust and Social Capital: Revisiting an Offshoring Failure Story of a Small German Software Company. In E. Balka and C. Simone (Eds): Proceedings of ECSCW 2009, Vienna Austria 7–11th September 2009, London: Springer, pp. 123–142.

  • Bourne, Kristina A., and Forman, Pamela J. (2013). Living in a Culture of Overwork: An Ethnographic Study of Flexibility. Journal of Management Inquiry, DOI: 10.1177/1056492613481245

    Google Scholar 

  • Crabtree, Andy (2003). Designing Collaborative Systems: A Practical Guide to Ethnography, London: Springer.

    Google Scholar 

  • Crary, Jonathan (2013). 24/7: Late Capitalism and the Ends of Sleep. Verso: London.

    Google Scholar 

  • Felsteiner, Alek (2011). Working the crowd: employment and labor law in the crowdsourcing industry. Berkeley Journal of Employment & Labor Law, vol. 32 no. 1, pp. 143–204.

    Google Scholar 

  • Fort, Karen., Gilles Adda and K. Bretonnel Cohen (2011). Amazon Mechanical Turk: Gold Mine or Coal Mine? Computational Linguisitcs, vol. 37, no. 2, pp. 413–420.

    Article  Google Scholar 

  • Gajera, Rinku and Jacki O’Neill (2014). Ethnography in Parallel. In C. Rossitto, L. Ciolfi, D. Martin and B. Conein (Eds): Proceedings of Coop 2014, 27th–30th May, 2014, Nice, France. London: Springer, pp. 259–276.

    Google Scholar 

  • Garfinkel, Harold (1967). Studies in ethnomethodology. Englewood Cliffs, N.J.: Prentice-Hall.

    Google Scholar 

  • Gawade, Mrunal, Rajan Vaish, Mercy Nduta Waihumbu and James Davis (2012). Exploring Employment Opportunities through Microtasks via Cybercafes. In Proceeding. of IEEE Global Humanitarian Technology Conference, 21 Oct - 24 Oct 2012, Seattle, WA, USA. Washington, DC: IEEE Computer Society, pp. 77–82.

  • Geertz, Clifford (2001). The World in Pieces: Culture and Politics at the End of the Century. In C. Geertz, Available Light: Anthropological Reflections on Philosophical Topics. Princeton, NJ: Princeton University Press, pp. 218–63.

    Google Scholar 

  • Graeber, David (2015). The Utopia of Rules: On Technology, Stupidity and the Secret Joys of Bureaucracy. Brooklyn, NY: First Melville House

    Google Scholar 

  • Gray, Mary L., Siddharth Suri, Syed S. Ali, and Deepti Kulkarni (2016). The Crowd is a Collaborative Network. To appear in P. Bjorn and D. Konstan (Eds): Proceedings of Computer-Supported Cooperative Work 2016, February 27 – March 2 2016. New York: ACM press.

  • Gupta, Neha, David Martin, Benjamin V. Hanrahan & Jacki O’Neill (2014). Turk-Life In India. In D. McDonald and P. Bjorn (Eds): Proceedings of Group 2014, Sanibel Island, Florida, 9-12th November. New York: ACM press, pp. 1–11.

    Google Scholar 

  • Hanrahan, Benjamin V., Jutta Willamowski, Saiganesh Swaminathan, and David Martin (2015). Turkbench: Rendering the Market of Turkers. In B. Begole and J. Kim (Eds): Proceedings of CHI 2015, Seoul, South Korea, April 18–23, 2015. New York: ACM press, pp. 1613-1616.

    Google Scholar 

  • Harper, Richard, Christian Bird, Tom Zimmermann and Brendan Murphy (2013). Dwelling in Software: Aspects of the felt-life of engineers in large software projects. In O. Bertelson, L. Ciofli, M. A. Grasso and A. Papadopoulos: Proceedings of ECSCW 13, Paphos, Cyprus, September 21–25, 2013. London: Springer, pp. 161–180.

    Google Scholar 

  • Held, David, Anthony McGrew, David Goldblatt, and Jonathan Perraton (1999). Global Transformations. Politics, Economics and Culture. Cambridge: Polity Press

    Google Scholar 

  • Howe, Jeff. Crowdsourcing: A definition. (2006). http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html

  • Hughes, John A., David Randall, & Dan Shapiro (1992). Faltering from ethnography to design. In M. Mantel and R. Baeker (Eds): Proceedings of Computer-Supported Cooperative Work 92, Toronto, Canada, November 1–4, 1992. New York: ACM press, pp. 115–122.

    Google Scholar 

  • Hughes, John, Val King, Tom Rodden and Hans Andersen (1994). Moving Out from the Control Room: Ethnography in System Design. In J. Smith, F. Smith and T. Malone (Eds): Proceedings of Computer-Supported Cooperative Work 94, Chapel Hill, NC, USA. New York: ACM press, pp. 429–439.

    Google Scholar 

  • Huwart, Jean-Yves and Loic Verdier (2013). OECD Report “Does Globalisation Promote Employment?” http://dx.doi.org/10.1787/9789264111905-en

  • Ipeirotis, Panos. (2010a). Analyzing the Amazon Mechanical Turk marketplace. Ceder Working Papers No. CeDER-10-04 (2010a). http://hdl.handle.net/2451/29801

  • Ipeirotis, Panos. (2010b). Demographics of Mechanical Turk. CeDER Working Papers No. CeDER-10-01 (2010b) http://hdl.handle.net/2451/29585

  • Irani, Lilly and Max Six Silberman (2013). Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk. In S. Brewster and S. Bødker (Eds): Proceedings of CHI 13, Paris, France, 22 April – 2 May 2013. New York: ACM press, pp. 611–620.

    Google Scholar 

  • Isaac, Mike & Natasha Singer. (2015) California Says Uber Driver Is Employee, Not a Contractor http://tinyurl.com/nc3n7z8

  • Khanna, Shashank, Aishwarya Ratan, James Davis and William Thies (2010). Evaluating and Improving the Usability of Mechanical Turk for Low-Income Workers in India. In A. Dearden, J. Parihk and L. Subrimanian (Eds): Proceedings of ACM DEV 2010, December 17–18, 2010, London, United Kingdom, New York: ACM press, article no. 12.

  • Kittur, Aniket, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease and John Horton (2013). The future of crowd work. In Proceedings of Computer-Supported Cooperative Work’13, February 23–27, 2013. New York: ACM press, pp. 1301–1318.

    Google Scholar 

  • Kuek, Siou C., Cecilia Paradi-Guilford, Toks Fayomi, Saori Imaizumi, Panos Ipeirotis, Patricia Pina, and Manpreet Singh (2015). The Global Opportunity in Online Outsourcing. World Bank Studies. http://documents.worldbank.org/curated/en/2015/06/24702763/global-opportunity-online-outsourcing

  • Lee, Min K., Daniel Kusbit, Evan Metsky and Laura Dabbish (2015). Working with Machines: The Impact of Algorithmic and Data Driven Management on Human Workers. In B. Begole and J. Kim (Eds): Proceedings of CHI 2015, Seoul, South Korea, April 18–23, 2015. New York: ACM press, pp. 1603–1612.

    Google Scholar 

  • Marcus, George E. (1995). Ethnography in/of the World System: The Emergence of Multi-Sited Ethnography. Annual Review of Anthropology, Vol. 24, pp. 95--117

  • Martin, David and Jacki O’Neill (2011). Talk: Talking the Organisation into Being. In M. Rouncefield & P. Tolmie (Eds): Ethnomethodology at Work, Farnham, England: Ashgate, pp. 109–129.

    Google Scholar 

  • Martin, D., Benjamin Hanrahan, Jacki O’Neill, J. and Neha Gupta (2014). Being A Turker. In M. Ringel Morris and M. Reddy (Eds): Proceedings of Computer-Supported Cooperative Work 14, Baltimore US, 15-19th February. New York: ACM press, pp. 224–235

    Google Scholar 

  • Matthiesen, Stina, Pernille Bjorn, and Lise Møller Peterson, L. (2014). Figure out how to code with the hands of others": recognizing cultural blind spots in global software development. In M. Ringel Morris and M. Reddy (Eds): Proceedings of Computer-Supported Cooperative Work 14, Baltimore US, 15-19th February. New York: ACM press, pp. 1107–1119.

    Google Scholar 

  • Morozoz, E. (2014). To Save Everything Click Here: Technology, solutionism and the urge to fix problems that don’t exist. Penguin (first published in 2013).

  • O’Neill, Jacki & David Martin (2003). Text Chat In Action. In M. Tremaine and C. Simone (Eds): Proceedings of the 2003 international ACM SIGGROUP conference on Supporting group work, Sanibel Island, Florida, GROUP ‘03. New York: ACM Press, pp. 40–49.

    Chapter  Google Scholar 

  • O’Neill, Jacki & David Martin (2013). Relationship-Based Business Process Crowdsourcing. In P. Kotzé, G. Marsdan, G. Lingaard, J. Wesson and M. Winckler (Eds): Proceedings of INTERACT 2013, 14th IFIP TC 13 International Conference, Cape Town, South Africa, September 2-6, 2013, Proceedings, Part IV, pp. 429--446

  • Olson, Gary, & Judith Olson (2000). Distance matters. Human-computer interaction, vol. 15, no. 2, pp. 139–178.

    Article  Google Scholar 

  • Raval, Noopur and Paul Dourish. Standing Out from the Crowd: Emotional Labor, Body Labor, and Temporal Labor in Ridesharing. To appear in P. Bjorn and D. Konstan (Eds): Proceedings of Computer-Supported Cooperative Work 2016, February 27 – March 2 2016. New York: ACM press.

  • Robinson, William. I. (2009) Saskia Sassen and the Sociology of Globalization: A Critical Appraisal. Sociological Analysis. Spring 2009.

  • Ross, Joel, Lilly Irani, Max Six Silberman, Andrew Zaldivar and Bill Tomlinson (2010). Who are the crowdworkers?: shifting demographics in mechanical turk. In G. Fitzpatrick and S. Hudson (Eds): Proceedings of CHI EA 10, 10–15 April, Atlanta, GA, USA. New York: ACM press, pp. 2863–2872.

    Google Scholar 

  • Salehi, Niloufar, Lilly Irani, Michael Bernstein, Ali Alkhatib, Eva Ogbe, and Kristy Milland (2015). We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers. In B. Begole and J. Kim (Eds): Proceedings of CHI 2015, Seoul, South Korea, April 18–23, 2015. New York: ACM press, pp. 1621–1630.

    Google Scholar 

  • Sassen, Saskia (1996). Losing Control? Sovereignty in the Age of Globalization. New York: Columbia University Press.

    Google Scholar 

  • Sassen, Saskia (1998). Globalization and its Discontents: Essays on the New Mobility of People and Money. New York: The New Press.

    Google Scholar 

  • Sassen, Saskia (2007). A Sociology of Globalization. W. W. Norton

  • Schmidt, Kjeld, and Liam Bannon (1992). ‘Taking CSCW Seriously: Supporting Articulation Work’, Computer Supported Cooperative Work (CSCW): An International Journal, vol. 1, no. 1–2, pp. 7–40.

    Article  Google Scholar 

  • Silberman, Max Six (2010). What’s fair? Rational action and its residuals in an electronic market. Unpublished manuscript. 2010 http://www.scribd.com/doc/86592724/Whats-Fair

  • Silberman, Max Six, Lilly Irani and Joel Ross (2010). J. Ethics and Tactics of Professional Crowdwork. ACM XRDS, 2010 vol. 17, no. 2. 2010.

  • Søderberg, Anne-Marie, S. Krishna and Pernille Bjørn (2013). ‘Global Software Development: Commitment, Trust and Cultural Sensitivity in Strategic Partnerships,’ International journal of Management, vol. 19, no. 4, 2013, pp. 347–361. http://www.sciencedirect.com/science/article/pii/S1075425313000513

  • Sommerville, Ian, Tom Rodden, Pete Sawyer, & Richard Bentley, (1992). Sociologists can be surprisingly useful in interactive systems design. In proceedings of HCI ‘92, CUP, pp. 341–341.

  • Star, Susan Leigh and Strauss, Anselm L. (1998). Layers of silence, arenas of voice: The ecology of visible and invisible work’. Computer Supported Cooperative Work (CSCW): An International Journal, vol. 8, no. 1, pp. 9–30.

    Google Scholar 

  • Steger, Manfred (2013). Globalization: A very short introduction (3rd Edition). Oxford: Oxford University Press.

    Book  Google Scholar 

  • Suchman, Lucy A. (1987). Plans and situated actions: the problem of human-machine communication. Cambridge: CUP.

    Google Scholar 

  • Thomsen, Steven, Joseph Straubhaar and Drew Bolyard (1998). Ethnomethodology and the study of online communities: exploring the cyber streets, Information Research, vol. 4 no. 1, July 1998 http://www.informationr.net/ir/4-1/paper50.html

Download references

Acknowledgments

Thanks to all the members of Turker Nation and our Indian Turkers for participating in and enabling this research.

Author information

Authors and Affiliations

  1. Xerox Research Centre Europe, Meylan, France

    David Martin

  2. Microsoft India, Bangalore, India

    Jacki O’Neill

  3. Nottingham University, Nottingham, UK

    Neha Gupta

  4. Pennsylvania State University, University Park, PA, USA

    Benjamin V. Hanrahan

Authors
  1. David Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacki O’Neill
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neha Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin V. Hanrahan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Martin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martin, D., O’Neill, J., Gupta, N. et al. Turking in a Global Labour Market. Comput Supported Coop Work 25, 39–77 (2016). https://doi.org/10.1007/s10606-015-9241-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10606-015-9241-6

Key words

Search

Navigation