This paper examines how working in the global labour market of Amazon Mechanical Turk (AMT) impacts upon and is understood by two different groups of workers. To do this we report on two qualitative studies; one of US and another of Indian crowdworkers (or ‘Turkers’) that we analysed from an ethnomethodological orientation. Our data is naturalistic and comes from a variety of sources—interviews, observations and forum posts—where Turkers describe their work, and their understandings of that work and of the transnational market they work in. We compare and contrast their situations, their reflections on the marketplace and its global reach, and we take a look at their understandings of one another. Our focus is on ‘the work to making turking work’ (Martin et al. 2014). That is, the work that turkers do to organise and make sense of their work as they operate in the AMT marketplace, such that they can do so effectively. AMT is a technologically mediated marketplace—that is the distribution, completion and payment of work is done online, almost completely through the AMT platform. The design of the platform has important consequences for how Turkers experience and understand the market (including its global or transnational nature). We discuss how our findings relate to a variety of CSCW issues and provide an initial examination of how they relate to globalisation both as a mundanely experienced phenomenon and as a topic of academic interest. We finish the article by drawing on our own experiences in research and design to look at how technology can be used to intervene in a market like this to try to address imbalances in power and agency between employers and workers.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
See Felsteiner (2011) for a detailed examination of wages and the legal issues.
Ipeirotis offers more up-to-date and dynamic demographic and statistical data with MTurk Tracker: http://www.mturk-tracker.com/#/general
“The current federal minimum wage is $7.25 per hour.” Source: http://poverty.ucdavis.edu/faq/what-are-annual-earnings-full-time-minimum-wage-worker
All the US Turker usernames have been changed apart from those of Spamgirl, taintturk and tigger (as we are in direct contact with them and have their permission to use their actual usernames). All the Indian Turker names are pseudonymous.
http://www.census.gov/content/dam/Census/library/publications/2014/demo/p60-249.pdf - page 43, $12,119 for a single person under 65 years of age.
Suddenly Amazon made electronic payments available to Indian Turkers in March 2015, in a move that we welcome, finally redressing this asymmetry: http://www.mturkgrind.com/threads/mturk-earnings-withdrawal-transfer-options.27510/
One of the clearest benefits of the Masters qualification is that Turkers do not have to complete captchas on many tasks that usually require them – this saves quite a lot of time over repetitive bulk HITs.
Although at the time of writing we have heard of considerable rumours about the widespread closing of Indian accounts. Unfortunately the lack of transparency at Amazon means it is impossible at this stage to verify this.
MMOs provide significant commercial opportunities (selling virtual currency and items). It has been reported that in e.g. India and China people work to accumulate these under duress so they can be sold to other gamers to profit ‘ringleaders’. Aleppo suggests something similar may be happening with AMT.
It was reported to us that a steady flow of unreasonable requests (i.e. find me good HITs, get my suspension lifted) accompanied by some abuse had led to a blanket ban [personal communication from Turker Nation admin]. It seems that this was a product of misunderstanding on some Indian Turkers’ part (that the forum had an influential relationship with AMT, that US workers did not need the money so much).
Source: State University of New York, Levin Institute: http://www.globalization101.org/what-is-globalisation
Ahmed, Syed Ishtiaque, Nicola Bidwell, Himanshu Zade, Srihari Muralidhar, Anupama Dhareshwar, Baneen Karachiwala, Cedrick Tandong, Jacki O’Neill (2016). Peer-to-peer in the workplace: A view from the road. To appear in C. Lampe and K. Inkpen (Eds): Proceedings of CHI 2016, San Jose, CA, USA, May 7–12, 2016. New York: ACM Press.
Bederson, Benjamin. B. and Alexander. J. Quinn (2011). Web workers unite! Addressing challenges of online laborers. In B. Begole and W. Kellog (Eds): Proceedings of CHI ‘11 Extended Abstracts. Vancouver, Canada 7–12 May 2011. New York: ACM press, 97–106.
Bjørn, Pernille and Nina Boulus-Rødje (2015). The multiple intersecting sites of design in CSCW research. Computer Supported Cooperative Work (CSCW): An International Journal, vol. 24, no. 3, 2015, pp. 319–351.
Blomberg, Jeanette and Helen Karasti (2013). Reflections on 25 years or ethnography in CSCW. Computer Supported Cooperative Work (CSCW): An International Journal, vol. 22, nos. 4–6, 2013, pp. 373–423.
Boden, Deirdre. (1994). The Business of Talk: organisations in action. Cambridge: Polity Press.
Boden, Alexander, Bernhard Nett, and Volker Wolf (2009). Trust and Social Capital: Revisiting an Offshoring Failure Story of a Small German Software Company. In E. Balka and C. Simone (Eds): Proceedings of ECSCW 2009, Vienna Austria 7–11th September 2009, London: Springer, pp. 123–142.
Bourne, Kristina A., and Forman, Pamela J. (2013). Living in a Culture of Overwork: An Ethnographic Study of Flexibility. Journal of Management Inquiry, DOI: 10.1177/1056492613481245
Crabtree, Andy (2003). Designing Collaborative Systems: A Practical Guide to Ethnography, London: Springer.
Crary, Jonathan (2013). 24/7: Late Capitalism and the Ends of Sleep. Verso: London.
Felsteiner, Alek (2011). Working the crowd: employment and labor law in the crowdsourcing industry. Berkeley Journal of Employment & Labor Law, vol. 32 no. 1, pp. 143–204.
Fort, Karen., Gilles Adda and K. Bretonnel Cohen (2011). Amazon Mechanical Turk: Gold Mine or Coal Mine? Computational Linguisitcs, vol. 37, no. 2, pp. 413–420.
Gajera, Rinku and Jacki O’Neill (2014). Ethnography in Parallel. In C. Rossitto, L. Ciolfi, D. Martin and B. Conein (Eds): Proceedings of Coop 2014, 27th–30th May, 2014, Nice, France. London: Springer, pp. 259–276.
Garfinkel, Harold (1967). Studies in ethnomethodology. Englewood Cliffs, N.J.: Prentice-Hall.
Gawade, Mrunal, Rajan Vaish, Mercy Nduta Waihumbu and James Davis (2012). Exploring Employment Opportunities through Microtasks via Cybercafes. In Proceeding. of IEEE Global Humanitarian Technology Conference, 21 Oct - 24 Oct 2012, Seattle, WA, USA. Washington, DC: IEEE Computer Society, pp. 77–82.
Geertz, Clifford (2001). The World in Pieces: Culture and Politics at the End of the Century. In C. Geertz, Available Light: Anthropological Reflections on Philosophical Topics. Princeton, NJ: Princeton University Press, pp. 218–63.
Graeber, David (2015). The Utopia of Rules: On Technology, Stupidity and the Secret Joys of Bureaucracy. Brooklyn, NY: First Melville House
Gray, Mary L., Siddharth Suri, Syed S. Ali, and Deepti Kulkarni (2016). The Crowd is a Collaborative Network. To appear in P. Bjorn and D. Konstan (Eds): Proceedings of Computer-Supported Cooperative Work 2016, February 27 – March 2 2016. New York: ACM press.
Gupta, Neha, David Martin, Benjamin V. Hanrahan & Jacki O’Neill (2014). Turk-Life In India. In D. McDonald and P. Bjorn (Eds): Proceedings of Group 2014, Sanibel Island, Florida, 9-12th November. New York: ACM press, pp. 1–11.
Hanrahan, Benjamin V., Jutta Willamowski, Saiganesh Swaminathan, and David Martin (2015). Turkbench: Rendering the Market of Turkers. In B. Begole and J. Kim (Eds): Proceedings of CHI 2015, Seoul, South Korea, April 18–23, 2015. New York: ACM press, pp. 1613-1616.
Harper, Richard, Christian Bird, Tom Zimmermann and Brendan Murphy (2013). Dwelling in Software: Aspects of the felt-life of engineers in large software projects. In O. Bertelson, L. Ciofli, M. A. Grasso and A. Papadopoulos: Proceedings of ECSCW 13, Paphos, Cyprus, September 21–25, 2013. London: Springer, pp. 161–180.
Held, David, Anthony McGrew, David Goldblatt, and Jonathan Perraton (1999). Global Transformations. Politics, Economics and Culture. Cambridge: Polity Press
Howe, Jeff. Crowdsourcing: A definition. (2006). http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html
Hughes, John A., David Randall, & Dan Shapiro (1992). Faltering from ethnography to design. In M. Mantel and R. Baeker (Eds): Proceedings of Computer-Supported Cooperative Work 92, Toronto, Canada, November 1–4, 1992. New York: ACM press, pp. 115–122.
Hughes, John, Val King, Tom Rodden and Hans Andersen (1994). Moving Out from the Control Room: Ethnography in System Design. In J. Smith, F. Smith and T. Malone (Eds): Proceedings of Computer-Supported Cooperative Work 94, Chapel Hill, NC, USA. New York: ACM press, pp. 429–439.
Huwart, Jean-Yves and Loic Verdier (2013). OECD Report “Does Globalisation Promote Employment?” http://dx.doi.org/10.1787/9789264111905-en
Ipeirotis, Panos. (2010a). Analyzing the Amazon Mechanical Turk marketplace. Ceder Working Papers No. CeDER-10-04 (2010a). http://hdl.handle.net/2451/29801
Ipeirotis, Panos. (2010b). Demographics of Mechanical Turk. CeDER Working Papers No. CeDER-10-01 (2010b) http://hdl.handle.net/2451/29585
Irani, Lilly and Max Six Silberman (2013). Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk. In S. Brewster and S. Bødker (Eds): Proceedings of CHI 13, Paris, France, 22 April – 2 May 2013. New York: ACM press, pp. 611–620.
Isaac, Mike & Natasha Singer. (2015) California Says Uber Driver Is Employee, Not a Contractor http://tinyurl.com/nc3n7z8
Khanna, Shashank, Aishwarya Ratan, James Davis and William Thies (2010). Evaluating and Improving the Usability of Mechanical Turk for Low-Income Workers in India. In A. Dearden, J. Parihk and L. Subrimanian (Eds): Proceedings of ACM DEV 2010, December 17–18, 2010, London, United Kingdom, New York: ACM press, article no. 12.
Kittur, Aniket, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease and John Horton (2013). The future of crowd work. In Proceedings of Computer-Supported Cooperative Work’13, February 23–27, 2013. New York: ACM press, pp. 1301–1318.
Kuek, Siou C., Cecilia Paradi-Guilford, Toks Fayomi, Saori Imaizumi, Panos Ipeirotis, Patricia Pina, and Manpreet Singh (2015). The Global Opportunity in Online Outsourcing. World Bank Studies. http://documents.worldbank.org/curated/en/2015/06/24702763/global-opportunity-online-outsourcing
Lee, Min K., Daniel Kusbit, Evan Metsky and Laura Dabbish (2015). Working with Machines: The Impact of Algorithmic and Data Driven Management on Human Workers. In B. Begole and J. Kim (Eds): Proceedings of CHI 2015, Seoul, South Korea, April 18–23, 2015. New York: ACM press, pp. 1603–1612.
Marcus, George E. (1995). Ethnography in/of the World System: The Emergence of Multi-Sited Ethnography. Annual Review of Anthropology, Vol. 24, pp. 95--117
Martin, David and Jacki O’Neill (2011). Talk: Talking the Organisation into Being. In M. Rouncefield & P. Tolmie (Eds): Ethnomethodology at Work, Farnham, England: Ashgate, pp. 109–129.
Martin, D., Benjamin Hanrahan, Jacki O’Neill, J. and Neha Gupta (2014). Being A Turker. In M. Ringel Morris and M. Reddy (Eds): Proceedings of Computer-Supported Cooperative Work 14, Baltimore US, 15-19th February. New York: ACM press, pp. 224–235
Matthiesen, Stina, Pernille Bjorn, and Lise Møller Peterson, L. (2014). Figure out how to code with the hands of others": recognizing cultural blind spots in global software development. In M. Ringel Morris and M. Reddy (Eds): Proceedings of Computer-Supported Cooperative Work 14, Baltimore US, 15-19th February. New York: ACM press, pp. 1107–1119.
Morozoz, E. (2014). To Save Everything Click Here: Technology, solutionism and the urge to fix problems that don’t exist. Penguin (first published in 2013).
O’Neill, Jacki & David Martin (2003). Text Chat In Action. In M. Tremaine and C. Simone (Eds): Proceedings of the 2003 international ACM SIGGROUP conference on Supporting group work, Sanibel Island, Florida, GROUP ‘03. New York: ACM Press, pp. 40–49.
O’Neill, Jacki & David Martin (2013). Relationship-Based Business Process Crowdsourcing. In P. Kotzé, G. Marsdan, G. Lingaard, J. Wesson and M. Winckler (Eds): Proceedings of INTERACT 2013, 14th IFIP TC 13 International Conference, Cape Town, South Africa, September 2-6, 2013, Proceedings, Part IV, pp. 429--446
Olson, Gary, & Judith Olson (2000). Distance matters. Human-computer interaction, vol. 15, no. 2, pp. 139–178.
Raval, Noopur and Paul Dourish. Standing Out from the Crowd: Emotional Labor, Body Labor, and Temporal Labor in Ridesharing. To appear in P. Bjorn and D. Konstan (Eds): Proceedings of Computer-Supported Cooperative Work 2016, February 27 – March 2 2016. New York: ACM press.
Robinson, William. I. (2009) Saskia Sassen and the Sociology of Globalization: A Critical Appraisal. Sociological Analysis. Spring 2009.
Ross, Joel, Lilly Irani, Max Six Silberman, Andrew Zaldivar and Bill Tomlinson (2010). Who are the crowdworkers?: shifting demographics in mechanical turk. In G. Fitzpatrick and S. Hudson (Eds): Proceedings of CHI EA 10, 10–15 April, Atlanta, GA, USA. New York: ACM press, pp. 2863–2872.
Salehi, Niloufar, Lilly Irani, Michael Bernstein, Ali Alkhatib, Eva Ogbe, and Kristy Milland (2015). We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers. In B. Begole and J. Kim (Eds): Proceedings of CHI 2015, Seoul, South Korea, April 18–23, 2015. New York: ACM press, pp. 1621–1630.
Sassen, Saskia (1996). Losing Control? Sovereignty in the Age of Globalization. New York: Columbia University Press.
Sassen, Saskia (1998). Globalization and its Discontents: Essays on the New Mobility of People and Money. New York: The New Press.
Sassen, Saskia (2007). A Sociology of Globalization. W. W. Norton
Schmidt, Kjeld, and Liam Bannon (1992). ‘Taking CSCW Seriously: Supporting Articulation Work’, Computer Supported Cooperative Work (CSCW): An International Journal, vol. 1, no. 1–2, pp. 7–40.
Silberman, Max Six (2010). What’s fair? Rational action and its residuals in an electronic market. Unpublished manuscript. 2010 http://www.scribd.com/doc/86592724/Whats-Fair
Silberman, Max Six, Lilly Irani and Joel Ross (2010). J. Ethics and Tactics of Professional Crowdwork. ACM XRDS, 2010 vol. 17, no. 2. 2010.
Søderberg, Anne-Marie, S. Krishna and Pernille Bjørn (2013). ‘Global Software Development: Commitment, Trust and Cultural Sensitivity in Strategic Partnerships,’ International journal of Management, vol. 19, no. 4, 2013, pp. 347–361. http://www.sciencedirect.com/science/article/pii/S1075425313000513
Sommerville, Ian, Tom Rodden, Pete Sawyer, & Richard Bentley, (1992). Sociologists can be surprisingly useful in interactive systems design. In proceedings of HCI ‘92, CUP, pp. 341–341.
Star, Susan Leigh and Strauss, Anselm L. (1998). Layers of silence, arenas of voice: The ecology of visible and invisible work’. Computer Supported Cooperative Work (CSCW): An International Journal, vol. 8, no. 1, pp. 9–30.
Steger, Manfred (2013). Globalization: A very short introduction (3rd Edition). Oxford: Oxford University Press.
Suchman, Lucy A. (1987). Plans and situated actions: the problem of human-machine communication. Cambridge: CUP.
Thomsen, Steven, Joseph Straubhaar and Drew Bolyard (1998). Ethnomethodology and the study of online communities: exploring the cyber streets, Information Research, vol. 4 no. 1, July 1998 http://www.informationr.net/ir/4-1/paper50.html
Thanks to all the members of Turker Nation and our Indian Turkers for participating in and enabling this research.
Rights and permissions
About this article
Cite this article
Martin, D., O’Neill, J., Gupta, N. et al. Turking in a Global Labour Market. Comput Supported Coop Work 25, 39–77 (2016). https://doi.org/10.1007/s10606-015-9241-6
- transnational markets
- amazon mechanical turk (AMT)