Training Workers for Improving Performance in Crowdsourcing Microtasks

Conference paper

DOI: 10.1007/978-3-319-24258-3_8

Part of the Lecture Notes in Computer Science book series (LNCS, volume 9307)
Cite this paper as:
Gadiraju U., Fetahu B., Kawase R. (2015) Training Workers for Improving Performance in Crowdsourcing Microtasks. In: Conole G., Klobučar T., Rensing C., Konert J., Lavoué É. (eds) Design for Teaching and Learning in a Networked World. Lecture Notes in Computer Science, vol 9307. Springer, Cham

Abstract

With the advent and growing use of crowdsourcing labor markets for a variety of applications, optimizing the quality of results produced is of prime importance. The quality of the results produced is typically a function of the performance of crowd workers. In this paper, we investigate the notion of treating crowd workers as ‘learners’ in a novel learning environment. This learning context is characterized by a short-lived learning phase and immediate application of learned concepts. We draw motivation from the desire of crowd workers to perform well in order to maintain a good reputation, while attaining monetary rewards successfully. Thus, we delve into training workers in specific microtasks of different types. We exploit (i) implicit training, where workers are provided training when they provide erraneous responses to questions with priorly known answers, and (ii) explicit training, where workers are required to go through a training phase before they attempt to work on the task itself. We evaluated our approach in 4 different types of microtasks with a total of 1200 workers, who were subjected to either one of the proposed training strategies or baseline case of no training. The results show that workers who undergo training depict an improvement in performance upto 5 %, and a reduction in the task completion time upto 41 %. Additionally, crowd training led to the elimination of malicious workers and a costs-benefit gain upto nearly 15 %.

Keywords

Crowdsourcing Workers Training Learning Microtask Performance 

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.L3S Research CenterLeibniz Universität HannoverHannoverGermany
  2. 2.Mobile.deKleinmachnowGermany

Personalised recommendations