Efficient Task Decomposition in Crowdsourcing
In order to facilitate crowdsourcing-based task solving, complex tasks are decomposed into smaller subtasks that can be executed either sequentially or in parallel by workers. These two task decompositions attract a plenty of empirical explorations in crowdsourcing. However the absence of formal study makes difficulty in providing task requesters with explicit guidelines on task decomposition. In this paper, we formally present and analyze those two task decompositions as vertical and horizontal task decomposition models. Our focus is on addressing the efficiency (i.e., the quality of the task’s solution) of task decomposition when the self-interested workers are paid in two different ways — equally paid and paid based on their contributions. By combining the theoretical analyses on worker’s behavior and simulation-based exploration on the efficiency of task decomposition, our study 1) shows the superiority of vertical task decomposition over horizontal task decomposition in improving the quality of the task’s solution; 2) gives explicit instructions on strategies for optimal vertical task decomposition under both revenue sharing schemes to maximize the quality of the task’s solution.
KeywordsTask decomposition task dependence task difficulty solution quality efficient crowdsourcing
Unable to display preview. Download preview PDF.
- 1.Kittur, A., Smus, B., Khamkar, S., Kraut, R.E.: Crowdforge: crowdsourcing complex work. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 16–19 (2011)Google Scholar
- 2.Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K.: Soylent: a word processor with a crowd inside. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 313–322 (2010)Google Scholar
- 3.Little, G., Chilton, L.B., Goldman, M., Miller, R.C.: Exploring iterative and parallel human computation processes. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, p. 25 (2010)Google Scholar
- 4.Kulkarni, A., Can, M., Hartmann, B.: Collaboratively crowdsourcing workflows with turkomatic. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, pp. 1003–1012 (2012)Google Scholar
- 5.Tran-Thanh, L., Huynh, T.D., Rosenfeld, A., Ramchurn, S.D., Jennings, N.R.: Budgetfix: budget limited crowdsourcing for interdependent task allocation with quality guarantees. In: AAMAS, pp. 477–484 (2014)Google Scholar
- 6.Tran-Thanh, L., Venanzi, M., Rogers, A., Jennings, N.R.: Efficient budget allocation with accuracy guarantees for crowdsourcing classification tasks. In: AAMAS, pp. 901–908 (2013)Google Scholar