Crowdsourcing Coordination: A Review and Research Agenda for Crowdsourcing Coordination Used for Macro-tasks
Crowdsourcing has become a widely accepted approach to leveraging the skills and expertise of others to accomplish work. Despite the potential of crowdsourcing to tackle complex problems, it has often been used to address simple micro-tasks. To tackle more complex macro-tasks, more attention is needed to better comprehend crowd coordination. Crowd coordination is defined as the synchronization of crowd workers in an attempt to direct and align their efforts in pursuit of a shared goal. The goal of this chapter is to advance our understanding of crowd coordination to tackle complex macro-tasks. To accomplish this, we have three objectives. First, we review popular theories of coordination. Second, we examine the current approaches to crowd coordination in the HCI and CSCW literature. Finally, the chapter identifies shortcomings in the literature and proposes a research agenda directed at advancing our understanding of crowd coordination needed to address complex macro-tasks.
This book chapter was supported in part by the National Science Foundation [grant CHS-1617820].
- Anderson, E. W., Potter, K. C., Matzen, L. E., Shepherd, J. F., Preston, G. A., & Silva, C. T. (2011). A user study of visualization effectiveness using EEG and cognitive load. In Computer graphics forum (Vol. 30, No. 3, pp. 791–800). Oxford, UK: Blackwell Publishing Ltd.Google Scholar
- Bolici, F., Howison, J., & Crowston, K. (2009). Coordination without discussion? Socio-technical congruence and stigmergy in free and open source software projects. Paper presented at the International Conference on Software Engineering, Vancouver, BC, Canada. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.193.7473&rep=rep1&type=pdf.
- Crowston, K. (1994). A taxonomy of organisational dependencies and coordination mechanisms. MIT Center for Coordination Science Working Paper. Massachusetts Institute of Technology, August 1994.Google Scholar
- Crowston, K., Howison, J., & Rubleske, J. (2006). Coordination theory: A ten year retrospective. In P. Zhang & D. Galletta (Eds.), Human-computer interaction in management information systems—foundations (pp. 120–138). Armonk, NY: M. E. Sharpe Inc.Google Scholar
- Gittell, J. H. (2006). Relational coordination: Coordinating work through relationships of shared goals, shared knowledge and mutual respect. In O. Kyriakidou & M. F. Özbilgin (Eds.), Relational perspectives in organizational studies: A research companion (pp. 74–94). Cheltenham, UK: Edward Elgar Publishers.Google Scholar
- Gittell, J. H. (2011). New directions for relational coordination theory. In K. S. Cameron & G. M. Spreitzer (Eds.), The Oxford handbook of positive organizational scholarship (pp. 400–411). New York, NY: Oxford University Press.Google Scholar
- Gittell, J. H. (2016). Transforming relationships for high performance: The power of relational coordination. Palo Alto, CA: Stanford University Press.Google Scholar
- Heylighen, F. (2015). Stigmergy as a universal coordination mechanism: Components, varieties and applications. In T. Lewis & L. Marsh (Eds.), Human stigmergy: Theoretical developments and new applications. New York, NY: Springer.Google Scholar
- Howe, J. (2006). The rise of crowdsourcing. Wired, 14(6), 1–4.Google Scholar
- Jung, J. Y., & Mellers, B. A. (2016). American attitudes toward nudges. Judgment & Decision Making, 11(1), 62–74.Google Scholar
- Kaur, H., Williams, A. C., Thompson, A. L., Lasecki, W. S., Iqbal, S. T., & Teevan, J. (2018). Creating better action plans for writing tasks via vocabulary-based planning. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 86.Google Scholar
- Kim, J., Sterman, S., Cohen, A. A. B., & Bernstein, M. S. (2017). Mechanical novel: Crowdsourcing complex work through reflection and revision. In Proceedings of the 2017 ACM Conference on Computer-supported Cooperative Work and Social Computing (pp. 233–245). New York, NY: ACM.Google Scholar
- Kittur, A., Smus, B., Khamkar, S., & Kraut, R. E. (2011). Crowdforge: Crowdsourcing complex work. In Proceedings of the 24th annual ACM symposium on User Interface Software and Technology (pp. 43–52). New York, NY: ACM.Google Scholar
- Kulkarni, A., Can, M., & Hartmann, B. (2012). Collaboratively crowdsourcing workflows with Turkomatic. In Proceedings of the ACM 2012 Conference on Computer-supported Cooperative Work (pp. 1003–1012). New York, NY: ACM.Google Scholar
- Lave, J. (2009). The practice of learning. In K. Illeris (Ed.), Contemporary learning theories (pp. 200–208). London, UK: Routledge.Google Scholar
- Montoya-Weiss, M. M., Massey, A. P., & Song, M. (2001). Getting it together: Temporal coordination and conflict management in global virtual teams. Academy of Management Journal, 44(6), 1251–1262.Google Scholar
- Munson, S. A., Kervin, K., & Robert Jr., L. P. (2014). Monitoring email to indicate project team performance and mutual attraction. In Proceedings of the 17th ACM Conference on Computer-supported Cooperative Work & Social Computing (pp. 542–549). New York, NY: ACM.Google Scholar
- Okhuysen, G. A., & Bechky, B. A. (2009). Coordination in organizations: An integrative perspective. In J. P. Walsh & A. P. Brief (Eds.), Academy of management annals (Vol. 3, pp. 463–502). Essex, UK: Routledge.Google Scholar
- Retelny, D., Bernstein, M. S., & Valentine, M. A. (2017). No workflow can ever be enough: How crowdsourcing workflows constrain complex work. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 89.Google Scholar
- Retelny, D., Robaszkiewicz, S., To, A., Lasecki, W. S., Patel, J., Rahmati, N.,… Bernstein, M. S. (2014). Expert crowdsourcing with flash teams. In Proceedings of the 27th annual ACM Symposium on User Interface Software and Technology (pp. 75–85). New York, NY: ACM.Google Scholar
- Rezgui, A., & Crowston, K. (2018). Stigmergic coordination in Wikipedia. In Proceedings of the 14th International Symposium on Open Collaboration (pp. 1–12). Paris, France: ACM Press.Google Scholar
- Robert, L. P. (2016). Far but near or near but far?: The effects of perceived distance on the relationship between geographic dispersion and perceived diversity. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 2461–2473). New York, NY: ACM.Google Scholar
- Robert, L. P., Dennis, A. R., & Ahuja, M. (2008). Social capital and knowledge integration in digitally enabled teams. Information Systems Research, 19(3), 314–334. http://pubsonline.informs.org/doi/abs/10.1287/isre.1080.0177.
- Robert, L. P., Dennis, A. R., & Ahuja, M. (2018). Differences are different: Examining the effects of communication media on the impacts of racial and gender diversity in decision-making teams. Information Systems Research, 29(3), 525–545. https://doi.org/10.1287/isre.2018.0773.CrossRefGoogle Scholar
- Robert, L. P., & Romero, D. M. (2015). Crowd size, diversity and performance. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1379–1382). New York, NY: ACM.Google Scholar
- Salehi, N., McCabe, M., Valentine, M., & Bernstein, M. S. (2017). Huddler: Convening stable and familiar crowd teams despite unpredictable availability. In Proceedings of the 20th ACM Conference on Computer-supported Cooperative Work & Social Computing (CSCW’17). New York, NY: ACM.Google Scholar
- Suchman, L. (1987). Plans and situated actions: The problem of human–machine communication. Cambridge, MA: Cambridge University Press.Google Scholar
- Teevan, J., Iqbal, S. T., & Von Veh, C. (2016). Supporting collaborative writing with microtasks. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 2657–2668). New York, NY: ACM.Google Scholar
- Valentine, M. A., Retelny, D., To, A., Rahmati, N., Doshi, T., & Bernstein, M. S. (2017, May). Flash organizations: Crowdsourcing complex work by structuring crowds as organizations. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 3523–3537). New York, NY: ACM.Google Scholar