Advertisement

Addressing Cooperation Issues in Situated Crowdsourcing

  • Jorge GoncalvesEmail author
  • Simo Hosio
  • Niels van Berkel
  • Simon Klakegg
Chapter
Part of the Human–Computer Interaction Series book series (HCIS)

Abstract

Situated crowdsourcing has been growing in popularity as an alternative way to collect complex and often creative crowd work. However, previous situated crowdsourcing deployments have not successfully leveraged cooperation possibilities with their audiences, which can improve the data quality of deployed macrotasks. In this chapter, we present three situated crowdsourcing case studies that used different situated technologies and identify the reasons behind their missteps regarding promoting cooperation between workers. Then, based on the identified issues, we propose the design of a novel situated crowdsourcing platform that aims to effectively support cooperation without alienating solo workers. In order to gather insights on our proposed design, we built a prototype platform and evaluated it using a laboratory study with 24 participants. In general, participants were positive about the idea as it provided an easy way to cooperate with friends when completing tasks, while also allowing them to adjust the working environment to their liking. Finally, we conclude by offering insights towards improving cooperation in future situated crowdsourcing deployments and how this can assist in completing macrotasks.

Notes

Acknowledgements

This work is partially funded by the Academy of Finland (286386-CPDSS, 285459-iSCIENCE), the European Commission (Grant 6AIKA-A71143-AKAI), and Marie Skłodowska-Curie Actions (645706-GRAGE).

References

  1. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.CrossRefGoogle Scholar
  2. Brignull, H., & Rogers, Y. (2003). Enticing people to interact with large public displays in public spaces. In M. Rauterberg, M. Menozzi, & J. Wesson (Eds.), Proceedings of 9th IFIP TC13 International Conference on Human-Computer Interaction, INTERACT ‘03, Zurich, Switzerland, 1–5 September 2003 (pp. 17–24). Amsterdam: IOS Press.Google Scholar
  3. Cheng, J., Teevan, J., Iqbal, S. T., & Bernstein, M. S. (2015). Break it down: A comparison of macro- and microtasks. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘15) (pp. 4061–4064). ACM Press.Google Scholar
  4. Felstiner, A. (2012). The weakness of crowds. Crowds and Clouds. Retrieved March 15, 2017, from http://limn.it/the-weakness-of-crowds/.
  5. Goncalves, J., Ferreira, D., Hosio, S., Liu, Y., Rogstadius, J., Kukka, H., et al. (2013). Crowdsourcing on the spot: Altruistic use of public displays, feasibility, performance, and behaviours. In J. F. Canny, M. Langheinrich, & J. Rekimoto (Eds.), Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ‘13, Zurich, Switzerland, 8–12 September 2013 (pp. 753–762). New York: ACM Press.Google Scholar
  6. Goncalves, J., Pandab, P., Ferreira, D., Ghahramani, M., Zhao, G., & Kostakos, V. (2014a). Projective testing of diurnal collective emotion. In J. Kientz, J. Scott, & J. Song (Eds.), Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ‘14, Seattle, USA, 13–17 September 2014 (pp. 487–497). New York: ACM Press.Google Scholar
  7. Goncalves, J., Hosio, S., Ferreira, D., & Kostakos, V. (2014b). Game of words: Tagging places through crowdsourcing on public displays. In C. Neustaedter, S. Bardzell, & E. Paulos (Eds.), Proceedings of the 2014 Conference on Designing Interactive Systems, DIS ‘14, Vancouver, Canada, 21–25 June 2014 (pp. 705–714). New York: ACM Press.Google Scholar
  8. Goncalves, J., Kukka, H., Sanchez, I., & Kostakos, V. (2016). Crowdsourcing queue estimations in situ. In P. Bjørn & J. Konstan (Eds.), Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ‘16, San Francisco, USA, 27 February–2 March 2016 (pp. 1040–1051). New York: ACM Press.Google Scholar
  9. Goncalves, J., Hosio, S., & Kostakos, V. (2017). Eliciting structured knowledge from situated crowd markets. ACM Transactions on Internet Technology, 17(2), Article 14.Google Scholar
  10. Gray, M. L., Suri, S., Ali, S. S., & Kulkarni, D. (2016). The crowd is a collaborative network. In P. Bjørn & J. Konstan (Eds.), Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ‘16, San Francisco, USA, 27 February–2 March 2016 (pp. 134–147). New York: ACM Press.Google Scholar
  11. Gupta, N., Martin, D., Hanrahan, B. V., & O’Neill, J. (2014). Turk-life in India. In D. W. McDonald & P. Bjørn (Eds.), Proceedings of the 18th International Conference on Supporting Group Work, GROUP ‘14, Sanibel Island, USA, 9–12 November 2014 (pp. 1–11). New York: ACM Press.Google Scholar
  12. Haas, D., Ansel, J., Gu, L., & Marcus, A. (2015, August). Argonaut: Macrotask crowdsourcing for complex data processing. In Proceedings of the VLDB Endow (Vol. 8, no. 12, pp. 1642–1653). http://dx.doi.org/10.14778/2824032.2824062.
  13. Heikkinen, T., Goncalves, J., Kostakos, V., Elhart, I., & Ojala, T. (2014). Tandem browsing toolkit: Distributed multi-display interfaces with web technologies. In A. Quigley (Ed.), Proceedings of the International Symposium on Pervasive Displays, PerDis ‘14, Copenhagen, Denmark, 3–4 June 2014 (pp. 142–147). New York: ACM Press.Google Scholar
  14. Heimerl, K., Gawalt, B., Chen, K., Parikh, T., & Hartmann, B. (2012). Community sourcing: Engaging local crowds to perform expert work via physical kiosks. In H. Chi & K. Höök (Eds.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ‘12, Austin, USA, 5–10 May 2012 (pp. 1539–1548). New York: ACM Press.Google Scholar
  15. Hosio, S., Goncalves, J., & Kostakos, V. (2013). Application discoverability on multipurpose public displays: Popularity comes at a price. In Proceedings of International Symposium on Pervasive Displays 2013 (PerDis’13) (pp. 31–36).Google Scholar
  16. Hosio, S., Goncalves, J., Kostakos, V., & Riekki, J. (2015a). Crowdsourcing public opinion using urban pervasive technologies: Lessons from real-life experiments in Oulu. Policy & Internet, 7(2), 203–222.CrossRefGoogle Scholar
  17. Hosio, S., Goncalves, J., Lehdonvirta, V., Ferreira, D., & Kostakos, V. (2014). Situated crowdsourcing using a market model. In M. Dontcheva & D. Wigdor (Eds.), Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ‘14, Honolulu, USA, 5–8 October 2014 (pp. 55–64). New York: ACM Press.Google Scholar
  18. Hosio, S., Harper, R., O’Hara, K., Goncalves, J., & Kostakos, V. (2015). Life through the lens: A qualitative investigation of human behaviour with an urban photography service. In P. Olivier & D. Foster (Eds.), Proceedings of the 2015 British HCI Conference, British HCI ‘15, Lincoln, United Kingdom, 13–17 July 2015 (pp. 157–164). New York: ACM Press.Google Scholar
  19. Hosio, S., Kukka, H., Goncalves, J., Kostakos, V., & Ojala, T. (2016). Toward meaningful engagement with pervasive displays. IEEE Pervasive Computing, 15(3), 24–31.CrossRefGoogle Scholar
  20. Huan, Y., Shema, A., & Xia, H. (2017). A proposed genome of mobile and situated crowdsourcing and its design implications for encouraging contributions. International Journal of Human-Computer Studies, 102, 69–80.CrossRefGoogle Scholar
  21. Huang, Y.-C. (2015). Designing a micro-volunteering platform for situated crowdsourcing. In L. Ciolfi & D. McDonald (Eds.), Proceedings of the 19th ACM Conference Companion on Computer-Supported Cooperative Work and Social Computing, CSCW ‘15, Vancouver, Canada, 14–18 March 2015 (pp. 73–76). New York: ACM Press.Google Scholar
  22. Ipeirotis, P. G., & Gabrilovich, E. (2014). Quizz: Targeted crowdsourcing with a billion (potential) users. In A. Broder, K. Shim, & T. Suel (Eds.), Proceedings of the 23rd International Conference on World Wide Web, WWW ‘14, Seoul, South Korea, 7–11 April 2014 (pp. 143–154). New York: ACM Press.Google Scholar
  23. Jurmu, M., Goncalves, J., Riekki, J., & Ojala, T. (2014). Exploring use and appropriation of a non-moderated community display. In S. W. Loke, L. Kulik, & E. Pitoura (Eds.), Proceedings of the 13th International Conference on Mobile and Ubiquitous Multimedia, MUM ‘14, Melbourne, Australia, 25–28 November 2014 (pp. 107–115). New York: ACM Press.Google Scholar
  24. Kittur, A. (2010). Crowdsourcing, collaboration and creativity. XRDS: Crossroads, The ACM Magazine for Students, 17(2), 22–26.CrossRefGoogle Scholar
  25. Kittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., et al. (2013). The future of crowd work. In C. Lampe & L. Terveen (Eds.), Proceedings of the 2013 Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ‘13, San Antonio, USA, 23–27 February 2013 (pp. 1301–1318). New York: ACM Press.Google Scholar
  26. Kuikkaniemi, K., Jacucci, G., Turpeinen, M., Hoggan, E., & Müller, J. (2011). From space to stage: How interactive screens will change urban life. Computer, 44(6), 40–47.CrossRefGoogle Scholar
  27. Ludwig, T., Kotthaus, C., Reuter, C., van Dongen, S., & Pipek, V. (2017). Situated crowdsourcing during disasters: Managing the tasks of spontaneous volunteers through public displays. International Journal of Human-Computer Studies, 102, 103–121.CrossRefGoogle Scholar
  28. Malone, T. W., Laubacher, R., & Dellarocas, C. (2010). The collective intelligence genome. MIT Sloan Management Review, 51(3), 21–31.Google Scholar
  29. Martin, D., Hanrahan, B. V., O’Neill, J., & Gupta, N. (2014). Being a turker. In M. R. Morris, & M. Reddy (Eds.), Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW ‘14, Baltimore, USA, 15–19 February 2013 (pp. 224–235). New York: ACM Press.Google Scholar
  30. Martin, D., O’Neill, J., Gupta, N., & Hanrahan, B. V. (2016). Turking in a global labour market. Computer Supported Cooperative Work, 25(1), 39–77.CrossRefGoogle Scholar
  31. Müller, J., Alt, F., Michelis, D., & Schmidt, A. (2010). Requirements and design space for interactive public displays. In Smeulders, A. (Ed.), Proceedings of the 18th ACM International Conference on Multimedia, MM ‘10, Firenze, Italy, 25–29 October 2010 (pp. 1285–1294). New York: ACM Press.Google Scholar
  32. Müller, J., Wilmsmann, D., Exeler, J., Buzeck, M., Schmidt, A., Jay, T., et al. (2009). Display blindness: The effect of expectations on attention towards digital signage. In H. Tokuda, M. Beigl, A. Friday, A. J. Brush, & Y. Tobe (Eds.), Proceedings of the International Conference on Pervasive Computing, Pervasive ‘09, Nara, Japan, 17–20 May 2010 (pp. 1–8). Berlin, Heidelberg: Springer.Google Scholar
  33. Peltonen, P., Kurvinen, E., Salovaara, A., Jacucci, G., Ilmonen, T., Evans, J., et al. (2008). It’s mine, don’t touch!: Interactions at a large multi-touch display. In D. Tan (Ed.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ‘08, Florence, Italy, 5–10 April 2008 (pp. 1285–1294). New York: ACM Press.Google Scholar
  34. Retelny, D., Robaszkiewicz, S., To, A., Lasecki, W. S., Patel, J.. Rahmati, N., et al. (2014). Expert crowdsourcing with flash teams. In M. Dontcheva & D. Wigdor (Eds.), Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ‘14, Honolulu, USA, 5–8 October 2014 (pp. 75–85). New York: ACM Press.Google Scholar
  35. Salehi, N., McCabe, A., Valentine, M., & Bernstein, M. (2017). Huddler: Convening stable and familiar crowd teams despite unpredictable availability. In L. Barkhuus, M. Borges, & W. Kellogg (Eds.), Proceedings of the 2017 ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ‘17, San Francisco, USA, 25 February–1 March 2017 (pp. 1700–1713). New York: ACM Press.Google Scholar
  36. Saxton, G. D., Onook, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2–20.CrossRefGoogle Scholar
  37. Schmitz, H., & Lykourentzou, I. (2018, January). Online sequencing of non-decomposable macrotasks in expert crowdsourcing. ACM Transactions on Social Computing, 1, 1, Article 1, 33 pp.  https://doi.org/10.1145/3140459.CrossRefGoogle Scholar
  38. Shen, C., Everitt, K., & Ryall, K. (2003). UbiTable: Impromptu face-to-face collaboration on horizontal interactive surfaces. In A. K. Dey, A. Schimdt, & J. F. McCarthy (Eds.), Proceedings of the International Conference on Ubiquitous Computing, UbiComp ‘03, Seattle, USA, 12–15 October 2003 (pp. 281–288). Berlin, Heidelberg: Springer.Google Scholar
  39. Shirky, C. (2010). Cognitive surplus: How technology makes consumers into collaborators. Penguin.Google Scholar
  40. Terrenghi, L., Quigley, A., & Dix, A. (2009). A taxonomy for and analysis of multi-person-display ecosystems. Personal and Ubiquitous Computing, 13(8), 583–598.CrossRefGoogle Scholar
  41. Weißker, T., Berst, A., Hartmann, J., & Echtler, F. (2016). The massive mobile multiuser framework: Enabling ad-hoc realtime interaction on public displays with mobile devices. In J. Muller & N. Memarovic (Eds.), Proceedings of the 5th ACM International Symposium on Pervasive Displays, PerDis ‘16, Oulu, Finland, 20–22 June 2016 (pp. 168–174). New York: ACM Press.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jorge Goncalves
    • 1
    Email author
  • Simo Hosio
    • 2
  • Niels van Berkel
    • 1
  • Simon Klakegg
    • 2
  1. 1.School of Computing and Information SystemsThe University of MelbourneParkvilleAustralia
  2. 2.Center for Ubiquitous Computing, University of OuluOuluFinland

Personalised recommendations