Advertisement

Interactive Crowds: Real-Time Crowdsourcing and Crowd Agents

  • Walter S. LaseckiEmail author
  • Jeffrey P. Bigham
Chapter

Abstract

Crowdsourcing using independent tasks provides an effective means of leveraging human intelligence to solve discretized problems. However, this model cannot handle acquiring input on an ongoing task from workers. In order to expand the power of crowd algorithms, we present an overview of approaches to continuous real-time crowdsourcing that engages workers for longer periods of time, allowing workers to receive feedback from the system as the task evolves due to their and other’s input. We describe how models of continuous crowdsourcing can be used to enable task completion using both synchronous and asynchronous groups of workers. We then explore a new model of continuous real-time crowdsourcing called a “crowd agent” that allows groups of workers to interact with users and their environment as if they were a single individual. This model provides a means of abstracting away the collective in crowdsourcing by making it appear as a single intelligence.

Keywords

Prospective Memory Ongoing Task Blind User Multiple Worker Prior Interaction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Bernstein MS, Little G, Miller RC, Hartmann B, Ackerman MS, Karger DR, Crowell D, Panovich K (2010) Soylent: a word processor with a crowd inside. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology, UIST’10, ACM, New York, pp 313–322Google Scholar
  2. Bernstein MS, Brandt JR, Miller RC, Karger DR (2011) Crowds in two seconds: enabling real-time crowd-powered interfaces. In: Proceedings of the 24th annual ACM symposium on user interface software and technology, UIST’11, ACM, New York, pp 33–42Google Scholar
  3. Bernstein MS, Karger DR, Miller RC, Brandt J (2012) Analytic methods for optimizing real-time crowdsourcing. In: Proceedings of the collective intelligence, Boston, MAGoogle Scholar
  4. Bigham JP, Jayant C, Ji H, Little G, Miller A, Miller RC, Miller R, Tatarowicz A, White B, White S, Yeh T (2010) Vizwiz: nearly real-time answers to visual questions. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology, UIST’10, ACM, New York, pp 333–342Google Scholar
  5. Brady E, Morris MR, Zhong Y, Bigham JP (2013) Visual challenges in the everyday lives of blind people. In: Proceedings of the ACM SIGCHI conference on human factors in computing systems (CHI 2013), ParisGoogle Scholar
  6. Chilton L (2009) Seaweed: a web application for designing economic games. Master’s thesis, MITGoogle Scholar
  7. Cooper S, Khatib F, Treuille A, Barbero J, Lee J, Beenen M, Leaver-Fay A, Baker D, Popovic Z, Players F (2010) Predicting protein structures with a multiplayer online game. Nature 466(7307):756–760CrossRefGoogle Scholar
  8. Dai P, Mausam, Weld DS (2010) Decision-theoretic control of crowd-sourced workflows. In: Twenty-fourth AAAI conference on artificial intelligence (AAAI 2010), AtlantaGoogle Scholar
  9. Kokkalis N, Köhn T, Pfeiffer C, Chornyi D, Bernstein MS, Klemmer SR (2013) EmailValet: managing email overload through private, accountable crowdsourcing. In: Proceedings of the 2013 conference on computer supported cooperative work (CSCW ’13), ACM, New YorkGoogle Scholar
  10. Lasecki WS, Murray K, White S, Miller RC, Bigham JP (2011) Real-time crowd control of existing interfaces. In: Proceedings of the ACM symposium on User Interface Software and Technology, UIST’11, ACM, New York, pp 23–32Google Scholar
  11. Lasecki WS, Miller CD, Sadilek A, Abumoussa A, Borrello D, Kushalnagar R, Bigham JP (2012a) Real-time captioning by groups of non-experts. In: Proceedings of the ACM symposium on User Interface Software and Technology (UIST 2012), Boston, MA, pp 23–34Google Scholar
  12. Lasecki WS, White S, Murray K, Bigham JP (2012b) Crowd memory: learning in the collective. In: Proceedings of collective intelligence (CI’12), BostonGoogle Scholar
  13. Lasecki WS, Miller CD, Bigham JP (2013a) Warping time for more effective real-time crowdsourcing. In: Proceedings of the international ACM conference on human factors in computing systems, CHI’13, page to appear, Paris, FranceGoogle Scholar
  14. Lasecki WS, Song Y, Kautz H, Bigham J (2013b) Real-time crowd labeling for deployable activity recognition. In: Proceedings of the international ACM conference on computer supported cooperative work and social computing (CSCW 2013), San Antonio, TX, pp 1203–1212Google Scholar
  15. Lasecki WS, Wesley R, Nichols J, Kulkarni A, Allen JF, Bigham J (2013c) Chorus: a crowd-powered conversational assistant. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology (UIST’13), St. Andrews, UK. UIST 2013. PP 151-162Google Scholar
  16. Little G, Chilton LB, Goldman M, Miller RC (2010) Turkit: human computation algorithms on mechanical Turk. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology, UIST’10, ACM, New York, pp 57–66Google Scholar
  17. Naim I, Lasecki WS, Bigham JP, Gildea D (2013) Text alignment for real-time crowd captioning. In: Proceedings of the North American Chapter of the association for computational linguistics (NAACL 2013), To appear, Atlanta, GAGoogle Scholar
  18. Nielsen J (1993) Usability engineering. Morgan Kaufmann, San FranciscozbMATHGoogle Scholar
  19. von Ahn L, Dabbish L (2004) Labeling images with a computer game. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’04, ACM, New York, pp 319–326Google Scholar
  20. Zhang H, Law E, Miller RC, Gajos K, Parkes DC, Horvitz E (2012) Human computation tasks with global constraints. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI’12), pp 217–226, Austin, TXGoogle Scholar
  21. Zhong Y, Thiha P, He G, Lasecki WS, Bigham JP (2012) In: Proceedings of the ACM conference on human factors in computing systems work-in-progress (CHI 2012), AustinGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of RochesterRochesterUSA
  2. 2.Human-Computer Interaction InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations