QoS-Based Task Scheduling in Crowdsourcing Environments

  • Roman Khazankin
  • Harald Psaier
  • Daniel Schall
  • Schahram Dustdar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7084)


Crowdsourcing has emerged as an important paradigm in human-problem solving techniques on the Web. One application of crowdsourcing is to outsource certain tasks to the crowd that are difficult to implement as solutions based on software services only. Another benefit of crowdsourcing is the on-demand allocation of a flexible workforce. Businesses may outsource certain tasks to the crowd based on workload variations. The paper addresses the monitoring of crowd members’ characteristics and the effective use of monitored data to improve the quality of work. Here we propose the extensions of standards such as Web Service Level Agreement (WSLA) to settle quality guarantees between crowd consumers and the crowdsourcing platform. Based on negotiated agreements, we provide a skill-based crowd scheduling algorithm. We evaluate our approach through simulations.


crowdsourcing skill monitoring scheduling QoS agreements 


  1. 1.
    Agichtein, E., Castillo, C., Donato, D., Gionis, A., Mishne, G.: Finding high-quality content in social media. In: WSDM 2008, pp. 183–194. ACM (2008)Google Scholar
  2. 2.
    Agrawal, A., et al.: WS-BPEL Extension for People (BPEL4People) (2007) Google Scholar
  3. 3.
    Amazon Mechnical Turk (May 2011),
  4. 4.
    Brabham, D.: Crowdsourcing as a model for problem solving: An introduction and cases. Convergence 14(1), 75 (2008)Google Scholar
  5. 5.
    Caprara, A., Monaci, M., Toth, P.: Models and algorithms for a staff scheduling problem. Math. Program. 98(1-3), 445–476 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Castellano, C., Fortunato, S., Loreto, V.: Statistical physics of social dynamics. Reviews of Modern Physics 81(2), 591–646 (2009)CrossRefGoogle Scholar
  7. 7.
    Che, Y.: Design competition through multidimensional auctions. The RAND Journal of Economics 24(4), 668–680 (1993)CrossRefGoogle Scholar
  8. 8.
    Doan, A., Ramakrishnan, R., Halevy, A.Y.: Crowdsourcing systems on the world- wide web. Commun. ACM 54, 86–96 (2011)CrossRefGoogle Scholar
  9. 9.
    Ernst, A.T., Jiang, H., Krishnamoorthy, M., Sier, D.: Staff scheduling and rostering: A review of applications, methods and models. European Journal of Operational Research 153(1), 3–27 (2004), timetabling and RosteringGoogle Scholar
  10. 10.
    Ford, M., et al.: Web Services Human Task (WS-HumanTask), Version 1.0. (2007) Google Scholar
  11. 11.
    Howe, J.: The rise of crowdsourcing (June 2006),
  12. 12.
    Ipeirotis, P.G.: Analyzing the Amazon Mechanical Turk Marketplace. SSRN eLibrary 17(2), 16–21 (2010)Google Scholar
  13. 13.
    Kern, R., Thies, H., Satzger, G.: Statistical Quality Control for Human-Based Electronic Services. In: Maglio, P.P., Weske, M., Yang, J., Fantinato, M. (eds.) ICSOC 2010. LNCS, vol. 6470, pp. 243–257. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Khazankin, R., Schall, D., Dustdar, S.: Adaptive request prioritization in dynamic service-oriented systems. In: Proceedings of the 8th International Conference on Services Computing (to appear, 2011) Google Scholar
  15. 15.
    La Vecchia, G., Cisternino, A.: Collaborative Workforce, Business Process Crowd- sourcing as an Alternative of BPO. In: Daniel, F., Facca, F.M. (eds.) ICWE 2010. LNCS, vol. 6385, pp. 425–430. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Leymann, F.: Workflow-Based Coordination and Cooperation in a Service World. In: Meersman, R., Tari, Z., et al. (eds.) OTM 2006. LNCS, vol. 4275, pp. 2–16. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    LiveOps (May 2011),
  18. 18.
    Parkes, D., Kalagnanam, J.: Models for iterative multiattribute procurement auctions. Management Science 51(3), 435–451 (2005)CrossRefzbMATHGoogle Scholar
  19. 19.
    Schall, D., Dustdar, S.: Dynamic Context-Sensitive Pagerank for Expertise Mining. In: Bolc, L., Makowski, M., Wierzbicki, A. (eds.) SocInfo 2010. LNCS, vol. 6430, pp. 160–175. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  20. 20.
    Sousa, J., Poladian, V., Garlan, D., Schmerl, B., Shaw, M.: Task-based adaptation for ubiquitous computing. IEEE Transactions on Systems, Man, and Cybernetics 36(3), 328–340 (2006)CrossRefGoogle Scholar
  21. 21.
    Vukovic, M.: Crowdsourcing for Enterprises. In: Proceedings of the 2009 Congress on Services, pp. 686–692. IEEE (2009) Google Scholar
  22. 22.
    Yahoo! Answers (May 2011),

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Roman Khazankin
    • 1
  • Harald Psaier
    • 1
  • Daniel Schall
    • 1
  • Schahram Dustdar
    • 1
  1. 1.Distributed Systems GroupVienna University of TechnologyViennaAustria

Personalised recommendations