Worker-Centric Design for Software Crowdsourcing: Towards Cloud Careers

  • Dave Murray-RustEmail author
  • Ognjen Scekic
  • Donghui Lin
Part of the Progress in IS book series (PROIS)


Crowdsourcing is emerging as a compelling technique for the cost-effective creation of software, with tools such as ODesk and TopCoder supporting large scale distributed development. From the point of view of the commissioners of software, there are many advantages to crowdsourcing work—as well as cost, it can be a more scalable process, as there is the possibility of selecting from a large pool of expertise. From the point of view of workers, there is a different set of benefits, including choice of when and how to work, providing a means to build a portfolio, and a lower level of commitment to any particular employer. The crowdsourcing of software development—in common with some other activities such as design—represents an alternative to existing mechanisms that require skilled workers. However, if crowdsourcing were to replace traditional employment for a significant proportion of software developers, the reduced levels of commitment between workers and commissioners could prove problematic for workers over time. In this paper, explore three areas of interest: (i) trust and reputation development; (ii) team selection and team building; (iii) contextualisation of the work carried out. By drawing together work in these areas from the point of view of workers rather than commissioners, we highlight some of the incipient issues with the growth of crowdsourced labour. We also explore ways in which crowdsourcing of software development—and other skilled practices—differers from microtasking.


Software Development Minimum Wage Reputation System Career Ladder Development Open Source Software 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Anagnostopoulos, A., Becchetti, L., Castillo, C., Gionis, A., Leonardi, S.: Online team formation in social networks. In: Proceedings of the 21st International Conference on World Wide Web, pp. 839–848. ACM (2012)Google Scholar
  2. 2.
    Bederson, B.B., Quinn, A.J.: Web workers unite! addressing challenges of online laborers. In: Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems—CHI EA’11, pp. 97–106. ACM Press, New York (2011). doi: 10.1145/1979742.1979606
  3. 3.
    Chilton, L.B., Horton, J.J., Miller, R.C., Azenkot, S.: Task search in a human computation market. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp. 1–9. ACM (2010)Google Scholar
  4. 4.
    Crowston, K., Li, Q., Wei, K., Eseryel, U.Y., Howison, J.: Self-organization of teams for free/libre open source software development. Inf. Softw. Technol. 49(6), 564–575 (2007).
  5. 5.
    Dai, P., Weld, D.S.: Others: decision-theoretic control of crowd-sourced workflows. In: Twenty-Fourth AAAI Conference on Artificial Intelligence (2010)Google Scholar
  6. 6.
    De Alfaro, L., Kulshreshtha, A., Pye, I., Adler, B.T.: Reputation systems for open collaboration. Commun. ACM 54(8), 81–87 (2011)CrossRefGoogle Scholar
  7. 7.
    Dustdar, S., Bhattacharya, K.: The social compute unit. IEEE Internet Comput. 15(3), 64–69 (2011). doi: 10.1109/MIC.2011.68
  8. 8.
    Felstiner, A.: Working the crowd : employment and labor law in the crowdsourcing industry (2010)Google Scholar
  9. 9.
  10. 10.
    Gassenheimer, J.B., Siguaw, J.A., Hunter, G.L.: Exploring motivations and the capacity for business crowdsourcing. AMS Rev. 1–12 (2013). doi: 10.1007/s13162-013-0055-8
  11. 11.
    Harris, C.G.: Dirty deeds done dirt cheap. In: 2011 IEEE International Conference on Privacy, Security, Risk and Trust, pp. 1314–1317 (2011)Google Scholar
  12. 12.
    Horton, J.J.: The condition of the turking class: are online employers fair and honest? Econ. Lett. 111(1), 10–12 (2011).
  13. 13.
    Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)Google Scholar
  14. 14.
    Josang, A., Ismail, R., Boyd, C.: A survey of trust and reputation systems for online service provision. Decis. Support Syst. 43(2), 618–644 (2007). doi: 10.1016/j.dss.2005.05.019.
  15. 15.
    Jøsang, A., Ismail, R., Boyd, C.: A survey of trust and reputation systems for online service provision. Decis. Support Syst. 43(2), 618–644 (2007)Google Scholar
  16. 16.
    Kittur, A., Nickerson, J.V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., Horton, J.: The future of crowd work. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work, pp. 1301–1318. ACM (2013)Google Scholar
  17. 17.
    Malone, T.W., Laubacher, R., Dellarocas, C.: The collective intelligence genome. MIT Sloan Manag. Rev. 51(3), 21–31 (2010).
  18. 18.
    Marti, S., Garcia-Molina, H.: Taxonomy of trust: categorizing P2P reputation systems. Comput. Netw. (April 2005), 1–20 (2006).
  19. 19.
    Prendergast, C.: The provision of incentives in firms. J. Econ. Lit. 37(1), 7–63 (1999).
  20. 20.
    Resnick, P., Kuwabara, K., Zeckhauser, R., Friedman, E.: Reputation systems. Commun. ACM 43(12), 45–48 (2000)CrossRefGoogle Scholar
  21. 21.
    Ross, J., Irani, L., Silberman, M., Zaldivar, A., Tomlinson, B.: Who are the crowdworkers? Shifting demographics in mechanical turk. In: CHI’10 Extended Abstracts on Human Factors in Computing Systems, pp. 2863–2872 (2010).
  22. 22.
    Schenk, E., Guittard, C.: Towards a characterization of crowdsourcing practices. J. Innov. Econ. Manag. 1(7), 93–107 (2011).
  23. 23.
    Serugendo, G.D.M.: Self-organisation and emergence in multi-agent systems. Knowl. Eng. Rev. 20(2), 165–189 (2005)CrossRefGoogle Scholar
  24. 24.
    Shaw, A.D., Horton, J.J., Chen, D.L.: Designing incentives for inexpert human raters. In: Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work, pp. 275–284. ACM (2011)Google Scholar
  25. 25.
    Silberman, M.S., Irani, L., Ross, J.: Ethics and tactics of professional crowdwork. XRDS: Crossroads ACM Mag. Stud. 17(2), 39 (2010). doi: 10.1145/1869086.1869100
  26. 26.
    Skopik, F.: Dynamic trust in mixed service-oriented systems. Ph.D. Thesis (2010).
  27. 27.
    Software developer salary in United States. Accessed 21 Nov 2013
  28. 28.
    Zittrain, J.: Ubiquitous human computing. Philos. Trans. Ser. A Math. Phys. Eng. Sci. 366(1881), 3813–3821 (2008). doi: 10.1098/rsta.2008.0116

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.CISA, School of InformaticsUniversity of EdinburghEdinburghUK
  2. 2.Distributed Systems GroupVienna University of TechnologyViennaAustria
  3. 3.Department of Social InformaticsKyoto UniversityKyotoJapan

Personalised recommendations