Skip to main content

Crowdsourcing: a new tool for policy-making?


Crowdsourcing is rapidly evolving and applied in situations where ideas, labour, opinion or expertise of large groups of people is used. Crowdsourcing is now used in various policy-making initiatives; however, this use has usually focused on open collaboration platforms and specific stages of the policy process, such as agenda-setting and policy evaluations. Other forms of crowdsourcing have been neglected in policy-making, with a few exceptions. This article examines crowdsourcing as a tool for policy-making and explores the nuances of the technology and its use and implications for different stages of the policy process. The article addresses questions surrounding the role of crowdsourcing and whether it can be considered as a policy tool or as a technological enabler and investigates the current trends and future directions of crowdsourcing.

This is a preview of subscription content, access via your institution.

Fig. 1


  1. 1.

    These categorisations are not exclusive or exhaustive, but useful for considering the different roles crowdsourcing can take in the policy cycle. For a review of the state of the art in crowdsourcing, see Prpić (2016).

  2. 2.

    With respect to their offline identities. However, researchers such as Lease et al. (2013) have previously demonstrated that a significant amount of information can be exposed about the workers through the VLM websites.

  3. 3.

  4. 4.

  5. 5.

    Various classification attempts and corresponding models of the policy processes exist, of which perhaps the most popular is the use of sequential interrelated stages as a policy cycle. In this article, based on the efforts of Stone (1988) and Howlett et al. (1995), the policy cycle is seen as a sequence of steps in which agenda-setting, problem definition, policy design, policy implementation, policy enforcement and policy evaluations are carried out in an iterative manner (Taeihagh et al. 2009).

  6. 6.

    Even in the case of online surveys, the speed at which a worker can carry out a microtask is much faster than an online survey (Prpić et al. 2014).

  7. 7.

    Expert crowdsourcing, mainly through competition-based platforms (and future high-skilled VLMs sites once their use becomes more mainstream) and non-expert crowdsourcing through the use of VLMs. OC platforms provide access to both expert and non-expert crowds, but require a more sustained effort in attracting and maintaining them. It is worth nothing recent research by Bonazzi et al. (2017) demonstrates a successful combined engagement of expert and non-expert crowds in scenario planning.

  8. 8.

    OC platforms, for instance, have amplified unscientific and unsubstantiated claims regarding MMR vaccination, resulting in a significant increase in outbreaks of preventable diseases such as measles in the UK and the USA (Perry 2013).

  9. 9.

    A potential worrying development in case of massive adoption of crowdsourcing (such as in the examples italicized in Table 5) is the difficulty in upholding oversight and keeping organisations accountable in future, especially if block-chain technology is used as the level of anonymity can increase. Block-chain technology such as Bitcoin is not anonymous, but in comparison with traditional means of monetary exchange (in the hands of expert individuals) it has a higher level of anonymity as it does not require sending and receiving personally identifiable information:

  10. 10.

    As a crude measure at the time of finalising this manuscript in November 2017, 469 papers have the term “crowdsourcing” in the title AND mention the term “revolution” in their text. There are also 16,800 academic papers that mention crowdsourcing AND revolution in their text.

  11. 11.

  12. 12.

    Different forms of crowdsourcing and sharing economy share commonalities in terms of the use of reputation systems and IT, the reliance on crowds and the exchange of information and currency (Taeihagh 2017a). The literature in one domain, however, often ignores the other or treats it in a singular form rather than considering the different types that fall under the umbrella term. Sometimes, moreover, a platform is categorised both as a sharing economy and as a crowdsourcing platform by different scholars, particularly when the topic of the study relates to VLMs and OCs (particularly commons such as Wikipedia). Westerbeek (2016) explicitly differentiates between crowdsourcing and sharing economy platforms by stating the one-on-one, peer-to-peer aspect to be the most important part of a sharing economy, and that this is not present in crowdsourcing. Other scholars distinguish between them by pointing out that if a labour market platform for instance provides a virtual service that can be performed online (such as Amazon Mturk), that platform is a crowdsourcing platform; in contrast, if it provides a physical service to be performed locally, it is a sharing economy platform (such as TaskRabbit) (Gansky 2010; De Groen, Maselli and Fabo 2016; Aloisi 2015; Rauch and Schleicher 2015). With these new developments in crowdsourcing, however, the line between crowdsourcing and sharing economy platforms seems to be gradually blurring which provides further evidence that as Prpić and Shukla (2016) point out, there is a potential for unifying these fields with development of generalisable frameworks for studying IT-mediated crowds.


  1. Aitamurto, T. (2012). Crowdsourcing for democracy: New era in policy-making. Publications of the Committee for the Future, Parliament of Finland, 1/2012. Helsinki, Finland.

  2. Aitamurto, T. (2016a). Collective intelligence in law reforms: When the logic of the crowds and the logic of policymaking collide. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 2780–2789). IEEE.

  3. Aitamurto, T., Chen, K., Cherif, A., Galli, J. S., & Santana, L. (2016, October). Civic CrowdAnalytics: Making sense of crowdsourced civic input with big data tools. In Proceedings of the 20th International Academic Mindtrek Conference (pp. 86–94). ACM.

  4. Aitamurto, T., & Landemore, H. (2016). Crowdsourced deliberation: The case of the law on off-road traffic in Finland. Policy and Internet, 8(2), 174–196.

    Article  Google Scholar 

  5. Aloisi, A. (2015). Commoditized workers: Case study research on labour law issues arising from a set of ‘on-demand/gig economy’ platforms.

  6. Asmolov, G. (2015). Vertical crowdsourcing in Russia: Balancing governance of crowds and state-citizen partnership in emergency situations. Policy & Internet, 7(3), 292–318.

    Article  Google Scholar 

  7. Bayus, B. L. (2013). Crowdsourcing new product ideas over time: An analysis of the Dell IdeaStorm community. Management Science, 59(1), 226–244.

    Article  Google Scholar 

  8. Bennett, C. J., & Howlett, M. (1992). The lessons of learning: Reconciling theories of policy learning and policy change. Policy Sciences, 25(3), 275–294.

    Article  Google Scholar 

  9. Berners-Lee, T., Fischetti, M. & Foreword By-Dertouzos, M.L. (2000). Weaving the web: The original design and ultimate destiny of the World Wide Web by its inventor. Harper Information.

  10. Boer, D. (2016). Stealthy McStealthface reports for service. The Times. Retrieved May/June 2017.

  11. Bonazzi, R., Viscusi, G., & Barbey, V. (2017). Crowd and experts’ knowledge: Connection and value through the notion of prism. In European, Mediterranean, and Middle Eastern Conference on Information Systems (pp. 646–654). Springer, Cham.

  12. Brabham, D. C. (2008). Crowdsourcing as a model for problem-solving an introduction and cases. Convergence, 14(1), 75–90.

    Article  Google Scholar 

  13. Brabham, D. C. (2013a). Crowdsourcing. Cambridge: MIT Press.

    Google Scholar 

  14. Brabham, D.C. (2013b). Using crowdsourcing in government. IBM Center for The Business of Government.

  15. Budhathoki, N. R., & Haythornthwaite, C. (2013). Motivation for open collaboration crowd and community models and the case of OpenStreetMap. American Behavioral Scientist, 57(5), 548–575.

    Article  Google Scholar 

  16. Certoma, C., Corsini, F., & Rizzi, F. (2015). Crowdsourcing urban sustainability. Data, people and technologies in participatory governance. Futures, 74, 93–106.

    Article  Google Scholar 

  17. Chappell, B. (2017). Footy McFooty face is stomping competition in vote For MLS team name. National Public Radio (NPR), Retrieved March 2017.

  18. Cha, M., Haddadi, H., Benevenuto, F., & Gummadi, P. K. (2010). Measuring user influence in Twitter: The million follower fallacy. ICWSM, 10(10–17), 30.

    Google Scholar 

  19. Codagnone, C., Abadie, F. & Biagi, F.(2016). The future of work in the ‘Sharing Economy’. Market efficiency and equitable opportunities or unfair precarisation? Institute for Prospective Technological Studies.

  20. Crowdflower (2016) Crowdflower Inc. Retrieved 20 May 2016.

  21. Crowley, C., Daniels, W., Bachiller, R., Joosen, W., & Hughes, D. (2014). Increasing user participation: An exploratory study of querying on the Facebook and Twitter platforms. First Monday, 19(8), 1.

    Article  Google Scholar 

  22. Crump, J. (2011). What are the police doing on Twitter? Social media, the police and the public. Policy and Internet, 3(4), 1–27.

    Article  Google Scholar 

  23. De Groen, W.P., Maselli, I. & Fabo, B. (2016). The digital market for local services: A one-night stand for workers? CEPS Special Report, No. 133.

  24. De Vreede, T., Nguyen, C., De Vreede, G.J., Boughzala, I., Oh, O. & Reiter-Palmon, R. (2013). A theoretical model of user engagement in crowdsourcing. In Collaboration and Technology Lecture Notes in Computer Science, 8224, pp. 94–109.

  25. De Winter, J.C.F., Kyriakidis, M., Dodou, D. & Happee, R. (2015). Using CrowdFlower to study the relationship between self-reported violations and traffic accidents. In Proceedings of the 6th International Conference on Applied Human Factors and Ergonomics (AHFE), Las Vegas, NV.

  26. Dutil, P. (2015). Crowdsourcing as a new instrument in the government’s arsenal: Explorations and considerations. Canadian Public Administration, 58(3), 363–383.

    Article  Google Scholar 

  27. Eichenwald, K. (2012). Microsoft’s lost Decade. Vanity Fair, Retrieved July 2017.

  28. Ellis-Petersen, H. (2016). Boaty McBoatface wins poll to name polar research vessel. The Guardian, Retrieved June 2017.

  29. Estrelles-Arolas, E., & Gonzalez-Ladron-De-Guevara, F. (2012). Towards an integrated crowdsourcing definition. Journal of Information Science, 38(2), 189–200.

    Article  Google Scholar 

  30. Fischer, F. (1993). Citizen participation and the democratization of policy expertise: From theoretical inquiry to practical cases. Policy Sciences, 26(3), 165–187.

    Article  Google Scholar 

  31. Gansky, L. (2010). The Mesh: Why the future of business is sharing. New York, NY: Penguin.

    Google Scholar 

  32. Gellers, J. C. (2016). Crowdsourcing global governance: Sustainable development goals, civil society, and the pursuit of democratic legitimacy. International Environmental Agreements. Politics, Law and Economics, 16(3), 415–432.

    Article  Google Scholar 

  33. Glaeser, E. L., Hillis, A., Kominers, S. D., & Luca, M. (2016). Crowdsourcing city government: Using tournaments to improve inspection accuracy. The American Economic Review, 106(5), 114–118.

    Article  Google Scholar 

  34. Goncalves, J., Hosio, S., Kostakos, V., Vukovic, M., & Konomi, S.I. (2015) Workshop on mobile and situated crowdsourcing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and International Symposium on Wearable Computers (pp. 1339–1342). ACM.

  35. Goodchild, M. F., & Glennon, J. A. (2010). Crowdsourcing geographic information for disaster response: A research frontier. International Journal of Digital Earth, 3(3), 231–241.

    Article  Google Scholar 

  36. Greengard, S. (2011). Following the crowd. Communications of the ACM, 54(2), 20–22.

    Article  Google Scholar 

  37. Gruzd, A., & Tsyganova, K. (2015). Information wars and online activism during the 2013/2014 Crisis in Ukraine: Examining the social structures of Pro-and Anti-Maidan Groups. Policy & Internet, 7(2), 121–158.

    Article  Google Scholar 

  38. Hendry, D. F., & Ericsson, N. R. (2003). Understanding economic forecasts. Cambridge, MA: MIT Press.

    Google Scholar 

  39. Hern, A. (2017). Trainy McTrainface: Swedish railway keeps Boaty’s legacy alive. The Guardian, Retrieved July 2017.

  40. Hira, A. (2017). Profile of the sharing economy in the developing world: Examples of companies trying to change the world. Journal of Developing Societies., 33(2), 244–271.

    Google Scholar 

  41. Hood, C. (1986). The tools of government. Chatham: Chatham House Publishers.

    Google Scholar 

  42. Hood, C. (2007). Intellectual obsolescence and intellectual makeovers: Reflections on the tools of government after two decades. Governance, 20(1), 127–144.

    Article  Google Scholar 

  43. Hood, C., & Margetts, H. (2007). The tools of government in the digital age. London: Palgrave Macmillan.

    Book  Google Scholar 

  44. Horton, J.J. & Chilton, L.B. (2010). The labour economics of paid crowdsourcing. In Proceedings of the 11th ACM Conference on Electronic Commerce (pp. 209–218). ACM.

  45. Hosio, S. (2016). Situated crowdsourcing. Retrieved 20 June 2017.

  46. Howe, J. (2006) Crowdsourcing: A definition.

  47. Howe, J. (2008) Crowdsourcing: Why the power of the crowd is driving the future of business. Crown Business.—%20Crowdsourcing.PDF.

  48. Howlett, M. (2000). Managing the hollow state: Procedural policy instruments and modern governance. Canadian Public Administration, 43(4), 412–431.

    Article  Google Scholar 

  49. Howlett, M. (2009). Government communication as a policy tool: A framework for analysis. Canadian Political Science Review, 3(2), 23–37.

    Google Scholar 

  50. Howlett, M. (2010). Designing public policies: Principles and instruments. London: Taylor and Francis.

    Google Scholar 

  51. Howlett, M., Ramesh, M., & Perl, A. (1995). Studying public policy: Policy cycles and policy subsystems. Toronto: Oxford University Press.

    Google Scholar 

  52. Jackson, P., & Klobas, J. (2013). Deciding to use an enterprise wiki: The role of social institutions and scripts. Knowledge Management Research and Practice, 11(4), 323–333.

    Article  Google Scholar 

  53. Jeppesen, L. B., & Lakhani, K. R. (2010). Marginality and problem-solving effectiveness in broadcast search. Organisation Science, 21(5), 1016–1033.

    Article  Google Scholar 

  54. Krumm, J. & Horvitz, E. (2014). Situated Crowdsourcing. Retrieved June 2017.

  55. Lakhani, K., Garvin, D. A., & Lonstein, E. (2010). Topcoder (a): Developing software through crowdsourcing (pp. 610–632). Cambridge: Harvard Business School General Management Unit Case.

    Google Scholar 

  56. Landemore, H. (2015). Inclusive constitution-making: The Icelandic experiment. Journal of Political Philosophy, 23(2), 166–191.

    Article  Google Scholar 

  57. Lease, M., Hullman, J., Bigham, J. P., Bernstein, M. S., Kim, J., Lasecki, W., Bakhshi, S., Mitra, T., & Miller, R. C. (2013). Mechanical turk is not anonymous. Rocheser, NY: Social Science Research Network.

    Google Scholar 

  58. Lehdonvirta, V., & Bright, J. (2015). Crowdsourcing for public policy and government. Policy and Internet, 7(3), 263–267.

    Article  Google Scholar 

  59. Liu, H. K. (2017a). Crowdsourcing design: A synthesis of literatures. In Proceedings of the 50th Hawaii International Conference on System Sciences.

  60. Liu, H. K. (2017b). Crowdsourcing government: Lessons from multiple disciplines. Public Administration Review, 77(5), 656–667.

    Article  Google Scholar 

  61. Longo, J., & Kelley, T. M. (2016). GitHub use in public administration in Canada: Early experience with a new collaboration tool. Canadian Public Administration, 59(4), 598–623.

    Article  Google Scholar 

  62. Luz, N., Silva, N., & Novais, P. (2015). A survey of task-oriented crowdsourcing. Artificial Intelligence Review, 44(2), 187–213.

    Article  Google Scholar 

  63. Marcus, A., & Parameswaran, A. (2015). Crowdsourced data management: Industry and academic perspectives. Foundations and Trends® in Databases, 6(1–2), 1–161.

    Google Scholar 

  64. Mazumdar, S., Wrigley, S., & Ciravegna, F. (2017). Citizen science and crowdsourcing for earth observations: An analysis of stakeholder opinions on the present and future. Remote Sensing, 9(1), 87.

    Article  Google Scholar 

  65. Mergel, I. (2015). Open collaboration in the public sector: The case of social coding on GitHub. Government Information Quarterly, 32(4), 464–472.

    Article  Google Scholar 

  66. Michel, F., Gil, Y. & Hauder, M. (2015). A virtual crowdsourcing community for open collaboration in science processes. In Americas Conference on Information Systems (AMCIS).

  67. Narula, P., Gutheim P., Rolnitzky, D., Kulkarni, A. & Hartmann, B. (2011). MobileWorks: A mobile crowdsourcing platform for workers at the bottom of the pyramid. In Proceedings of HCOMP.

  68. Nash, A. (2009). Web 2.0 applications for improving public participation in transport planning. In Transportation Research Board 89th Annual Meeting.

  69. Nordrum, A. (2016). Popular internet of things forecast of 50 billion devices by 2020 is outdated. IEEE Spectrum, 18.

  70. Okolloh, O. (2009). Ushahidi, or ‘testimony’: Web 2.0 tools for crowdsourcing crisis information. Participatory Learning and Action, 59(1), 65–70.

    Google Scholar 

  71. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5), 411–419.

    Google Scholar 

  72. Park, A. J., Ko, J. M., & Swerlick, R. A. (2017). Crowdsourcing dermatology: DataDerm, big data analytics, and machine learning technology. Journal of the American Academy of Dermatology. https:\\\10.1016/j.jaad.2017.08.053.

    Google Scholar 

  73. Perry, D. M. (2013). Destabilizing the Jenny McCarthy public health industrial complex. Atlantic Monthly July 11, Retrieved June 2017.

  74. Piller, F. T., & Walcher, D. (2006). Toolkits for idea competitions: a novel method to integrate users in new product development. R&D Management, 36(3), 307–318.

    Article  Google Scholar 

  75. Prpić, J. (2016). Next generation crowdsourcing for collective intelligence. In Collective intelligence conference: NYU Stern School of Business from June 13, 2016.

  76. Prpić, J., & Shukla, P. (2013). The theory of crowd capital. In Proceedings of the 46th annual Hawaii international conference on system sciences. Hawaii: Computer Society Press.

  77. Prpić, J. & Shukla, P. (2016). Crowd Science: Measurements, models, and methods. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 4365–4374). IEEE.

  78. Prpić, J., Taeihagh, A. & Melton, J. (2014) Experiments on crowdsourcing policy assessment. In Oxford Internet Policy and Politics Conference (IPP 2014), University of Oxford, 26–28 September 2014.

  79. Prpić, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy and Internet, 7(3), 340–361.

    Article  Google Scholar 

  80. Rauch, D.E. & Schleicher, D. (2015). Like Uber, but for local governmental policy: The future of local regulation of the “sharing economy”. George Mason Law & Economics Research Paper, 15–01.

  81. Rittel, H. W., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169.

    Article  Google Scholar 

  82. Rogstadius, J., Vukovic, M., Teixeira, C. A., Kostakos, V., Karapanos, E., & Laredo, J. A. (2013). CrisisTracker: Crowdsourced social media curation for disaster awareness. IBM Journal of Research and Development, 57(5), 1–4.

    Article  Google Scholar 

  83. Sabatier, P. A. (1988). An advocacy coalition framework of policy change and the role of policy-oriented learning therein. Policy Sciences, 21(2), 129–168.

    Article  Google Scholar 

  84. Sadat, D. R. (2014). M-government implementation evaluation in encouraging citizen participation in Indonesia: A case study of LAPOR!, Ph.D. Thesis, University of Manchester.

  85. Salus, P. H., & Vinton, G. (1995). Casting the net: From ARPANET to internet and beyond. Boston, MA: Addison-Wesley Longman Publishing.

    Google Scholar 

  86. Schweitzer, F. M., Buchinger, W., Gassmann, O., & Obrist, M. (2012). Crowdsourcing: Leveraging innovation through online idea competitions. Research-Technology Management, 55(3), 32–38.

    Article  Google Scholar 

  87. Seltzer, E., & Mahmoudi, D. (2013). Citizen participation, open innovation, and crowdsourcing challenges and opportunities for planning. Journal of Planning Literature, 28(1), 3–18.

    Article  Google Scholar 

  88. Stone, D. A. (1988). Policy Paradox and Political Reason. New York: Harper Collins.

    Google Scholar 

  89. Sun, X., Hu, S., Su, L., Abdelzaher, T., Hui, P., Zheng, W., et al. (2015). Participatory sensing meets opportunistic sharing: Automatic phone-to-phone communication in vehicles. IEEE Transactions on Mobile Computing, 15, 2550–2563.

    Article  Google Scholar 

  90. Taeihagh, A. (2017a). Crowdsourcing, sharing economy and development. Journal of Developing Societies, 33(2), 1–32.

    Google Scholar 

  91. Taeihagh, A. (2017b). Network centric policy design. Policy Sciences Journal, 50(2), 317–338.

    Article  Google Scholar 

  92. Taeihagh, A., Bañares-Alcántara, R., & Millican, C. (2009). Development of a novel framework for the design of transport policies to achieve environmental targets. Computers & Chemical Engineering, 33(10), 1531–1545.

    Article  Google Scholar 

  93. Taieb, S. B., & Hyndman, R. J. (2014). A gradient boosting approach to the Kaggle load forecasting competition. International Journal of Forecasting, 30(2), 382–394.

    Article  Google Scholar 

  94. The White House (2010). Guidance on the use of challenges and prizes to promote open government.

  95. Turner, A. M., Kirchhoff, K., & Capurro, D. (2012). Using crowdsourcing technology for testing multilingual public health promotion materials. Journal of Medical Internet Research, 14(3), e79.

    Article  Google Scholar 

  96. Westerbeek, J.B. (2016). Mapping the effects of peer-to-peer sharing economy platforms on society. Ph.D. Thesis, TU Delft.

  97. Zenonos, A., Stein, S. & Jennings, N. (2016). An algorithm to coordinate measurements using stochastic human mobility patterns in large-scale participatory sensing settings. Advances in Artificial Intelligence. In Association for the Advancement of Artificial Intelligence (

  98. Zhang, Y., Gu, Y., Song, L., Pan, M., Dawy, Z. & Han, Z. (2015). Tournament-based incentive mechanism designs for mobile crowdsourcing. In 2015 IEEE Global Communications Conference (GLOBECOM) (pp. 1–6). IEEE.

Download references

Author information



Corresponding author

Correspondence to Araz Taeihagh.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Taeihagh, A. Crowdsourcing: a new tool for policy-making?. Policy Sci 50, 629–647 (2017).

Download citation


  • Crowdsourcing
  • Public policy
  • Policy instrument
  • Policy tool
  • Policy process
  • Policy cycle
  • Open collaboration
  • Virtual labour markets
  • Tournaments
  • Competition