Crowdsourcing is rapidly evolving and applied in situations where ideas, labour, opinion or expertise of large groups of people is used. Crowdsourcing is now used in various policy-making initiatives; however, this use has usually focused on open collaboration platforms and specific stages of the policy process, such as agenda-setting and policy evaluations. Other forms of crowdsourcing have been neglected in policy-making, with a few exceptions. This article examines crowdsourcing as a tool for policy-making and explores the nuances of the technology and its use and implications for different stages of the policy process. The article addresses questions surrounding the role of crowdsourcing and whether it can be considered as a policy tool or as a technological enabler and investigates the current trends and future directions of crowdsourcing.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price includes VAT (USA)
Tax calculation will be finalised during checkout.
These categorisations are not exclusive or exhaustive, but useful for considering the different roles crowdsourcing can take in the policy cycle. For a review of the state of the art in crowdsourcing, see Prpić (2016).
With respect to their offline identities. However, researchers such as Lease et al. (2013) have previously demonstrated that a significant amount of information can be exposed about the workers through the VLM websites.
Various classification attempts and corresponding models of the policy processes exist, of which perhaps the most popular is the use of sequential interrelated stages as a policy cycle. In this article, based on the efforts of Stone (1988) and Howlett et al. (1995), the policy cycle is seen as a sequence of steps in which agenda-setting, problem definition, policy design, policy implementation, policy enforcement and policy evaluations are carried out in an iterative manner (Taeihagh et al. 2009).
Even in the case of online surveys, the speed at which a worker can carry out a microtask is much faster than an online survey (Prpić et al. 2014).
Expert crowdsourcing, mainly through competition-based platforms (and future high-skilled VLMs sites once their use becomes more mainstream) and non-expert crowdsourcing through the use of VLMs. OC platforms provide access to both expert and non-expert crowds, but require a more sustained effort in attracting and maintaining them. It is worth nothing recent research by Bonazzi et al. (2017) demonstrates a successful combined engagement of expert and non-expert crowds in scenario planning.
OC platforms, for instance, have amplified unscientific and unsubstantiated claims regarding MMR vaccination, resulting in a significant increase in outbreaks of preventable diseases such as measles in the UK and the USA (Perry 2013).
A potential worrying development in case of massive adoption of crowdsourcing (such as in the examples italicized in Table 5) is the difficulty in upholding oversight and keeping organisations accountable in future, especially if block-chain technology is used as the level of anonymity can increase. Block-chain technology such as Bitcoin is not anonymous, but in comparison with traditional means of monetary exchange (in the hands of expert individuals) it has a higher level of anonymity as it does not require sending and receiving personally identifiable information: https://bitcoin.org/en/protect-your-privacy.
As a crude measure at the time of finalising this manuscript in November 2017, 469 papers have the term “crowdsourcing” in the title AND mention the term “revolution” in their text. There are also 16,800 academic papers that mention crowdsourcing AND revolution in their text.
Different forms of crowdsourcing and sharing economy share commonalities in terms of the use of reputation systems and IT, the reliance on crowds and the exchange of information and currency (Taeihagh 2017a). The literature in one domain, however, often ignores the other or treats it in a singular form rather than considering the different types that fall under the umbrella term. Sometimes, moreover, a platform is categorised both as a sharing economy and as a crowdsourcing platform by different scholars, particularly when the topic of the study relates to VLMs and OCs (particularly commons such as Wikipedia). Westerbeek (2016) explicitly differentiates between crowdsourcing and sharing economy platforms by stating the one-on-one, peer-to-peer aspect to be the most important part of a sharing economy, and that this is not present in crowdsourcing. Other scholars distinguish between them by pointing out that if a labour market platform for instance provides a virtual service that can be performed online (such as Amazon Mturk), that platform is a crowdsourcing platform; in contrast, if it provides a physical service to be performed locally, it is a sharing economy platform (such as TaskRabbit) (Gansky 2010; De Groen, Maselli and Fabo 2016; Aloisi 2015; Rauch and Schleicher 2015). With these new developments in crowdsourcing, however, the line between crowdsourcing and sharing economy platforms seems to be gradually blurring which provides further evidence that as Prpić and Shukla (2016) point out, there is a potential for unifying these fields with development of generalisable frameworks for studying IT-mediated crowds.
Aitamurto, T. (2012). Crowdsourcing for democracy: New era in policy-making. Publications of the Committee for the Future, Parliament of Finland, 1/2012. Helsinki, Finland.
Aitamurto, T. (2016a). Collective intelligence in law reforms: When the logic of the crowds and the logic of policymaking collide. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 2780–2789). IEEE.
Aitamurto, T., Chen, K., Cherif, A., Galli, J. S., & Santana, L. (2016, October). Civic CrowdAnalytics: Making sense of crowdsourced civic input with big data tools. In Proceedings of the 20th International Academic Mindtrek Conference (pp. 86–94). ACM.
Aitamurto, T., & Landemore, H. (2016). Crowdsourced deliberation: The case of the law on off-road traffic in Finland. Policy and Internet, 8(2), 174–196.
Aloisi, A. (2015). Commoditized workers: Case study research on labour law issues arising from a set of ‘on-demand/gig economy’ platforms. http://ssrn.com/abstract=2637485.
Asmolov, G. (2015). Vertical crowdsourcing in Russia: Balancing governance of crowds and state-citizen partnership in emergency situations. Policy & Internet, 7(3), 292–318.
Bayus, B. L. (2013). Crowdsourcing new product ideas over time: An analysis of the Dell IdeaStorm community. Management Science, 59(1), 226–244.
Bennett, C. J., & Howlett, M. (1992). The lessons of learning: Reconciling theories of policy learning and policy change. Policy Sciences, 25(3), 275–294.
Berners-Lee, T., Fischetti, M. & Foreword By-Dertouzos, M.L. (2000). Weaving the web: The original design and ultimate destiny of the World Wide Web by its inventor. Harper Information.
Boer, D. (2016). Stealthy McStealthface reports for service. The Times. https://www.thetimes.co.uk/article/stealthy-mcstealthface-reports-for-service-tk6zwfz6g. Retrieved May/June 2017.
Bonazzi, R., Viscusi, G., & Barbey, V. (2017). Crowd and experts’ knowledge: Connection and value through the notion of prism. In European, Mediterranean, and Middle Eastern Conference on Information Systems (pp. 646–654). Springer, Cham.
Brabham, D. C. (2008). Crowdsourcing as a model for problem-solving an introduction and cases. Convergence, 14(1), 75–90.
Brabham, D. C. (2013a). Crowdsourcing. Cambridge: MIT Press.
Brabham, D.C. (2013b). Using crowdsourcing in government. IBM Center for The Business of Government. http://www.businessofgovernment.org/sites/default/files/Using%20Crowdsourcing%20In%20Government.pdf.
Budhathoki, N. R., & Haythornthwaite, C. (2013). Motivation for open collaboration crowd and community models and the case of OpenStreetMap. American Behavioral Scientist, 57(5), 548–575.
Certoma, C., Corsini, F., & Rizzi, F. (2015). Crowdsourcing urban sustainability. Data, people and technologies in participatory governance. Futures, 74, 93–106.
Chappell, B. (2017). Footy McFooty face is stomping competition in vote For MLS team name. National Public Radio (NPR), http://www.npr.org/sections/thetwo-way/2017/03/28/521810071/footy-mcfooty-face-is-stomping-competition-in-vote-for-mls-team-name. Retrieved March 2017.
Cha, M., Haddadi, H., Benevenuto, F., & Gummadi, P. K. (2010). Measuring user influence in Twitter: The million follower fallacy. ICWSM, 10(10–17), 30.
Codagnone, C., Abadie, F. & Biagi, F.(2016). The future of work in the ‘Sharing Economy’. Market efficiency and equitable opportunities or unfair precarisation? Institute for Prospective Technological Studies. http://ssrn.com/abstract=2784774.
Crowdflower (2016) Crowdflower Inc. www.clowdflower.com. Retrieved 20 May 2016.
Crowley, C., Daniels, W., Bachiller, R., Joosen, W., & Hughes, D. (2014). Increasing user participation: An exploratory study of querying on the Facebook and Twitter platforms. First Monday, 19(8), 1.
Crump, J. (2011). What are the police doing on Twitter? Social media, the police and the public. Policy and Internet, 3(4), 1–27.
De Groen, W.P., Maselli, I. & Fabo, B. (2016). The digital market for local services: A one-night stand for workers? CEPS Special Report, No. 133.
De Vreede, T., Nguyen, C., De Vreede, G.J., Boughzala, I., Oh, O. & Reiter-Palmon, R. (2013). A theoretical model of user engagement in crowdsourcing. In Collaboration and Technology Lecture Notes in Computer Science, 8224, pp. 94–109.
De Winter, J.C.F., Kyriakidis, M., Dodou, D. & Happee, R. (2015). Using CrowdFlower to study the relationship between self-reported violations and traffic accidents. In Proceedings of the 6th International Conference on Applied Human Factors and Ergonomics (AHFE), Las Vegas, NV. http://ssrn.com/abstract=2592903.
Dutil, P. (2015). Crowdsourcing as a new instrument in the government’s arsenal: Explorations and considerations. Canadian Public Administration, 58(3), 363–383.
Eichenwald, K. (2012). Microsoft’s lost Decade. Vanity Fair, https://www.vanityfair.com/news/business/2012/08/microsoft-lost-mojo-steve-ballmer. Retrieved July 2017.
Ellis-Petersen, H. (2016). Boaty McBoatface wins poll to name polar research vessel. The Guardian, https://www.theguardian.com/environment/2016/apr/17/boaty-mcboatface-wins-poll-to-name-polar-research-vessel. Retrieved June 2017.
Estrelles-Arolas, E., & Gonzalez-Ladron-De-Guevara, F. (2012). Towards an integrated crowdsourcing definition. Journal of Information Science, 38(2), 189–200.
Fischer, F. (1993). Citizen participation and the democratization of policy expertise: From theoretical inquiry to practical cases. Policy Sciences, 26(3), 165–187.
Gansky, L. (2010). The Mesh: Why the future of business is sharing. New York, NY: Penguin.
Gellers, J. C. (2016). Crowdsourcing global governance: Sustainable development goals, civil society, and the pursuit of democratic legitimacy. International Environmental Agreements. Politics, Law and Economics, 16(3), 415–432.
Glaeser, E. L., Hillis, A., Kominers, S. D., & Luca, M. (2016). Crowdsourcing city government: Using tournaments to improve inspection accuracy. The American Economic Review, 106(5), 114–118.
Goncalves, J., Hosio, S., Kostakos, V., Vukovic, M., & Konomi, S.I. (2015) Workshop on mobile and situated crowdsourcing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and International Symposium on Wearable Computers (pp. 1339–1342). ACM.
Goodchild, M. F., & Glennon, J. A. (2010). Crowdsourcing geographic information for disaster response: A research frontier. International Journal of Digital Earth, 3(3), 231–241.
Greengard, S. (2011). Following the crowd. Communications of the ACM, 54(2), 20–22.
Gruzd, A., & Tsyganova, K. (2015). Information wars and online activism during the 2013/2014 Crisis in Ukraine: Examining the social structures of Pro-and Anti-Maidan Groups. Policy & Internet, 7(2), 121–158.
Hendry, D. F., & Ericsson, N. R. (2003). Understanding economic forecasts. Cambridge, MA: MIT Press.
Hern, A. (2017). Trainy McTrainface: Swedish railway keeps Boaty’s legacy alive. The Guardian, https://www.theguardian.com/world/2017/jul/19/trainy-mctrainface-boaty-mcboatface-legacy-lives-on-in-swedish-train. Retrieved July 2017.
Hira, A. (2017). Profile of the sharing economy in the developing world: Examples of companies trying to change the world. Journal of Developing Societies., 33(2), 244–271.
Hood, C. (1986). The tools of government. Chatham: Chatham House Publishers.
Hood, C. (2007). Intellectual obsolescence and intellectual makeovers: Reflections on the tools of government after two decades. Governance, 20(1), 127–144.
Hood, C., & Margetts, H. (2007). The tools of government in the digital age. London: Palgrave Macmillan.
Horton, J.J. & Chilton, L.B. (2010). The labour economics of paid crowdsourcing. In Proceedings of the 11th ACM Conference on Electronic Commerce (pp. 209–218). ACM.
Hosio, S. (2016). Situated crowdsourcing. http://simohosio.com/situated-crowdsourcing/. Retrieved 20 June 2017.
Howe, J. (2006) Crowdsourcing: A definition. http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html.
Howe, J. (2008) Crowdsourcing: Why the power of the crowd is driving the future of business. Crown Business. http://www.bizbriefings.com/Samples/IntInst%20—%20Crowdsourcing.PDF.
Howlett, M. (2000). Managing the hollow state: Procedural policy instruments and modern governance. Canadian Public Administration, 43(4), 412–431.
Howlett, M. (2009). Government communication as a policy tool: A framework for analysis. Canadian Political Science Review, 3(2), 23–37.
Howlett, M. (2010). Designing public policies: Principles and instruments. London: Taylor and Francis.
Howlett, M., Ramesh, M., & Perl, A. (1995). Studying public policy: Policy cycles and policy subsystems. Toronto: Oxford University Press.
Jackson, P., & Klobas, J. (2013). Deciding to use an enterprise wiki: The role of social institutions and scripts. Knowledge Management Research and Practice, 11(4), 323–333.
Jeppesen, L. B., & Lakhani, K. R. (2010). Marginality and problem-solving effectiveness in broadcast search. Organisation Science, 21(5), 1016–1033.
Krumm, J. & Horvitz, E. (2014). Situated Crowdsourcing. https://www.microsoft.com/en-us/research/project/spatial-crowdsourcing/. Retrieved June 2017.
Lakhani, K., Garvin, D. A., & Lonstein, E. (2010). Topcoder (a): Developing software through crowdsourcing (pp. 610–632). Cambridge: Harvard Business School General Management Unit Case.
Landemore, H. (2015). Inclusive constitution-making: The Icelandic experiment. Journal of Political Philosophy, 23(2), 166–191.
Lease, M., Hullman, J., Bigham, J. P., Bernstein, M. S., Kim, J., Lasecki, W., Bakhshi, S., Mitra, T., & Miller, R. C. (2013). Mechanical turk is not anonymous. Rocheser, NY: Social Science Research Network.
Lehdonvirta, V., & Bright, J. (2015). Crowdsourcing for public policy and government. Policy and Internet, 7(3), 263–267.
Liu, H. K. (2017a). Crowdsourcing design: A synthesis of literatures. In Proceedings of the 50th Hawaii International Conference on System Sciences.
Liu, H. K. (2017b). Crowdsourcing government: Lessons from multiple disciplines. Public Administration Review, 77(5), 656–667.
Longo, J., & Kelley, T. M. (2016). GitHub use in public administration in Canada: Early experience with a new collaboration tool. Canadian Public Administration, 59(4), 598–623.
Luz, N., Silva, N., & Novais, P. (2015). A survey of task-oriented crowdsourcing. Artificial Intelligence Review, 44(2), 187–213.
Marcus, A., & Parameswaran, A. (2015). Crowdsourced data management: Industry and academic perspectives. Foundations and Trends® in Databases, 6(1–2), 1–161.
Mazumdar, S., Wrigley, S., & Ciravegna, F. (2017). Citizen science and crowdsourcing for earth observations: An analysis of stakeholder opinions on the present and future. Remote Sensing, 9(1), 87.
Mergel, I. (2015). Open collaboration in the public sector: The case of social coding on GitHub. Government Information Quarterly, 32(4), 464–472.
Michel, F., Gil, Y. & Hauder, M. (2015). A virtual crowdsourcing community for open collaboration in science processes. In Americas Conference on Information Systems (AMCIS).
Narula, P., Gutheim P., Rolnitzky, D., Kulkarni, A. & Hartmann, B. (2011). MobileWorks: A mobile crowdsourcing platform for workers at the bottom of the pyramid. In Proceedings of HCOMP.
Nash, A. (2009). Web 2.0 applications for improving public participation in transport planning. In Transportation Research Board 89th Annual Meeting.
Nordrum, A. (2016). Popular internet of things forecast of 50 billion devices by 2020 is outdated. IEEE Spectrum, 18.
Okolloh, O. (2009). Ushahidi, or ‘testimony’: Web 2.0 tools for crowdsourcing crisis information. Participatory Learning and Action, 59(1), 65–70.
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5), 411–419.
Park, A. J., Ko, J. M., & Swerlick, R. A. (2017). Crowdsourcing dermatology: DataDerm, big data analytics, and machine learning technology. Journal of the American Academy of Dermatology. https:\\doi.org\10.1016/j.jaad.2017.08.053.
Perry, D. M. (2013). Destabilizing the Jenny McCarthy public health industrial complex. Atlantic Monthly July 11, https://www.theatlantic.com/health/archive/2013/07/destabilizing-the-jenny-mccarthy-public-health-industrial-complex/277695/. Retrieved June 2017.
Piller, F. T., & Walcher, D. (2006). Toolkits for idea competitions: a novel method to integrate users in new product development. R&D Management, 36(3), 307–318.
Prpić, J. (2016). Next generation crowdsourcing for collective intelligence. In Collective intelligence conference: NYU Stern School of Business from June 1–3, 2016.
Prpić, J., & Shukla, P. (2013). The theory of crowd capital. In Proceedings of the 46th annual Hawaii international conference on system sciences. Hawaii: Computer Society Press.
Prpić, J. & Shukla, P. (2016). Crowd Science: Measurements, models, and methods. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 4365–4374). IEEE.
Prpić, J., Taeihagh, A. & Melton, J. (2014) Experiments on crowdsourcing policy assessment. In Oxford Internet Policy and Politics Conference (IPP 2014), University of Oxford, 26–28 September 2014. http://ipp.oii.ox.ac.uk/sites/ipp/files/documents/IPP2014_Taeihagh.pdf.
Prpić, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy and Internet, 7(3), 340–361. https://doi.org/10.1002/poi3.102.
Rauch, D.E. & Schleicher, D. (2015). Like Uber, but for local governmental policy: The future of local regulation of the “sharing economy”. George Mason Law & Economics Research Paper, 15–01.
Rittel, H. W., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169.
Rogstadius, J., Vukovic, M., Teixeira, C. A., Kostakos, V., Karapanos, E., & Laredo, J. A. (2013). CrisisTracker: Crowdsourced social media curation for disaster awareness. IBM Journal of Research and Development, 57(5), 1–4.
Sabatier, P. A. (1988). An advocacy coalition framework of policy change and the role of policy-oriented learning therein. Policy Sciences, 21(2), 129–168.
Sadat, D. R. (2014). M-government implementation evaluation in encouraging citizen participation in Indonesia: A case study of LAPOR!, Ph.D. Thesis, University of Manchester.
Salus, P. H., & Vinton, G. (1995). Casting the net: From ARPANET to internet and beyond. Boston, MA: Addison-Wesley Longman Publishing.
Schweitzer, F. M., Buchinger, W., Gassmann, O., & Obrist, M. (2012). Crowdsourcing: Leveraging innovation through online idea competitions. Research-Technology Management, 55(3), 32–38.
Seltzer, E., & Mahmoudi, D. (2013). Citizen participation, open innovation, and crowdsourcing challenges and opportunities for planning. Journal of Planning Literature, 28(1), 3–18.
Stone, D. A. (1988). Policy Paradox and Political Reason. New York: Harper Collins.
Sun, X., Hu, S., Su, L., Abdelzaher, T., Hui, P., Zheng, W., et al. (2015). Participatory sensing meets opportunistic sharing: Automatic phone-to-phone communication in vehicles. IEEE Transactions on Mobile Computing, 15, 2550–2563.
Taeihagh, A. (2017a). Crowdsourcing, sharing economy and development. Journal of Developing Societies, 33(2), 1–32. https://doi.org/10.1177/0169796X17710072.
Taeihagh, A. (2017b). Network centric policy design. Policy Sciences Journal, 50(2), 317–338. https://doi.org/10.1007/s11077-016-9270-0.
Taeihagh, A., Bañares-Alcántara, R., & Millican, C. (2009). Development of a novel framework for the design of transport policies to achieve environmental targets. Computers & Chemical Engineering, 33(10), 1531–1545.
Taieb, S. B., & Hyndman, R. J. (2014). A gradient boosting approach to the Kaggle load forecasting competition. International Journal of Forecasting, 30(2), 382–394.
The White House (2010). Guidance on the use of challenges and prizes to promote open government. https://www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/m10-11.pdf.
Turner, A. M., Kirchhoff, K., & Capurro, D. (2012). Using crowdsourcing technology for testing multilingual public health promotion materials. Journal of Medical Internet Research, 14(3), e79.
Westerbeek, J.B. (2016). Mapping the effects of peer-to-peer sharing economy platforms on society. Ph.D. Thesis, TU Delft.
Zenonos, A., Stein, S. & Jennings, N. (2016). An algorithm to coordinate measurements using stochastic human mobility patterns in large-scale participatory sensing settings. Advances in Artificial Intelligence. In Association for the Advancement of Artificial Intelligence (www.aaai.org).
Zhang, Y., Gu, Y., Song, L., Pan, M., Dawy, Z. & Han, Z. (2015). Tournament-based incentive mechanism designs for mobile crowdsourcing. In 2015 IEEE Global Communications Conference (GLOBECOM) (pp. 1–6). IEEE.
About this article
Cite this article
Taeihagh, A. Crowdsourcing: a new tool for policy-making?. Policy Sci 50, 629–647 (2017). https://doi.org/10.1007/s11077-017-9303-3
- Public policy
- Policy instrument
- Policy tool
- Policy process
- Policy cycle
- Open collaboration
- Virtual labour markets