Abstract
How to combine expert judgment and mass intelligence? This study can be characterized as methodological concept development. We combine the relevant theories and methodological approaches from expert-dominated Delphi methods and layman-dominated crowdsourcing approaches to introduce a novel Crowdsourcing Delphi research approach. The key contributions of this study consist: (1) the definition of Crowdsourcing Delphi term, (2) identification of differences between crowdsourcing and Delphi methodologies, (3) developing a framework for selecting and evaluating Crowdsourcing Delphi research strategies and (4) identifying pros and cons for different research strategy combinations. One notable advantage for knowledge management need is that the Crowdsourcing methodology allows a large variety of different Delphi variants and different Crowdsourcing techniques. The Crowdsourcing Delphi allows also combination of Data pools/Big Data flows of crowds to the methodological practices of various Delphi expert-panel methods (Delphi variants). The Crowdsourcing Delphi is a new knowledge management methodology tool package.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Linstone, H.A., Turoff, M.: The Delphi Method: Techniques and Applications. Wesley Publishing Company, Reading (1975)
Expert in Cambridge Dictionary. https://dictionary.cambridge.org/. Accessed 10 Mar 2022
Baker, J., Lovell, K., Harris, N.: How expert are the experts? An exploration of the concept of ‘expert’ within Delphi panel techniques. Nurse Res. 14(1), 59–70 (2006)
Keeney, S., Hasson, F., McKenna, H.P.: A critical review of the Delphi technique as a research methodology for nursing. Int. J. Nurs. Stud. 38(2), 195–200 (2001)
Okoli, C., Pawlowski, S.D.: The Delphi method as a research tool: an example, design considerations and applications. Inf. Manag. 42(1), 15–29 (2004)
Van Zolingen, S.J., Klaassen, C.A.: Selection processes in a Delphi study about key qualifications in senior secondary vocational education. Technol. Forecast. Soc. Chang. 70(4), 317–340 (2003)
Goodman, C.M.: The Delphi technique: a critique. J. Adv. Nurs. 12(6), 729–734 (1987)
Sackman, H.: Delphi Critique, Expert Opinion, Forecasting and the Group Process. Lexington Books (1975)
McMillan, S.S., King, M., Tully, M.P.: How to use the nominal group and Delphi techniques. Int. J. Clin. Pharm. 38(3), 655–662 (2016). https://doi.org/10.1007/s11096-016-0257-x
Hussler, C., Muller, P., Rondé, P.: Is diversity in Delphi panelist groups useful? Evidence from a French forecasting exercise on the future of nuclear energy. Technol. Forecast. Soc. Change 78(9), 1642–1653 (2011)
Estellés-Arolas, E., González-Ladrón-De-Guevara, F.: Towards an integrated crowdsourcing definition. J. Inf. Sci. 38(2), 189–200 (2012)
Kaivo-oja, J., Santonen, T., Myllylä, Y.: The crowdsourcing Delphi: combining the Delphi methodology and crowdsourcing techniques. In: International Society for Professional Innovation Management. ISPIM Conference Proceedings (2013). URN:NBN:fi:amk-2016101015024
Halicka, K.: Innovative classification of methods of the Future-oriented Technology Analysis. Technol. Econ. Dev. Econ. 22(4), 574–597 (2016)
Strasser, A.: Delphi method variants in information systems research: taxonomy development and application. Electron. J. Bus. Res. Methods 15(2), 120–133 (2017)
Howe, J.: Crowdsourcing: How the Power of the Crowd is Driving the Future of Business. Business Books, Great Britain (2008)
Ghezzi, A., Gabelloni, D., Martini, A., Natalicchio, A.: Crowdsourcing: a review and suggestions for future research. Int. J. Manag. Rev. 20(2), 343–363 (2018)
Geiger, D., Rosemann, M., Fielt, E., Schader, M.: Crowdsourcing information systems-definition typology, and design. In: George, J.F. (ed.) Proceedings of the 33rd Annual International Conference on Information Systems, pp. 1–11. Association for Information Systems/AIS Electronic Library (AISeL) (2012). http://aisel.aisnet.org/
Prpić, J., Shukla, P.P., Kietzmann, J.H., McCarthy, I.P.: How to work a crowd: developing crowd capital through crowdsourcing. Bus. Horiz. 58(1), 77–85 (2015)
Dalkey, N., Helmer, O.: An experimental application of the Delphi method to use of experts. Manag. Sci. 9(3), 458–467 (1963)
Turoff, M., Linstone, H.A.: The Delphi method-techniques and applications (2002). https://web.njit.edu/~turoff/pubs/delphibook/delphibook.pdf. Accessed 10 Mar 2022
McKenna, H.P.: The Delphi technique: a worthwhile research approach for nursing? J. Adv. Nurs. 19(6), 1221–1225 (1994)
Mullen, P.M.: Delphi: myths and reality. J. Health Organ. Manag. 17(1), 37–52 (2003)
Keeney, S.: The Delphi technique. In: Gerrish, K., Lacey, A. (eds.) The Research Process in Nursing, 6th edn., pp. 227–236. Wiley-Blackwell, London (2010)
Turoff, M.: The design of a policy Delphi. Technol. Forecast. Soc. Change 2(2), 149–171 (1970)
Rauch, W.: The decision Delphi. Technol. Forecast. Soc. Change 15(3), 159–169 (1979)
Schmidt, R.C.: Managing Delphi surveys using nonparametric statistical techniques. Decis. Sci. 28(3), 763–774 (1997)
Kuusi, O.: Expertise in the future use of generic technologies. VATT research reports 59. VATT, Helsinki (1999)
Nowack, M., Endrikat, J., Guenther, E.: Review of Delphi-based scenario studies: quality and design considerations. Technol. Forecast. Soc. Change 78(9), 1603–1615 (2011)
Tapio, P.: Disaggregative policy Delphi: using cluster analysis as a tool for systematic scenario formation. Technol. Forecast. Soc. Change 70(1), 83–101 (2003)
Flostrand, A.: Finding the future: crowdsourcing versus the Delphi technique. Bus. Horiz. 60(2), 229–236 (2017)
Popper, K.: The Logic of Scientific Discovery. Routledge, London (2005)
Marchau, V., van de Linde, E.: The Delphi method. In: van der Duin, P. (ed.) Foresight in Organizations. Methods and Tools, pp. 59–79. Routledge, New York (2016)
Brabham, D.C.: Crowdsourcing as a model for problem solving: an introduction and cases. Convergence 14(1), 75–90 (2008)
Ranard, B.L., et al.: Crowdsourcing—harnessing the masses to advance health and medicine, a systematic review. J. Gen. Internal Med. 29(1), 187–203 (2014). https://doi.org/10.1007/s11606-013-2536-8
Leech, N.L., Onwuegbuzie, A.J.: A typology of mixed methods research designs. Qual. Quant. 43, 265–275 (2009). https://doi.org/10.1007/s11135-007-9105-3
Kaivo-oja, J.: Towards better participatory processes in technology foresight: how to link participatory foresight research to the methodological machinery of qualitative research and phenomenology? Futures 86, 94–106 (2017)
Jari, K.-O., Theresa, L.: Knowledge management and triangulation logic in the foresight research and analyses in business process management. In: Uden, L., Lu, W., Ting, I.-H. (eds.) KMO 2017. CCIS, vol. 731, pp. 228–238. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-62698-7_20
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Santonen, T., Kaivo-oja, J. (2022). The Crowdsourcing Delphi: A Method for Combing Expert Judgements and Wisdom of Crowds. In: Uden, L., Ting, IH., Feldmann, B. (eds) Knowledge Management in Organisations. KMO 2022. Communications in Computer and Information Science, vol 1593. Springer, Cham. https://doi.org/10.1007/978-3-031-07920-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-07920-7_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-07919-1
Online ISBN: 978-3-031-07920-7
eBook Packages: Computer ScienceComputer Science (R0)