Consensus Methodologies and Producing the Evidence

King, Michelle A.; Kelly, Fiona S.; McMillan, Sara S.

doi:10.1007/978-3-030-50247-8_75-1

Michelle A. King²,
Fiona S. Kelly² &
Sara S. McMillan²

115 Accesses

Abstract

Consensus methods help bridge the gap between what is known and what information needs to be known by combining the opinions of key stakeholders to create knowledge. The three consensus methods that are used most often in pharmacy research are the Delphi, Nominal Group Technique (NGT), and RAND/UCLA Appropriateness Method (RAM). All three draw together experts’ opinions through a voting or rating step, and exchange of ideas through a moderated meeting or feedback summarizing the comments of others, or both. The Delphi, NGT, and RAM were introduced to address limitations of existing exploratory research methods seeking to generate a wider range of ideas, engage more diverse stakeholders, and obtain a sense of priorities to inform resource allocation. While the opinions of a group may be more valid than the opinions of a single person, they are still opinions and not a gold standard. How the opinions are drawn together has a significant impact on the findings, and as such, minimizing bias and maximizing quality are key concerns. Each of these methods has been modified in a range of ways, and this entry provides an overview of the original method, selected modifications, and factors for researchers to consider in both application and reporting of these methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

Almanasreh E, Moles R, Chen TF. The medication discrepancy taxonomy (MedTax): the development and validation of a classification system for medication discrepancies identified through medication reconciliation. Res Soc Adm Pharm. 2020;16(2):142–8. https://doi.org/10.1016/j.sapharm.2019.04.005.
Article Google Scholar
Alshakrah MA, Steinke DT, Tully MP, Abuzour AS, Williams SD, Lewis PJ. Development of the adult complexity tool for pharmaceutical care (ACTPC) in hospital: a modified Delphi study. Res Soc Adm Pharm. 2021;17 https://doi.org/10.1016/j.sapharm.2021.02.009.
Andrew Van De, Ven and Andre L. Delbecq. Nominal Versus Interacting Group Processes for Committee Decision-Making Effectiveness. AMJ. 1971;14:203–212. https://doi.org/10.5465/255307
Barrios M, Guilera G, Nuño L, Gómez-Benito J. Consensus in the delphi method: what makes a decision change? Technol Forecast Soc Chang. 2021;163:120484. https://doi.org/10.1016/j.techfore.2020.120484.
Article Google Scholar
Basger BJ, Chen TF, Moles RJ. Validation of prescribing appropriateness criteria for older Australians using the RAND/UCLA appropriateness method. BMJ Open. 2012;2(5) https://doi.org/10.1136/bmjopen-2012-001431.
Bell BG, Spencer R, Avery AJ, Campbell SM. Tools for measuring patient safety in primary care settings using the RAND/UCLA appropriateness method. BMC Fam Pract. 2014;15(1):110. https://doi.org/10.1186/1471-2296-15-110.
Article Google Scholar
Belton I, MacDonald A, Wright G, Hamlin I. Improving the practical application of the Delphi method in group-based judgment: a six-step prescription for a well-founded and defensible process. Technol Forecast Soc Chang. 2019;147:72–82. https://doi.org/10.1016/j.techfore.2019.07.002.
Article Google Scholar
Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476. https://doi.org/10.1371/journal.pone.0020476.
Article CAS Google Scholar
Bourrée F, Michel P, Salmi LR. Consensus methods: review of original methods and their main alternatives used in public health. Revue d'epidemiologie et de sante publique. 2008;56(6):e13–21. https://doi.org/10.1016/j.respe.2008.10.005.
Article Google Scholar
Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J, Park RE. A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care. 1986;2(1):53–63. https://doi.org/10.1017/S0266462300002774.
Article CAS Google Scholar
Campbell S, Cantrill J. Consensus methods in prescribing research. J Clin Pharm Ther. 2001;26(1):5–14. https://doi.org/10.1111/j.1365-2710.2001.00331.x.
Article CAS Google Scholar
Chalmers J, Armour M. The Delphi technique. In: Liamputtong P, editor. Handbook of research methods in health social sciences. Singapore: Springer; 2019. p. 715–35. https://doi.org/10.1007/978-981-10-5251-4_99.
Chapter Google Scholar
Chartrand M, Guénette L, Brouillette D, Côté S, Huot R, Landry J, Martineau J, Perreault S, White-Guay B, Williamson D, Martin É, Gagnon MM, Lalonde L. Development of quality indicators to assess oral anticoagulant management in community pharmacies for patients with atrial fibrillation. J Managed Care Spec Pharm. 2018;24(4):357–65. https://doi.org/10.18553/jmcp.2018.24.4.357.
Chen EYH, Sluggett JK, Ilomaki J, Hilmer SN, Corlis M, Picton LJ, Dean L, Alderman CP, Farinola N, Gailer J, Grigson J, Kellie AR, Putsey PJC, Yu S, Bell JS. Development and validation of the Medication Regimen Simplification Guide for Residential Aged CarE (MRS GRACE). Clin Interv Aging. 2018;13:975–86. https://doi.org/10.2147/CIA.S158417.
Article CAS Google Scholar
Dalkey N, Helmer O. An experimental application of the Delphi method to the use of experts. Manag Sci. 1963;9(3):458–67. https://doi.org/10.1287/mnsc.9.3.458.
Article Google Scholar
De Loë RC, Melnychuk N, Murray D, Plummer R. Advancing the state of policy Delphi practice: a systematic review evaluating methodological evolution, innovation, and opportunities. Technol Forecast Soc Chang. 2016;104:78–88. https://doi.org/10.1016/j.techfore.2015.12.009.
Article Google Scholar
de Meyrick J. The Delphi method and health research. Health Educ. 2003;103(1):7–16. https://doi.org/10.1108/09654280310459112.
Article Google Scholar
De Villiers MR, De Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639–43. https://doi.org/10.1080/13611260500069947.
Article Google Scholar
Delbecq AL, Van de Ven A. Nominal group techniques for involving clients and resource experts in program planning. Proc Acad Manage. 1970;8(1):208–27. https://doi.org/10.5465/ambpp.1970.4981077.
Article Google Scholar
Delbecq AL, Van de Ven AH. A group process model for problem identification and program planning. J Appl Behav Sci. 1971;7(4):466–92. https://doi.org/10.1177/002188637100700404.
Article Google Scholar
Delbecq A, Van de Ven A, Gustafson D. Group techniques for program planning: a guide to nominal group and Delphi processes. Scott, Foresman and Company; 1975.
Google Scholar
Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, Wales PW. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–9. https://doi.org/10.1016/j.jclinepi.2013.12.002.
Article Google Scholar
Dijkstra NE, De Baetselier E, Dilles T, van Rompaey B, da Cunha Batalha LM, Filov I, Grøndahl VA, Heczkova J, Helgesen AK, Jordan S. Developing a competence framework for nurses in pharmaceutical care: a Delphi study. Nurse Educ Today. 2021;104926 https://doi.org/10.1016/j.nedt.2021.104926.
Dreischulte T, Grant AM, McCowan C, McAnaw JJ, Guthrie B. Quality and safety of medication use in primary care: consensus validation of a new set of explicit medication assessment criteria and prioritisation of topics for improvement. BMC Clin Pharmacol. 2012;12:5. https://doi.org/10.1186/1472-6904-12-5.
Article Google Scholar
Drumm S, Bradley C, Moriarty F. ‘More of an art than a science’? The development, design and mechanics of the Delphi technique. Res Soc Adm Pharm. 2021;18 https://doi.org/10.1016/j.sapharm.2021.06.027.
Duggan S. Evaluating one-to-one teaching in general practice. Med Teach. 1999;21(1):61–6. https://doi.org/10.1080/01421599980057.
Article Google Scholar
Edwards L, Monro M, Butterfield Y, Johl R, Loftsgard KC, Pelletier H, McGavin C, Lavergne MR. What matters most to patients about primary healthcare: mixed-methods patient priority setting exercises within the PREFeR (PRioritiEs For Research) project. BMJ Open. 2019;9(7):e025954. https://doi.org/10.1136/bmjopen-2018-025954.
Article Google Scholar
Fakih S, Marriott JL, Hussainy SY. Employing the nominal group technique to explore the views of pharmacists, pharmacy assistants and women on community pharmacy weight management services and educational resources. Int J Pharm Pract. 2016;24(2):86–96. https://doi.org/10.1111/ijpp.12218.
Article Google Scholar
Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74(9):979–83. https://doi.org/10.2105/AJPH.74.9.979.
Article CAS Google Scholar
Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR. The RAND/UCLA appropriateness method user's manual. Santa Monica: Rand Corp; 2001.
Google Scholar
Fletcher AJ. Applying critical realism in qualitative research: methodology meets method. Int J Soc Res Methodol. 2017;20(2):181–94. https://doi.org/10.1080/13645579.2016.1144401.
Article Google Scholar
Foth T, Efstathiou N, Vanderspank-Wright B, Ufholz L-A, Dütthorn N, Zimansky M, Humphrey-Murto S. The use of Delphi and nominal group technique in nursing education: a review. Int J Nurs Stud. 2016;60:112–20. https://doi.org/10.1016/j.ijnurstu.2016.04.015.
Article Google Scholar
Foubert K, Capiau A, Mehuys E, De Bolle L, Somers A, Petrovic M, Boussery K. Ghent older People’s prescriptions community pharmacy screening (GheOP 3 S)-tool version 2: update of a tool to detect drug-related problems in older people in primary care. Drugs & Aging. 2021:1–11. https://doi.org/10.1007/s40266-021-00862-6.
Gallagher M, Hares T, Spencer J, Bradshaw C, Webb I. The nominal group technique: a research tool for general practice? Fam Pract. 1993;10(1):76–81. https://doi.org/10.1093/fampra/10.1.76.
Article CAS Google Scholar
Gibbins AK, Wood PJ, Spark MJ. Managing inappropriate use of non-prescription combination analgesics containing codeine: a modified Delphi study. Res Soc Adm Pharm. 2017;13(2):369–77. https://doi.org/10.1016/j.sapharm.2016.02.015.
Article Google Scholar
Glaser EM. Using behavioral science strategies for defining the state-of-the-art. J Appl Behav Sci. 1980;16(1):79–92. https://doi.org/10.1177/002188638001600107.
Article Google Scholar
Greatorex J, Dexter T. An accessible analytical approach for investigating what happens between the rounds of a Delphi study. J Adv Nurs. 2000;32(4):1016–24. https://doi.org/10.1046/j.1365-2648.2000.t01-1-01569.x.
Article CAS Google Scholar
Green B, Jones M, Hughes D, Williams A. Applying the Delphi technique in a study of GPs’ information requirements. Health Soc Care Community. 1999;7(3):198–205. https://doi.org/10.1046/j.1365-2524.1999.00176.x.
Article Google Scholar
Guzys D, Dickson-Swift V, Kenny A, Threlkeld G. Gadamerian philosophical hermeneutics as a useful methodological framework for the Delphi technique. Int J Qual Stud Health Well Being. 2015;10(1):26291. https://doi.org/10.3402/qhw.v10.26291.
Article Google Scholar
Harb SI, Tao L, Peláez S, Boruff J, Rice DB, Shrier I. Methodological options of the nominal group technique for survey item elicitation in health research: a scoping review. J Clin Epidemiol. 2021;139 https://doi.org/10.1016/j.jclinepi.2021.08.008.
Hasson F, Keeney S. Enhancing rigour in the Delphi technique research. Technol Forecast Soc Chang. 2011;78(9):1695–704. https://doi.org/10.1016/j.techfore.2011.04.005.
Article Google Scholar
Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008–15. https://doi.org/10.1046/j.1365-2648.2000.t01-1-01567.x.
Article CAS Google Scholar
Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2016;27(4):591–608. https://doi.org/10.1177/1049732316665344.
Article Google Scholar
Hohmann E, Cote MP, Brand JC. Research pearls: expert consensus based evidence using the Delphi method. Arthroscopy J Arthroscopic Related Surgery. 2018;34(12):3278–82. https://doi.org/10.1016/j.arthro.2018.10.004.
Article Google Scholar
Hsu C-C, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12(1):10. https://doi.org/10.7275/pdz9-th90.
Article Google Scholar
Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and nominal group in medical education research. Med Teach. 2017;39(1):14–9. https://doi.org/10.1080/0142159X.2017.1245856.
Article Google Scholar
Hussainy SY, Crum MF, White PJ, Larson I, Malone DT, Manallack DT, Nicolazzo JA, McDowell J, Lim AS, Kirkpatrick CM. Developing a framework for objective structured clinical examinations using the nominal group technique. Am J Pharm Educ. 2016;80(9):158. https://doi.org/10.5688/ajpe809158.
Article Google Scholar
Jaam M, Awaisu A, El-Awaisi A, Stewart D, El Hajj MS. Use of Delphi technique in pharmacy practice research. Res Soc Adm Pharm. 2021;18 https://doi.org/10.1016/j.sapharm.2021.06.028.
James D, Warren-Forward H. Research methods for formal consensus development. Nurse Res. 2015;22(3):35. https://doi.org/10.7748/nr.22.3.35.e1297.
Article Google Scholar
Jenghua K, Chinwong S, Chinwong D, Kanjanarat P. Development of a list of potentially inappropriate medications for patients with heart failure (PIMHF). Res Soc Adm Pharm. 2021;17(5):894–903. https://doi.org/10.1016/j.sapharm.2020.07.021.
Article Google Scholar
Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706. https://doi.org/10.1177/0269216317690685.
Kahan JP, Morton SC, Farris HH, Kominski GF, Donovan AJ. Panel processes for revising relative values of physician work. A pilot study. Medical Care. 1994;32(11):1069–85. https://doi.org/10.1097/00005650-199411000-00001.
Article CAS Google Scholar
Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53(2):205–12. https://doi.org/10.1111/j.1365-2648.2006.03716.x.
Article Google Scholar
Keeney S, Hasson F, McKenna HP, Wiley B. The Delphi technique in nursing and health research. Chichester: Wiley-Blackwell; 2011. https://doi.org/10.1002/9781444392029.
Book Google Scholar
Knapp KK, Olson AW, Schommer JC, Gaither CA, Mott DA, Doucette WR. Retail clinics colocated with pharmacies: a Delphi study of pharmacist impacts and recommendations for optimization. J Am Pharm Assoc. 2020;60(2):311–8. https://doi.org/10.1016/j.japh.2019.04.014.
Article Google Scholar
Koehler TC, Bok H, Westerman M, Jaarsma D. Developing a competency framework for pharmacy technicians: perspectives from the field. Res Soc Adm Pharm. 2019;15(5):514–20. https://doi.org/10.1016/j.sapharm.2018.06.017.
Article Google Scholar
Kranjc Horvat A, Wiener J, Schmeling S, Borowski A. Learning goals of professional development programs at science research institutions: a Delphi study with different stakeholder groups. J Sci Teacher Educ. 2021:1–23. https://doi.org/10.1080/1046560X.2021.1905330.
Kremer IEH, Evers SMAA, Jongen PJ, van der Weijden T, van de Kolk I, Hiligsmann M. Identification and prioritization of important attributes of disease-modifying drugs in decision making among patients with multiple sclerosis: a nominal group technique and best-worst scaling. PLoS One. 2016;11(11):e0164862. https://doi.org/10.1371/journal.pone.0164862.
Article CAS Google Scholar
Kuhn H, Park A, Kim B, Lukesh W, Rose A. Proportion of work appropriate for pharmacy technicians in anticoagulation clinics. Am J Health Syst Pharm. 2016;73(5):322–7. https://doi.org/10.2146/ajhp150272.
Article Google Scholar
Kulczycki A, Shewchuk RM. Using internet-based nominal group technique meetings to identify provider strategies for increasing diaphragm use. BMJ Sexual & Reproductive Health. 2008;34:227. https://doi.org/10.1783/147118908786000550.
Article Google Scholar
Lange T, Kopkow C, Lützner J, Günther K-P, Gravius S, Scharf H-P, Stöve J, Wagner R, Schmitt J. Comparison of different rating scales for the use in Delphi studies: different scales lead to different consensus and show different test-retest reliability. BMC Med Res Methodol. 2020;20(1):28. https://doi.org/10.1186/s12874-020-0912-8.
Article Google Scholar
Liamputtong P. The science of words and the science of numbers. In: Liamputtong P, editor. Research methods in health: foundations for evidence-based practice. 3rd ed. South Melbourne: Oxford University Press; 2012.
Google Scholar
Lincoln Y, Guba E. Naturalistic inquiry. Sage; 1985. http://www.qualres.org/HomeLinc-3684.html
Book Google Scholar
Linstone HA, Turoff M. The Delphi method: techniques and applications. Reading: Addison-Wesley; 2002.
Google Scholar
Lomas J, Pickard L, Mohide A. Patient versus clinician item generation for quality-of-life measures: the case of language-disabled adults. Med Care. 1987;25(8):764–9. http://www.jstor.org/stable/3765712
Article CAS Google Scholar
Manera K, Hanson C, Gutman T, Tong A. Consensus methods: nominal group technique. In: Liamputtong P, editor. Handbook of research methods in health social sciences. Singapore: Springer; 2018. p. 1–14. https://doi.org/10.1007/978-981-10-2779-6_100-1.
Chapter Google Scholar
McGlynn EA, Kosecoff J, Brook RH. Format and conduct of consensus development conferences: a multination comparison. In: Goodman C, Baratz SR, editors. Improving consensus development for health technology assessment: an international perspective. Washington, DC: National Academies Press (US); 1990.
Google Scholar
McMillan SS, Kelly F, Sav A, Kendall E, King MA, Whitty JA, Wheeler AJ. Using the nominal group technique: how to analyse across multiple groups. Health Serv Outcomes Res Methodol. 2014;14(3):92–108. https://doi.org/10.1007/s10742-014-0121-1.
Article Google Scholar
McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016;38(3):655–62. https://doi.org/10.1007/s11096-016-0257-x.
Article Google Scholar
Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of Health Research reporting guidelines. PLoS Med. 2010;7(2):e1000217. https://doi.org/10.1371/journal.pmed.1000217.
Article Google Scholar
Mubarak N, Hatah E, Aris MAM, Shafie AA, Zin CS. Consensus among healthcare stakeholders on a collaborative medication therapy management model for chronic diseases in Malaysia; A Delphi study. PLOS ONE. 2019;14(5):e0216563. https://doi.org/10.1371/journal.pone.0216563.
Article Google Scholar
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, Marteau T. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i–iv, 1–88.
Google Scholar
Newlands RS, Power A, Young L, Watson M. Quality improvement of community pharmacy services: a prioritisation exercise. Int J Pharm Pract. 2018;26(1):39–48. https://doi.org/10.1111/ijpp.12354.
Article Google Scholar
Ourth HL, Folstad J, Mambourg SE, Hopwood G, Marchiando C, Combs C, Gonzalez C. Evaluation of the potential impact of pharmacy technician performance of anticoagulation clinic tasks on operational efficiency. Am J Health Syst Pharm. 2019;76(16):1248–53. https://doi.org/10.1093/ajhp/zxz126.
Article Google Scholar
Pickering C, Byrne J. The benefits of publishing systematic quantitative literature reviews for PhD candidates and other early-career researchers. Higher Educ Res Dev. 2014;33(3):534–48. https://doi.org/10.1080/07294360.2013.841651.
Article Google Scholar
Pouliot A, Vaillancourt R, Stacey D, Suter P. Defining and identifying concepts of medication literacy: an international perspective. Res Soc Adm Pharm. 2018;14(9):797–804. https://doi.org/10.1016/j.sapharm.2017.11.005.
Article Google Scholar
Preferred Reporting Items for Systematic Reviews and Meta-Analyses. (2021) PRISMA. http://prisma-statement.org/. Accessed 17 Aug 2021.
Reimer D et al. Pre-clerkship medical students’ perceptions of medical professionalism. BMC Medical Education 2019;19(1):239.
Google Scholar
Schulick RD. Sometimes consensus and expert opinion is the best we can do. JAMA Surg. 2014;149(1):25. https://doi.org/10.1001/jamasurg.2013.3987.
Article Google Scholar
Sharma A, Minh Duc NT, Luu Lam Thang T, Nam NH, Ng SJ, Abbas KS, Huy NT, Marušić A, Paul CL, Kwok J, Karbwang J, de Waure C, Drummond FJ, Kizawa Y, Taal E, Vermeulen J, Lee GHM, Gyedu A, To KG, Verra ML, Jacqz-Aigrain ÉM, Leclercq WKG, Salminen ST, Sherbourne CD, Mintzes B, Lozano S, Tran US, Matsui M, Karamouzian M. A consensus-based checklist for reporting of survey studies (CROSS). J Gen Intern Med. 2021;36 https://doi.org/10.1007/s11606-021-06737-1.
Shawahna R. Which information on women's issues in epilepsy does a community pharmacist need to know? A Delphi consensus study. Epilepsy Behav. 2017;77:79–89. https://doi.org/10.1016/j.yebeh.2017.09.026.
Article Google Scholar
Shekelle P. The appropriateness method. Med Decis Mak. 2004;24(2):228–31. https://doi.org/10.1177/0272989X04264212.
Article Google Scholar
Shekelle PG, Chassin MR, Park RE. Assessing the predictive validity of the RAND/UCLA appropriateness method criteria for performing carotid endarterectomy. Int J Technol Assess Health Care. 1998a;14(4):707–27. https://doi.org/10.1017/S0266462300012022.
Article CAS Google Scholar
Shekelle PG, Kahan JP, Bernstein SJ, Leape LL, Kamberg CJ, Park RE. The reproducibility of a method to identify the overuse and underuse of medical procedures. N Engl J Med. 1998b;338(26):1888–95. https://doi.org/10.1056/nejm199806253382607.
Article CAS Google Scholar
Spassiani NA, Sawyer AR, Abou Chacra MS, Koch K, Muñoz YA, Lunsky Y. “Teaches people that I’m more than a disability”: using nominal group technique in patient-oriented research for people with intellectual and developmental disabilities. Intellect Dev Disabil. 2016;54(2):112–22. https://doi.org/10.1352/1934-9556-54.2.112.
Spinks JM, Kalisch Ellett LM, Spurling G, Theodoros T, Williamson D, Wheeler AJ. Adaptation of potentially preventable medication-related hospitalisation indicators for indigenous populations in Australia using a modified Delphi technique. BMJ Open. 2019;9(11):e031369. https://doi.org/10.1136/bmjopen-2019-031369.
Article Google Scholar
Stewart J. Is the Delphi technique a qualitative method? Med Educ. 2001;35(10):922–3. https://doi.org/10.1111/j.1365-2923.2001.01045.x.
Article CAS Google Scholar
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042.
Article Google Scholar
Tseng K-H, Lou S-J, Diez CR, Yang H-J. Using online nominal group technique to implement knowledge transfer. J Eng Educ. 2006;95(4):335–45. https://doi.org/10.1002/j.2168-9830.2006.tb00908.x.
Article Google Scholar
Tuffrey-Wijne I, Bernal J, Butler G, Hollins S, Curfs L. Using nominal group technique to investigate the views of people with intellectual disabilities on end-of-life care provision. J Adv Nurs. 2007;58(1):80–9. https://doi.org/10.1111/j.1365-2648.2007.04227.x.
Article Google Scholar
Turoff M. The policy Delphi. In: Turoff M, Linstone HA, editors. The Delphi method: techniques and applications. Reading: New Jersey Institute of Technology; 1975.
Google Scholar
Van De Ven A, Delbecq AL. Nominal versus interacting group processes for committee decision-making effectiveness. Acad Manag J. 1971;14(2):203–12. https://doi.org/10.2307/255307. https://journals.aom.org/action/showCitFormats?doi=10.5465%2F255641
Van De Ven AH, Delbecq A. The effectiveness of nominal, Delphi, and interacting group decision making processes. Acad Manag J. 1974;17(4):605–21. https://doi.org/10.5465/255641.
Article Google Scholar
Ven A, Delbecq A. The nominal group as a research instrument for exploratory health studies. Am J Public Health. 1972;62:337–42. https://doi.org/10.2105/AJPH.62.3.337.
Article Google Scholar
Watson KE, Singleton JA, Tippett V, Nissen LM. Defining pharmacists’ roles in disasters: a Delphi study. PLOS ONE. 2019;14(12):e0227132. https://doi.org/10.1371/journal.pone.0227132.
Article CAS Google Scholar
Weijters B, Cabooter E, Schillewaert N. The effect of rating scale format on response styles: the number of response categories and response category labels. Int J Res Mark. 2010;27(3):236–47. https://doi.org/10.1016/j.ijresmar.2010.02.004.
Article Google Scholar

Download references

Author information

Authors and Affiliations

School of Pharmacy and Medical Sciences, Griffith University, Gold Coast, Queensland, Australia
Michelle A. King, Fiona S. Kelly & Sara S. McMillan

Authors

Michelle A. King
View author publications
You can also search for this author in PubMed Google Scholar
Fiona S. Kelly
View author publications
You can also search for this author in PubMed Google Scholar
Sara S. McMillan
View author publications
You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michelle A. King .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar
Derek Charles Stewart

Rights and permissions

Reprints and permissions

Copyright information

About this entry

Cite this entry

King, M.A., Kelly, F.S., McMillan, S.S. (2022). Consensus Methodologies and Producing the Evidence. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_75-1

Download citation

DOI: https://doi.org/10.1007/978-3-030-50247-8_75-1
Received: 02 September 2021
Accepted: 25 April 2022
Published: 12 February 2023
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50247-8
Online ISBN: 978-3-030-50247-8
eBook Packages: Springer Reference Biomedicine and Life SciencesReference Module Biomedical and Life Sciences

Publish with us

Policies and ethics