Abstract
Understanding and addressing the priorities of relevant stakeholders is needed in healthcare decision-making, particularly when there is uncertainty due to insufficient evidence, healthcare resources are limited, or to optimize the delivery and assessment of interventions. Consensus methods have diverse applications that include prioritizing research questions that meet the needs of all stakeholders, outcomes for trials, clinical practice guidelines, topics for medical education, and the allocation of scarce resources. Formal consensus methods generate agreement among stakeholders in a transparent, inclusive, and rigorous way. In general, this is achieved by identifying, exploring, and clarifying the set of issues or opinions followed by a process to reconcile opinions (i.e., consensus development). Common features of consensus methods include the provision of standardized materials, moderated interactions, private polling, and iterative feedback from the group. This chapter reviews the three most commonly used formal consensus methods: the nominal group technique, the Delphi method, and stakeholder workshops, and discusses their use in healthcare research in a global setting.
This is a preview of subscription content, log in via an institution to check access.
References
Aspinal F, Hughes R, Dunckley M, Addington-Hall J (2006) What is important to measure in the last months and weeks of life?: a modified nominal group study. Int J Nurs Stud 43(4):393–403. https://doi.org/10.1016/j.ijnurstu.2005.06.005
Banks PA, Bollen TL, Dervenis C, Gooszen HG, Johnson CD, Sarr MG et al (2013) Classification of acute pancreatitis--2012: revision of the Atlanta classification and definitions by international consensus. Gut 62(1):102–111. https://doi.org/10.1136/gutjnl-2012-302779
Barrington H, Young B, Williamson PR (2021) Patient participation in Delphi surveys to develop core outcome sets: systematic review. BMJ Open 11(9):e051066. https://doi.org/10.1136/bmjopen-2021-051066
Bayley EW, Richmond T, Noroian EL, Allen LR (1994) A Delphi study on research priorities for trauma nursing. Am J Crit Care 3(3):208–216. https://www.ncbi.nlm.nih.gov/pubmed/8038850
Beuscart JB, Dalleur O, Boland B, Thevelin S, Knol W, Cullinan S et al (2017) Development of a core outcome set for medication review in older patients with multimorbidity and polypharmacy: a study protocol. Clin Interv Aging 12:1379–1389. https://doi.org/10.2147/CIA.S135481
Biggane AM, Brading L, Ravaud P, Young B, Williamson PR (2018) Survey indicated that core outcome set development is increasingly including patients, being conducted internationally and using Delphi surveys. Trials 19(1):113. https://doi.org/10.1186/s13063-018-2493-y
Bishop DV, Snowling MJ, Thompson PA, Greenhalgh T, consortium C. CATALISE: A Multinational and Multidisciplinary Delphi Consensus Study. Identifying Language Impairments in Children. PLoS ONE. 2016;11(7):e0158753.
Breuer E, De Silva MJ, Fekadu A, et al. Using workshops to develop theories of change in five low and middle income countries: lessons from the programme for improving mental health care (PRIME). Int J Ment Health Syst. 2014;8:15.
Brookes ST, Macefield RC, Williamson PR, McNair AG, Potter S, Blencowe NS et al (2016) Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development. Trials 17(1):409. https://doi.org/10.1186/s13063-016-1479-x
Carter SA, Lightstone L, Cattran D, Bagga A, Barbour SJ, Barratt J et al (2019) Standardized Outcomes in Nephrology-Glomerular Disease (SONG-GD): establishing a core outcome set for trials in patients with glomerular disease. Kidney Int 95(6):1280–1283. https://doi.org/10.1016/j.kint.2019.01.047
Carter SA, Gutman T, Logeman C, Cattran D, Lightstone L, Bagga A et al (2020) Identifying outcomes important to patients with glomerular disease and their caregivers. Clin J Am Soc Nephrol 15(5):673–684. https://doi.org/10.2215/CJN.13101019
Carter SA, Logeman C, Howell M, Cattran D, Lightstone L, Bagga A et al (2021) Development of an international Delphi survey to establish core outcome domains for trials in adults with glomerular disease. Kidney Int 100(4):881–893. https://doi.org/10.1016/j.kint.2021.04.027
Chevance A, Tran VT, Ravaud P (2020) Controversy and Debate Series on Core Outcome Sets. Paper 1: Improving the generalizability and credibility of core outcome sets (COS) by a large and international participation of diverse stakeholders. J Clin Epidemiol 125:206–212.e1. https://doi.org/10.1016/j.jclinepi.2020.01.004
Cho Y, Sautenet B, Gutman T, Rangan G, Craig JC, Ong AC et al (2019) Identifying patient-important outcomes in polycystic kidney disease: an international nominal group technique study. Nephrology (Carlton) 24:1214. https://doi.org/10.1111/nep.13566
Cho Y, Rangan G, Logeman C, Ryu H, Sautenet B, Perrone RD et al (2020) Core outcome domains for trials in autosomal dominant polycystic kidney disease: an international Delphi survey. Am J Kidney Dis 76(3):361–373. https://doi.org/10.1053/j.ajkd.2020.01.005
Collins JA, Lewin S, Shmueli-Blumberg D, Hoffman KA, Terashima JP, Korthuis PT et al (2021) The process and challenges of language translation and cultural adaptation of study instruments: a case study from the NIDA CTN CHOICES-2 trial. J Ethn Subst Abus:1–16. https://doi.org/10.1080/15332640.2021.1952129
Corner J, Wright D, Hopkinson J, Gunaratnam Y, McDonald JW, Foster C (2007) The research priorities of patients attending UK cancer treatment centres: findings from a modified nominal group study. Br J Cancer 96(6):875–881. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2360101/pdf/6603662a.pdf
Daines L, Lewis S, Schneider A, Sheikh A, Pinnock H (2020) Defining high probability when making a diagnosis of asthma in primary care: mixed-methods consensus workshop. BMJ Open 10(4):e034559. https://doi.org/10.1136/bmjopen-2019-034559
de Wolf-Linder S, Dawkins M, Wicks F, Pask S, Eagar K, Evans CJ et al (2019) Which outcome domains are important in palliative care and when? An international expert consensus workshop, using the nominal group technique. Palliat Med 33(8):1058–1068. https://doi.org/10.1177/0269216319854154
Dening KH, Jones L, Sampson EL (2012) Preferences for end-of-life care: a nominal group study of people with dementia and their family carers. Palliat Med, Nov 5 2012, Online First
Dening KH, Jones L, Sampson EL (2013) Preferences for end-of-life care: a nominal group study of people with dementia and their family carers. Palliat Med 27(5):409–417. https://doi.org/10.1177/0269216312464094
Epstein J, Santo RM, Guillemin F (2015) A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. J Clin Epidemiol 68(4):435–441. https://doi.org/10.1016/j.jclinepi.2014.11.021
Evangelidis N, Craig JC, Tong A, SONG Executive Committee and Investigators (2015) Standardised outcomes in nephrology-haemodialysis (SONG-HD): using the Delphi method to gain consensus on core outcomes for haemodialysis trials. J Ren Care 41(4):211–212. https://doi.org/10.1111/jorc.12144
Evangelidis N, Tong A, Manns B, Hemmelgarn B, Wheeler DC, Tugwell P et al (2017) Developing a set of core outcomes for trials in hemodialysis: an international Delphi survey. Am J Kidney Dis 70(4):464–475. https://doi.org/10.1053/j.ajkd.2016.11.029
Fink A, Kosecoff J, Chassin MR, Brook RH (1991) Consensus methods: characteristics and guidelines for use. https://www.rand.org/pubs/notes/N3367.html
Flynn TN, Louviere JJ, Peters TJ, Coast J (2008) Estimating preferences for a dermatology consultation using Best-Worst Scaling: comparison of various methods of analysis. BMC Med Res Methodol 8:76. https://doi.org/10.1186/1471-2288-8-76
Fried BJ, Boers M, Baker PR (1993) A method for achieving consensus on rheumatoid arthritis outcome measures: the OMERACT conference process. J Rheumatol 20(3):548–551. https://www.ncbi.nlm.nih.gov/pubmed/8478870
Hall CL, Brown S, James M, et al. Consensus workshops on the development of an ADHD medication management protocol using QbTest: developing a clinical trial protocol with multidisciplinary stakeholders. BMC Med Res Methodol. 2019;19(1):126.
Hall DA, Smith H, Heffernan E, Fackrell K, Core Outcome Measures in Tinnitus International Delphi Research Steering Group (2018) Recruiting and retaining participants in e-Delphi surveys for core outcome set development: evaluating the COMiT’ID study. PLoS One 13(7):e0201378. https://doi.org/10.1371/journal.pone.0201378
Hanson CS, Chapman JR, Gill JS, Kanellis J, Wong G, Craig JC et al (2018) Identifying outcomes that are important to living kidney donors: a nominal group technique study. Clin J Am Soc Nephrol 13(6):916–926. https://doi.org/10.2215/CJN.13441217
Hanson CS, Gutman T, Craig JC, et al. Identifying Important Outcomes for Young People With CKD and Their Caregivers: A Nominal Group Technique Study. Am J Kidney Dis. 2019;74(1):82-94.
Humphrey-Murto S, de Wit M (2019) The Delphi method-more research please. J Clin Epidemiol 106:136–139. https://doi.org/10.1016/j.jclinepi.2018.10.011
Humphrey-Murto S, Varpio L, Wood TJ, Gonsalves C, Ufholz LA, Mascioli K et al (2017) The use of the Delphi and other consensus group methods in medical education research: a review. Acad Med 92(10):1491–1498. https://doi.org/10.1097/ACM.0000000000001812
Irving MJ, Jan S, Tong A, Wong G, Craig JC, Chadban S et al (2014) What factors influence people’s decisions to register for organ donation? The results of a nominal group study. Transpl Int 27(6):617–624. https://onlinelibrary.wiley.com/doi/pdf/10.1111/tri.12307
Jaeschke R, Guyatt GH, Dellinger P, Schunemann H, Levy MM, Kunz R et al (2008) Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. BMJ 337:a744. https://doi.org/10.1136/bmj.a744
James Lind Alliance (2021a). https://www.jla.nihr.ac.uk. Accessed 21 Dec 2021
James Lind Alliance (2021b). https://www.jla.nihr.ac.uk/jla-lab/Report-on-JLA-PSP-online-priority-setting-workshops.pdf. Accessed 20 Dec 2021
Jones J, Hunter D (1995) Consensus methods for medical and health services research. BMJ 311(7001):376–380. https://doi.org/10.1136/bmj.311.7001.376
King KM, Khan N, Leblanc P, Quan H (2011) Examining and establishing translational and conceptual equivalence of survey questionnaires for a multi-ethnic, multi-language study. J Adv Nurs 67(10):2267–2274. https://doi.org/10.1111/j.1365-2648.2011.05679.x
Lee A, Davies A, Young AE (2020) Systematic review of international Delphi surveys for core outcome set development: representation of international patients. BMJ Open 10(11):e040223. https://doi.org/10.1136/bmjopen-2020-040223
Levey AS, Eckardt KU, Dorman NM, Christiansen SL, Hoorn EJ, Ingelfinger JR et al (2020) Nomenclature for kidney function and disease: report of a Kidney Disease: Improving Global Outcomes (KDIGO) Consensus Conference. Kidney Int 97(6):1117–1129. https://doi.org/10.1016/j.kint.2020.02.010
Logeman C, Guha C, Howell M, Hanson CS, Craig JC, Samuel S et al (2020) Developing consensus-based outcome domains for trials in children and adolescents with CKD: an international Delphi survey. Am J Kidney Dis 76(4):533–545. https://doi.org/10.1053/j.ajkd.2020.03.014
Lomas J, Pickard L, Mohide A (1987) Patient versus clinician item generation for quality-of-life measures. The case of language-disabled adults. Med Care 25(8):764–769. https://doi.org/10.1097/00005650-198708000-00009
MacLennan S, Williamson PR, Bekema H, Campbell M, Ramsay C, N’Dow J et al (2017) A core outcome set for localised prostate cancer effectiveness trials. BJU Int 120(5B):E64–E79. https://doi.org/10.1111/bju.13854
Manera KE, Johnson DW, Craig JC, Shen JI, Ruiz L, Wang AY et al (2019a) Patient and caregiver priorities for outcomes in peritoneal dialysis: multinational nominal group technique study. Clin J Am Soc Nephrol 14(1):74–83. https://doi.org/10.2215/CJN.05380518
Manera KE, Tong A, Craig JC, Shen J, Jesudason S, Cho Y et al (2019b) An international Delphi survey helped develop consensus-based core outcome domains for trials in peritoneal dialysis. Kidney Int 96(3):699–710. https://doi.org/10.1016/j.kint.2019.03.015
Mason S, Ling J, Mosoiu D, Arantzamendi M, Tserkezoglou AJ, Predoiu O et al (2021) Undertaking research using online nominal group technique: lessons from an international study (RESPACC). J Palliat Med 24(12):1867–1871. https://doi.org/10.1089/jpm.2021.0216
Minas H, Jorm AF (2010) Where there is no evidence: use of expert consensus methods to fill the evidence gap in low-income countries and cultural minorities. Int J Ment Health Syst 4:33. https://doi.org/10.1186/1752-4458-4-33
Miskovic D, Foster J, Agha A, Delaney CP, Francis N, Hasegawa H et al (2015) Standardization of laparoscopic total mesorectal excision for rectal cancer: a structured international expert consensus. Ann Surg 261(4):716–722. https://doi.org/10.1097/SLA.0000000000000823
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J et al (1998) Consensus development methods, and their use in clinical guideline development. Health Technol Assess 2(3):i–iv, 1–88. https://www.ncbi.nlm.nih.gov/pubmed/9561895; https://core.ac.uk/download/217366600.pdf
Porter J (2013) Be careful how you ask! Using focus groups and nominal group technique to explore the barriers to learning. Int J Res Method Educ 36(1):33–51. https://doi.org/10.1080/1743727x.2012.675554
Robert G, Harlock J, Williams I (2014) Disentangling rhetoric and reality: an international Delphi study of factors and processes that facilitate the successful implementation of decisions to decommission healthcare services. Implement Sci 9:123. https://doi.org/10.1186/s13012-014-0123-y
Sautenet B, Tong A, Manera KE, Chapman JR, Warrens AN, Rosenbloom D et al (2017) Developing consensus-based priority outcome domains for trials in kidney transplantation: a multinational Delphi survey with patients, caregivers, and health professionals. Transplantation 101(8):1875–1886. https://doi.org/10.1097/TP.0000000000001776
Shekelle PG, Goldman DP (2018) Development of a group judgment process for forecasts of health care innovations. JAMA Netw Open 1(7):e185108. https://doi.org/10.1001/jamanetworkopen.2018.5108
Singh JA, Qu H, Yazdany J, Chatham W, Dall’era M, Shewchuk RM (2015) Barriers to medication decision making in women with lupus nephritis: a formative study using nominal group technique. J Rheumatol 42(9):1616–1623. https://doi.org/10.3899/jrheum.150168
Sinha IP, Gallagher R, Williamson PR, Smyth RL (2012) Development of a core outcome set for clinical trials in childhood asthma: a survey of clinicians, parents, and young people. Trials 13:103. https://doi.org/10.1186/1745-6215-13-103
SONG Initiative (2017) The SONG handbook. Version 1.0, June 2017, Sydney, Australia. Available at songinitiative.org/reports-and-publications/
Strauss G, Chassin M, Lock J (1995) Can experts agree when to hospitalize adolescents? J Am Acad Child Adolesc Psychiatry 34(4):418–424. https://www.ncbi.nlm.nih.gov/pubmed/7751255
Tong A, Gill J, Budde K, Marson L, Reese PP, Rosenbloom D et al (2017a) Toward establishing core outcome domains for trials in kidney transplantation: report of the standardized outcomes in nephrology-kidney transplantation consensus workshops. Transplantation 101(8):1887–1896. https://doi.org/10.1097/TP.0000000000001774
Tong A, Manns B, Hemmelgarn B, Wheeler DC, Evangelidis N, Tugwell P et al (2017b) Establishing core outcome domains in hemodialysis: report of the standardized outcomes in nephrology-hemodialysis (SONG-HD) consensus workshop. Am J Kidney Dis 69(1):97–107. https://doi.org/10.1053/j.ajkd.2016.05.022
Tong A, Manns B, Wang AYM, Hemmelgarn B, Wheeler DC, Gill J et al (2018) Implementing core outcomes in kidney disease: report of the Standardized Outcomes in Nephrology (SONG) implementation workshop. Kidney Int 94(6):1053–1068. https://doi.org/10.1016/j.kint.2018.08.018
Urquhart-Secord R, Craig JC, Hemmelgarn B, Tam-Tham H, Manns B, Howell M et al (2016) Patient and caregiver priorities for outcomes in hemodialysis: an international nominal group technique study. Am J Kidney Dis 68(3):444–454. https://doi.org/10.1053/j.ajkd.2016.02.037
Van de Ven AH, Delbecq AL (1972) The nominal group as a research instrument for exploratory health studies. Am J Public Health 62(3):337–342. https://doi.org/10.2105/ajph.62.3.337
van Rijssen LB, Gerritsen A, Henselmans I, Sprangers MA, Jacobs M, Bassi C et al (2019) Core set of patient-reported outcomes in pancreatic cancer (COPRAC): an international Delphi study among patients and health care providers. Ann Surg 270(1):158–164. https://doi.org/10.1097/SLA.0000000000002633
Varma DS, Young ME, Kreider CM, Williams K, Vaddiparti K, Parisi C et al (2021) Practical considerations in qualitative health research during the COVID-19 pandemic. Int J Qual Methods 20:16094069211043755. https://doi.org/10.1177/16094069211043755
Veugelers R, Gaakeer MI, Patka P, Huijsman R (2020) Improving design choices in Delphi studies in medicine: the case of an exemplary physician multi-round panel study with 100% response. BMC Med Res Methodol 20(1):156. https://doi.org/10.1186/s12874-020-01029-4
Waggoner J, Carline JD, Durning SJ (2016) Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Acad Med 91(5):663–668. https://doi.org/10.1097/ACM.0000000000001092
Wallace SJ, Worrall L, Rose T, Le Dorze G, Cruice M, Isaksen J et al (2017) Which outcomes are most important to people with aphasia and their families? An international nominal group technique study framed within the ICF. Disabil Rehabil 39(14):1364–1379. https://doi.org/10.1080/09638288.2016.1194899
Weissenborn M, Krass I, Van C, Dahne A, Ritter CA, Haefeli WE et al (2020) Process of translation and cross-cultural adaptation of two Australian instruments to evaluate the physician-pharmacist collaboration in Germany. Res Social Adm Pharm 16(1):74–83. https://doi.org/10.1016/j.sapharm.2019.03.009
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E et al (2012) Developing core outcome sets for clinical trials: issues to consider. Trials 13:132. https://doi.org/10.1186/1745-6215-13-132
Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST et al (2017) The COMET handbook: version 1.0. Trials 18(Suppl 3):280. https://doi.org/10.1186/s13063-017-1978-4
Wood L, Bishop A, Lewis M, Smeets R, Bronfort G, Hayden JA et al (2021) Treatment targets of exercise for persistent non-specific low back pain: a consensus study. Physiotherapy 112:78–86. https://doi.org/10.1016/j.physio.2021.03.005
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this entry
Cite this entry
Carter, S.A., Tong, A., Craig, J.C., Teixeira-Pinto, A., Manera, K.E. (2023). Consensus Methods for Health Research in a Global Setting. In: Liamputtong, P. (eds) Handbook of Social Sciences and Global Public Health. Springer, Cham. https://doi.org/10.1007/978-3-030-96778-9_65-2
Download citation
DOI: https://doi.org/10.1007/978-3-030-96778-9_65-2
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-96778-9
Online ISBN: 978-3-030-96778-9
eBook Packages: Springer Reference Social SciencesReference Module Humanities and Social SciencesReference Module Business, Economics and Social Sciences
Publish with us
Chapter history
-
Latest
Consensus Methods for Health Research in a Global Setting- Published:
- 13 June 2023
DOI: https://doi.org/10.1007/978-3-030-96778-9_65-2
-
Original
Consensus Methods for Health Research in a Global Setting- Published:
- 04 June 2023
DOI: https://doi.org/10.1007/978-3-030-96778-9_65-1