Abstract
Consensus development sprang from a desire to synthesize clinician and expert opinions on clinical practice and research agendas in the 1950s. And since the American Institute of Medicine formally defined “guidelines” in 1990, there has been a proliferation of clinical practice guidelines (CPG) both formally and informally. This modern decision-making tool used by both physicians and patients, requires extensive planning to overcome the challenges of consensus development while reaping its rewards. Consensus allows for a group approach of multiple experts sharing ideas to form consensus on topics ranging from appropriateness of procedures to research agenda development. Disagreements can shed light on areas of controversy and launch further discussions. It has five main components: three inputs (defining the task, participant identification and recruitment, and information synthesis), the approach (consensus development by explicit or implicit means), and the output (dissemination of results). Each aspect requires extensive planning a priori as they influence the entire process, from how information will be interpreted, the interaction of participants, the resulting judgment, to whether there will be uptake of results. Implicit approaches utilize qualitative methods and/or a simple voting structure of majority wins, and are used in informal consensus development methods and consensus development conferences. Explicit approaches aggregate results or judgments using explicit rules set a priori with definitions of “agreement” or consensus. Because the implicit process can be more opaque, unforeseen challenges can emerge such as the undue influence of a minority. And yet, the logistics of explicit approaches may be more time consuming and not appropriate when speed is a priority. In determining which method to use, it is important to understand the pros and cons of different approaches and how it will affect the overall input, approach, and outcome.
Similar content being viewed by others
References
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, Marteau T (1998) Consensus development methods, and their use in clinical guideline development. Health Technol Assess 2(3):i–iv (1–88)
Bayley EW, Richmond T, Noroian EL, Allen LR (1994) A Delphi study on research priorities for trauma nursing. Am J Crit Care 3(3):208–216
Bond S, Bond J (1982) A Delphi survey of clinical nursing research priorities. J Adv Nurs 7(6):565–575
Kors JA, van Herpen G, van Bemmel JH (1992) Variability in ECG computer interpretation. Analysis of individual complexes vs analysis of a representative complex. J Electrocardiol 25(4):263–271
Rinaldi RC, Steindler EM, Wilford BB, Goodwin D (1988) Clarification and standardization of substance abuse terminology. JAMA 259(4):555–557
Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J, Park RE (1986) A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care 2(1):53–63
Park RE, Fink A, Brook RH, Chassin MR, Kahn KL, Merrick NJ, Kosecoff J, Solomon DH (1986) Physician ratings of appropriate indications for six medical and surgical procedures. Am J Public Health 76(7):766–772
Shekelle PG, Chassin MR, Park RE (1998) Assessing the predictive validity of the RAND/UCLA appropriateness method criteria for performing carotid endarterectomy. Int J Technol Assess Health Care 14(4):707–727
Field M, Lohr K (1990) Committe to Advise the Public Health Service on clinical practice guidelines IoM: Clinical practice guidelines: directions for a new program. Institute of Medicine, Washington
Weisz G, Cambrosio A, Keating P, Knaapen L, Schlich T, Tournay VJ (2007) The emergence of clinical practice guidelines. Milbank Q 85(4):691–727. doi:10.1111/j.1468-0009.2007.00505.x
Hurwitz B (1994) Clinical guidelines: proliferation and medicolegal significance. Qual Health Care 3(1):37–44
HIaQ Authority (2011) National quality assurance criteria for clinical guidelines. Health Information and Quality Authority, Dublin
Greenfield S, Steinberg EP, Auerbach A, Avorn JL, Lau J, Morrow M, Mulrow CD, Rosoff AJ, Santa J, Shiffman RN, smitt WR, Stewart WR, Stovall E, Storm BL, Titler MG (2011) Clinical practice guidelines we can trust. Institute of Medicine, Washington
Pagliari C, Grimshaw J (2002) Impact of group structure and process on multidisciplinary evidence-based guideline development: an observational study. J Eval Clin Pract 8(2):145–153
Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, Baldeh T, Zhang Y, Raid U, Neumann I, Norris SL, Thornton J, Harbour R, Treweek S, Guyatt G, Alonso-Coello P, Reinap M, Brozek J, Oxman A, Akl EA (2014) Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. Can Med Assoc J (journal de l’Association medicale canadienne) 186:E123–E142. doi:10.1503/cmaj.131237
Guyatt G, Vandvik PO (2013) Creating clinical practice guidelines: problems and solutions. Chest 144(2):365–367. doi:10.1378/chest.13-0463
Shekelle P, Woolf S, Grimshaw JM, Schunemann HJ, Eccles MP (2012) Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci 7:62. doi:10.1186/1748-5908-7-62
Eccles MP, Grimshaw JM, Shekelle P, Schunemann HJ, Woolf S (2012) Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implement Sci 7:60. doi:10.1186/1748-5908-7-60
Woolf S, Schunemann HJ, Eccles MP, Grimshaw JM, Shekelle P (2012) Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implement Sci 7:61. doi:10.1186/1748-5908-7-61
PCORI (2014) Engagement in research. Patient-centered outcomes research institute. http://www.pcori.org/get-involved/what-is-engagement-in-research/. Accessed 23 June 2014
Hogarth R (1978) A note on aggregating opinions. Organ Behav Hum Perform 21:40–46
Sun BC, Thiruganasambandamoorthy V, Cruz JD, EDSRSR Consortium to Standardize (2012) Standardized reporting guidelines for emergency department syncope risk-stratification research. Acad Emerg Med 19(6):694–702. doi:10.1111/j.1553-2712.2012.01375.x
Eichacker PQ, Natanson C, Danner RL (2006) Surviving sepsis–practice guidelines, marketing campaigns, and Eli Lilly. New Engl J Med 355(16):1640–1642. doi:10.1056/NEJMp068197
Bernard GR, Vincent JL, Laterre PF, LaRosa SP, Dhainaut JF, Lopez-Rodriguez A, Steingrub JS, Garber GE, Helterbrand JD, Ely EW, Fisher CJ Jr, CWEiSSsg Recombinant human protein (2001) Efficacy and safety of recombinant human activated protein C for severe sepsis. New Engl J Med 344(10):699–709. doi:10.1056/NEJM200103083441001
Abraham E (2005) Effects of recombinant human activated protein C in human models of endotoxin administration. Proc Am Thorac Soc 2(3):243–247. doi:10.1513/pats.200501-004AC
Nadel S, Goldstein B, Williams MD, Dalton H, Peters M, Macias WL, Abd-Allah SA, Levy H, Angle R, Wang D, Sundin DP, Giroir B, REs Sepsis, g Organ dysfunction in children: a gLobal perspective study (2007) Drotrecogin alfa (activated) in children with severe sepsis: a multicentre phase III randomised controlled trial. Lancet 369(9564):836–843. doi:10.1016/S0140-6736(07)60411-5
Ranieri VM, Thompson BT, Barie PS, Dhainaut JF, Douglas IS, Finfer S, Gardlund B, Marshall JC, Rhodes A, Artigas A, Payen D, Tenhunen J, Al-Khalidi HR, Thompson V, Janes J, Macias WL, Vangerow B, Williams MD, P-SS Group (2012) Drotrecogin alfa (activated) in adults with septic shock. New Engl J Med 366(22):2055–2064. doi:10.1056/NEJMoa1202290
Jacoby I (1988) Evidence and consensus. JAMA 259(20):3039
Fink A, Kosecoff J, Chassin M, Brook RH (1984) Consensus methods: characteristics and guidelines for use. Am J Public Health 74(9):979–983
Shea BJ, Bouter LM, Peterson J, Boers M, Andersson N, Ortiz Z, Ramsay T, Bai A, Shukla VK, Grimshaw JM (2007) External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS One 2(12):e1350. doi:10.1371/journal.pone.0001350
Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10. doi:10.1186/1471-2288-7-10
Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, Atkins D, Kunz R, Montori V, Jaeschke R, Rind D, Dahm P, Akl EA, Meerpohl J, Vist G, Berliner E, Norris S, Falck-Ytter Y, Schunemann HJ (2013) GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol 66(2):151–157. doi:10.1016/j.jclinepi.2012.01.006
Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, Brozek J, Norris S, Meerpohl J, Djulbegovic B, Alonso-Coello P, Post PN, Busse JW, Glasziou P, Christensen R, Schunemann HJ (2013) GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes. J Clin Epidemiol 66(2):158–172. doi:10.1016/j.jclinepi.2012.01.012
Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanicolas P, Akl EA, Vist G, Kunz R, Brozek J, Kupper LL, Martin SL, Meerpohl JJ, Alonso-Coello P, Christensen R, Schunemann HJ (2013) GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. J Clin Epidemiol 66(2):173–183. doi:10.1016/j.jclinepi.2012.08.001
Guyatt G, Eikelboom JW, Akl EA, Crowther M, Gutterman D, Kahn SR, Schunemann H, Hirsh J (2013) A guide to GRADE guidelines for the readers of JTH. J Thromb Haemost 11(8):1603–1608. doi:10.1111/jth.12320
Andrews JC, Schunemann HJ, Oxman AD, Pottie K, Meerpohl JJ, Coello PA, Rind D, Montori VM, Brito JP, Norris S, Elbarbary M, Post P, Nasser M, Shukla V, Jaeschke R, Brozek J, Djulbegovic B, Guyatt G (2013) GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation’s direction and strength. J Clin Epidemiol 66(7):726–735. doi:10.1016/j.jclinepi.2013.02.003
Andrews J, Guyatt G, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, Nasser M, Meerpohl J, Post PN, Kunz R, Brozek J, Vist G, Rind D, Akl EA, Schunemann HJ (2013) GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. J Clin Epidemiol 66(7):719–725. doi:10.1016/j.jclinepi.2012.03.013
Guyatt GH, Oxman AD, Schunemann HJ (2013) GRADE guidelines-an introduction to the 10th-13th articles in the series. J Clin Epidemiol 66(2):121–123. doi:10.1016/j.jclinepi.2012.05.011
Brunetti M, Shemilt I, Pregno S, Vale L, Oxman AD, Lord J, Sisk J, Ruiz F, Hill S, Guyatt GH, Jaeschke R, Helfand M, Harbour R, Davoli M, Amato L, Liberati A, Schunemann HJ (2013) GRADE guidelines: 10. Considering resource use and rating the quality of economic evidence. J Clin Epidemiol 66(2):140–150. doi:10.1016/j.jclinepi.2012.04.012
Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, Devereaux PJ, Montori VM, Freyschuss B, Vist G, Jaeschke R, Williams JW Jr, Murad MH, Sinclair D, Falck-Ytter Y, Meerpohl J, Whittington C, Thorlund K, Andrews J, Schunemann HJ (2011) GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol 64(12):1283–1293
Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, Alonso-Coello P, Glasziou P, Jaeschke R, Akl EA, Norris S, Vist G, Dahm P, Shukla VK, Higgins J, Falck-Ytter Y, Schunemann HJ, GW Group (2011) GRADE guidelines: 7. Rating the quality of evidence–inconsistency. J Clin Epidemiol 64(12):1294–1302. doi:10.1016/j.jclinepi.2011.03.017
Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, Alonso-Coello P, Djulbegovic B, Atkins D, Falck-Ytter Y, Williams JW Jr, Meerpohl J, Norris SL, Akl EA, Schunemann HJ (2011) GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemio 64(12):1277–1282. doi:10.1016/j.jclinepi.2011.01.011
Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, Alonso-Coello P, Falck-Ytter Y, Jaeschke R, Vist G, Akl EA, Post PN, Norris S, Meerpohl J, Shukla VK, Nasser M, Schunemann HJ, GW Group (2011) GRADE guidelines: 8. Rating the quality of evidence—indirectness. J Clin Epidemiol 64(12):1303–1310. doi:10.1016/j.jclinepi.2011.04.014
Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello P, Atkins D, Kunz R, Brozek J, Montori V, Jaeschke R, Rind D, Dahm P, Meerpohl J, Vist G, Berliner E, Norris S, Falck-Ytter Y, Murad MH, Schunemann HJ, GW Group (2011) GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol 64(12):1311–1316. doi:10.1016/j.jclinepi.2011.06.004
Scott IA, Guyatt GH (2011) Clinical practice guidelines: the need for greater transparency in formulating recommendations. The Medical journal of Australia 195(1):29–33
Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, Montori V, Akl EA, Djulbegovic B, Falck-Ytter Y, Norris SL, Williams JW Jr, Atkins D, Meerpohl J, Schunemann HJ (2011) GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol 64(4):407–415. doi:10.1016/j.jclinepi.2010.07.017
Brozek JL, Akl EA, Compalati E, Kreis J, Terracciano L, Fiocchi A, Ueffing E, Andrews J, Alonso-Coello P, Lang DM, Jaeschke R, Williams JW Jr, Phillips B, Lethaby A, Bossuyt P, Glasziou P, Helfand M, Watine J, Afilalo M, Welch V, Montedori A, Abraha I, Horvath AR, Bousquet J, Guyatt GH, Schunemann HJ, GW Group (2011) Grading quality of evidence and strength of recommendations in clinical practice guidelines part 3 of 3. The GRADE approach to developing recommendations. Allergy 66(5):588–595. doi:10.1111/j.1398-9995.2010.02530.x
Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH (2011) GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 64(4):401–406. doi:10.1016/j.jclinepi.2010.07.015
Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schunemann HJ (2011) GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 64(4):383–394. doi:10.1016/j.jclinepi.2010.04.026
Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, Alderson P, Glasziou P, Falck-Ytter Y, Schunemann HJ (2011) GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol 64(4):395–400. doi:10.1016/j.jclinepi.2010.09.012
Guyatt GH, Oxman AD, Schunemann HJ, Tugwell P, Knottnerus A (2011) GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol 64(4):380–382. doi:10.1016/j.jclinepi.2010.09.011
Guyatt G, Akl EA, Hirsh J, Kearon C, Crowther M, Gutterman D, Lewis SZ, Nathanson I, Jaeschke R, Schunemann H (2010) The vexing problem of guidelines and conflict of interest: a potential solution. Ann Intern Med 152(11):738–741. doi:10.7326/0003-4819-152-11-201006010-00254
Fahey T, Griffiths S, Peters TJ (1995) Evidence based purchasing: understanding results of clinical trials and systematic reviews. BMJ 311(7012):1056–1059 (discussion 1059–1060)
Bucher HC, Weinbacher M, Gyr K (1994) Influence of method of reporting study results on decision of physicians to prescribe drugs to lower cholesterol concentration. BMJ 309(6957):761–764
Laupacis A, Sackett DL, Roberts RS (1988) An assessment of clinically useful measures of the consequences of treatment. New Engl J Med 318(26):1728–1733. doi:10.1056/NEJM198806303182605
Bettman JW Jr, Bettman JW Sr (1977) One-day hospitalization for extracapsular cataract surgery without phacoemulsification. Ophthalmic Surg 8(4):81–86
Jarvenpaa SL (1990) Graphic displays in decision making-the visual salience effect. Behav Decision Mak 3(4):247–262. doi:10.1002/bdm.3960030403
U.S. Preventive Services Task Force (2009) Summaries for patients. Screening for breast cancer: U.S. Preventive Services Task Force recommendation. Ann Intern Med 151(10):I-44. doi:10.7326/0003-4819-151-10-200911170-00002
Goodman A (2011) Attitudes toward mammography guidelines influenced by media. Medscape. http://www.medscape.com/viewarticle/744107. Accessed 18 June 2014
Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM (2011) How can we improve guideline use? A conceptual framework of implementability. Implement Sci 6:26. doi:10.1186/1748-5908-6-26
Penciner R, Langhan T, Lee R, McEwen J, Woods RA, Bandiera G (2011) Using a Delphi process to establish consensus on emergency medicine clerkship competencies. Med Teach 33(6):e333–e339. doi:10.3109/0142159X.2011.575903
Lindsay P, Schull M, Bronskill S, Anderson G (2002) The development of indicators to measure the quality of clinical care in emergency departments following a modified-Delphi approach. Acad Emerg Med 9(11):1131–1139
Solberg LI, Asplin BR, Weinick RM, Magid DJ (2003) Emergency department crowding: consensus development of potential measures. Ann Emerg Med 42(6):824–834. doi:10.1016/S0196064403008163
Cairns CB, Maier RV, Adeoye O, Baptiste D, Barsan WG, Blackbourne L, Burd R, Carpenter C, Chang D, Cioffi W, Cornwell E, Dean JM, Dyer C, Jaffe D, Manley G, Meurer WJ, Neumar R, Silbergleit R, Stevens M, Wang M, Weiner D, Wright D, Roundtable External P, Roundtable Steering C, Federal P (2010) NIH Roundtable on Emergency Trauma Research. Ann Emerg Med 56(5):538–550. doi:10.1016/j.annemergmed.2010.05.029
D’Onofrio G, Jauch E, Jagoda A, Allen MH, Anglin D, Barsan WG, Berger RP, Bobrow BJ, Boudreaux ED, Bushnell C, Chan YF, Currier G, Eggly S, Ichord R, Larkin GL, Laskowitz D, Neumar RW, Newman-Toker DE, Quinn J, Shear K, Todd KH, Zatzick D, Roundtable External P, Roundtable Steering C, Federal P (2010) NIH Roundtable on Opportunities to Advance Research on Neurologic and Psychiatric Emergencies. Ann Emerg Med 56(5):551–564. doi:10.1016/j.annemergmed.2010.06.562
Kaji AH, Lewis RJ, Beavers-May T, Berg R, Bulger E, Cairns C, Callaway C, Camargo CA Jr, Carcillo J, DeBiasi R, Diaz T, Ducharme F, Glickman S, Heilpern K, Hickey R, Hoek TV, Hollander J, Janson S, Jurkovich G, Kellermann A, Kingsmore S, Kline J, Kuppermann N, Lowe R, McLario D, Nathanson L, Nichol G, Peitzman A, Richardson L, Sanders A, Shah M, Shapiro N, Silverman R, Than M, Wilber S, Yealy DM, Roundtable External P, Roundtable Steering C, Federal P (2010) Summary of NIH Medical-Surgical Emergency Research Roundtable held on April 30 to May 1, 2009. Ann Emerg Med 56(5):522–537. doi:10.1016/j.annemergmed.2010.03.014
Sun BC, Costantino G, Barbic F, Bossi I, Casazza G, Dipaola F, McDermott D, Quinn J, Reed M, Sheldon RS, Solbiati M, Thiruganasambandamoorthy V, Krahn AD, Beach D, Bodemer N, Brignole M, Casagranda I, Duca P, Falavigna G, Ippoliti R, Montano N, Olshansky B, Raj SR, Ruwald MH, Shen WK, Stiell I, Ungar A, van Dijk JG, van Dijk N, Wieling W, Furlan R (2014) Priorities for Emergency Department Syncope Research. Ann Emerg Med. doi:10.1016/j.annemergmed.2014.04.014
Conflict of interest
None.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kea, B., Sun, B.CA. Consensus development for healthcare professionals. Intern Emerg Med 10, 373–383 (2015). https://doi.org/10.1007/s11739-014-1156-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11739-014-1156-6