A critical appraisal of the quality of critical care pharmacotherapy clinical practice guidelines and their strength of recommendations
- First Online:
Clinical practice guideline (CPG) quality assessment is important before applying their recommendations. Determining whether recommendation strength is consistent with supporting quality of evidence is also essential. We aimed to determine quality of critical care pharmacotherapy CPGs and to assess whether high quality evidence supports strong pharmacotherapy recommendations.
MEDLINE (1966–February 2008), EMBASE (1980–February 2008), National Guideline Clearinghouse (February 2008) and personal files were searched to identify CPGs. Four appraisers evaluated each guideline using the appraisal of guidelines, research and evaluation (AGREE) instrument. AGREE assesses 23 items in six domains that include scope/purpose, stakeholder involvement, rigor of development, clarity, applicability and editorial independence. Standardized domain scores (0–100%) were determined to decide whether to recommend a guideline for use. One appraiser extracted strong pharmacotherapy recommendations and supporting evidence quality.
Twenty-four CPGs were included. Standardized domain scores were clarity [69% (95% confidence interval (CI) 62–76%)], scope/purpose [62% (95% CI 55–68%)], rigor of development [51% (95% CI 42–60%)], editorial independence [39% (95% CI 26–52%)], stakeholder involvement [32% (95% CI 26–37%)] and applicability [19% (95% CI 12–26%)]. The proportion of guidelines that could be strongly recommended, recommended with alterations and not recommended was 25, 37.5 and 37.5%, respectively. High quality evidence supported 36% of strong pharmacotherapy recommendations.
Variation in AGREE domain scores explain why one-third of critical care pharmacotherapy CPGs cannot be recommended. Only one-third of strong pharmacotherapy recommendations were supported by high quality evidence. We recommend appraisal of guideline quality and the caliber of supporting evidence prior to applying recommendations.
KeywordsClinical practice guideline Critical appraisal Quality of evidence Critical care Pharmacotherapy
- 1.Field MJ, Lohr KN (1992) Guidelines for clinical practice: from development to use. National Academy Press, WashingtonGoogle Scholar
- 5.The AGREE Collaboration (2001) Appraisal of guidelines for research and evaluation (AGREE) instrument. http://www.agreecollaboration.org/pdf/agreeinstrumentfinal.pdf. Accessed 23 Jul 2007
- 6.The AGREE Collaboration (2003) Appraisal of guidelines for research and evaluation (AGREE) training manual. http://www.agreecollaboration.org/pdf/aitraining.pdf. Accessed 23 Jul 2007
- 7.Oxford Centre for Evidence Based Medicine (2009) Levels of evidence. http://www.cebm.net/index.aspx?o=1025. Accessed 8 Jul 2009
- 9.Anonymous (2001) Guidelines for the management of penetrating brain injury. J Trauma 51:S3–S6, S34–S43Google Scholar
- 10.Brain Trauma Foundation (2007) Guidelines for the management of severe traumatic brain injury, 3rd edn. J Neurotrauma 24 Suppl 1:S1–S106Google Scholar
- 11.Bromberg WJ, Collier B, Diebel L, Dwyer K, Holevar M, Jacobs D, Kurek S, Schreiber M, Shapiro M, Vogel T (2007) Blunt cerebrovascular injury practice management guidelines. The Eastern Association for the Surgery of Trauma. http://www.east.org/tpg/BluntCVInjury.pdf. Accessed 3 Oct 2007
- 12.Mayberg MR, Batjer HH, Dacey R, Diringer M, Haley EC, Heros RC, Sternau LL, Torner J, Adams HP, Feinberg W (1994) Guidelines for the management of aneurysmal subarachnoid hemorrhage. A statement for healthcare professionals from a special writing group of the Stroke Council, American Heart Association. Circulation 90:2592–2605PubMedGoogle Scholar
- 15.Jacobi J, Fraser GL, Coursin DB, Riker RR, Fontaine D, Wittbrodt ET, Chalfin DB, Masica MF, Bjerke HS, Coplin WM, Crippen DW, Fuchs BD, Kelleher RM, Marik PE, Nasraway SA, Murray MJ, Peruzzi WT, Lumb PD (2002) Clinical practice guidelines for the sustained used of sedatives and analgesics in the critically ill adult. Crit Care Med 30:119–141CrossRefPubMedGoogle Scholar
- 18.Niederman MS, Craven DE, Bonten MJ, Chastre J, Craig WA, Fagon J, Hall J, Jacoby GA, Kollef MH, Luna CM, Mandell LA, Torres A, Wunderink RG (2005) Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med 171:388–416CrossRefGoogle Scholar
- 19.Mandell LA, Wunderink RG, Anzueto A, Bartlett JG, Campbell GD, Dean NC, Dowell SF, File TM, Musher DM, Niederman MS, Torres A, Whitney CG (2007) Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis 44:S27–S72CrossRefPubMedGoogle Scholar
- 20.Articas A, Bernard GR, Carlet J, Dreyfuss D, Gattinoni L, Hudson L, Lamy M, Marini JJ, Matthay MA, Pinsky MR, Spragg R, Suter PM (1998) The American-European consensus conference on ARDS, part 2. Am J Respir Crit Care Med 157:1332–1347Google Scholar
- 21.Lim WS, Anderson SR, Read RC (2004) Hospital management of adults with severe acute respiratory syndrome (SARS) if SARS re-emerges. British Thoracic Society. http://www.brit-thoracic.org.uk/c2/uploads/sars0304.pdf. Accessed 3 Oct 2007
- 22.Germann P, Braschi A, Della Rocca G, Dinh-Xuan AT, Falke K, Frostell C, Gustafsson LE, Herve P, Jolliet P, Kaisers U, Litvan H, Macrae DJ, Maggiorini M, Marczin N, Mueller B, Payen D, Ranucci M, Schranz D, Zimmermann R, Ullrich R (2005) Inhaled nitric oxide therapy in adults: European expert recommendations. Intensive Care Med 31:1029–1041CrossRefPubMedGoogle Scholar
- 23.Dellinger RP, Levy MM, Carlet JM, Bion J, Parker MM, Jaeschke R, Reinhart K, Angus DC, Brun-Buisson C, Beale R, Calandra T, Dhainaut JF, Gerlach H, Harvey M, Marini JJ, Marshall J, Ranieri M, Ramsay G, Sevransky J, Thompson BT, Townsend S, Vender JS, Zimmerman JL, Vincent JL (2008) Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock: 2008. Intensive Care Med 34:17–60 [Erratum in: Intensive Care Med 2008 34:783–785]Google Scholar
- 24.Hollenberg SM, Ahrens TS, Annane D, Astiz ME, Chalfin DB, Dasta JF, Heard SO, Martin C, Napolitano LM, Susla GM, Totaro R, Vincent JL, Zanotti-Cavazzoni S (2004) Practice parameters for hemodynamic support of sepsis in adult patients: 2004 update. Crit Care Med 32:1928–1948CrossRefPubMedGoogle Scholar
- 25.Martin GS, Matthay MA, Abraham ER, Evans TW, Matuschak GM, Slutsky AS, Bhattacharya J, Dellinger RP, Dematte JE, Matthay MA, Sibbald WJ, Bernard GR, Cook DJ, Dubois M, Martin GS, Stewart TE, Vincent JL (2004) Evidence-based colloid use in the critically ill: American Thoracic Society consensus statement. Am J Respir Crit Care Med 170:1247–1259CrossRefGoogle Scholar
- 26.ECC Committee, Subcommittees and Task Forces of the American Heart Association (2005) American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 112:IV1–IV5, IV58–IV77Google Scholar
- 27.Erstad BL, Grant KL, Boucher BA, Hatton J, Jacobi J, Jones R, Olsen KM, Reines HD, Quinn Wilson A, Dotson Jaggers L (1999) ASHP therapeutic guidelines on stress ulcer prophylaxis. Am J Health Syst Pharm 56:347–379Google Scholar
- 36.Brozek J, Oxman A, Schunemann H (2008) GRADEpro [Computer Program]. Version 3.2 for WindowsGoogle Scholar
- 39.Dupuis LL (2008) Knowledge transfer: adoption and adaptation of clinical practice guidelines. Can J Hosp Pharm 61:177–178Google Scholar
- 40.The ADAPTE Collaboration (2007) ADAPTE http://www.adapte.org/rubrique/the-adapte-collaboration.php. Accessed 15 Aug 2008