Intensive Care Medicine

, Volume 36, Issue 10, pp 1636–1643

A critical appraisal of the quality of critical care pharmacotherapy clinical practice guidelines and their strength of recommendations

  • Sean K. Gorman
  • Michelle Ho Chung
  • Richard S. Slavik
  • Peter J. Zed
  • Kerry Wilbur
  • Vinay K. Dhingra
Review

Abstract

Objective

Clinical practice guideline (CPG) quality assessment is important before applying their recommendations. Determining whether recommendation strength is consistent with supporting quality of evidence is also essential. We aimed to determine quality of critical care pharmacotherapy CPGs and to assess whether high quality evidence supports strong pharmacotherapy recommendations.

Methods

MEDLINE (1966–February 2008), EMBASE (1980–February 2008), National Guideline Clearinghouse (February 2008) and personal files were searched to identify CPGs. Four appraisers evaluated each guideline using the appraisal of guidelines, research and evaluation (AGREE) instrument. AGREE assesses 23 items in six domains that include scope/purpose, stakeholder involvement, rigor of development, clarity, applicability and editorial independence. Standardized domain scores (0–100%) were determined to decide whether to recommend a guideline for use. One appraiser extracted strong pharmacotherapy recommendations and supporting evidence quality.

Results

Twenty-four CPGs were included. Standardized domain scores were clarity [69% (95% confidence interval (CI) 62–76%)], scope/purpose [62% (95% CI 55–68%)], rigor of development [51% (95% CI 42–60%)], editorial independence [39% (95% CI 26–52%)], stakeholder involvement [32% (95% CI 26–37%)] and applicability [19% (95% CI 12–26%)]. The proportion of guidelines that could be strongly recommended, recommended with alterations and not recommended was 25, 37.5 and 37.5%, respectively. High quality evidence supported 36% of strong pharmacotherapy recommendations.

Conclusion

Variation in AGREE domain scores explain why one-third of critical care pharmacotherapy CPGs cannot be recommended. Only one-third of strong pharmacotherapy recommendations were supported by high quality evidence. We recommend appraisal of guideline quality and the caliber of supporting evidence prior to applying recommendations.

Keywords

Clinical practice guideline Critical appraisal Quality of evidence Critical care Pharmacotherapy 

Supplementary material

134_2010_1786_MOESM1_ESM.doc (34 kb)
Supplementary material 1 (DOC 34 kb)

References

  1. 1.
    Field MJ, Lohr KN (1992) Guidelines for clinical practice: from development to use. National Academy Press, WashingtonGoogle Scholar
  2. 2.
    Gundersen L (2000) The effect of clinical practice guidelines on variations in care. Ann Intern Med 133:317–318PubMedGoogle Scholar
  3. 3.
    Vlayen J, Aertgeerts B, Hannes K, Sermeus W, Ramaekers D (2005) A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. Int J Qual Health Care 17:235–242CrossRefPubMedGoogle Scholar
  4. 4.
    Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC (2009) Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 301:831–841CrossRefPubMedGoogle Scholar
  5. 5.
    The AGREE Collaboration (2001) Appraisal of guidelines for research and evaluation (AGREE) instrument. http://www.agreecollaboration.org/pdf/agreeinstrumentfinal.pdf. Accessed 23 Jul 2007
  6. 6.
    The AGREE Collaboration (2003) Appraisal of guidelines for research and evaluation (AGREE) training manual. http://www.agreecollaboration.org/pdf/aitraining.pdf. Accessed 23 Jul 2007
  7. 7.
    Oxford Centre for Evidence Based Medicine (2009) Levels of evidence. http://www.cebm.net/index.aspx?o=1025. Accessed 8 Jul 2009
  8. 8.
    The AGREE Collaboration (2003) Development and validation of an international appraisal instrument for assessing the quality of guidelines: the AGREE project. Qual Saf Health Care 12:18–23CrossRefGoogle Scholar
  9. 9.
    Anonymous (2001) Guidelines for the management of penetrating brain injury. J Trauma 51:S3–S6, S34–S43Google Scholar
  10. 10.
    Brain Trauma Foundation (2007) Guidelines for the management of severe traumatic brain injury, 3rd edn. J Neurotrauma 24 Suppl 1:S1–S106Google Scholar
  11. 11.
    Bromberg WJ, Collier B, Diebel L, Dwyer K, Holevar M, Jacobs D, Kurek S, Schreiber M, Shapiro M, Vogel T (2007) Blunt cerebrovascular injury practice management guidelines. The Eastern Association for the Surgery of Trauma. http://www.east.org/tpg/BluntCVInjury.pdf. Accessed 3 Oct 2007
  12. 12.
    Mayberg MR, Batjer HH, Dacey R, Diringer M, Haley EC, Heros RC, Sternau LL, Torner J, Adams HP, Feinberg W (1994) Guidelines for the management of aneurysmal subarachnoid hemorrhage. A statement for healthcare professionals from a special writing group of the Stroke Council, American Heart Association. Circulation 90:2592–2605PubMedGoogle Scholar
  13. 13.
    Roderick J, Connolly S, Feldmann E, Hanley D, Kase C, Krieger D, Mayberg M, Morgenstern L, Ogilvy CS, Vespa P, Zuccarello M (2007) Guidelines for the management of spontaneous intracerebral hemorrhage in adults. Stroke 38:2001–2023CrossRefGoogle Scholar
  14. 14.
    Meierkord H, Boon P, Engelsen B, Gocke K, Shorvon S, Tinuper P, Holtkamp M (2006) EFNS guideline on the management of status epilepticus. Eur J Neurology 13:445–450CrossRefGoogle Scholar
  15. 15.
    Jacobi J, Fraser GL, Coursin DB, Riker RR, Fontaine D, Wittbrodt ET, Chalfin DB, Masica MF, Bjerke HS, Coplin WM, Crippen DW, Fuchs BD, Kelleher RM, Marik PE, Nasraway SA, Murray MJ, Peruzzi WT, Lumb PD (2002) Clinical practice guidelines for the sustained used of sedatives and analgesics in the critically ill adult. Crit Care Med 30:119–141CrossRefPubMedGoogle Scholar
  16. 16.
    Murray MJ, Cowen J, DeBlock H, Erstad B, Gray AW, Tescher AN, McGee WT, Prielipp RC, Susla G, Jacobi J, Nasraway SA, Lumb PD (2002) Clinical practice guidelines for sustained neuromuscular blockade in the adult critically ill patient. Crit Care Med 30:142–156CrossRefPubMedGoogle Scholar
  17. 17.
    Dodek P, Keenan S, Cook D, Heyland D, Jacka M, Hand L, Muscedere J, Foster D, Mehta N, Hall R, Brun-Buisson C (2004) Evidence-based clinical practice guideline for the prevention of ventilator-associated pneumonia. Ann Intern Med 141:305–313PubMedGoogle Scholar
  18. 18.
    Niederman MS, Craven DE, Bonten MJ, Chastre J, Craig WA, Fagon J, Hall J, Jacoby GA, Kollef MH, Luna CM, Mandell LA, Torres A, Wunderink RG (2005) Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med 171:388–416CrossRefGoogle Scholar
  19. 19.
    Mandell LA, Wunderink RG, Anzueto A, Bartlett JG, Campbell GD, Dean NC, Dowell SF, File TM, Musher DM, Niederman MS, Torres A, Whitney CG (2007) Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis 44:S27–S72CrossRefPubMedGoogle Scholar
  20. 20.
    Articas A, Bernard GR, Carlet J, Dreyfuss D, Gattinoni L, Hudson L, Lamy M, Marini JJ, Matthay MA, Pinsky MR, Spragg R, Suter PM (1998) The American-European consensus conference on ARDS, part 2. Am J Respir Crit Care Med 157:1332–1347Google Scholar
  21. 21.
    Lim WS, Anderson SR, Read RC (2004) Hospital management of adults with severe acute respiratory syndrome (SARS) if SARS re-emerges. British Thoracic Society. http://www.brit-thoracic.org.uk/c2/uploads/sars0304.pdf. Accessed 3 Oct 2007
  22. 22.
    Germann P, Braschi A, Della Rocca G, Dinh-Xuan AT, Falke K, Frostell C, Gustafsson LE, Herve P, Jolliet P, Kaisers U, Litvan H, Macrae DJ, Maggiorini M, Marczin N, Mueller B, Payen D, Ranucci M, Schranz D, Zimmermann R, Ullrich R (2005) Inhaled nitric oxide therapy in adults: European expert recommendations. Intensive Care Med 31:1029–1041CrossRefPubMedGoogle Scholar
  23. 23.
    Dellinger RP, Levy MM, Carlet JM, Bion J, Parker MM, Jaeschke R, Reinhart K, Angus DC, Brun-Buisson C, Beale R, Calandra T, Dhainaut JF, Gerlach H, Harvey M, Marini JJ, Marshall J, Ranieri M, Ramsay G, Sevransky J, Thompson BT, Townsend S, Vender JS, Zimmerman JL, Vincent JL (2008) Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock: 2008. Intensive Care Med 34:17–60 [Erratum in: Intensive Care Med 2008 34:783–785]Google Scholar
  24. 24.
    Hollenberg SM, Ahrens TS, Annane D, Astiz ME, Chalfin DB, Dasta JF, Heard SO, Martin C, Napolitano LM, Susla GM, Totaro R, Vincent JL, Zanotti-Cavazzoni S (2004) Practice parameters for hemodynamic support of sepsis in adult patients: 2004 update. Crit Care Med 32:1928–1948CrossRefPubMedGoogle Scholar
  25. 25.
    Martin GS, Matthay MA, Abraham ER, Evans TW, Matuschak GM, Slutsky AS, Bhattacharya J, Dellinger RP, Dematte JE, Matthay MA, Sibbald WJ, Bernard GR, Cook DJ, Dubois M, Martin GS, Stewart TE, Vincent JL (2004) Evidence-based colloid use in the critically ill: American Thoracic Society consensus statement. Am J Respir Crit Care Med 170:1247–1259CrossRefGoogle Scholar
  26. 26.
    ECC Committee, Subcommittees and Task Forces of the American Heart Association (2005) American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 112:IV1–IV5, IV58–IV77Google Scholar
  27. 27.
    Erstad BL, Grant KL, Boucher BA, Hatton J, Jacobi J, Jones R, Olsen KM, Reines HD, Quinn Wilson A, Dotson Jaggers L (1999) ASHP therapeutic guidelines on stress ulcer prophylaxis. Am J Health Syst Pharm 56:347–379Google Scholar
  28. 28.
    Solomkin JS, Mazuski JE, Baron EJ, Sawyer RG, Nathens AB, DiPiro JT, Buchman T, Dellinger EP, Jernigan J, Gorbach S, Chow AW, Bartlett J (2003) Guidelines for the selection of anti-infective agents for complicated intra-abdominal infections. Clin Infect Dis 37:997–1005CrossRefPubMedGoogle Scholar
  29. 29.
    Mazuski JE, Sawyer RG, Nathens AB, Dipiro JT, Schein M, Kudsk KA, Yowler C (2002) The Surgical Infection Society guidelines on antimicrobial therapy for intra-abdominal infections: an executive summary. Surg Infect 3:161–173CrossRefGoogle Scholar
  30. 30.
    Nathens AB, Curtis JR, Beale RJ, Cook DJ, Moreno RP, Romand JA, Skerrett SJ, Stapleton RD, Ware LB, Waldmann CS (2004) Management of the critically ill patient with severe acute pancreatitis. Crit Care Med 32:2524–2536CrossRefPubMedGoogle Scholar
  31. 31.
    O’Grady NP, Alexander M, Dellinger EP, Gerberding JL, Heard SO, Maki DG, Masur H, McCormick RD, Mermel LA, Pearson ML, Raad II, Randolph A, Weinstein RA (2002) Guidelines for the prevention of intravascular catheter-related infections. Clin Infect Dis 35:1281–1307CrossRefGoogle Scholar
  32. 32.
    Mermel LA, Farr BM, Sherertz RJ, Raad II, O’Grady N, Harris JS, Craven DE (2001) Guidelines for the management of intravascular catheter-related infections. Clin Infect Dis 32:1249–1272CrossRefPubMedGoogle Scholar
  33. 33.
    Sinuff T, Patel RV, Adhikari NK, Meade MO, Schunemann HJ, Cook DJ (2008) Quality of professional society guidelines and consensus conference statements in critical care. Crit Care Med 36:1049–1058CrossRefPubMedGoogle Scholar
  34. 34.
    Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ (2008) What is the “quality of evidence” and why is it important to clinicians? BMJ 336:995–998CrossRefPubMedGoogle Scholar
  35. 35.
    Upshur REG (2003) Are all evidence-based practices alike? Problems in ranking of evidence. CMAJ 169:672–673PubMedGoogle Scholar
  36. 36.
    Brozek J, Oxman A, Schunemann H (2008) GRADEpro [Computer Program]. Version 3.2 for WindowsGoogle Scholar
  37. 37.
    Shaneyfelt TM, Centor RM (2009) Reassessment of clinical practice guidelines: go gently into that good night. JAMA 301:868–869CrossRefPubMedGoogle Scholar
  38. 38.
    Cates JR, Young DN, Bowerman DS, Porter RC (2006) An independent AGREE evaluation of the occupational medicine practice guidelines. Spine J 6:72–77CrossRefPubMedGoogle Scholar
  39. 39.
    Dupuis LL (2008) Knowledge transfer: adoption and adaptation of clinical practice guidelines. Can J Hosp Pharm 61:177–178Google Scholar
  40. 40.
    The ADAPTE Collaboration (2007) ADAPTE http://www.adapte.org/rubrique/the-adapte-collaboration.php. Accessed 15 Aug 2008

Copyright information

© Copyright jointly held by Springer and ESICM 2010

Authors and Affiliations

  • Sean K. Gorman
    • 1
  • Michelle Ho Chung
    • 2
  • Richard S. Slavik
    • 3
  • Peter J. Zed
    • 1
    • 4
  • Kerry Wilbur
    • 5
  • Vinay K. Dhingra
    • 6
    • 7
  1. 1.Department of Pharmacy, Capital District Health Authority, College of PharmacyDalhousie UniversityHalifaxCanada
  2. 2.Department of PharmacySt. Paul’s HospitalVancouverCanada
  3. 3.Pharmacy Department, Interior Health Authority, Faculty of Pharmaceutical SciencesUniversity of British ColumbiaKelownaCanada
  4. 4.Department of Emergency MedicineDalhousie UniversityHalifaxCanada
  5. 5.College of PharmacyQatar UniversityDohaQatar
  6. 6.Intensive Care UnitVancouver General HospitalVancouverCanada
  7. 7.Faculty of MedicineUniversity of British ColumbiaVancouverCanada

Personalised recommendations