Clinical practice guidelines (CPGs) are intended to support clinical decisions and should be based on high-quality evidence. The objective of the study was to evaluate the quality of evidence supporting the recommendations issued in CPGs for therapy, diagnosis, and prevention of hospital-acquired and ventilator-associated pneumonia (HAP/VAP). CPGs released by international scientific societies after year 2000, using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) methodology, were analyzed. Number and strength of recommendations and quality of evidence (high, moderate, low, and very low) were extracted and indexed in the aforementioned sections. High-quality evidence was based on randomized control trials (RCT) without important limitations and exceptionally on rigorous observational studies. Eighty recommendations were assessed, with 7 (8.7%), 24 (30.0%), 29 (36.3%), and 20 (25.0%) being supported by high, moderate, low, and very low-quality evidence, respectively. Highest evidence degree was reported for 26 prevention recommendations, with 7 (26.9%) supported by high-quality evidence and no recommendation based on very low-quality evidence. In contrast, among 9 recommendations for diagnosis and 45 for therapy, none was supported by high-quality evidence, in spite of being recommended as strong in 33.3% and 46.7%, respectively. Among HAP/VAP diagnosis recommendations, the majority of evidence was rated as low or very low-quality (55.6% and 22.2%, respectively) whereas among HAP/VAP therapy recommendations, 4/5 were rated as low and very low-quality (40% each). In conclusion, among HAP/VAP international guidelines, most recommendations, particularly in therapy, remain supported by observational studies, case reports, and expert opinion. Well-designed RCTs are urgently needed.
The first international guidelines on pneumonia were released in 1993 by the American Thoracic Society (ATS), addressing community-acquired pneumonia (CAP), and the first ventilator-associated pneumonia (VAP) and hospital-acquired pneumonia (HAP) guidelines were released later in 1996 by ATS [1, 2]. In Europe, the first guideline on CAP was released in 1998 by the European Respiratory Society (ERS); in 2001, an ERS task force released an expert review on VAP, followed by clinical practice guidelines (CPGs) on HAP/VAP in 2009 [3,4,5]. These early CPGs were based primarily on experts’ opinion, derived from clinical experience. Since the first editions, several versions of the guidelines were released, and throughout the years the infectious diseases societies joined the respiratory, thoracic, and intensive care medicine associations, to release shared documents.
A major change in guideline writing was the need for evidence stratification. In 2004, the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) methodology was released . This method was introduced with the aim of standardizing patient management, guiding everyday clinic decision, and improving the quality of care. Recommendations are based on literature analysis and experts’ opinion and, depending on the quality of the reviewed literature, recommendations with different strength are made. With the GRADE method, the quality of literature can be downgraded, based on the presence of important limitations, inconsistencies or publications bias, or upgraded if a large magnitude effect is recognized [6, 7].
The modern aim towards evidence-based medicine (EBM) should rely on a broad variety of high-quality clinical studies to support the clinician’s decisions and CPG recommendations . Nevertheless several CPGs in different medical fields remain associated with little high-quality evidence, including in infectious diseases [9,10,11,12,13,14,15,16,17]. HAP/VAP guidelines and their quality of evidence have not been analyzed in the past. Our hypothesis was that the majority of recommendations, in spite of efforts to review evidence, are based on moderate/low-quality evidence. The primary objective was to analyze the quality of evidence supporting treatment recommendations in international societies guidelines on HAP/VAP; secondary objectives covered HAP/VAP diagnosis and prevention.
Materials and methods
Guidelines on HAP/VAP released by international societies between the year 2000 and 2019 were identified and downloaded from the societies websites, to identify issued recommendations on treatment, diagnosis, and prevention of HAP/VAP. Inclusion criteria were international guidelines on adult HAP/VAP, with explicit recommendations that used the GRADE method to rate the reviewed evidence for each stated recommendation. Recommendations without evidence rating were excluded, as well as good practice statements and recommendations on ventilator-associated tracheobronchitis (VAT) and for pediatric patients.
A single reviewer (LC) comprehensively reviewed all the documents and collected all the grades of recommendation and quality of evidence clearly displayed in the documents. Other two reviewers (ST and JFC) validated the information.
For each document, reviewers recorded the total number of recommendations and by consensus chose those related with treatment, diagnosis, and prevention of HAP/VAP, when not clearly stated in the original document. For the study purpose, therapy included all treatment recommendations, choice and duration of antibiotic, and the assessment of pneumonia evolution and response to treatment; diagnosis included microbiological tests and serological markers used to diagnose HAP/VAP. Prevention involved indications of infection control in general and specific indication to reduce HAP/VAP incidence. Strength of each issued recommendation and the rating of quality of evidence were also recorded.
The GRADE system rates the quality of evidence in high, moderate, low, and very low, and the strength of recommendations as strong and weak. Grades of recommendation and quality of evidence were extracted as reported in original documents. Randomized controlled trials (RCTs) are the highest quality literature available, given the ability to eliminate confounding factors through the randomization process and analyze cause-effect correlations. GRADE evaluation of quality of evidence is resumed in Fig. 1.
Quality of evidence for each recommendation, classified as high, moderate, low, and very low, was reported for treatment, diagnosis, and prevention. The quality of evidence data was also stratified by strength of recommendation. Data were presented both as absolute numbers and relative proportions.
Ethical approval was not collected, since the study does not involve patients directly.
Three CPGs from different scientific societies were identified and included (Table 1) [18,19,20]. SHEA/IDSA/AHA/APIC guidelines address specifically HAP/VAP prevention. In this CPG, strength of recommendations is not reported, hence only the rating of quality of evidence was analyzed. Strength of recommendations and level of evidence for recommendations specific for each guideline are reported in supplementary e-Table 1.
A total of 127 recommendations were identified. Of these, 38 were excluded because involved pediatric patients, 4 did not report explicit evidence rating, 4 were reported as good practice statements and 1 was on VAT. The flow chart of the recommendation selection, following the four PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) phases , is reported in Fig. 2. Eighty recommendations were finally enclosed and analyzed; strength was graded for 54 of the recommendations. The majority of recommendations were issued on HAP/VAP treatment (n = 45), followed by prevention (n = 26) and diagnosis (n = 9). Overall, 7 recommendations (8.7%) were supported by high-quality evidence, 24 (30.0%) by moderate quality, 29 (36.3%) by low, and 20 (25.0%) by very low–quality evidence (Table 2). Among 24 strong recommendations (44.4%), only 9 (37.5%) were supported by moderate quality of evidence. Low and very low quality of evidence supported comparable proportion of strong evidence (33.3% and 29.2%, respectively) (Table 3).
Forty-five recommendations were abstracted for HAP/VAP treatment. These were equally based on low and very low quality of evidence (40% each, n = 18); none was supported by high-quality evidence (Table 2). The majority of recommendations were issued as weak (53.3%), based mostly on low and very low quality of evidence (45.8% each). Strong recommendations (46.7%) were supported by moderate quality of evidence in only 1/3 of issued recommendations (Table 3).
For HAP/VAP diagnosis, a total of 9 recommendations were recorded. The majority was based on low quality of evidence (55.6%, n = 5); none was supported by high-quality evidence (Table 2). The majority of recommendations were issued as weak (66.7%). Among strong recommendations (33.3%), 2/3 were supported by moderate quality of evidence (Table 3).
HAP/VAP prevention recommendations (n = 26) were based on high-quality evidence in 29.6% of cases. The majority of evidence was rated moderate quality (50%); no recommendation was based on very low quality of evidence. Strength of recommendations was not reported (Table 2).
Overall, less than 10% of all recommendations were supported by high-quality evidence, the majority were based on low-quality evidence (36.3%), despite being graded strong in 44.4%. For HAP/VAP treatment and diagnosis, none of the recommendations issued was based on high-quality evidence; the majority of evidence was rated low and very low. Nonetheless, strong recommendations were 46.7% in treatment and 33.3% in diagnosis, supported by moderate quality of evidence in 1/3 and 2/3, respectively. A summary of the results is detailed in Fig. 3.
In current HAP and VAP international guidelines, less than 10% of recommendations were supported by high-quality evidence. Low and very low evidence predominate in HAP/VAP treatment and diagnosis recommendations. Guidelines are redacted by groups of experts, who can give strong recommendations based on personal opinions, not only on literature evidence. The major implication is that further randomized controlled trials are an unmet clinical need, urgently required.
With the GRADE methodology, high-quality evidence is based on RCTs without major limitations, and exceptionally on observational studies with large magnitude effect [6, 7]. In the infectious diseases field, with rare diseases and heterogeneous clinical presentations, conducting RCTs might be difficult, lowering the overall quality of evidence. A study conducted on IDSA guidelines from 1994 to 2010 found that only 14% of the recommendations were linked to high-quality evidence (level of evidence A in the original article, based on RCTs), while the majority of the recommendations relied on expert opinion and case studies . In HAP and VAP, despite being common hospital-acquired infections and a major concern for morbidity and mortality in intensive care unit [18, 19, 22], the involvement of critically ill patients might represent a limitation for conducting interventional studies and to generalize conclusions from observational cohorts in non-critically ill patients. Individual recommendations issued in guidelines may not always be supported by high-quality evidence, and evidence-based medicine practice should be based on profound knowledge of CPGs and literature, to properly evaluate bedside patient management.
Feasibility of interventional studies has also to be considered, together with ethical implications, appropriateness of the study design, practical limitations, and costs. The majority of recommendations were issued on HAP/VAP treatment. In HAP and VAP, often caused by multidrug-resistant pathogens, the delay in appropriate antimicrobial treatment represents a poor prognostic factor and is mainly driven by the pathogen extended resistance pattern itself [23,24,25,26,27]. Appropriate antibiotic treatment is hence pivotal and challenging. The majority of treatment recommendations regarded antibiotic choice and duration (data not shown), addressing key bedside clinical decisions. Nonetheless, for HAP/VAP treatment, none of the recommendations issued was based on high-quality evidence. The majority of evidence was rated low and very low, and only a third of strong recommendations was supported by moderate quality of evidence. The lack of a gold standard for HAP/VAP diagnosis and the geographical and in-hospital variability of etiology and antimicrobial susceptibility are partly responsible for the lack of high-quality evidence supporting these practices (Table 4). Interventional studies on general hygiene measures, effectiveness of epidemiological surveillance, and collateral care, such as bed positioning, might be performed with less difficulties, explaining the greater quantity of high-quality evidence recorded in HAP/VAP prevention. These data point out the need of collaboration to realize large RCTs, increasing the quality of evidence.
The revision of clinical guidelines in several medical fields, such as cardiology, emergency medicine, gastroenterology, interventional medicine, hepatology, and nephrology, reported similar findings, with an overall scarcity of high-quality evidence [9,10,11,12,13,14,15, 17, 28, 29]. This low amount of high-quality evidence represents a call for collaboration to plan large well-conducted new research and simultaneously shows the importance of expert opinion in particular fields.
In the EBM era, CPGs represent an important and valuable tool in bedside clinical decisions and for patient management. Our study results reinforce the importance of literature knowledge to accurately translate CPGs and their recommendations in clinical practice and represent a call of wide collaboration to realize well-designed research in treatment, diagnosis, and prevention of HAP and VAP. Importantly, some clinical questions cannot be answered with high-quality evidence, because the clinical questions themselves are too complex and the antibiotic resistance profiles are profoundly different between sites and keep evolving. Furthermore, among the earliest HAP/VAP guidelines in the pre-GRADE era, with the possible exception of single antibiotic coverage for patients with Pseudomonas aeruginosa and shock absence, few recommendations based on lower grade of evidence were revised by subsequent studies. This implies the pivotal importance of high-quality studies in areas where there is a high degree of uncertainty, with potential adverse implications.
Several limitations can be found in the study. First, the more recent assessed guideline was written in 2017, including literature published until September 2016. Since then, new evidence might have been published on the addressed topic. Most CPGs are updated in periods longer than 5 years, with difficult incorporation of emerging evidence until next iteration. Published literature should be periodically reassessed, to update CPGs references and recommendations, creating a dynamic document. The creation of an adaptation framework for integration of new evidence in CPGs  would avoid outdated recommendations and will improve implementation in clinical practice. Second, literature supporting each recommendation was not revised and quality of evidence was not reassessed. Evidence and recommendation rating analyses were based on the classification reported in original papers, with no changes in the rating assigned to the literature by CPGs writing committees. Lastly, CPGs on HAP/VAP issued by other scientific societies without explicitly reported recommendations or quality of evidence assessment were excluded from the analysis. Despite these limitations, there are important strengths. The study provides a detailed analysis of quality of evidence used to write HAP/VAP international guidelines, an important evaluation in evidence-based medicine. The results point out the difficulty of performing RCTs and providing high-quality evidence in critically ill patients. Areas in which higher quality evidence are scarce reveal research opportunities, to assess unmet clinical needs in HAP/VAP management.
In the more recent guidelines on HAP/VAP treatment, diagnosis, and prevention, only a minority of issued recommendations are supported by high-quality evidence. Most recommendations, particularly related to treatment, remain supported by observational studies, case reports, and expert opinion. These data stress the need to direct efforts towards planning large, well-conducted, multicentric randomized controlled trials on HAP and VAP. Research areas that need further study are summarized in Table 5.
American Hospital Association
Asociacion Latinoamericana del Torax
Association for Professionals in Infection Control and Epidemiology
American Thoracic Association
Clinical Practice Guideline
European Respiratory Society
European Society of Clinical Microbiology and Infectious Diseases
European Society of Intensive Care Medicine
Grades of Recommendation, Assessment, Development, and Evaluation
Infectious Diseases Society of America
Preferred Reporting Items for Systematic reviews and Meta-Analysis
Randomized Controlled Trial
Society of Healthcare Epidemiology of America
Niederman MS, Bass JB, Campbell GD et al (1993) Guidelines for the initial management of adults with community-acquired pneumonia: diagnosis, assessment of severity, and initial antimicrobial therapy. American Thoracic Society. Medical section of the American Lung Association. Am Rev Respir Dis 148(5):1418–1426
Hospital-acquired pneumonia in adults: diagnosis, assessment of severity, initial antimicrobial therapy, and preventive strategies. A consensus statement, American Thoracic Society, November 1995. Am J Respir Crit Care Med. 1996;153(5):1711–25
Society ER (1998) ERS task force report. Guidelines for management of adult community-acquired lower respiratory tract infections. European Respiratory Society. Eur Respir J 11(4):986–991
Members of the Task Force, Bouza E, Brun-Buisson C, Chastre J et al (2001) Ventilator-associated pneumonia: European Task Force on ventilator-associated pneumonia Chairmen of the Task Force: A. Torres and J. Carlet. Eur Respir J 17(5):1034–1045
Torres A, Ewig S, Lode H, Carlet J (2009) For the European HAP working group. Defining, treating and preventing hospital acquired pneumonia: European perspective. Intensive Care Med 35(1):9–29
Atkins D, Best D, Briss PA et al (2004) Grading quality of evidence and strength of recommendations. BMJ 328(7454):1490
Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ et al (2008) What is ‘quality of evidence’ and why is it important to clinicians? BMJ 336(7651):995–998
Evidence-Based Medicine Working Group (1992) Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 268(17):2420–2425
Fanaroff AC, Califf RM, Windecker S, Smith SC, Lopes RD (2019) Levels of evidence supporting American College of Cardiology/American Heart Association and European Society of Cardiology Guidelines, 2008-2018. JAMA 321(11):1069–1080
Schumacher RC, Nguyen OK, Desphande K, Makam AN (2019) Evidence-based medicine and the American Thoracic Society clinical practice guidelines. JAMA Intern Med
Meyer C, Bowers A, Wayant C, Checketts J, Scott J, Musuvathy S, Vassar M (2018) Scientific evidence underlying the American College of Gastroenterology’s clinical practice guidelines. PLoS One 13(10):e0204720
Venkatesh AK, Savage D, Sandefur B, Bernard KR, Rothenberg C, Schuur JD (2017) Systematic review of emergency medicine clinical practice guidelines: implications for research and policy. PLoS One 12(6):e0178456
Alseiari M, Meyer KB, Wong JB (2016) Evidence underlying KDIGO (kidney disease: improving global outcomes) guideline recommendations: a systematic review. Am J Kidney Dis Off J Natl Kidney Found 67(3):417–422
Koh C, Zhao X, Samala N, Sakiani S, Liang TJ, Talwalkar JA (2013) AASLD clinical practice guidelines: a critical review of scientific evidence and evolving recommendations. Hepatol Baltim Md 58(6):2142–2152
Poonacha TK, Go RS (2011) Level of scientific evidence underlying recommendations arising from the National Comprehensive Cancer Network clinical practice guidelines. J Clin Oncol Off J Am Soc Clin Oncol 29(2):186–191
Khan AR, Khan S, Zimmerman V, Baddour LM, Tleyjeh IM (2010) Quality and strength of evidence of the Infectious Diseases Society of America clinical practice guidelines. Clin Infect Dis Off Publ Infect Dis Soc Am 51(10):1147–1156
Lee DH, Vielemeyer O (2011) Analysis of overall level of evidence behind Infectious Diseases Society of America practice guidelines. Arch Intern Med 171(1):18–22
Torres A, Niederman MS, Chastre J, Ewig S, Fernandez-Vandellos P, Hanberger H et al (2017) International ERS/ESICM/ESCMID/ALAT guidelines for the management of hospital-acquired pneumonia and ventilator-associated pneumonia: Guidelines for the management of hospital-acquired pneumonia (HAP)/ventilator-associated pneumonia (VAP) of the European Respiratory Society (ERS), European Society of Intensive Care Medicine (ESICM), European Society of Clinical Microbiology and Infectious Diseases (ESCMID) and Asociación Latinoamericana del Tórax (ALAT). Eur Respir J 50(3)
Kalil AC, Metersky ML, Klompas M, Muscedere J, Sweeney DA, Palmer LB et al (2016) Management of Adults with Hospital-acquired and Ventilator-associated Pneumonia: 2016 clinical practice guidelines by the Infectious Diseases Society of America and the American Thoracic Society. Clin Infect Dis Off Publ Infect Dis Soc Am 63(5):e61–e111
Klompas M, Branson R, Eichenwald EC, Greene LR, Howell MD, Lee G et al (2014) Strategies to prevent ventilator-associated pneumonia in acute care hospitals: 2014 update. Infect Control Hosp Epidemiol 35(8):915–936
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 339:b2700
Borgatta B, Rello J (2014) How to approach and treat VAP in ICU patients. BMC Infect Dis 14:211
Luna CM, Vujacich P, Niederman MS, Vay C, Gherardi C, Matera J et al (1997) Impact of BAL data on the therapy and outcome of ventilator-associated pneumonia. Chest. 111(3):676–685
Kumar A, Zarychanski R, Light B, Parrillo J, Maki D, Simon D et al (2010) Early combination antibiotic therapy yields improved survival compared with monotherapy in septic shock: a propensity-matched analysis. Crit Care Med 38(9):1773–1785
Kuti EL, Patel AA, Coleman CI (2008) Impact of inappropriate antibiotic therapy on mortality in patients with ventilator-associated pneumonia and blood stream infection: a meta-analysis. J Crit Care 23(1):91–100
Muscedere JG, Shorr AF, Jiang X, Day A, Heyland DK, Canadian Critical Care Trials Group (2012) The adequacy of timely empiric antibiotic therapy for ventilator-associated pneumonia: an important determinant of outcome. J Crit Care 27(3):322.e7–322.14
Rello J, Ulldemolins M, Lisboa T, Koulenti D, Mañez R, Martin-Loeches I et al (2011) Determinants of prescription and choice of empirical therapy for hospital-acquired and ventilator-associated pneumonia. Eur Respir J 37(6):1332–1339
Feuerstein JD, Akbari M, Gifford AE, Hurley CM, Leffler DA, Sheth SG et al (2014) Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in interventional medicine subspecialty guidelines. Mayo Clin Proc 89(1):16–24
Lorente L, Blot S, Rello J (2007) Evidence on measures for the prevention of ventilator-associated pneumonia. Eur Respir J 30(6):1193–1207
Alhazzani W, Møller MH, Belley-Cote E, Citerio G (2019) Intensive care medicine rapid practice guidelines (ICM-RPG): paving the road of the future. Intensive Care Med
This study is supported in part by CIBERES, Instituto Salud Carlos III as part of the PCI of pneumonia (CB06-06-036).
JR served on speakers’ bureau and consultant for Pfizer and Astellas.
MIR served as co-author in redacting IDSA/ATS guidelines on HAP/VAP (2016).
The remaining authors, LC, ST, JFC, declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Campogiani, L., Tejada, S., Ferreira-Coimbra, J. et al. Evidence supporting recommendations from international guidelines on treatment, diagnosis, and prevention of HAP and VAP in adults. Eur J Clin Microbiol Infect Dis 39, 483–491 (2020). https://doi.org/10.1007/s10096-019-03748-z
- Clinical practice guidelines
- Hospital-acquired pneumonia
- Ventilator-associated pneumonia
- Quality of evidence