Introduction

The first international guidelines on pneumonia were released in 1993 by the American Thoracic Society (ATS), addressing community-acquired pneumonia (CAP), and the first ventilator-associated pneumonia (VAP) and hospital-acquired pneumonia (HAP) guidelines were released later in 1996 by ATS [1, 2]. In Europe, the first guideline on CAP was released in 1998 by the European Respiratory Society (ERS); in 2001, an ERS task force released an expert review on VAP, followed by clinical practice guidelines (CPGs) on HAP/VAP in 2009 [3,4,5]. These early CPGs were based primarily on experts’ opinion, derived from clinical experience. Since the first editions, several versions of the guidelines were released, and throughout the years the infectious diseases societies joined the respiratory, thoracic, and intensive care medicine associations, to release shared documents.

A major change in guideline writing was the need for evidence stratification. In 2004, the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) methodology was released [6]. This method was introduced with the aim of standardizing patient management, guiding everyday clinic decision, and improving the quality of care. Recommendations are based on literature analysis and experts’ opinion and, depending on the quality of the reviewed literature, recommendations with different strength are made. With the GRADE method, the quality of literature can be downgraded, based on the presence of important limitations, inconsistencies or publications bias, or upgraded if a large magnitude effect is recognized [6, 7].

The modern aim towards evidence-based medicine (EBM) should rely on a broad variety of high-quality clinical studies to support the clinician’s decisions and CPG recommendations [8]. Nevertheless several CPGs in different medical fields remain associated with little high-quality evidence, including in infectious diseases [9,10,11,12,13,14,15,16,17]. HAP/VAP guidelines and their quality of evidence have not been analyzed in the past. Our hypothesis was that the majority of recommendations, in spite of efforts to review evidence, are based on moderate/low-quality evidence. The primary objective was to analyze the quality of evidence supporting treatment recommendations in international societies guidelines on HAP/VAP; secondary objectives covered HAP/VAP diagnosis and prevention.

Materials and methods

Guidelines on HAP/VAP released by international societies between the year 2000 and 2019 were identified and downloaded from the societies websites, to identify issued recommendations on treatment, diagnosis, and prevention of HAP/VAP. Inclusion criteria were international guidelines on adult HAP/VAP, with explicit recommendations that used the GRADE method to rate the reviewed evidence for each stated recommendation. Recommendations without evidence rating were excluded, as well as good practice statements and recommendations on ventilator-associated tracheobronchitis (VAT) and for pediatric patients.

A single reviewer (LC) comprehensively reviewed all the documents and collected all the grades of recommendation and quality of evidence clearly displayed in the documents. Other two reviewers (ST and JFC) validated the information.

For each document, reviewers recorded the total number of recommendations and by consensus chose those related with treatment, diagnosis, and prevention of HAP/VAP, when not clearly stated in the original document. For the study purpose, therapy included all treatment recommendations, choice and duration of antibiotic, and the assessment of pneumonia evolution and response to treatment; diagnosis included microbiological tests and serological markers used to diagnose HAP/VAP. Prevention involved indications of infection control in general and specific indication to reduce HAP/VAP incidence. Strength of each issued recommendation and the rating of quality of evidence were also recorded.

The GRADE system rates the quality of evidence in high, moderate, low, and very low, and the strength of recommendations as strong and weak. Grades of recommendation and quality of evidence were extracted as reported in original documents. Randomized controlled trials (RCTs) are the highest quality literature available, given the ability to eliminate confounding factors through the randomization process and analyze cause-effect correlations. GRADE evaluation of quality of evidence is resumed in Fig. 1.

Fig. 1
figure 1

Quality of evidence assessment, based on GRADE methodology [6, 7]

Full size image

Data presentation

Quality of evidence for each recommendation, classified as high, moderate, low, and very low, was reported for treatment, diagnosis, and prevention. The quality of evidence data was also stratified by strength of recommendation. Data were presented both as absolute numbers and relative proportions.

Ethical approval was not collected, since the study does not involve patients directly.

Results

Three CPGs from different scientific societies were identified and included (Table 1) [18,19,20]. SHEA/IDSA/AHA/APIC guidelines address specifically HAP/VAP prevention. In this CPG, strength of recommendations is not reported, hence only the rating of quality of evidence was analyzed. Strength of recommendations and level of evidence for recommendations specific for each guideline are reported in supplementary e-Table 1.

Table 1 Included clinical practice guidelines and number of total, excluded, and included recommendations for each CPGs
Full size table

A total of 127 recommendations were identified. Of these, 38 were excluded because involved pediatric patients, 4 did not report explicit evidence rating, 4 were reported as good practice statements and 1 was on VAT. The flow chart of the recommendation selection, following the four PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) phases [21], is reported in Fig. 2. Eighty recommendations were finally enclosed and analyzed; strength was graded for 54 of the recommendations. The majority of recommendations were issued on HAP/VAP treatment (n = 45), followed by prevention (n = 26) and diagnosis (n = 9). Overall, 7 recommendations (8.7%) were supported by high-quality evidence, 24 (30.0%) by moderate quality, 29 (36.3%) by low, and 20 (25.0%) by very low–quality evidence (Table 2). Among 24 strong recommendations (44.4%), only 9 (37.5%) were supported by moderate quality of evidence. Low and very low quality of evidence supported comparable proportion of strong evidence (33.3% and 29.2%, respectively) (Table 3).

Fig. 2
figure 2

Recommendation selection process from the included CPGs, flow chart following the four PRISMA phases

Full size image
Table 2 Proportion of high, moderate, low, and very low quality of evidence and of strength of recommendation, overall and for HAP/VAP treatment, diagnosis, and prevention. Data shown as number (%)
Full size table
Table 3 Proportion of strong and weak recommendations stratified by quality of evidence high, moderate, low, and very low
Full size table

Forty-five recommendations were abstracted for HAP/VAP treatment. These were equally based on low and very low quality of evidence (40% each, n = 18); none was supported by high-quality evidence (Table 2). The majority of recommendations were issued as weak (53.3%), based mostly on low and very low quality of evidence (45.8% each). Strong recommendations (46.7%) were supported by moderate quality of evidence in only 1/3 of issued recommendations (Table 3).

For HAP/VAP diagnosis, a total of 9 recommendations were recorded. The majority was based on low quality of evidence (55.6%, n = 5); none was supported by high-quality evidence (Table 2). The majority of recommendations were issued as weak (66.7%). Among strong recommendations (33.3%), 2/3 were supported by moderate quality of evidence (Table 3).

HAP/VAP prevention recommendations (n = 26) were based on high-quality evidence in 29.6% of cases. The majority of evidence was rated moderate quality (50%); no recommendation was based on very low quality of evidence. Strength of recommendations was not reported (Table 2).

Overall, less than 10% of all recommendations were supported by high-quality evidence, the majority were based on low-quality evidence (36.3%), despite being graded strong in 44.4%. For HAP/VAP treatment and diagnosis, none of the recommendations issued was based on high-quality evidence; the majority of evidence was rated low and very low. Nonetheless, strong recommendations were 46.7% in treatment and 33.3% in diagnosis, supported by moderate quality of evidence in 1/3 and 2/3, respectively. A summary of the results is detailed in Fig. 3.

Figure. 3
figure 3

Proportion of strong and weak recommendations, overall and for treatment and diagnosis, stratified by quality of evidence (high, moderate, low, and very low). Recommendations on prevention were excluded

Full size image

Discussion

In current HAP and VAP international guidelines, less than 10% of recommendations were supported by high-quality evidence. Low and very low evidence predominate in HAP/VAP treatment and diagnosis recommendations. Guidelines are redacted by groups of experts, who can give strong recommendations based on personal opinions, not only on literature evidence. The major implication is that further randomized controlled trials are an unmet clinical need, urgently required.

With the GRADE methodology, high-quality evidence is based on RCTs without major limitations, and exceptionally on observational studies with large magnitude effect [6, 7]. In the infectious diseases field, with rare diseases and heterogeneous clinical presentations, conducting RCTs might be difficult, lowering the overall quality of evidence. A study conducted on IDSA guidelines from 1994 to 2010 found that only 14% of the recommendations were linked to high-quality evidence (level of evidence A in the original article, based on RCTs), while the majority of the recommendations relied on expert opinion and case studies [17]. In HAP and VAP, despite being common hospital-acquired infections and a major concern for morbidity and mortality in intensive care unit [18, 19, 22], the involvement of critically ill patients might represent a limitation for conducting interventional studies and to generalize conclusions from observational cohorts in non-critically ill patients. Individual recommendations issued in guidelines may not always be supported by high-quality evidence, and evidence-based medicine practice should be based on profound knowledge of CPGs and literature, to properly evaluate bedside patient management.

Feasibility of interventional studies has also to be considered, together with ethical implications, appropriateness of the study design, practical limitations, and costs. The majority of recommendations were issued on HAP/VAP treatment. In HAP and VAP, often caused by multidrug-resistant pathogens, the delay in appropriate antimicrobial treatment represents a poor prognostic factor and is mainly driven by the pathogen extended resistance pattern itself [23,24,25,26,27]. Appropriate antibiotic treatment is hence pivotal and challenging. The majority of treatment recommendations regarded antibiotic choice and duration (data not shown), addressing key bedside clinical decisions. Nonetheless, for HAP/VAP treatment, none of the recommendations issued was based on high-quality evidence. The majority of evidence was rated low and very low, and only a third of strong recommendations was supported by moderate quality of evidence. The lack of a gold standard for HAP/VAP diagnosis and the geographical and in-hospital variability of etiology and antimicrobial susceptibility are partly responsible for the lack of high-quality evidence supporting these practices (Table 4). Interventional studies on general hygiene measures, effectiveness of epidemiological surveillance, and collateral care, such as bed positioning, might be performed with less difficulties, explaining the greater quantity of high-quality evidence recorded in HAP/VAP prevention. These data point out the need of collaboration to realize large RCTs, increasing the quality of evidence.

Table 4 Challenges for high-quality studies in the infectious diseases field, in ICU patients and in HAP/VAP
Full size table

The revision of clinical guidelines in several medical fields, such as cardiology, emergency medicine, gastroenterology, interventional medicine, hepatology, and nephrology, reported similar findings, with an overall scarcity of high-quality evidence [9,10,11,12,13,14,15, 17, 28, 29]. This low amount of high-quality evidence represents a call for collaboration to plan large well-conducted new research and simultaneously shows the importance of expert opinion in particular fields.

In the EBM era, CPGs represent an important and valuable tool in bedside clinical decisions and for patient management. Our study results reinforce the importance of literature knowledge to accurately translate CPGs and their recommendations in clinical practice and represent a call of wide collaboration to realize well-designed research in treatment, diagnosis, and prevention of HAP and VAP. Importantly, some clinical questions cannot be answered with high-quality evidence, because the clinical questions themselves are too complex and the antibiotic resistance profiles are profoundly different between sites and keep evolving. Furthermore, among the earliest HAP/VAP guidelines in the pre-GRADE era, with the possible exception of single antibiotic coverage for patients with Pseudomonas aeruginosa and shock absence, few recommendations based on lower grade of evidence were revised by subsequent studies. This implies the pivotal importance of high-quality studies in areas where there is a high degree of uncertainty, with potential adverse implications.

Several limitations can be found in the study. First, the more recent assessed guideline was written in 2017, including literature published until September 2016. Since then, new evidence might have been published on the addressed topic. Most CPGs are updated in periods longer than 5 years, with difficult incorporation of emerging evidence until next iteration. Published literature should be periodically reassessed, to update CPGs references and recommendations, creating a dynamic document. The creation of an adaptation framework for integration of new evidence in CPGs [30] would avoid outdated recommendations and will improve implementation in clinical practice. Second, literature supporting each recommendation was not revised and quality of evidence was not reassessed. Evidence and recommendation rating analyses were based on the classification reported in original papers, with no changes in the rating assigned to the literature by CPGs writing committees. Lastly, CPGs on HAP/VAP issued by other scientific societies without explicitly reported recommendations or quality of evidence assessment were excluded from the analysis. Despite these limitations, there are important strengths. The study provides a detailed analysis of quality of evidence used to write HAP/VAP international guidelines, an important evaluation in evidence-based medicine. The results point out the difficulty of performing RCTs and providing high-quality evidence in critically ill patients. Areas in which higher quality evidence are scarce reveal research opportunities, to assess unmet clinical needs in HAP/VAP management.

Conclusion

In the more recent guidelines on HAP/VAP treatment, diagnosis, and prevention, only a minority of issued recommendations are supported by high-quality evidence. Most recommendations, particularly related to treatment, remain supported by observational studies, case reports, and expert opinion. These data stress the need to direct efforts towards planning large, well-conducted, multicentric randomized controlled trials on HAP and VAP. Research areas that need further study are summarized in Table 5.

Table 5 Research areas for HAP/VAP
Full size table