Introduction

The diagnosis of bacterial pneumonia in mechanically ventilated patient represents a difficult dilemma for the clinician [1]. One option is to treat every patient clinically suspected of having a pulmonary infection with new antibiotics, even when the likelihood of infection is low, arguing that several studies showed that immediate initiation of appropriate antibiotics was associated with reduced mortality [2, 3, 4, 5]. However, this “clinical” approach leads to an overestimation of the incidence of ventilator-associated pneumonia (VAP) because tracheobronchial colonization and non-infectious processes mimicking it are included. Ironically, most antibiotics are given in the intensive care unit (ICU) for clinically suspected and not proven respiratory tract infections, exposing many patients to unnecessary toxicity, increasing hospital costs and favoring the emergence of resistant microorganisms [6, 7, 8, 9]. In addition, antibiotic overuse in these patients may delay the diagnosis of the true cause of fever and pulmonary infiltrate [10].

Concern about the inaccuracy of clinical approaches to VAP recognition led investigators to postulate that “invasive” diagnostic methods, including quantitative cultures of specimens obtained with bronchoscopic bronchoalveolar lavage (BAL) and/or protected specimen brush (PSB), could improve identification of patients with true VAP and facilitate decisions whether or not to treat, and thus improve clinical outcome [1, 5, 11, 12, 13]. However, these procedures require rigorous adherence to bronchoscopic and microbiologic techniques, and are not universally available; for these reasons, their use in everyday practice remains controversial [14].

In an attempt to minimize overuse of antibacterial agents, but still allow clinicians flexibility in managing patients with a perceived treatable infection, Singh et al. [15] recently proposed a new strategy, in which decisions concerning antibiotic therapy are based on a modified version of the clinical pulmonary infection score (CPIS) that was originally described by Pugin et al. [16]. This score is calculated at baseline, when VAP is clinically suspected, and 3 days later, by adding the points accorded to the following variables: body temperature, leukocyte count, tracheal secretion characteristics, oxygenation, pulmonary radiography, progression of pulmonary infiltrate from day 1 to day 3 and tracheal aspirate culture results. The first five criteria are used for CPIS calculation on day 1 and all seven criteria for calculation on day 3 (Table 1). Using the algorithm based on this score (Fig. 1), patients with CPIS more than 6 are treated as having VAP, i.e. with antibiotics for 10−21 days, while antibiotics are discontinued when the score remains at 6 or less 3 days later.

Fig. 1
figure 1

Diagnostic and therapeutic strategy applied to patients managed according to the strategy proposed by Singh et al. [15]

Full size image

Pertinently, in a randomized study on 81 ICU patients clinically suspected of having developed nosocomial pneumonia, the authors were able to demonstrate that this strategy led to significantly fewer antimicrobial therapy costs, microbial resistance and super infections without adversely affecting the length of stay or mortality, compared to a clinical strategy in which the choice and duration of antibiotics were left to the discretion of physicians [15]. However, only 58% of the 81 patients included in that study required mechanical ventilation (MV). The new algorithm was compared to a clinical strategy that required patients included in the control group to receive prolonged antimicrobial treatment despite a low probability of infection, thereby according it a potential capability to reduce inappropriate antibiotic use. Thus, it remains to be precisely determined whether this algorithm can perform as well when it is applied to ventilated patients and in comparison with an invasive strategy in which decisions concerning antibiotic prescriptions are based on results of quantitative bronchoscopic specimen cultures.

Accordingly, we designed this study to determine if a strategy based on the modified CPIS algorithm [15] would lead to the same antibiotic policy as our invasive strategy [11] for identifying and treating ventilated patients with suspected VAP. To do so, the CPIS was retrospectively determined on days 1 and 3 for a large series of ventilated patients clinically suspected of having developed pneumonia, and compared to patients identified as having developed VAP or not, based on quantitative cultures of specimens obtained by bronchoscopy. Then, we calculated the sensitivity and specificity of the CPIS algorithm to identify patients with VAP and estimate who among them should have an adaptation of antibiotic therapy.

Methods

Study location and patients

The original study was designed to compare survival on day 14, antibiotic use and organ failure(s) in patients managed for VAP with an invasive strategy versus those managed with a non-invasive strategy (clinical criteria and isolation of microorganisms by non-quantitative cultures of tracheal aspirates) [11]. Accordingly, all patients clinically suspected of having VAP who were randomized to the invasive-strategy arm underwent immediate fiberoptic bronchoscopy (day 1) with either PSB and/or BAL, according to each center’s protocol. Patients were considered to have VAP if more than 5% of the cells in cytocentrifuge preparations of BAL fluid contained intracellular bacteria or at least one bacterial species grew at a significant concentration from the PSB sample (≥103 cfu/ml) or from BAL fluid (≥104 cfu/ml).

Data collection

The following variables were prospectively recorded and analyzed in the original study: patient age and sex; severity of underlying disease according to the criteria of McCabe and Jackson [17]; classification as a medical patient or surgical patient with or without trauma, according to the admitting diagnosis; the reason for initiating MV [18]; Simplified Acute Physiology Score II at admission to the ICU and at baseline; time elapsed between the beginning of MV and the suspicion of VAP and duration of antimicrobial treatment. We also recorded, on days 1 and 3 after inclusion in the study, the following variables: temperature; leukocyte count; oxygenation assessed by the PaO2/FIO2 ratio; tracheal secretion characteristics (volume and aspect); radiologic score (range, 0−12 according to the density of the radiologically detected infiltrate) [11] and its evolution from day 1 to day 3. Any antibiotic use was recorded daily until day 28.

Definitions

Among the 204 patients included in the invasive strategy arm of the original study, we were able to calculate the CPIS automatically based on the data that were prospectively collected for the initial study for 201 (98%) (Table 1). The other three patients died between days 1 and 3, and thus the day-3 CPIS could not be determined; they were excluded from the present study. Because tracheal aspirate-culture results were not available for all patients managed with the invasive strategy, we modified the last criterion proposed by Singh et al. as follows: “pathogenic bacteria cultured in rare or light quantity or no growth” was replaced by “no bacterial growth of PSB or BAL fluid”; “pathogenic bacteria grown in moderate or heavy quantity” was replaced by “pathogenic bacteria cultured at non-significant concentration(s) (<103 for PSB or <104 for BAL fluid)” and “same pathogenic bacteria seen on Gram stain” by “pathogenic bacteria grown at significant concentration(s) (≥103 for PSB or ≥104 for BAL fluid)”. The CPIS at baseline included the first five variables and it was re-calculated with all seven variables 3 days later. As described by Singh et al. [15], CPIS more than 6 on days 1 and/or 3 was considered suggestive of VAP and thus justified an adaptation of antibiotic therapy.

Table 1 Clinical pulmonary infection score (CPIS) calculationa,b
Full size table

Statistical analyses

The data are expressed as means ± SD or the number with the percent in parentheses. The chi-square or Fisher exact test was used to compare categorical variables. Continuous variables were compared using the Student’s t-test, or the Mann−Whitney U-test when they were not normally distributed. Correlations were assessed using Spearman’s test. CPIS operating characteristics to identify patients with VAP and the kappa coefficient for concordance were calculated according to standard definitions, using microbiologically proven pneumonia as the reference test. We also assessed the accuracy of the CPIS to detect VAP using the area under the receiver operating characteristic (ROC) curve. For all tests, a value of p less than 0.05 was considered significant.

Results

Among the 201 patients, microbiologic cultures of 63 of 170 PSB samples and 46 of 137 BAL samples were positive, for a total of 88 (44%) cases of bacteriologically confirmed VAP. Clinical characteristics of patients at ICU admission and baseline are reported in Tables 2 and 3, respectively. Of the five clinical variables used to determine the CPIS at baseline (day 1), only the percentages of patients with a localized infiltrate differed between patients with and without VAP (67 versus 50%, respectively, p=0.02) (Table 3).

Table 2 Intensive care unit admission characteristics of study patientsa
Full size table
Table 3 Characteristics of study patients at baseline, when ventilator-associated pneumonia (VAP) was clinically suspecteda
Full size table

The day-1 CPIS were similar for the two groups (6.4±1.4 versus 6.2±1.6 in patients with and without VAP, respectively; p>0.2) (Fig. 2A). However, when the CPIS was calculated on day 3, based on all seven variables including radiologic progression of infiltrate and microbiologic culture results, the mean CPIS was higher for patients with VAP (8.7±1.8) than those without (7.0±1.9, p<0.0001) (Fig. 2B). The results of PSB (r=0.46; p<0.001) and BAL (r=0.54; p<0.001) quantitative cultures were significantly correlated with the CPIS, even though no threshold could accurately discriminate between the different CPIS groups (data not shown).

Fig. 2
figure 2

Clinical pulmonary infection score on day 1 (A) and 3 (B) for patients with and without ventilator-associated pneumonia defined by microbiologic results of bronchoscopic specimens. The boxes represent the 25th–75th percentiles, with the 50th percentile (solid line) shown within the boxes. The 10th and 90th percentiles are shown as capped bars, with dots marking the outliers

Full size image

As indicated on Fig. 3, 138 patients (69%) had a CPIS more than 6 on day 1 or day 3, that would have required 10−21 days of antimicrobial therapy according to the proposed algorithm. While the sensitivity of CPIS more than 6 to identify patients with VAP, as defined by bronchoscopic results, was 89%, its specificity was only 47% (Table 4). Positive- and negative-predictive values of CPIS more than 6 were 57% and 84%, respectively, for a 44% frequency of VAP in the study population. Thus, the CPIS strategy was in agreement with bronchoscopic results for only 131 (65%) of the 201 patients, with a kappa coefficient of 0.33, indicating poor agreement between the two approaches. Using the proposed algorithm, 11% of VAP patients (10/88) as identified by bronchoscopy would not have been identified as having VAP and 60/113 (53%) patients without VAP would have received antibiotics for 10−21 days (Fig. 3). Similar results were obtained for different subgroups of patients, including patients with short (<8 days) or prolonged (≥8 days) duration of MV before study entry, those with or without prior antimicrobial treatment or those with localized or diffuse pulmonary infiltrates (Table 4).

Fig. 3
figure 3

Number of patients assessed and enrolled in the trial. Actual numbers of patients falling into each category are reported

Full size image
Table 4 Operating characteristics of clinical pulmonary infection score (CPIS) more than 6 for detecting ventilator-associated pneumonia (VAP) diagnosed based on microbiologic results of bronchoscopic specimens
Full size table

Using the proposed algorithm and assuming that patients with a CPIS more than 6 would have been treated for 14 days and those with a CPIS of 6 or less for 3 days, the total number of antibiotic days received by the 201 patients would have been 2,121 days (i.e. 11±5 antibiotic days per patient). Interestingly, these patients actually received only 1,773 days of antimicrobial treatment during the first 14 days after inclusion in our trial (i.e. 8.8±5 antibiotic days per patient, p<0.0001). In the 60 patients with a day-1 or day-3 CPIS more than 6 and negative bronchoscopic results, the total use of antibiotics for the first 14 days would have been 840 days using the Singh strategy; whereas only 424 days of antibiotic (7.1±5.2 days per patient) were actually prescribed using the invasive strategy (p<0.0001). Among these 60 patients, 18 (30%) did not receive any antibiotics within the first 2 weeks after inclusion in the study and 13 (22%) received antibiotics only after at least 7 days had elapsed from bronchoscopy, leaving only 29 patients (48%) who were treated within the first week, mostly for a clearly documented extrapulmonary infection.

The ROC curve for CPIS detection of VAP was plotted (Fig. 4). Based on this curve, for our study population the best cutoff for the CPIS to identify patients with VAP was more than 7, with an overall accuracy of 70%, a sensitivity of 75% and a specificity of 66%. Using this threshold, 25% of the patients with VAP (22/88) would not have been identified as having a lung infection and would not have received new antibiotics, while 38/113 (34%) patients without VAP would have been unduly treated.

Fig. 4
figure 4

Receiver operating characteristic curve of clinical pulmonary infection score calculated on day 3 for the identification of patients with ventilator-associated pneumonia

Full size image

Discussion

To evaluate the potential usefulness of the CPIS to identify and treat ICU patients clinically suspected of having developed VAP, we studied a large group of patients who required MV for more than 48 h and for whom strict bronchoscopic criteria were applied to diagnose or exclude pneumonia. The CPIS assessed at baseline according to the methodology proposed by Singh et al. [15] did not differ significantly for patients with or without VAP and, of the five variables used to calculate it, only the percentage of patients with localized infiltrate was significantly higher in VAP patients. On day 3, when microbiologic culture results were taken into consideration, patients with VAP had higher CPIS than patients without pneumonia, but no threshold could accurately discriminate between the two groups. Despite good CPIS sensitivity for identifying patients with VAP on day 3, when the proposed cutoff of more than 6 was chosen to define the presence of VAP, application of the CPIS algorithm would have meant treating a total of 138/201 (69%) patients with prolonged administration of antibiotics, while only 88 of these 138 patients had VAP as diagnosed by bronchoscopy. Using the cutoff established by our ROC curve (>7) as giving the best overall accuracy, 25% of our patients with VAP would not have been identified as such and thus would not have received new necessary antibiotics, while 38 other patients without VAP would have been prescribed potentially unneeded antibiotics.

To the best of our knowledge, only a few studies [16, 19, 20, 21, 22] have assessed the usefulness of the CPIS for patients with suspected VAP to distinguish those with microbiologically confirmed VAP from those with only proximal airway colonization and all but one of these used the score described by Pugin et al. [16], which does not consider the same variables and does not use the same definitions as the modified CPIS proposed by Singh et al. [15]. Recently, Schurink et al. evaluated the ability of the CPIS to recognize VAP, diagnosed by the quantitative BAL culture results of 99 patients as the reference test, and obtained a sensitivity and a specificity value of only 41% and 77%, respectively, for a CPIS value more 7 [21]. Poor clinical predictions were also obtained by Fartoukh et al. in a series of 79 episodes of suspected pneumonia, when a modified CPIS based on clinical criteria recorded on the day of clinical suspicion was used, with approximately one-half of the patients being incorrectly classified [22]. Only incorporating the results of specimens with Gram stain increased the physicians’ diagnostic accuracy, with a sensitivity of 85% and a specificity of 49%.

At least two factors may explain the inability of the CPIS to detect accurately pneumonia in ICU patients requiring MV. First, several investigators have clearly documented that the clinical, radiologic and laboratory variables used to calculate the CPIS are frequently inconclusive for patients clinically suspected of having VAP [23, 24, 25, 26]. In one study [26], even a mathematical model constructed from the results of a multivariate analysis based on a total of 15 variables, including temperature, blood leukocyte and blood lymphocyte counts, PaO2/FiO2, radiologic score and changes in these parameters during the 3 days preceding suspicion of pneumonia, was unable accurately to separate patients who had pneumonia from those who did not, thereby confirming previous conclusions that no objective clinical criteria exist for differentiating patients with or without pneumonia and that the use of microbiologic data is needed to increase the CPIS accuracy [1, 22]. Second, this score is quite tedious to calculate and difficult to use in clinical practice, since several variables, such as pulmonary radiography, tracheal secretion characteristics, progression of pulmonary infiltrates and results of semiquantitative cultures of tracheal secretions are observer dependent [21].

Our study was limited by uncertainty about the value of the reference test we chose for diagnosing VAP. Using bronchoscopic techniques for this purpose, we might have missed some VAP episodes or, on the contrary, classified some patients as having developed VAP while they might just have needed a short course of antimicrobial treatment. However, despite the need for cautious interpretation, the results of many studies have indicated that those techniques offer a rather sensitive and specific approach to identifying the microorganisms involved in pneumonia in critically ill patients and to differentiating between colonization of the upper respiratory tract and distal lung infection [1]. Pooling the results of the 18 studies evaluating the PSB technique in a total of 795 critically ill patients showed the overall accuracy of this technique for diagnosing nosocomial pneumonia to be high, with a sensitivity of 89% (95% CI: 87−93%) and a specificity of 94% (95% CI: 92−97%) [1, 27]. Second, because of the retrospective nature of our study, we were unable to use the exact definitions established by Singh et al. [15] for the seventh variable on which the CPIS is based and, instead of using the results of tracheal culture results to calculate it, we used the results of quantitative bronchoscopic specimen cultures. While this modification undoubtedly linked the CPIS we calculated to the reference test, and thus might have artificially increased its sensitivity, we must emphasize that this bias actually favored the CPIS rather than the contrary. However, the substitution of tracheal aspirate culture results for those of BAL or PSB could also have falsely increased the number of false negative results observed with the CPIS, since endotracheal aspirate cultures have higher sensitivity that bronchoscopic techniques for diagnosing VAP.

Third, patients who had received antibiotics during the 3 days before collection of respiratory samples were not included in the initial study [11]. Not taking into account these patients could have falsely lowered the number of days of antibiotic use with the invasive strategy, as compared to the CPIS strategy, in patients with suspected VAP. However, it could be argued that this particular group of patients who required urgent introduction or modification of antimicrobial treatment instigated by new clinical symptoms might have had a CPIS more than 6 in many cases and would have been treated with at least 14 days of antibiotics using such a strategy. Finally, it is important to acknowledge that our study was not designed directly to test the hypothesis that a strategy based on the CPIS to decide which patients should receive new antibiotics is inferior to a strategy based on bronchoscopy in terms of improving clinical outcomes and minimizing antibiotic use. We have only simulated the application of the CPIS algorithm to the group of patients who had been randomized to the invasive strategy arm in our original study [11]. Only a prospective, randomized study comparing these two approaches would be able to answer such a question.

In summary, when microbiologic culture results are taken into consideration on day 3, patients with VAP have higher CPIS than patients without pneumonia and a cutoff of more than 6 is able to identify most patients with lung infection. Based on this high sensitivity (89%) and negative predictive value (84%), a strategy applying this clinical score to decide which patients suspected of having VAP should receive prolonged administration of antibiotics may represent a valid alternative to the clinical strategy, minimizing unnecessary antibiotic use to some extent. However, because the CPIS calculated at day 1, based on five clinical variables does not discriminate patients with from those without VAP, the use of the CPIS requires treating all patients with clinically suspected pneumonia for at least 3 days, even when the likelihood of infection is low, which can render more difficult the search for another (the true) site of infection. Furthermore, as many as 53% of the patients without VAP, as diagnosed by bronchoscopy, would then receive prolonged antimicrobial treatment after day 3, leading to potential over-prescription of antibiotics compared to a strategy based on quantitative cultures of bronchoscopic specimens.