Introduction

Healthcare-associated pneumonia (HCAP), a relatively new category of pneumonia, refers to infections that occur prior to hospital admission in patients with contact or exposure to a healthcare environment [1]. Compared to community-acquired pneumonia (CAP), HCAP is a distinct type of pneumonia with unique microbiological and epidemiological characteristics and outcomes [2–6].

In the current era of rising healthcare costs, the decision to hospitalize adults with CAP has received considerable attention and many pneumonia severity prediction rules have been designed to stratify patients with CAP into risk groups [7, 8]. Severity assessment is not only the key to deciding the site of care but also in guiding both general management and antibiotic treatment. Of the prominent tools for this purpose are the Pneumonia Severity Index (PSI) developed by Fine and colleagues [9] and the CURB (confusion, urea, respiratory rate, blood pressure) score proposed by the British Thoracic Society, and Infectious Diseases Society of America/American Thoracic Society Consensus Guidelines on the Management of Community-Acquired Pneumonia in Adults [10]. Other clinical prediction rules for severe community-acquired pneumonia, like the severe community acquired pneumonia (SCAP) score were also developed, and were seemingly better at identifying severe CAP. The SCAP is validated to predict 30-day mortality among two cohorts of consecutive adult patients with CAP and identifies more patients as low risk for potential outpatient care [11]. The need for ICU care was better identified with the SOAR (systolic blood pressure, oxygenation, age, respiratory rate) model compared to the other scoring rules (CURB (confusion, urea, respiratory rate, blood pressure), CURB-65 (confusion, urea, respiratory rate, blood pressure, age 65), CRB-65 (confusion, respiratory rate, blood pressure, age 65)) in patients with nursing home acquired pneumonia [12], a subgroup of HCAP.

Each scoring system has its strengths and weaknesses. As demonstrated by the studies on heterogeneous populations, validation studies of algorithms for HCAP therapy will be difficult [13]. It would be very helpful if we can apply the existing scoring systems to HCAP. However, to the best of our knowledge, none of these prediction rules has been validated in patients hospitalized with HCAP. Therefore, we sought to compare the performance of the current scoring indices to predict mortality and ICU admission in patients with HCAP.

Materials and methods

Setting and study design

This multi-center study was conducted at six medical centers in Taiwan (Taipei Veterans General Hospital, National Taiwan University Hospital, Taichung Veterans General Hospital, China Medical University Hospital, National Cheng Kung University Hospital, and Kaohsiung Chang Gung Memorial Hospital). All adult patients presenting to one of the study hospitals with pneumonia who were discharged between 1 January 2007 and 31 December 2007 were reviewed. According to the 2005 IDSA/ATS (Infectious Diseases Society of America/American Thoracic Society) guidelines [14], a patient with HCAP is defined as one having pneumonia and any of the following historical features: (1) hospitalization for two or more days in an acute care facility within 90 days of infection, (2) being a resident of a nursing home or long-term care facility, (3) attending a hospital or hemodialysis clinic, (4) having received intravenous antibiotic, chemotherapy, or wound care within 30 days of infection. The patients were excluded if they had any one of the following conditions: (1) they were younger than 18 years old; (2) their pneumonia developed two days after admission or within 14 days after discharge; (3) they had lung cancer with obstructive pneumonia; (4) they were HIV positive with a CD4+ <200; (5) there were inadequate data for scoring. A total of 551 HCAP patients were recruited and 444 patients with adequate data (with all variables for calculating all scoring indices we compared available at admission) were studied. The study was approved by the institutional review board of each medical center and informed consent was waived.

Microbiology evaluation

The specimens obtained within 72 h of admission were eligible for etiologic evaluation, including sputum, tracheal aspirate, bronchoalveolar lavage fluid, pleural effusion, blood, and urine for Legionellae antigen test or Streptococcus pneumoniae antigen test. The HCAP pathogens were defined according to the principles proposed by Lauderdale et al. [15].

In brief, etiology was determined based on laboratory data from blood and sputum cultures plus serology from paired serum and urine antigen detection tests. Blood cultures were accepted if the same microorganism was identified in a respiratory specimen and no other source for the positive blood culture could be identified. If the patients received bronchoscopic study, the definite organisms were confirmed by quantitative bacterial cultures BAL (bronchoalveolar lavage) >104/cfu or PSB (protected sheath brushing) >103/cfu. The probable pathogen was the organism isolated as a predominant organism from sputum or endotracheal aspirate.

Definition of co-morbidities

The co-morbidities were defined according to the definition in the study by Fine et al. [9], including neoplastic disease, liver disease, congestive heart failure, cerebrovascular disease, and renal disease.

Outcomes

The primary outcomes include 30-day all-cause mortality and ICU admission after 3 days and 14 days. The lengths of both the ICU and hospital stay were also determined.

Scoring indices

The modified ATS rule was met if at least two of three minor criteria assessed at admission (systolic blood pressure <90 mmHg, multilobar (>2 lobes) involvement, PaO2/FiO2 <250), or one of two major criteria assessed at admission or during follow-up (requirement for mechanical ventilation or septic shock) were present [16, 17].

IDSA/ATS refers to the Infectious Diseases Society of America/American Thoracic Society Consensus Guidelines on the Management of Community-Acquired Pneumonia in Adults [10]. In addition to the two major criteria (need for mechanical ventilation and septic shock), an expanded set of minor criteria (respiratory rate ≥30 breaths/minute; arterial oxygen pressure/fraction of inspired oxygen (PaO2/FiO2) ratio ≤250; multilobar infiltrates; confusion; blood urea nitrogen level ≥20 mg/dL; leukopenia resulting from infection; thrombocytopenia; hypothermia; or hypotension requiring aggressive fluid resuscitation) is proposed. The presence of at least three of these criteria suggests the need for ICU care.

SOAR comprises systolic blood pressure, oxygenation, age, and respiratory rate [18]. We then defined severe pneumonia as the presence of two or more out of the four criteria. A score of 1 was given for the presence of each of the following (dichotomized variables): systolic BP <90 mmHg; PaO2:FiO2 <250; age ≥65 years; and RR ≥30/minute.

SCAP was proposed by Espana [19]. The evaluation of SCAP is based on the presence of one major criterion (PS) or two or more minor criteria (CURXO80). P = arterial pH <7.3; S = systolic pressure <90 mmHg; C = confusion; U = blood urea nitrogen >30 mg/dL; R = respiratory rate >30/minute; X = X-ray multilobar bilateral; O = PaO2 <54 or PaO2/FiO2 <250 mmHg; and 80 = Age ≥80 years.

SMART-COP (systolic blood pressure, multilobar involvement, albumin respiratory rate, tachycardia, confusion, oxygenation, pH) scores were calculated as presented by Charles [20], and consisted of systolic blood pressure (<90 mmHg, two points); multilobar chest radiography involvement (one point); low albumin level (<3.5 g/dL, one point); high respiratory rate (≤50 years: ≥25 br/minute, >50 years: ≥30 br/minute; one point); tachycardia (≥125 bpm; one point); confusion (new onset; one point); poor oxygenation (≤50 years: PaO2 <70 mmHg or O2 saturation ≤93%, >50 years: PaO2 <60 mmHg or O2 saturation ≤90%; two points); and low arterial pH (<7.35; two points).

SMRT-CO (Simplified SMART-COP was designed for use by primary care physicians, and it excludes the results for albumin, arterial pH, and PaO2 [20]).

CURB-65 score is a six-point score, with one point for each of: confusion; urea >7 mmol/l; respiratory rate ≥30/minute; low systolic (<90 mmHg) or diastolic (≤60 mmHg) blood pressure; and age ≥65 years [21].

The pneumonia severity index (PSI) was calculated as presented in the study by Fine et al. [9], and is comprised of the following variables: age, gender, co-morbidity, and vital sign abnormalities, together with several laboratory, blood gas, and radiographic parameters. The PSI results in a five-class point scoring system reflecting the increasing risk of mortality.

Statistical analysis

Categorical variables were analyzed using a chi-square test or Fisher's exact test where appropriate, and continuous variables were compared using Student's t-test or the Mann-Whitney U test. The discriminatory power of each scoring index was measured by receiver operating characteristic (ROC) curves. The areas under the ROC curve (AUC) was calculated to give an estimate of the overall accuracy of each scoring index in predicting different patient outcomes (3-day ICU admission, 14-day ICU admission and 30-day mortality). An area of 0.50 implies that the scoring index is no better than chance, whereas an area of 1 implies perfect accuracy. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were also calculated as well with their 95% confidence intervals for all the scoring indices. The Hanley-McNeil test was used for testing the statistical significance of the difference between the two AUC figures. All tests were two-tailed, and P- value <0.05 was considered to be statistically significant. All statistical analyses were performed using the SPSS 14.0 software (SPSS Inc., Chicago, IL, USA) and the MedCalc 9.6.2.0 package (MedCalc Software, Mariakerke, Belgium).

Results

Enrolled background

A total of 444 patients met the inclusion criteria for HCAP. Among these patients, there were 40 (9%) patients receiving regular hemodialysis, peritoneal dialysis, or infusion therapy. The enrolled patient backgrounds are provided in Table 1. The all-cause mortality rate at 30 days was 20.9%, and the 3-day ICU admission and 14-day ICU admission rates were 25% and 29.1%, respectively.

Table 1 Background of patients with healthcare-associated pneumonia
Full size table

Patient demographics, clinical characteristics, and bacterial pathogens

The demographic and clinical characteristics of the patients with HCAP are provided in Tables 2 and 3. There are no significant differences for gender and age between survivors and non-survivors at 30 days post admission. Patients who smoke have higher all-cause mortality rates than non-smokers.

Table 2 Patient demographics characteristics (three-day ICU)
Full size table
Table 3 Patient clinical characteristics (three-day ICU)
Full size table

Neoplasm disease is the most important co-morbidity which causes higher mortality. Other co-morbidities--cerebrovascular disorders, renal disease, liver disease, and diabetes mellitus--can predict a higher need for ICU admission at Day 3.

Many of the predictors that were checked within two days were associated with higher all-cause mortality and the need for ICU admission. The predictors include a patient's requirement for mechanical ventilation, septic shock status, altered mental status, presence of pleural effusion, pneumonia with multilobar involvement, high fever or hypothermia, high BUN level, arterial blood acidosis, and hypoxemia.

The pathogen yielded in patients who were admitted to the ICU at 3 days and at 14 days tended to be Gram negative bacteria. Initial antibiotic choice is crucial and inadequate antibiotic administration could cause higher mortality. Pseudomonas aeruginosa was the most frequently found pathogen, followed by Klebsiella spp. (Table 4).

Table 4 Etiology of healthcare-associated pneumonia
Full size table

Scoring indices to predict mortality and ICU admission hospital LOS

As shown in Table 5, the scoring indices originally designed for CAP were tested to be applied to HCAP. The adverse outcome rate increased steadily from low to high, meeting the criteria for all scores. The average LOS increased steadily from low to high, either for risk class or meeting criteria. PSI can offer moderate discriminating ability for separating patients between survivors and non-survivors at 30 days, as well as for predicting the need for ICU admission. The performance of each index in predicting 3-day and 14-day ICU admission and 30-day mortality were also determined (Tables 6 and 7). PSI (>90) has the highest sensitivity to predicting mortality (AUC: 0.70), followed by CURB-65 (≥2) (AUC: 0.66), and SCAP (>9) (AUC: 0.71). For predicting ICU admission (Day 3 and Day 14), modified ATS (AUC: 0.84, 0.82), SMART-COP (AUC: 0.84, 0.82), SCAP (AUC: 0.82, 0.80) and IDSA/ATS (AUC: 0.80, 0.79) performed better (statistically significant difference) than PSI, CURB-65, SOAR and SMRT-CO.

Table 5 ICU admission, mortality, and hospital LOS according to different prediction rules
Full size table
Table 6 Measure of performance predicting 3-day and 14-day ICU admission and 30-day mortality by using different prediction rules
Full size table
Table 7 Pairwise comparison of ROC curves (the number represents the p-value)
Full size table

Discussion

HCAP is a heterogeneous disease that includes patient populations with varying severities of illness [22]. The mortality associated with HCAP was similar to that of nosocomial pneumonia, higher than that of CAP, and lower than ventilator-associated pneumonia [13]. As shown in Table 1, each subgroup contributes to different parts of overall HCAP mortality. There is increased mortality of groups II (34.4%) and III (47.3%) of patients with HCAP, indicating that HCAP is a heterogeneous disease. As has already been reported by Brito and Niederman, all patients with HCAP should be identified and then divided on the basis of severity of illness to guide initial therapy [13]. Severe pneumonia has been defined by the requirement for admission to an ICU [16]. The decision to admit a patient with HCAP to an ICU depends on subjective clinical views and the peculiarities of the local healthcare setting. The availability of valid criteria for defining severe pneumonia would provide a more reliable basis for improving patient risk assessments. The severity on admission can affect hospital mortality, the need for ICU admission, and even 90-day mortality after hospital discharge [23]. A number of prognostic scoring tools have been developed to predict mortality and the need for ICU care for patients with CAP; the two tools that have been studied the most are the PSI and CURB-65. However, they are not ideal for assessing the need for ICU care, and other scoring systems, such as those developed by the IDSA/ATS guideline group, and the SMART-COP tool, are available for this purpose [24]. So far, and to the best of our knowledge, no severity index has been developed and validated for patients with HCAP.

The AUC is a measure of the accuracy of a test to correctly classify patients with and without a particular outcome and is used frequently in studies of severity assessment in CAP. The AUC describes the relationships between sensitivity and specificity, a higher AUC implies a less steep trade-off between sensitivity and specificity. An AUC is considered to have moderate discriminating power from a value of 0.70 on up. We conducted this retrospective chart review of 444 records and assessed the validity of PSI, CURB-65, SCAP, and others and constructed an ROC.

The PSI scoring system has been shown to be a powerful tool for assigning the risk of death from CAP in different populations [17]. This scoring system was primarily designed to identify patients with a low mortality risk who could safely be treated as outpatients. However, it is complicated to use, requiring computation of a score based on 20 variables. To ensure that the final prediction rule remained simple to use and practical, prognostic features not usually available at the time of initial assessment post hospital admission were excluded from the CURB-65 model [21]. The CURB-65 model does not consider decompensated co-morbidity due to CAP and results in limited application in the elderly [24]. Since the majority of patients were elderly, the data are not much different from what is published in the literature regarding CAP; that is, CURB-65 may not be a good index for predicting mortality in this population.

The modified ATS rule provides simple clinical criteria for those patients who require ICU admission [16]. According to the authors' description, the modified ATS rule can serve as a useful counterpart to the prediction by Fine et al. The modified ATS rule was good in terms of sensitivity (89.9%) and the area under the receiver operator curve graph (0.823) for predicting 14-day ICU admission in HCAP patients. The modified ATS severe CAP definition published in 2001 was superseded by the 2007 IDSA-ATS severe CAP definition (IDSA/ATS). The newer definition was based on a series of papers and on re-evaluation by the guideline committee of data published since the 2001 definition was made. Therefore, we also tested the two indices and found that modified ATS as well as IDSA/ATS can be applied for defining severe HCAP.

The strongest clinical predictors of SCAP were pH <7.30 and systolic pressure <90 mmHg [19]. A depressed pH, which is likely a side effect of metabolic acidosis derived from sepsis, is not included in other prediction rules, such as CURB-65 or modified ATS. In our series, a low pH was associated with poor outcomes in patients with HCAP. The SCAP score is as accurate as, or better than, other current scoring systems (for example, CURB-65 and PSI) in predicting adverse outcomes in patients hospitalized with CAP [25]. We found that SCAP also works well with HCAP. The discriminatory power of SCAP, as measured by AUC, was 0.81 for ICU admission in our HCAP patients, compared with the 0.75 in CAP patients from another study [25].

The PSI and CURB-65 have been used to guide the need for ICU care, but they are not ideal for this purpose [24]. Some of these indices were originally designed to assess ICU admission rather than mortality. Therefore, a poor performance could be found if applied in predicting mortality. Compared to PSI, modified ATS, IDSA/ATS, SCAP, and SMART-COP were easy to calculate. For predicting ICU admission (Day 3 and Day 14), modified ATS (AUC: 0.84, 0.82), SMART-COP (AUC: 0.84, 0.82), SCAP (AUC: 0.82, 0.80) and IDSA/ATS (AUC: 0.80, 0.79) performed better (showing a statistically significant difference) than PSI, CURB-65, SOAR and SMRT-CO.

The main strength of the study is the relatively large sample size. The limitations of the study include possible selection bias as all patients who were included in our analysis consist of a heterogenic variety of sources. There may be different patient characteristics in each study site. On the other hand, it can reflect the reality of HCAP coming from heterogeneous populations. In addition, there are a huge number of patients that received microbiologically adequate therapy (sensitive to the antibiotic administered) and their clinical conditions do not improve because of other possible factors (for example, incorrect dosing, interval of administration, pharmacokinetic/pharmacodynamic features, hypoalbuminemia in critically ill patients) which were not investigated in this study. However, those were beyond the scope of the study.

Conclusions

The utility of the scoring indices for risk assessment in patients with healthcare-associated pneumonia shows that the scoring indices originally designed for CAP can be applied to HCAP. The promising results offer the clinician an adjunctive tool when making site-of-treatment decisions for patients and when stratifying patients with HCAP into risk groups.

Key messages

  • There is currently no scoring index to predict the outcomes of patients with HCAP, a type of pneumonia that occurs prior to hospital admission in patients with specific risk factors following contact or exposure to a healthcare environment.

  • We applied and compared different community acquired pneumonia (CAP) scoring indices to predict 30-day mortality and 3-day and 14-day intensive care unit (ICU) admission in patients with HCAP.

  • PSI has the highest sensitivity in predicting mortality, followed by CURB-65 (≥2) and SCAP (>9) (SCAP score (AUC: 0.71), PSI (AUC: 0.70) and CURB-65 (AUC: 0.66)).

  • For predicting ICU admission (Day 3 and Day 14), modified ATS (AUC: 0.84, 0.82), SMART-COP (AUC: 0.84, 0.82), SCAP (AUC: 0.82, 0.80) and IDSA/ATS (AUC: 0.80, 0.79) performed better (statistically significant difference) than PSI, CURB-65, SOAR and SMRT-CO.

  • The promising results offer the clinician an adjunctive tool when making site-of-treatment decisions for patients and when stratifying patients with HCAP into risk groups.