Multicenter retrospective development and validation of a clinical prediction rule for nosocomial invasive candidiasis in the intensive care setting

  • L. Ostrosky-Zeichner
  • C. Sable
  • J. Sobel
  • B. D. Alexander
  • G. Donowitz
  • V. Kan
  • C. A. Kauffman
  • D. Kett
  • R. A. Larsen
  • V. Morrison
  • M. Nucci
  • P. G. Pappas
  • M. E. Bradley
  • S. Major
  • L. Zimmer
  • D. Wallace
  • W. E. Dismukes
  • J. H. Rex
Concise Article

DOI: 10.1007/s10096-007-0270-z

Cite this article as:
Ostrosky-Zeichner, L., Sable, C., Sobel, J. et al. Eur J Clin Microbiol Infect Dis (2007) 26: 271. doi:10.1007/s10096-007-0270-z

Abstract

The study presented here was performed in order to create a rule that identifies subjects at high risk for invasive candidiasis in the intensive care setting. Retrospective review and statistical modelling were carried out on 2,890 patients who stayed at least 4 days in nine hospitals in the USA and Brazil; the overall incidence of invasive candidiasis in this group was 3% (88 cases). The best performing rule was as follows: Any systemic antibiotic (days 1–3) OR presence of a central venous catheter (days 1–3) AND at least TWO of the following—total parenteral nutrition (days 1–3), any dialysis (days 1–3), any major surgery (days −7–0), pancreatitis (days −7–0), any use of steroids (days −7–3), or use of other immunosuppressive agents (days −7–0). The rate of invasive candidiasis among patients meeting the rule was 9.9%, capturing 34% of cases in the units, with the following performance: relative risk 4.36, sensitivity 0.34, specificity 0.90, positive predictive value 0.01, and negative predictive value 0.97. The rule may identify patients at high risk of invasive candidiasis.

Introduction

Candidemia is the fourth most common bloodstream infection in the USA and is independently associated with increased morbidity, mortality, and costs, particularly in the non-neutropenic critically ill adult [1, 2, 3]. Thus, strategies such as prophylaxis, preemptive, and empirical therapy in selected patients at high risk would appear appropriate. Three studies previously demonstrated the clinical utility of prophylaxis for invasive candidiasis (IC) in the ICU in single hospital or geographical settings [4, 5, 6]. These studies showed that prophylaxis may be useful in select high-risk populations. Although risk factors for IC have been described extensively [7, 8], predicting disease risk through the identification of single risk factors is nearly impossible due to the common nature of many of these factors in the ICU setting. Although very useful in single-center studies, defining colonization as a risk factor for IC is controversial [5, 7, 9, 10, 11].

The purpose of the study presented here was to develop a clinically relevant rule for the early prediction of IC using combinations of known risk factors other than colonization. This rule could potentially be applied in clinical trials that investigate prophylaxis, preemptive, or empirical therapy.

Materials and methods

A retrospective review of patient charts and electronic records was carried out for patients aged 19 or older who stayed in one of 12 participating medical and/or surgical ICUs in the USA and Brazil for more than 4 days during the 2000–2002 time period. The data were recorded using case-report forms designed specifically for this study. Patients with evidence of IC or who received systemic antifungal agents during the week prior to ICU admission through the first 3 days of ICU stay (days −7–3) were excluded. Study ICUs did not have specific policies regarding antifungal prophylaxis. The Institutional Review Board at each participating site approved the study; each board waived the need for subject informed consent.

Basic demographic details and hospital metrics were collected. Race information was not collected for this study. Data indicating the presence or absence of classically described risk factors were collected for the time period day −7 to day 3. Data on the rate of IC were collected during the outcome period, which was defined as day 4 of ICU stay through day 7 following ICU discharge. IC was defined as proven or probable using EORTC/MSG criteria [12].

The data were randomly split into “training” (75%) and “validation” (25%) sub-samples to develop and validate the prediction rule, respectively. Univariate analyses were used to assess the relationships between the presence or absence of each individual risk factor and subsequent development of IC. Tables detailing the frequency of risk factor versus infection status were constructed and the Cochran-Mantel-Haenszel chi-square test of association was performed; relative risk of infection was also explored. Then, several different prediction-rule “formats” (with different weights for the risk factors) were proposed. Within each format, all possible combinations of risk factors and time periods (days −7–0, days 1–3, days −7–3) were explored systematically, analyzing multiple combinations of increasing numbers of risk factors. The predictive ability of each rule was assessed using the methods described above as well as traditional performance measures such as sensitivity, specificity, positive predictive value, and negative predictive value. The “best” rule was selected by team consensus using the following criteria before it was applied to the validation sample: (1) the proportion of all IC cases contained in the subpopulation defined by the rule; (2) the risk of IC in the subpopulation defined by the rule; (3) the relative risk of IC in the subpopulation; and (4) the proportion of total ICU patients defined as high-risk by the rule. Statistical analysis was performed using SAS, version 8.2 (SAS Institute, Cary, NC, USA).

Results and discussion

The population analyzed consisted of 2,890 patients. Thirty-two percent of patients were female, and there were no significant differences between IC cases and non-IC cases with respect to sex. The mean age of patients was 59 years (range 19–97 years). Patients who developed infection were on average 5 years younger than those who did not (mean age 54.7 vs. 59.6 years, respectively; Wilcoxon rank sum test p=0.0033). The mean duration of ICU stay for the eligible patient population was 11 days (range 4–371 days, median 8 days). Patients who developed infection tended to stay in the ICU an average of 11 days longer than those who did not (mean length of stay 21.6 vs. 10.8 days, respectively; Wilcoxon rank sum test p<0.0001). The incidence of the most significant risk factors for IC in the population and the results of univariate analysis are shown in Table 1.
Table 1

Rates of invasive candidiasis based on single risk factors

Risk factor

No. of patients with risk factor (% of total)

No. of cases with risk factor (% of total)

Infection rate among patients

p-valuea

Without risk factor (%)

With risk factor (%)

Diabetes

772 (26.7)

26 (29.6)

2.9

3.4

0.5420

BMT

8 (0.3)

1 (1.1)

3.0

12.5

0.1192

TPN

 D −7–0

88 (3.1)

5 (5.8)

3.0

5.7

0.1449

 D 1–3

247 (8.6)

19 (21.6)

2.6

7.7

<0.0001

A Abx

 D −7–0

1,190 (41.7)

54 (62.1)

2.0

4.5

<0.0001

 D 1–3

2,525 (87.4)

86 (97.7)

0.6

3.4

0.0030

BS Abx

 D −7–0

1,066 (37.4)

48 (55.2)

2.2

4.5

0.0005

 D 1–3

2,304 (79.7)

84 (95.5)

0.7

3.7

0.0002

CVC

 D −7–0

503 (17.7)

22 (25.3)

2.8

4.4

0.0601

 D 1–3

1,981 (68.6)

79 (89.8)

1.0

4.0

<0.0001

Any surgery

 D −7–0

262 (9.2)

12 (13.8)

2.9

4.6

0.1309

 D 1–3

1,013 (35.1)

13 (14.8)

4.0

1.3

<0.0001

Abdominal surgery

 D −7–0

134 (4.7)

9 (10.3)

2.9

6.7

0.0115

 D 1–3

425 (14.7)

8 (9.1)

3.3

1.9

0.1310

Pancreatitis

 D −7–0

52 (1.8)

4 (4.6)

3.0

7.7

0.0495

 D 1–3

55 (1.9)

3 (3.4)

3.0

5.5

0.2938

Neutropenia

 D −7–0

23 (0.8)

1 (1.2)

3.0

4.4

0.7167

 D 1–3

26 (0.9)

2 (2.3)

3.0

7.7

0.1660

Malignancy

 D −7–0

457 (16.0)

13 (14.9)

3.1

2.8

0.7801

 D 1–3

428 (14.8)

13 (14.8)

3.1

3.0

0.9921

Steroids,

 D −7–0

453 (15.9)

24 (27.6)

2.6

5.3

0.0024

 D 1–3

717 (24.8)

35 (39.8)

2.4

4.9

0.0010

Other immunosuppressives

 D −7–0

120 (4.2)

10 (11.5)

2.8

8.3

0.0006

 D 1–3

122 (4.2)

8 (9.1)

2.9

6.6

0.0211

Any dialysis

 D −7–0

137 (4.8)

8 (9.2)

2.9

5.8

0.0516

 D 1–3

247 (8.6)

18 (20.5)

2.7

7.3

<0.0001

New dialysis, D −7–3

145 (5.1)

11 (12.5)

2.8

7.6

0.0013

BMT bone marrow transplant, TPN total parenteral nutrition, D day, A Abx any antibiotics, BS Abx broad-spectrum antibiotics, CVC central venous catheter

aCochran-Mantel-Haenszel chi-square test of association between risk factor and infection status

In the total sample, 88 cases of IC occurred from day 4 of ICU admission to day 7 after ICU discharge. There were 84 proven cases and four probable cases. Thus, the overall rate of proven-probable IC we observed was 3.0%. Of the 88 valid cases, Candida was recovered from blood in 72 cases and from sterile sites in 16. Infection rates ranged from 0.8 to 12.5% in the different units.

Descriptions and performances of the three finalist rules are presented in Table 2. As seen in the table, the rate of infection among patients identified by rule 3 was 9.9% vs. only 2.3% among those not identified (30/303 vs. 58/2,556 patients; p = <0.0001) in the overall population. This rule was also highly selective, enrolling only 11% of patients. However, just over one-third (34%) of cases were captured. Noteworthy is the fact that the C-statistic for the model with rule 3 only was 0.62, while an overall logistic model with all predictors included in rule 3 was 0.73, indicating only modest prediction; however, the performance of rule 3 on the validation sample was as follows: IC rate 11.8 vs. 1.9% with a RR of 6.11 (2,72, 13.7, p < 0.0001). This rule applied to 12% of patients, capturing 45.5% of IC cases.
Table 2

Post-hoc performance of selected predictive rules on the complete population analyzed

Rulea

Rule description

No. of patients selected by rule (% of total)

No. of cases selected by rule (% of total)

Infection rate among IC patients

Relative riskb

p-valuec

Sensitivity

Specificity

PPV

NPV

(n = 2,890)

Not selected by rule (%)

Selected by rule (%)

1 (n = 2,889)

Any antibiotic use (day 1–3) AND CVC (day 1–3)

1,801 (62.3)

78 (88.6)

0.9

4.3

4.71 (2.45, 9.06)

<0.001

0.89

0.38

0.04

0.99

2 (n = 2,879)

Any antibiotic use (day 1–3) AND CVC (day 1–3) AND at least one of the following additional risk factors: any surgery (day −7–0); immunosuppressive use (day −7–0); pancreatitis (day −7–0); TPN (day 1–3); any dialysis (day 1–3); steroid use (day −7–3)

916 (31.8)

58 (65.9)

1.5

6.3

4.14 (2.69, 6.39)

<0.001

0.66

0.69

0.06

0.98

3 (n = 2,859)

Any antibiotic use (day 1–3) OR CVC (day 1–3) AND at least two of the following additional risk factors: any surgery (day −7–0); immunosuppressive use (day −7–0); pancreatitis (day −7–0); TPN (day 1–3); any dialysis (day 1–3); steroid use (day −7–3)

303 (10.6)

30 (34.1)

2.3

9.9

4.36 (2.85, 6.67)

<0.001

0.34

0.90

0.09

0.97

IC invasive candidiasis, CVC central venous catheter, TPN total parenteral nutrition, PPV positive predictive value, NPV negative predictive value

aOutcome information was available for a total of 2,890 subjects. Because some of these subjects had missing information for some risk factors, the assessment of performance for each particular rule excluded patients with missing risk factor data that precluded patient risk status ascertainment for that rule

bRelative risk of infection (95% confidence interval) for patients selected by the rule vs. those not selected by the rule

cCochran-Mantel-Haenszel chi-square test of association between risk factor and infection status

Although not universally inclusive, the clinical prediction rule reported on here consistently identifies patients at increased risk for IC. Other clinical prediction rules for IC have been developed and published recently; among them, the most notable examples are those of Paphitou et al. [13] and Dupont et al. [14]. Other prediction rules can be inferred from the enrollment criteria of clinical trials of prophylaxis [4, 5, 6]. None of these rules have been validated systematically for risk prediction in a multicenter setting.

Aside from the retrospective nature of the study, a potential limitation of this data set is the exclusion of patients who were receiving antifungal agents or those whose antifungal drug status was unknown upon ICU admission through day 3. While this approach was methodologically necessary to exclude patients who may have had baseline fungal infections, a substantial number of high-risk patients may have been excluded. Another limitation is the lack of information on severity of illness for the patients and detailed microbiology of the Candida species, which would have allowed broader comparison to other centers and, in turn, wider applicability of the results.

The incidence of IC in this study is lower than the incidence of IC in classic prophylaxis studies. This may be related to the stricter definitions we chose, or to differences in the acuity, patient mix, and range of differences in the incidence of IC in the units we studied. It is also important to consider that the prophylaxis studies were conducted in single centers that could have a particularly high incidence of IC, consistent with some of the units in our study, but that represent facilities with a more narrow range of risk than those represented here. While some researchers might question the absence of important risk factors such as Candida colonization and severity of illness scores from our rule, we decided to exclude those risk factors in order to create a clinically useful and practical rule that would identify patients early in their ICU admission without causing substantial losses of time or cost for patients and the hospital, or undue laboratory burdens.

Although limited by its retrospective nature, and only capturing 34.1% of patients with IC, this study represents the first multicenter validation of a clinical prediction rule for identifying patients at increased risk of IC. The obvious next step is prospective validation of this clinical prediction rule for IC. Clinicians should wait for prospective validation before adopting this or any other rule in their routine clinical practice.

Acknowledgments

This project was supported by a grant from Merck and Co, Inc. and funded in part by the National Institute of Allergy and Infectious Diseases, National Institutes of Health under Contract Numbers N01-AI-15440 and N01-AI-15441. V.K. wishes to acknowledge the participation of the VA Medical Center, Washington, DC in this study. The authors wish to thank T. Nolen for her independent validation of the statistics and data-set and P. Stephenson for her independent statistical review of this manuscript. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  • L. Ostrosky-Zeichner
    • 1
  • C. Sable
    • 2
  • J. Sobel
    • 3
  • B. D. Alexander
    • 4
  • G. Donowitz
    • 5
  • V. Kan
    • 6
  • C. A. Kauffman
    • 7
  • D. Kett
    • 8
  • R. A. Larsen
    • 9
  • V. Morrison
    • 10
  • M. Nucci
    • 11
  • P. G. Pappas
    • 12
  • M. E. Bradley
    • 13
  • S. Major
    • 14
  • L. Zimmer
    • 14
  • D. Wallace
    • 14
  • W. E. Dismukes
    • 12
  • J. H. Rex
    • 15
  1. 1.University of Texas Health Science Center at HoustonHoustonUSA
  2. 2.Merck Research LaboratoriesBlue BellUSA
  3. 3.Wayne State UniversityDetroitUSA
  4. 4.Duke University Medical CenterDurhamUSA
  5. 5.University of Virginia Health SystemCharlottesvilleUSA
  6. 6.VA Medical CenterWashingtonUSA
  7. 7.University of Michigan Medical CenterAnn ArborUSA
  8. 8.Jackson Memorial HospitalMiamiUSA
  9. 9.Univeristy of Southern CaliforniaLos AngelesUSA
  10. 10.VA Medical CenterMinneapolisUSA
  11. 11.Hospital Universitário Clementino Fraga Filho Universidade Federal do Rio de Janeiro (UFRJ)Rio de JaneiroBrazil
  12. 12.University of Alabama at BirminghamBirminghamUSA
  13. 13.University of Alabama at BirminghamBirminghamUSA
  14. 14.RhoChapel HillUSA
  15. 15.AstraZeneca, Alderley HouseMacclesfieldUK

Personalised recommendations