Multicenter retrospective development and validation of a clinical prediction rule for nosocomial invasive candidiasis in the intensive care setting
- First Online:
- Cite this article as:
- Ostrosky-Zeichner, L., Sable, C., Sobel, J. et al. Eur J Clin Microbiol Infect Dis (2007) 26: 271. doi:10.1007/s10096-007-0270-z
- 989 Downloads
The study presented here was performed in order to create a rule that identifies subjects at high risk for invasive candidiasis in the intensive care setting. Retrospective review and statistical modelling were carried out on 2,890 patients who stayed at least 4 days in nine hospitals in the USA and Brazil; the overall incidence of invasive candidiasis in this group was 3% (88 cases). The best performing rule was as follows: Any systemic antibiotic (days 1–3) OR presence of a central venous catheter (days 1–3) AND at least TWO of the following—total parenteral nutrition (days 1–3), any dialysis (days 1–3), any major surgery (days −7–0), pancreatitis (days −7–0), any use of steroids (days −7–3), or use of other immunosuppressive agents (days −7–0). The rate of invasive candidiasis among patients meeting the rule was 9.9%, capturing 34% of cases in the units, with the following performance: relative risk 4.36, sensitivity 0.34, specificity 0.90, positive predictive value 0.01, and negative predictive value 0.97. The rule may identify patients at high risk of invasive candidiasis.
Candidemia is the fourth most common bloodstream infection in the USA and is independently associated with increased morbidity, mortality, and costs, particularly in the non-neutropenic critically ill adult [1, 2, 3]. Thus, strategies such as prophylaxis, preemptive, and empirical therapy in selected patients at high risk would appear appropriate. Three studies previously demonstrated the clinical utility of prophylaxis for invasive candidiasis (IC) in the ICU in single hospital or geographical settings [4, 5, 6]. These studies showed that prophylaxis may be useful in select high-risk populations. Although risk factors for IC have been described extensively [7, 8], predicting disease risk through the identification of single risk factors is nearly impossible due to the common nature of many of these factors in the ICU setting. Although very useful in single-center studies, defining colonization as a risk factor for IC is controversial [5, 7, 9, 10, 11].
The purpose of the study presented here was to develop a clinically relevant rule for the early prediction of IC using combinations of known risk factors other than colonization. This rule could potentially be applied in clinical trials that investigate prophylaxis, preemptive, or empirical therapy.
Materials and methods
A retrospective review of patient charts and electronic records was carried out for patients aged 19 or older who stayed in one of 12 participating medical and/or surgical ICUs in the USA and Brazil for more than 4 days during the 2000–2002 time period. The data were recorded using case-report forms designed specifically for this study. Patients with evidence of IC or who received systemic antifungal agents during the week prior to ICU admission through the first 3 days of ICU stay (days −7–3) were excluded. Study ICUs did not have specific policies regarding antifungal prophylaxis. The Institutional Review Board at each participating site approved the study; each board waived the need for subject informed consent.
Basic demographic details and hospital metrics were collected. Race information was not collected for this study. Data indicating the presence or absence of classically described risk factors were collected for the time period day −7 to day 3. Data on the rate of IC were collected during the outcome period, which was defined as day 4 of ICU stay through day 7 following ICU discharge. IC was defined as proven or probable using EORTC/MSG criteria .
The data were randomly split into “training” (75%) and “validation” (25%) sub-samples to develop and validate the prediction rule, respectively. Univariate analyses were used to assess the relationships between the presence or absence of each individual risk factor and subsequent development of IC. Tables detailing the frequency of risk factor versus infection status were constructed and the Cochran-Mantel-Haenszel chi-square test of association was performed; relative risk of infection was also explored. Then, several different prediction-rule “formats” (with different weights for the risk factors) were proposed. Within each format, all possible combinations of risk factors and time periods (days −7–0, days 1–3, days −7–3) were explored systematically, analyzing multiple combinations of increasing numbers of risk factors. The predictive ability of each rule was assessed using the methods described above as well as traditional performance measures such as sensitivity, specificity, positive predictive value, and negative predictive value. The “best” rule was selected by team consensus using the following criteria before it was applied to the validation sample: (1) the proportion of all IC cases contained in the subpopulation defined by the rule; (2) the risk of IC in the subpopulation defined by the rule; (3) the relative risk of IC in the subpopulation; and (4) the proportion of total ICU patients defined as high-risk by the rule. Statistical analysis was performed using SAS, version 8.2 (SAS Institute, Cary, NC, USA).
Results and discussion
Rates of invasive candidiasis based on single risk factors
No. of patients with risk factor (% of total)
No. of cases with risk factor (% of total)
Infection rate among patients
Without risk factor (%)
With risk factor (%)
New dialysis, D −7–3
In the total sample, 88 cases of IC occurred from day 4 of ICU admission to day 7 after ICU discharge. There were 84 proven cases and four probable cases. Thus, the overall rate of proven-probable IC we observed was 3.0%. Of the 88 valid cases, Candida was recovered from blood in 72 cases and from sterile sites in 16. Infection rates ranged from 0.8 to 12.5% in the different units.
Post-hoc performance of selected predictive rules on the complete population analyzed
No. of patients selected by rule (% of total)
No. of cases selected by rule (% of total)
Infection rate among IC patients
(n = 2,890)
Not selected by rule (%)
Selected by rule (%)
1 (n = 2,889)
Any antibiotic use (day 1–3) AND CVC (day 1–3)
4.71 (2.45, 9.06)
2 (n = 2,879)
Any antibiotic use (day 1–3) AND CVC (day 1–3) AND at least one of the following additional risk factors: any surgery (day −7–0); immunosuppressive use (day −7–0); pancreatitis (day −7–0); TPN (day 1–3); any dialysis (day 1–3); steroid use (day −7–3)
4.14 (2.69, 6.39)
3 (n = 2,859)
Any antibiotic use (day 1–3) OR CVC (day 1–3) AND at least two of the following additional risk factors: any surgery (day −7–0); immunosuppressive use (day −7–0); pancreatitis (day −7–0); TPN (day 1–3); any dialysis (day 1–3); steroid use (day −7–3)
4.36 (2.85, 6.67)
Although not universally inclusive, the clinical prediction rule reported on here consistently identifies patients at increased risk for IC. Other clinical prediction rules for IC have been developed and published recently; among them, the most notable examples are those of Paphitou et al.  and Dupont et al. . Other prediction rules can be inferred from the enrollment criteria of clinical trials of prophylaxis [4, 5, 6]. None of these rules have been validated systematically for risk prediction in a multicenter setting.
Aside from the retrospective nature of the study, a potential limitation of this data set is the exclusion of patients who were receiving antifungal agents or those whose antifungal drug status was unknown upon ICU admission through day 3. While this approach was methodologically necessary to exclude patients who may have had baseline fungal infections, a substantial number of high-risk patients may have been excluded. Another limitation is the lack of information on severity of illness for the patients and detailed microbiology of the Candida species, which would have allowed broader comparison to other centers and, in turn, wider applicability of the results.
The incidence of IC in this study is lower than the incidence of IC in classic prophylaxis studies. This may be related to the stricter definitions we chose, or to differences in the acuity, patient mix, and range of differences in the incidence of IC in the units we studied. It is also important to consider that the prophylaxis studies were conducted in single centers that could have a particularly high incidence of IC, consistent with some of the units in our study, but that represent facilities with a more narrow range of risk than those represented here. While some researchers might question the absence of important risk factors such as Candida colonization and severity of illness scores from our rule, we decided to exclude those risk factors in order to create a clinically useful and practical rule that would identify patients early in their ICU admission without causing substantial losses of time or cost for patients and the hospital, or undue laboratory burdens.
Although limited by its retrospective nature, and only capturing 34.1% of patients with IC, this study represents the first multicenter validation of a clinical prediction rule for identifying patients at increased risk of IC. The obvious next step is prospective validation of this clinical prediction rule for IC. Clinicians should wait for prospective validation before adopting this or any other rule in their routine clinical practice.
This project was supported by a grant from Merck and Co, Inc. and funded in part by the National Institute of Allergy and Infectious Diseases, National Institutes of Health under Contract Numbers N01-AI-15440 and N01-AI-15441. V.K. wishes to acknowledge the participation of the VA Medical Center, Washington, DC in this study. The authors wish to thank T. Nolen for her independent validation of the statistics and data-set and P. Stephenson for her independent statistical review of this manuscript. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.