Summary

Cancer patients have a high prevalence of risk factors for ESBL-PE infection. The developed scoring system did not adequately discriminate patients with ESBL-PE. In high-risk population other strategies should be sought to identify patients at risk of ESBL-PE infection.

Introduction

Infections caused by ESBL-PE (Escherichia coli, Klebsiella spp. and Proteus mirabilis) are a major clinical problem and represent a growing proportion of infections acquired in the community and associated with health care worldwide [1,2,3,4].

The delay in the initiation of adequate antibiotic treatment against infections caused by ESBL-PE is associated with increased morbidity, duration of hospitalizations, mortality and treatment-related costs [5].

Carbapenems are considered the antibiotics of choice for the treatment of ESBL-PE infections given their stability against the hydrolytic activity of ESBL. However, excessive use of carbapenems promotes the selection of microorganisms resistant to these antibiotics, leaving few therapeutic options to treat infections by resistant microorganisms [6,7,8].

Identification of patients at risk of ESBL-PE infections could allow timely and adequate selection of appropriate empirical antibiotic treatment, reducing treatment failure, complications, antibiotic costs and inappropriate use of carbapenems with the risk of selecting resistant microorganisms [9, 10].

Cancer patients represent an intrinsically vulnerable population to infections, particularly to ESBL-PE. It is recognized that the depressed immune system and frequent lesions in the gastrointestinal mucosa and skin barriers due to surgical interventions, invasive devices or cytotoxic chemotherapy, facilitate the translocation or invasion of tissues and bloodstream by ESBL-PE [11,12,13,14]. In addition, frequent use of antibiotics as prophylaxis or treatment has also been linked to increased ESBL-PE infections in cancer patients [15,16,17].

Several clinical scoring tools have been developed to identify patients at risk of ESBL-PE infection, with different populations and heterogeneous findings [18], although none of them evaluate specifically patients with solid or hematologic malignancies.

The objective of this study was to develop and validate a reliable and easy-to-use clinical scoring system to identify patients with solid or hematologic malignancies with a high risk of ESBL-PE infections at the National Cancer Institute of Colombia (Instituto Nacional de Cancerología).

Methods

Study design and study site

A retrospective, analytical study was conducted in cancer patients with documented microbiological isolation during hospitalization at the National Cancer Institute of Colombia; an institution located in Bogotá with 180 beds and an admission rate of 14,000 patients/year, national reference center for solid and hematological cancer care in the country.

A case-control study (Derivation phase) was carried out to identified risk factors in cancer patients for microbiological isolation of ESBL-PE in any clinical sample, then a scoring system was developed assigning weight to each factor. The scoring system was evaluated in a cohort admitted on a different date (Validation phase).

Inclusion criteria and case definition

Patients admitted to the hospital of any sex and age, with a confirmed oncological diagnosis (independent of stage or activity of cancer) and microbiological isolation of Enterobacteriaceae in any clinical sample since admission to the hospital were included. Patients with prior isolation of ESBL-PE or incomplete information of the study variables were excluded. Only the first culture (index) was taken into account in case of more than one isolation. The cases consisted of patients admitted between 1 January 2013 and 31 December 2015, with ESBL-PE isolation; two controls were sought for each case with isolation of non-ESBL-PE, paired for the same month and hospitalization ward at the time of index culture. Patients included in the Validation phase were admitted between 1 January 2016 and 31 December 2017, with the same eligibility criteria. Patients were only included in a single phase of the study.

Collection of data and variables

Patients were selected from the microbiology laboratory database and the inclusion performed chronologically until the sample size was completed. The closest control to the date of the index culture of the corresponding case was selected. Information was obtained from laboratory reports and electronic medical records. The study variables included demographic information (sex, admission site, hospitalization room), cancer-related (hematological or solid malignancy, active or remission malignancy according to the medical record of the treating physician), comorbidities (liver disease, kidney disease, diabetes mellitus, lung disease chronic obstructive, heart failure, cerebrovascular disease, dementia, connective tissue disease, acquired immunodeficiency syndrome), Charlson comorbidity index [19], healthcare-related (hospitalization during the last year, length of hospital stay (≥7 days), immunosuppression [prednisone 7.5 mg per day, tacrolimus, sirolimus, cyclosporine, mycophenolate, during the previous 2 weeks] radiation therapy in the previous 3 months, chemotherapy in the previous 3 months, neutropenia at the time of culture, use of invasive devices [central venous catheter, bladder catheter, surgical drains or nasogastric tube], surgeries during the last year, antimicrobial use in the last month) and microbiological (isolated microorganism, antimicrobial susceptibility). To reduce errors in data capture, the information was recorded electronically in REDCap® and was independently verified by an institutional monitor to identify discrepancies.

Microbiological analysis

Cultures were ordered according to criteria of the attending physician and carried out following institutional protocols. The Vitek 2 system (bioMerieux, Inc., Hazelwood, MO) or Phoenix (Becton Dickinson Microbiology Systems) fluorimetry and colorimetry for species identification and colorimetry and turbidimetry for susceptibility assessment was used. Phenotypic detection was performed according to recommendations and cut-off points for cefotaxime and ceftazidime and the combination of both with clavulanic acid according to the Vitek 2 and Phoenix 100 panel following the recommendations and current cut-off points of CLSI for Colombia without confirmation by molecular biology [20].

Sample size

For the derivation phase, a number of 10 events per variable included in the model was taken into account [21]. A total of 28 study variables were evaluated, 280 cases (ESBL-PE event) and 560 controls were estimated to be included, with a ratio of cases to controls of 1:2. For the validation phase, 985 patients were included, taking into account an estimated sensitivity and specificity of the scoring system of 80%, an approximate prevalence of ESBL-PE infections of 25% [3, 22] and an accuracy of 5% around the estimator.

Statistical analysis

For categorical variables absolute and relative frequencies were calculated, the Chi2 test or Fisher’s exact test was applied to estimate differences between groups. For continuous variables, normality assumptions were validated from the Shapiro-Wilk test. In variables with normal distribution, averages and standard deviations were calculated, otherwise, medians and ranges were estimated. To establish comparisons between groups, the T-Student or U-Mann-Whitney test was applied. In the bivariate analysis, those variables with a significance value of less than 0.15 were incorporated into the multivariable conditional logistic regression analysis. Variables incorporated in the final model were selected using a “stepwise” strategy, maintaining probability input values of 0.15. The final model was transformed into a raw score: assigning an equal score for each variable, and also a weighted score: each variable with a score calculated by dividing each regression coefficient by half of the smallest coefficient and rounded to the nearest integer [23]. The discriminatory power of the model was determined by calculating sensitivity, specificity and Area Under the Receiver Operating Characteristic Curve (AUC-ROC). The optimal cutting points were established by the method proposed by Liu [24]. Additionally, sensitivity, specificity, positive and negative predictive values (PPV and NPV, respectively) were estimated with different cut-off points. Except for the “stepwise” procedures, 5% significance values were used and two-tailed hypotheses were tested. All statistical analyses were performed with Stata 13®.

Ethical considerations

The study was approved by the Institutional Review Boards of the Universidad Nacional de Colombia and Instituto Nacional de Cancerología. Written informed consent was not required.

Results

Derivation phase

A total of 265 cases with a positive culture for ESBL-PE met the eligibility criteria. For 180 cases their respective 2 paired controls were found, for the remaining 85 cases, only 1 paired control met all the eligibility criteria. In total, 710 patients were included in the derivation phase. The median age for cases was 56 years, 56% were men, 231 (87%) were admitted through the emergency department, the majority had a solid tumor (70%) and active malignancy (92%). E. coli and Klebsiella spp. represented 98% of the microbiological isolates of the cases (Table 1).

Table 1 Characteristics of cancer patients with Enterobacteriaceae isolation. Derivation Phase
Full size table

The multivariable conditional logistic regression analysis showed eight variables independently associated with the microbiological isolation of ESBL-PE: Hospitalization during the last year (p = 0.000), Prolonged hospitalization ≥7 days (p = 0.000), Immunosuppressive therapy during the 3 months previous (p = 0.003), Neutropenia (p = 0.017), Exposure to Beta-lactam drugs in the previous month (p = 0.022), Use of invasive devices (p = 0.038), Neoplasm in remission (p = 0.012) and No use of chemotherapy in the last 3 months (p = 0.004). A crude and weighted scoring system was constructed with the identified variables (Table 2).

Table 2 Risk factors for ESBL-PE isolation. Derivation Phase
Full size table

The crude score achieved an acceptable ability to discriminate patients with ESBL-PE isolation from patients without ESBL-PE (AUC-ROC: 0.684 95% CI 0.646–0.722). The weighted score achieved a similar capacity (AUC-ROC: 0.687 95% CI 0.648–0.726). For simplicity, it was decided to apply the crude score in the validation cohort.

Validation phase

Consecutively, from 1 January 2016 to 26 December 2017, all patients were included until the sample size of 985 cancer patients with a positive culture for Enterobacteriaceae was completed. More than half of patients were older than 58 years, men represented 39%, the majority (51%) were admitted through the emergency department. E. coli and Klebsiella spp. accounted for 83% of microbiological isolates. The prevalence of ESBL-PE isolates was 19%. In the Validation cohort, there was a larger proportion of women (p = 004), lower frequency of hospitalization during the last year (p = 0.000), lower frequency of use of immunosuppressive therapy (p = 0.001) and lower frequency of isolation of Klebsiella spp. (p = 0.000) (Table 3).

Table 3 Comparison of patient in Derivation and Validation phase
Full size table

Scoring system application

In the Derivation phase, cases had an average and standard deviation (SD) of 4.05 +/− 1.11 points and controls 3.23 +/− 1.23 points (p = 0.000) (Fig. 1). The crude score with the best operating performance was ≥4 points, with a sensitivity of 68%, specificity of 61% and an accuracy of 63%.

Fig. 1
figure 1

Distribution of crude score. Derivation Phase

Full size image

In the Validation phase, patients with ESBL-PE had an average of 2.9 +/− 1.3 points and patients with non-ESBL-PE 2.4 +/− 1.2 points (p = 0.000) (Fig. 2). Using the cut-off point ≥4 points, a sensitivity of 32%, a specificity of 83% and accuracy of 73% was obtained (Table 4). The crude score achieved an acceptable ability to discriminate patients with ESBL-PE isolation (AUC-ROC: 0.6042 95% CI 0.560–0.647) (Fig. 3).

Fig. 2
figure 2

Distribution of crude score. Validation Phase

Full size image
Table 4 Classification of patients according to the crude score in the Derivation and Validation phase
Full size table
Fig. 3
figure 3

Receiver Operational Characteristic Curve Analysis with crude score. Derivation and validation phase

Full size image

Subgroup analysis of blood cultures (n = 184) within the Validation phase showed an average of 3.36 +/− 1.27 points for ESBL-PE and 2.59 +/− 1.21 for non- ESBL-PE (p = 0.002). With the cut-off point ≥4 points, a sensitivity of 43%, specificity of 83% and accuracy of 77% were obtained. The crude score achieved an acceptable ability to discriminate patients with ESBL-PE isolation (AUC-ROC: 0.6722 95% CI 0.569–0.774) (Supplement).

Discussion

ESBL-PE infections have increased in recent years and are a major cause of hospital and community-acquired infections [1, 3, 4, 12, 25, 26]. The carrier status in the healthy population is estimated at 14% worldwide, but in some regions, it can be as high as 46% [2]. Colonization status increases in populations at higher risk such as patients with solid or hematological malignancies, with a prevalence of 19% [25]. In Colombia, community-acquired urinary tract infections produced by Enterobacteriaceae with resistance to third-generation cephalosporins range from 3.4 to 17.2% [4].

Colonization by ESBL-PE is an important risk factor for subsequent infection or bacteremia by these microorganisms [12, 17, 27]. Other clinical characteristics of the patients, the epidemiological environment and the healthcare-related procedures are recognized as risk factors for ESBL-PE infection [9].

This study evaluated risk factors for isolation of ESBL-PE in cancer patients, derived and validated a scoring system to quickly identify these patients. The multivariable analysis identified eight variables associated with microbiological isolation of ESBL-PE, these included hospitalization in the last year, prolonged hospitalization greater than 7 days, immunosuppressive therapy by glucocorticoids or calcineurin inhibitors, presence of neutropenia, use of invasive devices, exposure to beta-lactam drugs in the previous month, remission neoplasia and no use of chemotherapy in the last 3 months. These factors confirm some already identified in previous studies [9]. The developed score includes simple variables to be evaluated clinically upon the first contact with the patient.

The crude scoring system achieved an acceptable ability to discriminate patients with ESBL-PE. Operational performance was better for the Derivation Phase; however, it fell in the Validation Phase (ROC-AUC 0.68 and 0.60). The cut-off point ≥4 points selected by the Liu method [24] that maximizes sensitivity and specificity showed generally low values in both phases (sensitivity 68 and 32%, specificity 61 and 83% respectively), also, the accuracy was low, with misclassification of 30%.

The scoring systems can not only be measured with the capacity of maximum discrimination with the ROC-AUC because the objective of their implementation must also be taken into account. If the goal is to screen hospitalized patients to determine their risk of ESBL-PE infection, a cut-off point with maximum sensitivity should be sought to capture the majority of patients at risk, with the price of lowering specificity; in this scenario, patients would receive treatment with carbapenems and contact isolation. The limitation of this approach is that many patients without ESBL-PE infection would end up receiving broad-spectrum antibiotics and contact isolation without needing. On the other hand, if the objective is to guide antibiotic therapy, specifically to avoid the indiscriminate use of carbapenems, a cut-off point with low sensitivity and high specificity should be sought [28, 29].

In Tumbarello study [23], the cut-off point of at least 3 points had high sensitivity (93%) and NPV (97%), but low specificity (62%) and PPV (45%); when the cut-off point rose to 6, sensitivity and NPV decreased to 63 and 88%, but specificity and PPV increased to 95 and 80%. In the Duke model [30], with a cut-off point of 8 points, good specificity and PPV (95 and 79%, respectively) were obtained, but low sensitivity (58%). In the Kengkla study [31] the 9-point cut-off obtained moderate sensitivity (74%) and NPV (68%) with inadequate specificity (66%). The most recent model proposed by Lee [32] obtained high sensitivity (84%) and specificity (92%) using a cut-off greater than 2 points. These studies included different populations and variables.

In the present study, setting the highest sensitivity (cut-off point ≥1) in the Derivation and Validation phases, showed that majority of patients had at least 1 risk factor (sensitivity of 100 and 99% respectively), but with a high percentage of false positives (100 and 97%). This makes the scale impractical to exclude patients without risk of ESBL-PE infection and would overestimate the empirical use of carbapenems and the need for contact isolation. Maximizing specificity (cut-off point ≥7) allows to identify patients without ESBL-PE (100% specificity), but only 1% of patients with ESBL-PE are captured, with low capacity to select patients at high risk of ESBL-PE. This way, a large number of patients who may require carbapenem treatment would not receive it and therefore the risk of mortality could increase. It is considered that with this scoring system there is no good cut-off point to predict a high risk of ESBL-PE infection nor to make decisions about the prescription of empirical antibiotics.

Previous described scoring systems may have better discriminatory performance because of the inclusion of more selected populations like the inclusion of patients within the first 48 h of admission, only with E. coli infection or presence of bacteremia [18]. This study included patients of all age ranges, patients with infection or colonization, all Enterobacteriaceae species, community and hospital-acquired infections, as well as infection of any organ. These characteristics reflect better the behavior of infections in the real life of an institution to make the results more generalizable. The subgroup analysis of blood cultures reflected similar performance to the whole validation cohort (Supplement).

The Validation cohort showed that the population attending the National Institute of Cancer of Colombia had a higher prevalence of ESBL-PE than the ambulatory population in Colombia, especially E.coli (19% vs. 6.4%) [4, 22]. Also, this population is highly morbid, approximately 70% required hospitalization in the last year, 40% received chemotherapy in the last 3 months, 45% had an invasive device and 25% was exposed to antibiotics in the last month.

In the context of patients with cancer and immunosuppression, the current therapeutic behavior is oriented to establishing the patient’s risk group and beginning empirical treatment with broad-spectrum antibiotics until microbiological isolation is achieved. This approach has been proposed recently, based on observational studies where patients are classified according to the immune status, source of infection and severity of disease presentation [7, 8]. For immunosuppressed patients (e.g. neutropenia, leukemia, lymphoma, HIV, solid organ or hematopoietic cell transplantation, cytotoxic chemotherapy or steroid use) and severely ill patients (high Pitt or APACHE II score, need for ICU or presence of septic shock) the beginning of empirical treatment with carbapenems has been recommended [8].

The results of this study allow us to conclude that in patients at high risk for ESBL-PE infections such as cancer patients, the risk score fails to adequately discriminate the patients and therefore other methods should be evaluated to early identify patients with ESBL-PE infections and to guide antibiotic therapy [33, 34].

The limitations of the study include its retrospective nature, historic medical records as source of information; the lack of inclusion of some specific variables of cancer patients (type of neoplasia, type of chemotherapy, ECOG and Karnofsky scores) which could have provided more information or allowed a better classification of patients; the absence of distinction between infection vs colonization, variables that are retrospectively difficult to discriminate and are proposed to be validated prospectively and non-discrimination between infection from the community or healthcare-associated. Finally, it is single specialized cancer institution in Colombia and therefore the results are not generalizable to other institutions with different characteristics. The findings of the neoplasia in remission and absence of chemotherapy in the previous 3 months as risk factors for ESBL-PE are inconsistent with previous studies and warrant confirmation in the context of cancer patients.

Conclusions

Cancer patients have a high prevalence of risk factors for ESBL-PE infection. The scoring system did not adequately discriminate patients with ESBL-PE. In a high-risk population, other strategies should be sought to identify patients at risk of resistant ESBL-PE infection.