Introduction

Febrile neutropenia (FN) is a major dose-limiting toxicity of systemic chemotherapy, associated with delays in treatment, hospitalization, higher costs [1, 2], and mortality ranging from 4% to 21% [1, 3]. Prophylactic administration of granulocyte-colony stimulating factor (G-CSF) decreases the risk of febrile neutropenia and infection [4, 5]. Given the significant costs associated with G-CSF, it is neither practical nor clinically appropriate to administer this agent to all patients receiving chemotherapy. Rates of FN vary substantially with different chemotherapy regimens and many commonly used regimens have a negligible risk of FN [6–8]. American Society of Clinical Oncology (ASCO) guidelines recommend G-CSF prophylaxis when the risk of FN is approximately 20% or higher [6]. These guidelines also recommend primary prophylaxis be considered in patients at increased risk due to advanced age (>65), poor performance status, pre-existing neutropenia, extensive prior chemotherapy, irradiation to a significant amount of bone marrow, a history of recurrent FN, and comorbid conditions that increase the risk of mortality with a serious infection. Despite these recommendations, limited data exist on the increase in risk associated with each of these conditions. In addition, little is known about how these risk factors may interact in the elderly who are already vulnerable [8].

In this study, we sought to create a clinical prediction model for the risk of FN in the first cycle of chemotherapy in the elderly. At a time when efforts are being made to provide more aggressive chemotherapy for older individuals, it is important to further identify those patients at highest risk of complications related to neutropenia.

Methods

Data sources

We used the SEER-Medicare database, which has been previously well-described [9]. The catchment areas (six metropolitan areas and five states) of the SEER population, not including the 2,000 expansion areas (Kentucky, Louisiana, New Jersey, and California), covers about 14% of the total US population. The SEER population is comparable to the US elderly population with respect to age, sex, and socioeconomic measures (education and poverty level). However, cancer mortality rates are slightly lower in the SEER-Medicare cohort as compared with the total US population [9].

The SEER registry has been found to capture almost all (97%) incident cancer cases based on comparisons with detailed reviews of hospital, pathology, and radiation oncology records [10, 11]. Medicare administrative data includes information on demographics, Medicare enrollment, and outpatient and inpatient claims. This includes part A coverage for inpatient care, skilled nursing facilities, home health and hospice. Part B Medicare covers outpatient care, to which 95% of Medicare beneficiaries subscribe. About 97% of all US adults age 65 years and older have Medicare as their primary insurer [9]. The combination of SEER and Medicare data together has been shown to be highly complete in determining which treatments a patient has received [12, 13].

The observation period for study inclusion was defined as 1 year before cancer diagnosis (in order to capture comorbid conditions) until 1 month after initial chemotherapy administration. We excluded those who did not receive chemotherapy within 11 months after cancer diagnosis. The Committee for the Protection of Human Subjects at the University of California Los Angeles approved this study.

Study population

We included subjects with breast, lung, prostate, and colorectal cancer, diagnosed from (1994–2005), who received chemotherapy within 11 months of their diagnosis. We excluded those with in situ carcinoma or unknown cancer stage, those eligible for Medicare coverage due to disability or end-stage renal disease, and individuals for whom Medicare claims data would not fully capture the health care services they received: (1) not enrolled in both Medicare parts A or B for ≥1 month during study period, (2) enrolled in Medicare health maintenance organization (HMO). Patients for whom the chemotherapy agents could not be identified—patients with a chemotherapy administration code but no claims for individual drugs or those receiving inpatient treatment only—were also excluded.

To allow sufficient follow-up, we excluded those who died within 28 days of initial chemotherapy administration unless they were hospitalized for FN prior to death. Finally, subjects who received G-CSF within 7 days of chemotherapy administration were excluded since this would alter their baseline risk of FN. Prophylactic antibiotic use to prevent FN use could not be evaluated within the dataset.

Predictor variables

Cancer stage was evaluated as a potential risk factor for FN using AJCC stage reported in SEER (stage I, II, III, and IV) for breast, lung, and colon cancer. After 1993, within the SEER dataset, prostate cancer was classified simply as 1 for local/regional disease and 4 for advanced disease. We examined patient demographic data from the Patient Entitlement and Diagnosis Summary File (PEDSF), including age (5 year intervals: 65–69, 70–74, 75–79, 80–84, 85 years and older), sex, and race/ethnicity (white, black, Latino, Asian, other).

Receipt of chemotherapy, as previously described [14], was identified by ICD-9 codes and J codes. The following chemotherapy agents were classified as myelosuppressive based upon a significant association with FN (p < 0.05 for Chi square): carboplatin, cisplatin, cladribine, etoposide, floxuridine, irinotecan, paclitaxel, pentostatin, streptozocin, vinblastine, vincristine, and vinorelbine.

Time between chemotherapy treatments (chemotherapy interval) was estimated using the time between the first four claims and coded as 1 (0–7 days), 2 (8-14), 3 (15–21), or 4 weeks (22–42). Subjects with variation in the length of chemotherapy intervals were classified by the initial interval. Subjects who had only one treatment were coded as “no interval.” Time from diagnosis to initial administration of chemotherapy was classified as less than 1 month, 1–3 months, and greater than 3 months.

Comorbid conditions

Since burden of illness has been shown to affect both cancer treatment [15, 16] and mortality [17], we included unweighted count of comorbid conditions in our model using conditions contained in the Charlson Comorbidity Index [18–20]. We considered a condition present if one inpatient or two outpatient claims at least 1 month apart included diagnostic codes in the 12 months prior to cancer diagnosis [19, 21]. Because individual conditions may be independent predictors of neutropenic events [1, 22], we also tested the association of FN with the individual conditions comprising the comorbidity index: myocardial infarction, congestive heart failure, peripheral vascular disease, other cardiovascular disease, dementia, chronic pulmonary disease, rheumatologic disease, peptic ulcer disease, liver disease, diabetes, paralysis, renal disease, and acquired immunodeficiency syndrome.

Defining febrile neutropenia

The outcome was limited to the first cycle of chemotherapy as more than 50% of episode of FN occur at this time and decisions to use prophylactic growth factor should be made before initiation of treatment [23, 24]. Also, chemotherapy regimens are often modified if complications occur after the first cycle of treatment. To ensure we examined FN associated with the first cycle only and not subsequent cycles, we limited the occurrence of FN to within 28 days of the first chemotherapy administration. Medicare claims do not have a specific code for FN, so we defined our outcome variable as any of the following admission diagnoses; neutropenia (ICD-9 288.0), fever of unknown origin (ICD-9 780.6), or various infectious complications (Appendix A). We also performed sensitivity analyses using hospitalization with neutropenia (288.0) alone as a narrower definition of FN.

Statistical analysis

We first randomly split the sample into a “training set” and a “validation set” with two thirds of the sample included in the training set and one third of the sample included in the validation set. We examined the bivariate relationship of demographic and clinical characteristics with febrile neutropenia using χ 2 for categorical variables and t tests for continuous variables. We used logistic regression to estimate the association of the predictor variables with febrile neutropenia. We used both forward and backward stepwise selection methods to identify the best predictors for the final model with p = 0.10 for inclusion in the model. All two-way interaction terms were also tested in the model.

We then created a prediction model using the beta coefficients from the logistic regression. To simplify the model, we multiplied the regression coefficients by a common multiplying factor (10) and rounded to the nearest integer [25]. The points assigned to each predictor ranged from −13 to 6.

Performance of the risk-stratification system in the training and validation set was quantified and compared using the receiver operating characteristic analysis [26]. The predictive accuracy of the model to identify patients at high risk of developing FN was estimated using the C-statistic, which ranges from 0.5, indicating a model that performs no better than chance alone, to 1.0, indicating perfect prediction [27]. We also calculated positive and negative predictive values for the ability of the model to evaluate a risk of FN greater than 10%.

Results

The final sample included 86,693 subjects, from an initial SEER-Medicare sample of 1,717,478 patients with breast, lung, prostate, and colorectal cancer. Subjects were excluded because they were not diagnosed between 1994–2005 (N = 490,106), were Medicare-eligible due to disability or end-stage renal disease (N = 97,590), were enrolled in managed care, not eligible for Medicare A and B, or disenrolled from Medicare for 1 or more months (N = 390,302), did not receive chemotherapy within 11 months of diagnosis (N = 320,689), were missing specific chemotherapy agents (N = 313,333) received G-CSF within 7 days of chemotherapy (N = 1,269), had in situ cancer or unknown stage at time of diagnosis (N = 5,198), or had unknown date of diagnosis (N = 11,347). We also excluded 681 men with breast cancer.

Patient characteristics are presented in Table 1. Chemotherapy regimens varied greatly between cancer types. Lung cancer was more commonly associated with comorbid conditions. The outcome variable (FN) also varied significantly between cancer sub-types with the most episodes of FN in the first cycle in subjects with lung cancer (10.2%). The initial total sample was divided randomly into the training sample (N = 58,053) and validation sample (N = 28,910). There were no significant differences in patient, disease, or treatment characteristics between the two samples.

Table 1 Patient characteristics according to type of cancer
Full size table

In bivariate analysis across all tumor types using the training dataset, younger age, female sex and white race were the only demographic characteristics correlated with the outcome (Table 2). Patients with lung and colon cancer diagnosis had an increased risk of FN while patients with prostate cancer diagnosis had markedly decreased risk of FN as compared with a breast cancer diagnosis. An increasing number of comorbid conditions were also associated with and increasing risk of FN. Being diagnosed with stage II or greater cancer was associated with increased risk of FN. In addition, having a shorter time from diagnosis to initiation of chemotherapy and use of any myelosuppressive chemotherapy agent were associated with FN.

Table 2 Bivariate correlates of febrile neutropenia in the first cycle of chemotherapy
Full size table

In multivariate analysis, the independent predictors of FN were cancer type, cancer stage (stage 2 or greater), an increasing number of comorbid conditions, and less than 1 month from time of diagnosis to initiation of chemotherapy (Table 3). Predictor variables and overall performance of the models were similar when looking at the more restrictive definition of FN (ICD-9 diagnosis of neutropenia only) and the less restrictive definition (ICD-9 diagnosis of neutropenia, infection, or fever; data not shown). The C-statistic for was 0.75 for both the training and validation datasets. We tested all 2-way interaction terms, a few of which were statistically significant, but had almost no effect on the predictive value of the model. Thus, we chose to exclude the interaction terms from the model for the sake of simplicity and ease of calculating a FN risk score.

Table 3 Multiple logistic regression predicting febrile neutropenia in the first cycle of chemotherapy (N = 63,033)
Full size table

The point values for the clinical prediction model were created from the multivariate model of the training dataset (Table 3). Correlation between the predicted probability from the multivariate model and the prediction model score was high (0.93). For each patient, individual risk score values were summed to create a total risk score. Maximum possible score was 19, and highest reached within the sample was 19.

Those with higher risk scores had a higher predicted and observed risk of FN in the first 28 days (Table 4). Among those with a score of 0 or less, the predicted risk of FN in the first 28 days was 1.6%, and the observed risk was 1.6% in the validation dataset. In contrast, those with a risk score of 13 or higher had a predicted risk of 15.0% and an observed risk of 15.5% in the validation dataset. Similar results were found in the training dataset.

Table 4 Observed and predicted proportion of patients with febrile neutropenia (FN) in the first cycle by prediction score in the derivation and validation datasets
Full size table

A cutoff of 10 points (score ≥ 10) on the FN risk score was associated with a predicted FN risk of greater than 10%. Using this cutoff, the sensitivity of the model was 24% and specificity of 93%. The positive predictive value was 12% and negative predictive value 97%.

Discussion

In this study, we created and internally validated a clinical prediction model for development of FN in the first cycle of chemotherapy among elderly patients with four common malignancies. With increasingly aggressive efforts to treat malignancies within this population, the model has the potential to help clinicians identify those patients at greatest risk of FN prior to initiation of myelosuppressive chemotherapy. Efforts were made to maintain the simplicity of the model by keeping myelosuppressive chemotherapy as a dichotomous variable and using a total number of comorbid conditions, to assure that it could be easily used with information readily available within the clinic setting. Ultimately, the model provided moderate predictive power to identify patients at higher risk of developing FN.

To our knowledge, a clinical prediction model of this nature has not previously been published. Given the difficulty in identifying a high-risk patient population, past studies have attempted to find predictors of FN and have identified patient specific factors as well as therapy and disease-related effects. Similar to previous studies, we found that increased risk of FN was associated with more advanced stage at diagnosis and comorbid conditions [24, 28]. Previous studies have found that persons older than 65 years have a higher risk of FN than those who are younger [24, 29]. Although we expected that the risk might increase with increasing age in the elderly, we did not find an association between age and FN among a sample of persons older than 65 years. Although the reason for this finding is not clear from our analysis, it may reflect the use of lower doses or less aggressive chemotherapy regimens for older persons. Alternatively, the risk of FN may be greater for those 65 years and older compared with those under age 65 years, but FN risk may not vary substantially among those older than 65 years.

A recent national cohort of prospectively enrolled patients undergoing chemotherapy found that neutropenic complications, defined as an absolute neutrophil count less than 500 or infection, were associated with anthracycline chemotherapy regimens, pre-treatment cytopenia, prior chemotherapy, low performance status, elevated blood urea nitrogen, and elevated alkaline phosphatase [30, 31]. We found a number of other agents in addition to anthracycline drugs associated with FN, which suggests that the elderly may have a different risk of FN with these agents than younger patients. However, a recent publication which summarized available clinical data noted a greater than 10% incidence of febrile neutropenia for numerous non-anthracycline based chemotherapeutic regimens [6]. We suspect that our extensive database also allowed us to identify other agents not normally captured in smaller studies. Our study lacked some of the clinical detail included in the prospective cohort; however, it is based upon a population-based sample and thus is less subject to selection bias that may result when practices and patients have to agree to participate.

Cancer type was also associated with FN in our multivariate analysis and was included in the prediction model. Numerous past studies that have evaluated risk factors for FN have focused on a single cancer type such as NHL and breast cancer [22]. Patients with hematologic malignancies have been demonstrated to have an increased risk of FN [6, 22]. The prospective study previously outlined did include a variety of solid tumor diagnosis, but they were not noted to be associated with FN in published results [31]. We suspect that the influence of cancer type on episodes of FN is related to both individual patient factors, such as overall health and performance status not captured with our comorbid illness score, as well as difference in treatment regimens that were not captured by our chemotherapy variables.

While the chemotherapy interval was not associated with FN, receiving chemotherapy within 1 month (as compared with 1–3 months or >3 months) of diagnosis was associated with a higher risk of FN. To our knowledge, timing of chemotherapy initiation has not been examined in past studies as a risk factor for development of FN. Although we do not have any supporting evidence, we suspect that timing of chemotherapy initiation may be a reflection of disease severity (i.e., more rapid therapy for more aggressive disease) or a proxy for patient-related factors that influence the choice or intensity of chemotherapy regimens.

It is important to note that the current study only identified episodes of FN in the first 28 days in order to approximate the first cycle of chemotherapy. We limited our prediction model to the first cycle because clinicians often make changes to the chemotherapy dose based upon patients' experience in the first cycle, and our dataset would not capture these changes. In addition, the decision to use G-CSF should ideally occur at the start of the first cycle since that is when half or more of the FN episodes occur and therefore should be based upon the data available to the clinician at that time [23, 24].

Given that approximately 50% of episodes of FN occur after initial chemotherapy cycle [30], we chose FN risk of 10% in the study as cutoff for our clinical prediction rule. This cutoff is based on the assumption that those with risk greater than 10% after initial cycle are similar to the population with cumulative risk greater than 20% across all cycles of chemotherapy. If the prediction rule is utilized across all four malignancies, any patient with a prediction score of 10 or greater would have 10% or greater predicted risk of developing FN with the initial chemotherapy cycle and should be considered for prophylactic growth factor administration.

This study has a number of limitations common to analysis of administrative datasets [32]. We identified chemotherapy through Medicare claims which do not accurately capture dose, which may be especially important in an elderly population where physicians may be more apt to reduce dose to avoid toxicity. Claims also lack clinical data, such as neutrophil count or functional status, which have been shown to be predictive of FN.

Because no specific ICD-9 codes exist for FN, we defined FN as a hospitalization for fever, infection, or neutropenia immediately following use of chemotherapy. Chen-Hardee used SEER-Medicare data and chart reviews to study FN in patients with non-Hodgkin's lymphoma [33]. With chart review data as the comparison, they found the ICD-9 code for neutropenia (288.0) from the Medicare data had 80% sensitivity for FN. Our definition of FN is likely more sensitive, but may misclassify others who have infections without neutropenia. However, the definition of FN did not appear to bias our estimates of the risk of FN, as the results were similar when the definition of FN was varied in a sensitivity analysis.

Despite these limitations, the current study provides important information and a potentially useful tool for clinicians treating elderly patients with chemotherapy. The clinical prediction model can easily be used with available data prior to initiation of chemotherapy. Ongoing efforts should be made with prospective cohorts with more detailed clinical data to improve the accuracy of prediction model. Specifically, laboratory data, performance status, and more detailed information on chemotherapy dosing will likely be valuable components to a final model. Before our model is used in clinical practice, our current prediction model should be tested in other cohorts to examine its performance and clarify its overall utility. Ultimately, implementation of an accurate prediction model into clinical practice would help further define the role of G-CSF on an individual patient basis and improve the appropriate use of growth factors to prevent FN after chemotherapy use.