Background

The median age at lung cancer diagnosis is 70 years [1]. Given the increasing probability of developing comorbidities with age, the prevalence of comorbidity is higher in lung cancer than in other cancers, with at least 50 to 70% of patients having at least one comorbidity at diagnosis [2, 3].

The negative affect of comorbidities on patient survival are well described [4,5,6]. Since the development of the Charlson Comorbidity Index (CCI) [7], several comorbidity scores have been developed and are currently used. They all differ by the origin of the initial data source (administrative data or physician-reported data), their purpose (measuring comorbidity, measuring the impact of comorbidity and physical function), and comorbidity measures (organ or system-based approaches, counts of individual conditions and weighted indices) [8] (Table S1 Supplementary Materials 1). Some have been developed using lung cancer patient cohorts [9,10,11]. Despite the large number of comorbidity scores available, the CCI is the most studied and used comorbidity index in the medical literature [12, 13].

Certain comorbidity scores are based on the International Statistical Classification of Diseases and Related Health Problems (ICD-10), as Quan et al. published ICD-10 codes relative to comorbidities in 2005 [14]. They include the CCI (updated by Quan et al. in 2011 [7, 15]), CCI for lung cancer (named later CCI-lung) (Klabunde et al. in 2007) [16], age-adjusted CCI (ACCI) [17], Elixhauser score (updated in 2009 by Van Wallraven et al. [18, 19]), Elixhauser for lung cancer (Elixhauser-lung) (Mehta et al. [20]), National Cancer Institute Combined Index (NCI) [1], and NCI for lung cancer (NCI-lung) (Klabunde et al. in 2007 [16, 21]). They differ in terms of the type of comorbidities considered, the cohort used for validation, and their initial purpose [22].

Yang et al. found that the ACCI was better at predicting three-year overall survival than the CCI and Elixhauser score in a cohort of resected lung-cancer patients [23] based on administrative data coded using the ICD-9. However, they only compared the ACCI, CCI, and Elixhauser score. More recently, Mehta et al. proposed an Elixhauser score adapted to the cancer type (breast, lung, prostate, and colorectal). The cancer-specific Elixhauser score appears to be a better prognostic score for two-year survival than the cancer-specific NCI (developed by Klabunde et al.) [20].

Although the CCI is the most widely used comorbidity score, it would be informative to assess which score is more predictive of mortality in cohorts with administrative data. Here, we compared the seven comorbidity scores available using administrative data coded using the ICD-10 in predicting four-month survival of our cohort of hospitalized lung-cancer patients.

Materials and methods

Data source and population

We included patients hospitalized in the Thoracic Oncology Unit of Grenoble Alpes University Hospital from 2011 to 2015 described in an earlier publication [24]. Lung-cancer patients were included at their first hospitalization during the studied period.

The study was approved by our institutional review board and ethics approval was obtained on September 1, 2021 (CECIC Rhône-Alpes-Auvergne, Clermont-Ferrand, IRB 5891).

The database contains information on individuals including their age, gender, lung cancer’ TNM staging, performance status at their first presentation case in multidisciplinary concertation meetings, and the histological type of the lung cancer.

Outcome and covariates

The outcome was median overall survival. Survival data were obtained from our district cancer registry, including the date of the last follow-up and the vital status at the last follow-up. Right censored date point was defined by median overall survival.

Age, gender, lung cancer metastatic status, histologic type, age at hospitalization, and age at diagnosis were included as covariates.

Comorbidity scores

Data concerning comorbidities were obtained by the Health Information Services Department and coded using the ICD-10. The diagnoses for comorbidities were recorded at the patients’ discharge in our medical unit. Seven comorbidity scores were calculated: CCI, ACCI, CCI-lung, NCI, NCI-lung, Elixhauser, and Elixhauser-lung. We did not record metastatic solid tumors and lung cancer as comorbid conditions. The seven scores are summarized in Table 1.

Table 1 Summary of differences between the seven comorbidity scores used
Full size table

Statistical analyses

For descriptive analysis, quantitative variables are expressed as medians [Interquartile ranges] and qualitative variables as n (%).

Comorbidity scores were calculated and survival estimated as the time between the day of hospitalization and the date of last follow-up (cut off at cohort’s estimated median overall survival which was our right censored date point). The Kaplan Meier estimator was used to estimate the probability of survival. Log-rank tests were used to compare the probability of the event (death) between populations. The model was adjusted for each score. A Cox proportional hazards regression model was used to perform multivariable analyses of prognostic factors and calculate hazard ratios (HRs) and 95% confidence intervals (95% CI) for median survival for the seven comorbidity scores. A median cut-off was used for continuous variables in the multivariate analysis. Proportional hazards assumptions were verified using the Martingale method [25]. Only covariables with a p-value < 0.2 were retained for multivariable analysis.

To compare comorbidity scores, Akaike Information Criteria (AIC), Bayesian Information Criteria (BIC) were used to compare the relative goodness-of-fit among different models. Then, a discrimination analysis using the c-statistic, Harrell’s c-statistic, sensitivity, specificity was performed from a base model containing significant covariables from the multivariate analysis. Sensitivity and specificity were respectively calculated as follow: (True positive = Death estimated by the model) / (True positive + false negative (= patient estimated as non-dead by the model although they are dead)); and (true negative = non-dead patients estimated by the model) / (true negative + false positive = estimated dead by the model although they are not).

The impact of the scores was compared using the base model (significant covariables in multivariable analysis) plus each index score alone by multivariable Cox regression. The model with the lowest AIC and BIC indicates which model was the best fit for the data and the highest c statistic and Harrell’s c statistic was considered to be the best predictive model.

A sensitivity analysis using a bootstrap method for each statistical indicator, with 1,000 samples from two thirds of the cohort, was performed. Each indicator (AIC, BIC, c-statistic, and Harrell’s c-statistic) was calculated for the 1,000 samples. Boxplots were generated for the four parameters.

All statistical analyses were performed using SAS 9.4 for Windows (SAS Institute, Inc., Cary, NC, USA). A p-value < 0.05 was considered significant.

Results

Descriptive analyses and adjusted hazard ratios for four-month survival using the seven comorbidity scores

In total, 633 patients were enrolled in the study. The demographic characteristics of the population are presented in Table 2. The median age [IQ25%;IQ75%] at diagnosis was 65 [58–72] years and 540 (71%) of the patients were men. The median survival from hospital admission was 4 [1; 11] months. A Kaplan Meier curve of follow-up time (which corresponds to survival) in this cohort (cut off at 4 months as 4 months was median overall survival) is represented in supplementary materials (Figure S1). Among the cohort, 428 patients (74%) had metastatic lung cancer and 295 (47%) had adenocarcinoma. The diagnosis of cancer was made before hospitalization for most of the patients (518, 82%).

Table 2 Baseline characteristics and adjusted hazard ratios of four-month survival in the cohort (n = 633)
Full size table

In multivariable analysis, only the presence of metastases, male gender, and histological type were prognostic factors of four-month survival.

We assessed the prevalence of each comorbidity that contributed to the score for each comorbidity score (Table 3 for the Elixhauser score and Elixhauser-lung and Tables S2 and S3 in Supplementary Materials for CCI, CCI-lung, NCI, and NCI-lung). For the Elixhauser score, weight loss, fluid and electrolyte disorders, and chronic pulmonary disease were the three most common comorbidities, whereas weight loss, chronic pulmonary disease, and peripheral vascular disease were the most common comorbidities for Elixhauser-lung. The common thread between the Elixhauser score, Elixhauser-lung, CCI, CCI-lung, NCI, and NCI-lung was the high prevalence of chronic pulmonary disease, which was among the three most common comorbidities in the cohort.

Table 3 Description of comorbidity in the population according to the Elixhauser score and Elixhauser-lung
Full size table

Among the seven scores, in terms of the p value and type 3 p value, an ACCI ≥ 5, Elixhauser score > 11, and Elixhauser-lung ≥ 4 were associated with lower survival. However, neither the CCI, CCI-lung, NCI, nor NCI-lung were associated with poorer survival (Table 4).

Table 4 Adjusted hazard ratios for four-month survival among the population (n = 633)
Full size table

Model comparison and discrimination analyses between the seven scores and the bootstrap method

We calculated the AIC, BIC, c-statistic, Harrell’s c-statistic, sensitivity and specificity (Table 5). The Elixhauser score had the lowest AIC and BIC. It also had highest c-statistic and Harrell’s c-statistic as discriminative parameters, indicating that this score is the best predictive model for estimating four-month survival in our cohort.

Table 5 Model comparison with AIC, BIC and discrimination parameters between the seven comorbidity scores
Full size table

We confirmed this trend by generating boxplots from the sensitivity analyses of the AIC, BIC, c-statistics, and Harrell’s c-statistic (Supplementary Figures S2).

Discussion

The CCI has been shown to be associated with poorer survival for all TNM stage lung-cancer patients [12, 26] and is the most widely used comorbidity score. Here, we performed the first study to compare seven comorbidity scores on a cohort of lung-cancer patients. Sarfati et al. suggested that the CCI, cancer-specific NCI, and Elixhauser score may be the preferred comorbidity scores when using administrative data [22]. In this study, we used the ICD-10 to identify comorbidity from administrative data and found the Elixhauser score to be the best score for predicting four-month mortality.

The Elixhauser score has already been compared to the CCI for patients with cancers other than lung cancer and found to be a better prognosis score for colorectal and oral cancer patients [27, 28]. Mehta et al. found that the lung cancer-specific Elixhauser performed better than the lung cancer-specific NCI and Elixhauser score [29]. The outcome of the aforementioned study was two-year non-cancer mortality, which had a consequence on the statistical analyses because the authors had to consider competing risks. In addition, they studied comorbidities prior to the lung cancer diagnosis. More interestingly, they also compared these scores to the individual Charlson and Elixhauser comorbidity scores. The Elixhauser individual comorbidity scores performed better than the Charlson individual comorbidity scores. However, scores have been shown to be good substitutes for individual comorbidity variables in health services research [30]. In a paper published by Yang et al., the ACCI predicted overall three-year survival better than the CCI or Elixhauser score [23]. In contrast to Mehta et al., they did not discriminate between death from cancer and other causes, but they did consider comorbidities prior to the diagnosis of lung cancer.

These scores differ not only in the way they were constructed (origin of the cohort and outcome chosen), but also in the weight assigned to each comorbidity; some use the beta coefficient obtained from the regression and others the hazard ratio. The beta coefficients and hazard ratios are related to each other by an exponential relationship, and although the use of beta coefficients is preferred when using a summary score [31], we calculated the comorbidity scores as they were described and published in the original papers.

There are several possible explanations concerning the better performance of the Elixhauser score. The Elixhauser score was developed using a short-term outcome: in-hospital mortality. The median overall survival in our cohort was four months, which is short relative to the other scores (i.e., the CCI), which were constructed using a long-term outcome, such as one- or two-year mortality. This result corroborates another publication concerning in-hospital mortality of non-cancer patients, in which the Elixhauser score outperformed the CCI [32]. Another possibility is the number of comorbidities taken into account in the Elixhauser score, which is more than for the other scores. Lung cancer patients have the most comorbidities at diagnosis relative to patients with other types of cancer, especially due to tobacco exposure [3, 33]. This could explain why the Elixhauser score best fit our cohort in predicting four-month mortality.

This study had several limitations. We assessed comorbidities that occurred both before and after lung cancer diagnosis and did not distinguish between death from lung cancer and that from other causes. Extension of this paper results should be done with one caution. Despite we had 71% of men and 74.2% of patients with metastatic status at diagnosis, which is similar to literature, age have a non-significant effect on survival. This may be due to the inclusion criteria which is hospitalized patients and therefore frailty ones with the shortest survivals, and high comorbidity burden (Median Elixhauser score of 6). Because performance test has been performed on the same data used to train the model there will be a need for external validity of the results. There may have also been unknown confounders. Moreover, this was a retrospective monocentric study.

The use of ICD-10 codes to identify the comorbidities was a strength of our study, as they can be used to query easily available structured datasets and allow the comparison of comorbidity scores, as well as sensitivity analyses, which confirmed the superiority of the Elixhauser score for estimating four-month survival in our cohort.

Conclusions

Despite the extensive use of the CCI in the literature, other comorbidity scores are available, including scores based on administrative data coded using the ICD-10. In this original study, in which we compared seven comorbidity scores using administrative data, the Elixhauser score was the comorbidity score best suited to hospitalized lung-cancer patients for predicting four-month mortality. It could be informative to repeat these analyses with a longer follow-up of the patients.