Background

Lung cancer is the leading cause of cancer incidence and mortality globally, of which non-small cell lung cancer (NSCLC) comprises 85% [1]. In Australia, the majority of NSCLC patients are diagnosed at stage I-III, with guidelines recommending curative radiotherapy in for those who are medically inoperable or refuse surgery [2].

Over the last two decades, while there has been an increase in the use of curative treatment [3], significant patterns of radiotherapy underutilisation persist across Australia and internationally [4]. Despite optimal radiotherapy rates being 61–76% of NSCLC patients, in practice this falls to 42–43% [5] due to a multitude of reasons including patient comorbidity and clinician bias. One strategy for addressing such variation is the development of a survival prediction model that integrates individual, medical and environmental factors unaccounted for by guidelines that commonly influence treatment decisions. This would have the potential to objectively evaluate treatment benefits in individual patients to facilitate shared decision-making, tailor patient management and optimise outcomes [6].

At present, the tumour, node, and metastasis (TNM) classification is considered gold standard for NSCLC prognostication. However stage alone is a poor predictor of overall survival, accounting for less than half of prognostic variance [7]. NSCLC patients within the same anatomic stratification are inherently heterogenous, with actual prognosis depending on a complex interplay of patient, tumour and treatment characteristics [8]. To accurately predict NSCLC survival beyond TNM stage and clinical judgement alone [9], quantitative survival prediction models that can be applied to specific patient profiles must account for a range of predictive factors, reflect current practice and demonstrate higher concordance than existing prognostication methods.

While several models have been published, none have demonstrated superior performance, applicability or global utility [11]. There is considerable discordance in the factors included in prognostic tools, with a systematic review discussing incomplete coverage of established predictors and the incorporation of variables that are difficult to measure as key shortcomings of published models [10]. Additionally, the discriminatory accuracies of existing tools have generally been insufficient to justify deviation from conventional staging systems [11, 12]. Furthermore, with the development of newer radiotherapy protocols, targeted therapies and immunotherapy, earlier studies fail to capture contemporary approaches to NSCLC and are no longer clinically relevant [13]. There is a need for prediction models that incorporate comprehensive data from cohorts treated with modern radiotherapy techniques, and that encompass emerging factors such as mutation and programmed cell death-ligand-1 (PD- L1) status [14].

The primary aim of this study was to develop and validate a 2-year survival prediction model in a contemporary cohort of stage I–III NSCLC patients treated with radiotherapy. Secondary aims were to compare model survival predictions to those predicted by TNM stage and validate published survival prediction models in a comparable cohort of patients.

Methods

Population

This retrospective cohort study included patients diagnosed with inoperable stage I-III NSCLC between January 2016 and December 2017 at three Australian radiotherapy treatment institutions. Tumour staging was performed based on multidisciplinary team recommendations. All patients treated radically were staged using positron emission tomography-computed tomography (PET-CT), with confirmation using endobronchial ultrasound and biopsy in instances of uncertainty or where there was the potential to influence management. Patients who received radiotherapy alone or chemoradiotherapy (concurrent or sequential) were eligible for inclusion. Patients treated surgically or for recurrent disease were excluded. The development cohort comprised patients from South Western Sydney Local Health District (SWSLHD). The validation cohort included patients from Blacktown Cancer and Haematology Centre and Illawarra Cancer Care Centre (BICC).

Data

Retrospective data were retrieved through automated and manual extraction methods using the electronic medical record systems MOSAIQ (Elekta AB, Stockholm, Sweden), ARIA (Varian Medical Systems, Palo Alto, CA) and Cerner Powerchart (Cerner Corp, North Kansas City, MO). Gross tumour volume (GTV) data were obtained from radiotherapy planning systems for patients who received radiotherapy.

Data were collected for all available predictive variables for survival as identified from a prior literature review. Patient-related variables included: age at diagnosis, sex, current smoking status, pack years smoked, weight loss, pre-treatment pulmonary function (percent predicted values for forced expiratory volume in 1 s (FEV1) and diffusing capacity of the lung for carbon monoxide (DLCO)), Eastern Cooperative Oncology Group (ECOG) performance status [15] and comorbidities as defined by the Simplified Comorbidity Score (SCS) [16].

Tumour-related variables were: TNM stage according to the International Association for the Study of Lung Cancer (IASLC) 8th edition [17], histology, tumour grade, GTV, tumour location, mutation status (epidermal growth factor receptor (EGFR), anaplastic lymphoma kinase (ALK) and V-raf murine sarcoma viral oncogene homolog B (BRAF)) and PD-L1 status [18]. GTV was defined as the sum of the GTV primary and GTV lymph nodes.

Treatment-related variables included radiotherapy technique, radiotherapy treatment duration, equivalent dose in 2 Gy fractions (EQD2) and use of chemotherapy. Radiotherapy across the three cohorts included both conventional and stereotactic ablative body radiotherapy (SABR), with radiotherapy technique classified as conformal, intensity-modulated radiotherapy (IMRT) and volumetric modulated arc therapy (VMAT).

The primary outcome of overall survival was recorded as patient status at 31/12/2019 with all follow-up data obtained before this study end date. Survival time was defined as the period from the start of radiation therapy until date of death or until 31/12/2019 for living patients.

Statistical analysis

The Kaplan–Meier method was used to predict survival in the study population. Variables missing > 50% of data were excluded from univariate and multivariate analysis. In the development cohort, univariate Cox proportional hazards regression was used to evaluate the predictive value of variables, with those demonstrating an association with overall survival (p < 0.20) considered for inclusion in multivariate analysis. Backward stepwise regression was applied to select the variables retained in the final multivariate model [19]. Model fit was evaluated using the Hosmer–Lemeshow goodness of fit test.

A scoring system for the South West Sydney Model (SWSM) was generated using the logarithm of the odds ratio (OR) to allocate points for each variable. Risk groups were defined according to total score quartiles in the development cohort. Kaplan–Meier curves were generated and log-rank test used to evaluate significant survival differences between subgroups. The model was applied to the SWSLHD cohort to assess internal validity, with discrimination estimated using Harrell’s concordance index (c-index) [20]. To assess the impact of missing DLCO data on model performance, validation was performed twice, initially with missing data excluded and again with simple mean imputation. Model calibration was assessed graphically by plotting observed survival probabilities against predicted probabilities and calculating the calibration slope and intercept for the development and validation cohorts [21]. External validation was performed by applying the SWSM to the BICC cohort.

The performance of the multivariate model was compared with predictions based on TNM staging alone. In addition, an existing prediction model was externally validated in the development and validation cohorts and compared to the SWSM. The discrimination of the model was also assessed using Harrell’s c-index.

Statistical analyses were performed using IBM SPSS Statistics, version 26.0 (IBM Corp, Armonk, NY), Matlab, version 9.3 (MathWorks, Natick, MA) and Python, version 3.6 (Python Software Foundation, Wilmington, DE). Ethics approval was obtained from the SWSLHD Human Research Ethics Committee.

Results

The patient, tumour and treatment characteristics of each study site are summarised in Table 1. There were a total of 261 patients included in the study. At the study end date, 47.5% of patients were alive. In the development cohort, mutation status was unknown in 65.0% and PD-L1 status in 85.5% of patients and these variables were excluded from analysis.

Table 1 Characteristics of the development and validation datasets
Full size table

By univariate analysis, the variables predictive of survival in the study population were ECOG performance status, DLCO, overall stage, histological diagnosis, tumour lobe, radiotherapy technique, radiotherapy treatment duration and EQD2 (Table 2). On multivariate analysis, the variables predictive of survival were ECOG performance status, DLCO, histological diagnosis, tumour lobe and EQD2 (Table 3). The Hosmer–Lemeshow test demonstrated good model fit using the selected predictors (p = 0.65). The scoring system for the SWSM is presented in Table 4.

Table 2 Survival predictors in the development cohort by univariate and multivariate Cox proportional hazards regression
Full size table
Table 3 Odds ratios of SWSM for overall survival by multivariate analysis
Full size table
Table 4 Scoring system for the SWSM
Full size table

Internal validation of the SWSM model demonstrated a c-index of 0.70, 95% CI [0.64, 0.75]. Discrimination on the external validation cohort resulted in a c-index of 0.72, 95% CI [0.68, 0.76] with imputation and 0.67, 95% CI [0.58, 0.74] with missing DLCO data excluded. Calibration of the SWSM yielded a calibration slope of 0.89, 95% CI [0.40, 1.34] and intercept of 0.19, 95% CI [0.01, 0.35] in the development cohort and a calibration slope of 0.98, 95% CI [0.77, 1.16] and intercept of 0.16, 95% CI [0.06, 0.26] in the validation cohort (Fig. 1).

Fig. 1
figure 1

Calibration plots for the SWSM. SWSM calibration plots in the A development cohort (n = 117) and B validation cohort (n = 144)

Full size image

The formation of four risk groups in the SWSLHD cohort according to SWSM quartiles resulted in no differences between groups 2 and 3. Subsequently these groups were combined, producing a total of three risk groups. Kaplan–Meier survival curves by SWSM risk group in the development and validation cohorts are presented in Fig. 2. Log-rank test identified significant differences (p < 0.05) between risk groups in both development and validation cohorts. Using the SWSM, 2-year survival probability in the development cohort was 63.3% in group 1, 55.0% in group 2 and 20.0% in group 3.

Fig. 2
figure 2

Survival curves according to SWSM risk group. Kaplan–Meier curves for cumulative survival of NSCLC patients by SWSM risk group in A development cohort (n = 117) and B validation cohort (n = 144). SWSM = South West Sydney Model

Full size image

The c-index of TNM staging alone for survival prediction was 0.53, 95% CI [0.48, 0.58] in the SWSLHD cohort and 0.59, 95% CI [0.55, 0.63] in the BICC cohort. An existing model from the MAASTRO clinic [13] was externally validated to give c-indexes of 0.62, 95% CI [0.56, 0.67] in the SWSLHD cohort and 0.66, 95% CI [0.61, 0.70]in the BICC cohort.

Discussion

To our knowledge, this study is the first to develop and validate a survival prediction model in a cohort of inoperable but potentially curable stage I-III NSCLC patients irrespective of radiotherapy intent. Developing a survival prediction model in this cohort may impact on management decisions as although in theory all patients should be treated with curative radiotherapy, this does not happen in real world practice. The SWSM incorporates a combination of well-established and novel predictors using routinely collected data to predict survival in a radiotherapy cohort. Notably, our incorporation of EQD2 enables the SWSM to be applied as a predictive models with the potential to facilitate treatment decisions and radiotherapy planning.

When evaluating the predictive performance of a model, a c-index greater than 0.6 generally reflects helpful discrimination [22]. In the development cohort, the SWSM demonstrated good predictive performance, and achieved similar results on external validation in one independent cohort. There was minimal difference between the c-indexes obtained on external validation when missing data were excluded or imputed. Importantly, the SWSM maintained discrimination superior to survival predictions based on TNM staging alone. This is in accordance with published models whereby the addition of demographic and clinical covariates translated into more robust survival predictions [14, 23, 24].

In prediction models, calibration illustrates the association between predicted and observed outcomes. A slope of one and intercept of zero indicate perfect calibration [25]. While our model calibration plot showed good agreement between expected and actual survival probabilities, the slope and positive intercept are suggestive of survival underestimation. Validation in a larger population is required for the generation of a more precise calibration curve.

Several models predicting survival in inoperable NSCLC cohorts have been published [11, 14, 15], however none have examined the entire inoperable stage I-III radiotherapy cohort in a contemporary setting. The MAASTRO model is a prognostic tool for 2-year survival of stage I-III NSCLC patients treated with curative chemoradiotherapy. However, since its 2009 publication, staging classifications and radiotherapy techniques have been updated. The c-indexes obtained during external validation of the SWSM were higher than those obtained on validation of the MAASTRO model [13]. In addition, the advantages of our model include its reflection of current practice and potential to be applied to patients receiving curative or palliative therapy.

Similarly, models published more recently have been limited to patients receiving curative radiotherapy or with early or localised disease. The STEPS (sex, T stage, Staging EBUS, performance status, N stage) score was developed in the UK to predict 2-year risk of death in stage I-III NSCLC and also contained five variables [26]. However, only patient and tumour-related factors were retained in this multivariate model, precluding its use in treatment decision-making. Another model developed in Japan only included patient-related variables (age, performance status, body mass index and Charlson comorbidity index) to predict non-lung cancer death in an elderly cohort receiving definitive SABR [27]. In contrast to these models, we deliberately chose to include a potentially curable population of patients with stage I-III NSCLC who were managed heterogeneously in order to develop a prediction model which can help decision-making in the real world.

During the development of the SWSM, the predictive variables considered for model inclusion encompassed patient, tumour and treatment factors as supported by current evidence. The patient-related determinants most commonly included in NSCLC models are age, sex and performance status. While a poorer prognosis has been associated with older age [27] and male sex [8] overall findings have been inconclusive. In line with previous models, our study found no significant associations between age and sex on the survival outcomes of the study population [28, 29]. In contrast, performance status has been associated with NSCLC survival in patients receiving curative radiotherapy [23, 26, 28, 30] and was retained in our final model. However, a recognised limitation of performance status as a predictive variable is its subjectivity and inter-observer variability [27].

The only other patient-related variable identified as a survival predictor was pre-treatment DLCO. This has been demonstrated in a model involving NSCLC patients treated with SABR [30]. Insufficient lung function is a common reason for medical inoperability and frequently determines the suitability of treatment, with one study identifying pre-treatment DLCO as the pulmonary function measure most strongly associated with overall survival [31]. While DLCO has been reported to influence NSCLC survival in the surgical literature [32], few studies have analysed its influence on inoperable cohorts. The results of this study could be used to support further research exploring this association.

The tumour-related characteristics included in the SWSM were histological diagnosis and tumour lobe. Consistent with prior studies, non-squamous cell tumours demonstrated better survival probability than squamous cell carcinomas [17, 33]. The inclusion of tumour lobe in the SWSM is novel. A recent systematic review concluded that tumours located in the upper lobe conferred improved survival compared to those in the middle or lower lobes [34], consistent with our findings. The increased treatment toxicities from higher cardiac dose for middle lobe tumours and higher lung dose for lower lobe tumours may explain this result. Furthermore, the lower lobe has been associated with an increased proportion of non-adenocarcinoma tumours and a lower frequency of EGFR mutations, both of which are unfavourable survival characteristics [35]. However, at present, evidence supporting the significance of tumour location as an independent predictor of survival is less established.

The inclusion of the treatment variable EQD2 in the SWSM allows its application as a predictive rather than a prognostic model [36]. By including EQD2 in the prediction model, using the scoring in Table 4, one can calculate survival depending on dosage regimen in an individual patient and counsel patients accordingly about risks and benefits of treatment. Some patients who only derive a small survival benefit from more intensive radiotherapy may not wish to risk the toxicities of treatment [37]. Others may choose to undergo higher dose radiotherapy for any survival benefit no matter how small. The model attempts to provide some objectivity to aid decision-making rather than relying on clinician judgement alone. Few studies have considered EQD2 as a variable [23] as most have been developed in a specific population only receiving curative treatment [11, 37]. Furthermore, while the survival benefit of increasing radiotherapy dose has been demonstrated, its ability to improve quality of life is yet to be established [38].

There are limitations to this study. The SWSM was developed using single institution data with a relatively small sample size, although the sample size is similar to other studies [37, 39]. The model was developed on the population demographic of South West Sydney, which has a higher proportion of overseas-born individuals compared to Australia as a whole, hence this may impact generalisability. This was a retrospective study relying on information documented in medical records resulting in inevitable missing data. Information not routinely collected within oncology information systems such as blood parameters were not analysed. Furthermore, data on the staging procedures used for individual patients were not collected, although this is reflective of a clinic cohort of patients. Survival may also have been impacted on by treatment at relapse which was not accounted for in this study. However the greatest impact on survival is initial treatment, and our methodology is similar to other survival prediction modelling studies [13, 39].

The findings of this study have implications for further research. External validation studies should be conducted by applying the SWSM to larger datasets to confirm findings and assess model generalisability. The current model may potentially be improved by the addition of mutation and PD-L1 status. Unfortunately, the influence of these predictors was unable to be evaluated as data on these markers were not routinely collected in stage I-III NSCLC patients during the time period of the study. Likewise, global advances in laboratory biomarkers and genomic parameters [40, 41] have been recently highlighted and may transform future NSCLC prognostication, however at present lack systematic investigation. In addition, data were not collected on cardiac radiation dose, which has been identified as an independent risk factor for all-cause mortality after radiotherapy in locally advanced NSCLC [42, 43]. Finally, impact analysis studies evaluating the acceptability, cost-effectiveness and practicality of the SWSM is required prior to clinical implementation [44].

We plan to develop and validate a survival prediction model in patients with Stage I-III NSCLC patients undergoing radiotherapy in a larger cohort of patients with distributed learning across multiple centres using the AusCAT network [45]. The factors found to be significant in this work will be considered alongside newer variables. The ultimate aim is to develop a tool to support radiotherapy decision-making in NSCLC using objective parameters rather than subjective clinical judgment. This will facilitate shared decision-making between patients and clinicians and reduce variability in treatment recommendations between clinicians and between institutions.

Conclusions

In conclusion, our study developed a survival prediction model in a real-world contemporary cohort of inoperable stage I–III NSCLC patients treated with radiotherapy. The SWSM utilises readily obtainable data and is convenient and simple to use by clinicians. The model exhibited good discrimination on both internal and external validation, and has the potential to guide treatment decisions. Further validation of this model is needed in a larger cohort of patients.