Background

Kidney cancer is one of the most prevalent genitourinary malignancies. In 2018 alone, it accounted for 2.2% of new cancer cases and 1.8% of overall cancer deaths worldwide [1]. In the United States alone, new cases and deaths due to kidney and renal pelvis cancer in 2019 were 73,820 and 14,770, respectively [2]. RCC is the most common subtype of kidney neoplasms, accounting for 90–95% of these cancers [3]. Based on the SEER database, 16% of kidney cancer patients presented with metastatic tumors at diagnosis. Also, the 5-year survival rate of metastatic kidney cancer patients was only 13.0% during the 2010–2016 period [4]. The national Swedish kidney cancer register reported comparable findings, in which between 2005 and 2010, 15–23% of the RCC patients presented with metastatic forms of the disease at diagnosis [5].

Accurate prognostic models are invaluable in designing clinical trials, patient psychological management, and governing therapeutic modalities. Currently, the Memorial Sloan-Kettering Cancer Center (MSKCC) and the International Metastatic RCC Database Consortium (IMDC) models are the most widely used prognostic models for metastatic RCC. Karnofsky performance, level of serum lactate dehydrogenase, hemoglobin, and corrected calcium as well as time from diagnosis to initiation of treatment are the independent prognostic factors in the MSKCC model [6]. The efficiency of the MSKCC model was validated in an independent study of 353 participants, previously untreated for metastatic RCC [7]. In the era of targeted therapy, Heng et al. identified 645 patients with metastatic RCC treated with sunitinib, sorafenib, or bevacizumab plus interferon from the IMDC during 2004–2008, and proposed the IMDC model including Karnofsky performance status, serum hemoglobin, corrected serum calcium, time from diagnosis to treatment together with neutrophils and platelets [8]. The two models provide the median OS for each group after stratifying metastatic RCC patients based on the prognostic risk factors.

Clinical and pathological characteristics of metastatic RCC patients are highly heterogeneous. In addition, they exhibit highly variable survival time [9]. Besides the MSKCC and IMDC models, individual survival prediction can also be implemented by risk-scoring systems. Unfortunately, best to our knowledge, there is no risk-scoring system for metastatic RCC patients. Accordingly, we aimed at constructing a predictive nomogram for patients newly diagnosed with metastatic RCC by combining clinical and pathological characteristics derived from the SEER database. In addition, we further aimed at evaluating the discrimination, calibration, and clinical use of this model.

Methods

Data source

We used data in the publicly available SEER database (http://www.seer.cancer.gov), which covers approximately 28% of the US population. SEER provides data on patient demographics, primary tumor site, tumor stage, surgical treatment, patient survival among others. Relevant data was retrieved using SEER*Stat software (version 8.3.5).

Study population

To be included in this study, participants must have first presented with metastasis RCC at diagnosis. The RCC must have been identified using universal morphology codes (8050/3, 8260/3, 8310/3, 8317/3, 8318/3, and 8319/3) based on the International Classification of Diseases for Oncology codes (3rd edition). Besides, the RCC must have been the first and only primary disease. The diagnoses were confirmed by histological examination and there was complete follow-up data of the patients. Patients under 18 years at diagnosis, missing data on follow-up, race, marital status, Fuhrman grade, tumor size, tumor stage, lymph node status, metastasis, and surgery were all excluded from the study. Autopsy or death certificate cases were also excluded. In the end, 2315 metastatic RCC patients fulfilled the inclusion criteria, and were included in the final analyses. The flow diagram for patient selection is presented in Fig. 1.

Fig. 1
figure 1

Diagrammatic flow of patient selection

Full size image

Measurable variables

Demographic and clinical variables such as age at diagnosis, race (black, white, other), sex, marital status (married, unmarried), histologic subtype (clear cell renal cell carcinoma (CCRCC), papillary renal cell carcinoma (PRCC), chromophobe renal cell carcinoma (CHRCC), sarcomatoid renal cell carcinoma (SRCC), collecting duct renal cell carcinoma (CDRCC)), Fuhrman grade (grade I, grade II, grade III, grade IV), Tumor classification (T1, T2, T3, T4, TX), lymph node status (N0, N1, NX), sarcomatoid feature (yes, no, unknown), cancer-directed surgery (recommended and performed, recommended but not performed, not recommended), bone, brain, liver, and lung metastasis status, survival time and vital status were all captured. The AJCC Cancer Staging Manual (7th edition, 2010) was employed to evaluate tumor stages.

Ascertainment of the outcome

OS was the primary outcome of this study, defined as survival time between metastatic RCC diagnoses to death, attributed to any cause. OS was ascertained based on the code “vital status” in the SEER database.

Statistical analysis

Selected patients were randomly and equally stratified into training and validation sets. Preliminary descriptive statistics were performed to describe the baseline characteristics of the patients in both sets. Thereafter, normally distributed continuous variables were expressed as mean ± the standard deviation, whereas non-normal continuous variables were presented by medians (interquartile range). Categorical variables were summarized in frequencies and percentages. Univariable and multivariable Cox regression analyses were performed on the training set to obtain crude and adjusted hazard ratios (HRs), important in identifying prognostic factors significant for OS. Prognostic factors were determined by a backward stepwise process using the Bayesian information criterion. Schoenfeld residuals were used to assess the proportional hazards assumption of Cox regression models.

Nomograms are pictorial representations that quantify risks and the probability of clinical events by scoring the involved factors. They have been demonstrated to generate more precise predictions than the conventional AJCC staging system in several types of cancers [10, 11]. In this study, a nomogram for predicting 1, 3 and 5-year OS was derived based on the findings of multivariable Cox regression analysis.

Discrimination and calibration, important properties in evaluating the performance of the model [12, 13], were both assessed in our study. C-index was applied to evaluate the discriminative ability of the nomogram, which depicted the probability of the predicted risk was higher for a random patient having an event than for a random patient not having an event. After comparing the predicted probability of events for all possible pairs of patients, C-index is 0.5 if the model can not discriminate the patients with and without events. Conversely, C-index is 1 if the probability predicted by the model is always higher for patients with events than those without events [14]. The robust performance of the model was assessed using the original and optimism-corrected C-indices. Calibration plot, the best visual representation of the relationship between predicted risk and actual risk, was presented using bootstrap resampling method [14]. Calibration plots fall on a 45-degree diagonal line, reflecting excellent absolute risk estimates. NRI and IDI usually assess and quantify the refinement in risk prediction between new and old models [15]. NRI is based on reclassification tables composed of patients with and without events, and can quantify the correct reclassification in categories. NRI is calculated by adding the percentage of patients with events who are correctly reclassified to the percentage of patients without events who are correctly reclassified [14]. IDI reflects the improvement in sensitivity and specificity of a model. It is also an integrated difference in Youden’s indices [15]. IDI is calculated by adding the increased probability predicted by new model compared to old model for patients with events to the decreased probability predicted by new model compared to old model for patients without events [15]. Therefore, NRI and IDI were both employed to compare the discriminative ability between the new model and the AJCC staging system. Notably, even though NRI and IDI have become increasingly popular, they should be interpreted with caution [16]. IPA is a promising metric that combines discrimination and calibration in one value, thus improves interpretability by adjusting for the benchmark model [17, 18]. IPA was also reported in this study to reflect the performance of the model. DCA is a method for evaluating the benefits of a diagnostic test across a range of patient preferences for accepting risk of undertreatment and overtreatment to facilitate decisions about test selection and use [19]. Unlike the sensitivity, specificity, and area under the curve, DCA directly assesses the utility of clinical risk prediction models for decision making [20]. Herein, DCA was plotted to evaluate the clinical value of the nomogram by quantifying the net benefit in comparison with the AJCC staging system.

Statistical tests were performed using R software (version 3.5.2, http://www.r-project.org/). All tests were two-sided, with statistical significance set at P < 0.05.

Results

Patient baseline characteristics

Data for 2315 eligible metastatic RCC patients collected between 2010 and 2016 were extracted from the SEER database, and then randomly stratified equally into two sets, with 1158 and 1157 patients in the training and validation sets, respectively. The median age of the participants was 61 (54–68) years. Whites accounted for 83.8% of the patients. With regard to gender, 1629 (70.4%) patients were males. The majority of the patients (67.3%) were married. The most common histologic subtype was CCRCC (84.6%), with Grade III (40.0%) being the most common Fuhrman grade. The majority of the tumors were T3 (56.8%), with T2 and T4 accounting for the rest (16.5 and 12.7%, respectively). Concerning treatment, 85.6% of the patients underwent cancer-directed surgery. Overall, at diagnosis, 31.5, 9.9, 11.0, and 61.7% of the patients presented with bone, brain, liver, and lung metastases, respectively.

The median follow-up time of the cohort was 16 months, whereas the median OS time was 19 months (95% CI, 18–21 months). The median follow-up time for patients without events was 30 months. By the end of the survey, 1563 patients had died, of which 1467 from RCC and 96 from other causes. The demographic and clinical characteristics of the patients were summarized in Table 1.

Table 1 Demographics and clinical characteristics for the metastatic RCC patients
Full size table

Independent prognostic factors

Univariable and multivariable Cox regression analyses were performed to explore independent risk factors for OS. Crude and adjusted HRs were presented in Table 2. After adjustment for other risk factors, identified eight variables: Fuhrman grade, lymph node status, sarcomatoid feature, cancer-directed surgery, bone, brain, liver, and lung metastases were significantly associated with OS. Subgroup analyses (Table 3) also showed patients not recommended for nephrectomy were more likely to succumb to death than those who underwent nephrectomy, indicating the robustness of the model.

Table 2 Univariable and multivariable Cox regression analysis of OS for metastatic RCC patients
Full size table
Table 3 Subgroup analyses for patients who underwent or did not undergo nephrectomy
Full size table

Nomogram construction

The 1, 3 and 5-year predictive nomogram for OS were constructed only incorporating the significant prognostic factors. As shown in the nomogram (Fig. 2), cancer-directed surgery was the most significant factor for prognosis, followed by Fuhrman grade, brain metastasis, and sarcomatoid feature. Lymph node status, lung, and liver metastases moderately impacted on OS. Meanwhile, Bone metastasis had the least effect on OS.

Fig. 2
figure 2

Nomogram for 1, 3, and 5-year prediction of OS of patients with metastasis RCC a Classical application of the nomogram b. Each category of the prognostic variables was assigned a score on the Points scale. After summing up the score of each variable and locating the total score on the Total Points scale, a line was vertically drawn to the 1, 3, and 5-year survival probability scale and estimated survival probability at each time point could be obtained

Full size image

Classical application of the nomogram

The practical use of the nomogram was presented in Fig. 2. Here, a patient was diagnosed with Fuhrman grade IV, N1 stage metastatic RCC. He lacked sarcomatoid differentiation and displayed no bone metastasis. After assessment, he underwent nephrectomy. The nomogram scored 39 points for Fuhrman grade IV, 31 for N1 stage, 0 for no sarcomatoid feature, 27 for bone metastasis, and 0 for cancer-directed surgery, totaling 97 points. This corresponded with a 1, 3 and 5-year survival probability of 57, 20, and 8%, respectively.

Nomogram performance

Discrimination and calibration were employed to evaluate the performance of the model. The C-indices of the nomogram were 0.701 (95% CI, 0.682–0.720) for the training set and 0.676 (95% CI, 0.655–0.697) for the validation set. With respect to the AJCC staging system, the C-indices were 0.612 (95% CI, 0.591–0.633) for the training set and 0.600 (95% CI, 0.578–0.622) for the validation set, significantly lower than those of the nomogram (P < 0.001). Across the 1000 bootstrap resamples, the optimism-corrected C-indices of the nomogram were 0.698 (95% CI: 0.677–0.717) and 0.677 (95% CI: 0.656–0.697) for the training and validation sets, respectively, indicative of robustness in the performance of the model. Calibrations of the nomogram were assessed using actual and predicted estimates after bootstrapping with 1000 resamples. Calibration plots (Fig. 3) revealed good consistency between the predicted and the actual survival in both sets. Category-free NRI for 1, 3 and 5-year follow-up in the training set were 0.546 (95% CI: 0.376–0.655), 0.551 (95% CI: 0.368–0.679) and 0.468 (95% CI: 0.276–0.710), respectively, and 0.455 (95% CI: 0.307–0.610), 0.454 (95% CI: 0.305–0.615) and 0.356 (95% CI: 0.172–0.555), respectively, in the validation set. In addition, IDI for 1, 3 and 5-year follow-up in the training set were 0.082 (P < 0.001), 0.084 (P < 0.001) and 0.063 (P < 0.001), respectively, and 0.052 (P < 0.001), 0.057 (P < 0.001) and 0.048 (P < 0.001), respectively, in the validation set. The IPA estimates for the nomogram were greater than AJCC staging estimates in training set (19.6% vs. 7.8, 18.2% vs. 5.9 and 12.8% vs. 0.8%, for 1, 3 and 5-year IPA) and validation set (15.4% vs. 6.7, 15.9% vs. 5.7 and 9.1% vs. 2.9%, for 1, 3 and 5-year IPA). These results demonstrated the superior predictive power of the nomogram for OS over the AJCC staging system.

Fig. 3
figure 3

Calibration curves for predicting 1, 3 and 5-year OS of patients with metastasis RCC in training set a, b, c and validation set d, e, f. The nomogram-predicted probability of OS was plotted on the x-axis, with actual OS plotted on the y-axis. The calibration curves were visual representations of the relationship between the predicted and actual absolute risk

Full size image

Clinical use

DCA plots showed that our nomogram had greater net benefits in comparison with the AJCC staging system for predicting 1, 3 and 5-year OS of patients with metastatic RCC (Fig. 4), demonstrating its application in guiding clinical decision for metastatic RCC patients.

Fig. 4
figure 4

DCA for the nomogram and the AJCC staging system in the training set a, b, c and validation set d, e, f for 1, 3, and 5-year survival, respectively. Horizontal coordinates represented the threshold of probability, whereas the vertical coordinates represented the net benefit rate. The red and black dash lines represented DCA of the nomogram, and the AJCC staging system, respectively. The black solid line assumed all patients were alive, whereas the gray solid one with a negative slope assumed all patients were dead. DCA plot depicted the model with superior clinical application

Full size image

Discussion

It is well accepted that clinical and pathological heterogeneity of metastatic RCC highly influences the prognosis of patients with the cancer [9]. Therefore, accurate prognosis prediction of the tumor continually frustrates efforts against the cancer. The IMDC model is the most widely used prognostic prediction model for metastatic RCC patients receiving targeted therapy. It is composed of six clinical and laboratory factors [8]. Using this model, patients can be stratified into three risk groups, each with different median OS. The efficacy of IMDC has been validated in metastatic RCC patients receiving first-line [8], second-line [21], and third-line targeted therapies [22]. Such models aside, risk-scoring systems including nomograms can also predict survival probability of individual patients. This study comprised of data for patients from multiple centers, archived in the SEER database. The focus was on the newly diagnosed metastatic RCC patients for whom the MSKCC and IMDC models might be unsuitable. Meanwhile, the predictive model proposed in this study is a nomogram, demonstrated to predict the OS more precisely. Apparently, accurate OS prediction is one of the most concerned issues for metastatic RCC patients. Our nomogram meets all the criteria necessary for endorsement by the AJCC [23], and has the potential to supplement the AJCC staging system.

With an HR of 1.54, lymph node metastasis negatively affected the survival of metastatic RCC patients. Using 2530 RCC patients treated with nephrectomy, Karakiewicz et al. constructed a nomogram for survival prediction based on six independent factors, including tumor stage, lymph node status, metastatic status, tumor size, Fuhrman grade, and type of symptoms [24]. Pati et al. also reported that lymph node metastasis strongly impacted on progression-free and overall survival of metastatic RCC patients [25], consistent with the findings at Cleveland Clinic [7]. Fuhrman grade, as the most widely accepted prognostic grading system, is not only associated with the survival of local RCC but also metastatic types [24]. In this study, compared with grade I, Fuhrman grade IV was associated with a worse OS after adjusting other clinical variables (HR = 1.80, 95% CI: 1.06–3.06, P = 0.029). Sarcomatoid differentiation is another predictor of short survival for metastatic RCC patients. Considering the small percentage of patients with sarcomatoid features (< 6%), sarcomatoid differentiation was not included in the IMDC model for OS prediction, though it is associated with adverse oncologic outcomes [8]. However, nearly 20% of patients included in this study exhibited sarcomatoid differentiation, with patients not displaying this feature shown to have superior OS (HR = 0.61, 95% CI: 0.50–0.75, P < 0.001). Therefore, it was included in our model.

Based on a Nationwide Inpatient epidemiologic study, lung was the most common metastatic site for metastatic RCC, accounting for 45.2% of such cases. Bone and liver metastases followed with 29.5 and 20.3%, respectively [26]. For this study, 61.7, 31.5, 11.1, and 9.9% of the patients in our cohort presented with lung, bone, liver, and brain metastases, respectively (Table 1). These four clinical variables were all significantly associated with adverse survival, and our findings were consistent with the growing evidence that the presence of distant metastasis predicted worse clinical outcomes. In one phase III trial of 375 metastatic CCRCC patients who received sunitinib, lung and liver metastases were significantly associated with poor progression-free survival [27]. In a separate study involving 281 metastatic RCC patients administered with third-line targeted therapy, it was demonstrated metastases at diagnosis were associated with adverse effects on survival [28]. In a separate retrospective study of over 2000 metastatic RCC patients who received first-line targeted therapy, bone or liver metastasis were independent risk factors for shorter OS. Intriguingly, compared with other metastatic sites, both bone and liver metastases were associated with worse survival. Also, the incorporation of bone and liver metastases significantly increased the predictive performance of the IMDC model [29].

In the era of cytokines, two randomized controlled trials demonstrated prolonged survival with nephrectomy plus interferon, relative to interferon alone among metastatic RCC patients [30, 31]. Early retrospective studies also corroborated the role of cytoreductive nephrectomy [32, 33]. However, the role of nephrectomy in treating metastatic RCC in the era of targeted therapy remains controversial. For instance, in one randomized trial (CARMENA), sunitinib monotherapy against intermediate- to poor-risk metastatic RCC was not inferior to a combination modality of nephrectomy followed by sunitinib [33]. Though the survival advantage to metastatic RCC patients with favorable characteristics was not evaluated in CARMENA, a significant proportion of metastatic RCC patients did not benefit from nephrectomy. Multiple retrospective studies investigated the association between nephrectomy and clinical outcomes among metastatic RCC patients receiving targeted therapy. Most of them supported this population could receive survival benefits from nephrectomy [34,35,36,37]. Using the IMDC database, Heng et al. reported metastatic RCC patients who received targeted therapy and cytoreductive nephrectomy had longer OS compared to those on targeted therapy alone (20.6 vs. 9.6 months, P < 0.001). After adjusting for other variables, the HR for death was 0.60 (95% CI: 0.52–0.69, P < 0.001) [34]. Comparable findings were reported following analysis of the National Cancer Data Base [35]. Most investigations analyzing the cytoreductive nephrectomy benefits are inherently limited by their retrospective nature. Notably, under any circumstances, patient selection and timing of surgery critically impact on nephrectomy benefits in metastatic RCC patients [33, 35]. Most often, patients with poor health status are unlikely to benefit from nephrectomy [38]. On the other side, patients with good performance status, minimal symptoms related to metastases, a resectable primary tumor, and a limited burden of metastatic disease are likely to highly benefit from the surgery [38]. One systematic review also found that rigorously selected patients based on prognostic factors, with less metastasis, and those displaying favorable responses to initial systemic therapy could highly benefit from nephrectomy [39]. In this study, patients recommended for surgery and who indeed underwent the procedure were more likely to benefit from nephrectomy relative to non operated patients. Subgroup analyses validated the robustness of the nomogram. It is noteworthy that there is no “one size fits all” approach for managing cancers [33]. Stated differently, the current findings do not recommend cytoreductive nephrectomy to all patients, but rather, clinicians should adopt rigorous selection criteria for surgery. The findings of this study are only based on retrospective data in the SEER database, thus should be interpreted with caution. As such, more investigations should be performed to strengthen the reliability of this model before its clinical application.

Our findings notwithstanding, this study suffered several limitations. First, because of the retrospective nature, inherent selection bias associated with such studies, in particular treatment selection bias was unavoidable. Because more relatively healthy patients were commonly recommended for surgery, it was more likely they would exhibit better clinical outcomes. Second, other independent RCC prognostic factors such as MSKCC or IMDC risk stratification, level of serum albumin, and lactate dehydrogenase were not captured in the SEER database. Third, some potentially valuable variables such as type of targeted therapy, timing of surgery, and other comorbidities were also not captured in the SEER database.

Conclusions

In summary, we established a nomogram for survival prediction of metastatic RCC patients. It incorporated eight clinical factors: Fuhrman grade, lymph node status, sarcomatoid feature, cancer-directed surgery, bone, brain, liver, and lung metastases. The nomogram displayed superior predictive capability and higher clinical application than the conventional AJCC staging system.