Introduction

Breast cancer (BC) is the most frequently diagnosed tumor and the leading cause of cancer death in women in Spain, in Europe and in the world1,2,3,4. A multitude of prognostic factors provide information about the behavior of BC (demographic, clinical, pathological and molecular) and enable comparisons of case series and the results of different treatments. The most important of these factors are lymph node involvement, tumor size and grade5,6,7.

When assessing the prognosis of a disease such as BC, predictive models are very useful because they combine different risk factors to determine the likelihood of patient survival. Regarding predictive models of BC mortality, a meta-analysis published in 2006 found that of all those published, only the Nottingham Prognostic Index had sufficient scientific evidence for clinical use7. However, other predictive models assessing BC mortality were not included in this meta-analysis8,9,10,11,12. Table 1 shows the main features of the models and of the Nottingham Prognostic Index8,9,10,11,12,13,14,15. The majority of these models focused on all-cause mortality, and they were developed for specific populations (excluding advanced stages, considering specific treatments) or are difficult to use in clinical practice. Analyzing the methodology used for their development, we note that, generally, no survival models were used, risk groupings were made without justifying their classification and internal validation was not performed in the most correct way (Table 1)8,9,10,11,12,13,14,15. This validation should confirm two issues: discrimination and calibration. The first corresponds to the ability of the model to distinguish between patients who experience an event and those who do not, while the second assesses whether the probabilities of an event predicted by the model correspond to reality. There are different levels of calibration, with moderate being one of the most recommended (“a risk model is moderately calibrated if, among patients with the same predicted risk, the observed event rate equals the predicted risk”)16. We recommend validation through bootstrapping, since the other methods (split-sample and cross-validation) are less accurate17. To more accurately assess calibration, smooth curves are preferable (splines or loess transformations)16, 18. Not all models evaluated both discrimination and calibration, two of six models did not use bootstrapping for internal validation and none used smooth curves to assess calibration (Table 1)8,9,10,11,12,13,14,15. Consequently, we conducted a study to construct a predictive model specific to mortality at 5 and 10 years in patients diagnosed with BC without excluding any conditions, applying bootstrapping, and determining calibration through smooth curves (spline transformations). To facilitate use in routine clinical practice, the model was adapted to a points system and integrated into an application for the Android mobile phone operating system.

Table 1 Published predictive models for mortality in breast cancer patients.
Full size table

Methods

Study population

Patients diagnosed with BC in the Elda Health Department were included. This Department provides health coverage to a region of southeastern Spain with a total of 184,195 inhabitants (2003). Health coverage is free and universal, for treatment of BC or any other disease.

Study design and participants

This cohort study included patients diagnosed with BC in the Elda Health Department between 2003 and 2006 (inclusive). Patients were diagnosed through histological analysis by the Department of Pathology either through biopsy or surgical specimen. Patients were followed at the University General Hospital of Elda (the only hospital of this Health Department) from the date of diagnosis until July 2016 or the date of last contact with the patient, including death.

Variables and measurements

The primary variable was BC-specific time-to-death. This variable was obtained in patient follow-up through the clinical history, which always includes the date of death and the cause, including BC. We must keep in mind that our variable considers each individual’s time to death from BC or no death from BC (last clinical contact or death from other causes), which is censored data19.

The secondary variables were collected at baseline (diagnosis date): age (years), personal history of breast surgery (none, benign pathology, or malignancy), personal history of any cancer, personal history of BC, premenopause, postmenopause, grade (low, intermediate/moderate or high)20, estrogen receptor, progesterone receptor, c-erbB2, TNM stage21, multicentricity/multifocality, and diagnosis (invasive ductal carcinoma, invasive lobular carcinoma, ductal/lobular in situ, or other). These study variables were selected because there is scientific evidence they are prognostic factors of BC5, 7, 21,22,23. Since a personal history of breast surgery affects the breast, it was suspected this could influence prognosis. Although not addressed as a prognostic factor in the scientific literature, it has been shown that patients with a personal history of BC who develop a second BC have a worse prognosis24, 25; thus, it is important to include this variable when developing a predictive model. All the secondary variables are easily obtained in clinical practice, which would facilitate their implementation in a predictive model.

Other factors to consider in the prognosis of breast cancer are the treatments given to the patient. For this reason the initial treatment was collected: neoadjuvant therapy, chemotherapy, hormone therapy, radiotherapy, breast surgery (none, mastectomy, or conserving) and lymphadenectomy. We note that sentinel lymph node biopsy does not appear in this list because it was introduced in our area after 2006.

Sample size

Since the primary objective was to develop a predictive model, the sample size of our study had to verify that the events-per-variable was greater than 1026. Our final sample size was 287 women, of whom 55 died from BC, allowing construction of a predictive model with five predictors.

Statistical methods

The qualitative variables were described by calculating absolute and relative frequencies and the quantitative variables through the mean and standard deviation. Given that our main variable was time-to-event, a multivariate Cox regression model was constructed with five predictors, verifying the assumption of proportional hazards by means of graphical and analytical tests19. Qualitative ordinal variables were considered quantitative predictors in the multivariate analysis (personal history of breast surgery, grade and TNM stage). Age was considered a quantitative continuous variable, as approximately 99% of our patient sample was over 35 years of age, which has been seen as a predictor of mortality5. These quantitative variables were analyzed linearly, as it was previously verified that they had no quadratic association with mortality (score test). Since we had a total of 22 possible predictors to introduce into the model, all combinations of 5, 4, 3, 2 and 1 predictors were determined, i.e., a total of 35,442. This approach is often called ‘Best-subsets’ model selection. In all of these, the optimism-adjusted estimate of the C-statistic was calculated and the combination yielding the highest result was chosen, i.e., the one with the highest discriminating capacity. Through the model estimated, the adjusted hazard ratios (HR) were obtained. The goodness of fit of this model was verified by the score test. The survival model was adapted to a points system using the Framingham Heart Study methodology27, which categorizes the predictors and, through a weighting system based on the model coefficients, gives a score to each category. The sum of the scores associated with each of the predictors gives a total score which correlates with a risk of event, so that the use in clinical practice of a points system is truly simple27. The points system methodology was used to calculate the risk of each total score at 5 and at 10 years from diagnosis of BC. To internally validate the points system we applied bootstrapping simulating 1000 samples as bootstrapping is the most recommended technique for this procedure17, 28 (although the relevant reference was published in 2001, its use is still recommended), and in each the C-statistic (discrimination) and the observed probability of event at 5 and 10 years was obtained (calibration) for each score through transformations by linear splines (smooth curves). These smooth curves are included in the recommended level of calibration (moderate)16, 18. A bootstrap sample is a random sample taken with replacement from the original sample with the same number of elements. The 1000 C-statistic values obtained were plotted on a histogram to analyze their distribution. We can say that the points system accurately discriminates the patient who died if the values of the distribution are high. The observed probabilities of mortality from BC obtained by smooth curves versus the probabilities predicted by the points system are represented on a Cartesian graph. To say that the model is well calibrated both probabilities should be very similar, i.e., the soft curve must conform to the diagonal line16, 18. Type I error was set at 5% for all analyses, and for each relevant parameter its associated confidence interval (CI) was calculated. All calculations were performed through IBM SPSS Statistics 19 and R 2.13.2.

Ethical issues

The study was approved by the Ethics Committee of the Elda Department of Health. As this was a study of routine clinical practice without any intervention, informed consent was not requested from the patients. The Ethics Committee approved this procedure. The study was conducted in accordance with the basic principles of the Declaration of Helsinki World Medical Association and met the standards described in the guidelines for good clinical practice of the European Union.

Mobile application

The points system was integrated into a mobile phone application using the Android operating system. Download of the application is free for all users from the store (Google Play) under the name Breast Cancer Mortality.

Results

The study began with 292 patients, with five lost to follow-up, leaving a final sample size of 287 patients. With a mean follow-up of 8.6 ± 3.5 years, 55 women died of BC. This represents an incidence of 222 deaths per 10,000 person-years (95% CI: 167–289). The descriptive characteristics of the sample (Table 2) revealed the women had an average age of 59 years, 2.8% had a history of BC, and 65.9% were postmenopausal. Regarding the histological features, the majority were intermediate/moderate grade (40.8%), estrogen receptor-positive (74.6%), progesterone receptor-positive (63.1%), stage IV (4.9%) and with a diagnosis of invasive ductal carcinoma (78.4%). Treatments varied ranging from neoadjuvant therapy (6.3%) to surgery (89.5%) (Table 2).

Table 2 Descriptive characteristics and adjusted hazard ratios for predicting breast cancer mortality.
Full size table

Analysis of 35,442 multivariate models provided a maximum C-statistic of 0.85 (standard error 0.039). Its associated model consisted of the following variables: older age (HR = 1.02, 95% CI: 1.00–1.04, p = 0.119), personal history of BC (HR = 4.78, 95% CI: 1.82–12.57, p = 0.002), higher grade (HR = 2.26, 95% CI: 1.45–3.51, p < 0.001), higher stage (HR = 1.83, 95% CI: 1.59–2.11, p < 0.001) and multicentricity (HR = 1.60, 95% CI: 0.86–2.97, p = 0.140). Adaptation to the points system model is shown in Fig. 1. As we can see, in the risk assessment there was apparently no difference between point sums 11–15. Nevertheless we have to take into account that differences did in fact exist, but they are given in decimals and after rounding the figures to 2 decimal places they appear to have the same risk. The reason they are similar is due to the exponentiality of the Cox model.

Figure 1
figure 1

Scoring system to predict breast cancer mortality.

Full size image

Internal validation through bootstrapping yielded a C-statistic of 0.83 (Fig. 2) and a slope within the linear calibration of 0.99 (Fig. 3). Smooth calibration curves at 5 and 10 years adjusted successfully to the diagonal line (Fig. 4). Although the observed risk may appear to be lower than the risk suggested by the model for most of the range of scores, it should be noted that in this type of calibration it is only necessary for the smooth curve to adjust to the straight line, whether it be oscillating, above or below16. In fact, the condition of having an underestimated risk exists in the calibration of survival models, even in very high sample sizes (1000 events), and the calibration is considered satisfactory when it does not deviate too far from the perfect condition29.

Figure 2
figure 2

C-statistic distribution for the validation of our scoring system using the bootstrap method.

Full size image
Figure 3
figure 3

Calibration slope distribution for the validation of our scoring system using the bootstrap method.

Full size image
Figure 4
figure 4

Smooth calibration plots for the validation of our scoring system using the bootstrap method. The black line represents perfect calibration and the grey line indicates the results of our calibration.

Full size image

Discussion

We constructed and internally validated a points system to predict mortality in BC at 5 and 10 years. The methodology followed used powerful statistical techniques for validating a predictive model: verification of more than 35,000 models, and discrimination considering censoring, bootstrapping and smooth calibration. The model can be applied to all patients with BC regardless of their clinical, histopathological or treatment characteristics, although it should be applied with caution in patients younger than 35 years, as our sample contained very few such patients.

The main strength of this study is the prediction model developed and internally validated (Table 1) that is applicable to all patients with BC (with caution in younger patients)8,9,10,11,12,13,14,15. This prediction model has improved upon existing models. Construction and internal validation were performed with the most accurate statistical techniques for predictive models. The predictor variables of our points system are easily obtained, allowing its routine use. Unlike other predictive models (Table 1)8,9,10,11,12,13,14,15, risk calculation using our model is very simple and is integrated in a mobile phone application that is free for all users of the Android operating system. This model fills the need for a predictive model for BC-specific mortality. Other models have generally been constructed to predict all-cause mortality.

Although it may seem that the sample size was insufficient to develop a predictive model, the size must be based on the events-per-variable ratio, which must be greater than 1026. This ratio was verified in our prediction model. Although the p-values associated with both age (p = 0.119) and multicentricity (p = 0.140) were not significant (p > 0.05), we must bear in mind that we did not assess each factor independently. All predictors were measured together, because our goal was to develop a prediction model. The model had high discriminatory capacity (C-statistic 0.85) and compared with the null model there were highly significant differences (χ2 = 165.1, p < 0.001). The model is therefore very satisfactory for prediction of mortality due to BC.

No other variables in the model evaluated in this study were prognostic factors (Table 2)5, 7, 21,22,23. Though other variables have been noted as prognostic factors they were not considered in this study as they were not collected correctly, such as the BRCA status. However, even without the inclusion of these other variables, our five predictors very successfully discriminated which patients would die of BC, in addition to successfully calibrating the model. Regarding selection bias, all patients with BC over a specific period were included in the study, i.e., none were excluded due to comorbidities, stage of disease or treatment received. Information bias was minimized through rigorous data collection. Additionally, the sentinel lymph node technique was not available when the baseline data for this study were recorded. Nonetheless, it should be recalled that this technique detects the presence of affected lymph nodes in the axilla and is therefore less aggressive than lymphadenectomy, though the resulting diagnosis is the same for both techniques. Consequently, this lack of inclusion of the sentinel lymph node among the study variables cannot be considered a limitation. Finally, concerning the duration of hormone therapy, it should be noted that our model is designed to predict mortality at the time of diagnosis. Thus, at this time we are unaware of the duration of hormone therapy, so we cannot include it as an explanatory variable in our scoring system. Whilst estrogen receptor-positive patients who receive hormone therapy have a better prognosis, we are estimating a multivariate model that considers this issue; that is, it correctly weights these factors in the patient prognosis. This can also be seen in other conditions, like cardiovascular diseases, where the general population does not have a very high prevalence of left ventricular hypertrophy (good prognosis), but the factor is still taken into account in the construction of the model, which can then be applied in the general population.

When we compare our model with those reported in the literature (Table 1)8,9,10,11,12,13,14,15, we note that they are generally applicable to selected populations, i.e., not all patients with BC in general. Additionally, some studies used models that did not consider censoring or the events-per-variable ratio19, 26, and constructed risk groups without any justification for their classification (Table 1)8,9,10,11,12,13,14,15. Regarding validation, some studies did not employ bootstrapping, when this is the most recommended technique17. In some papers discrimination was not analyzed and calibration, when it was done, was not done through smooth calibration (Table 1)8,9,10,11,12,13,14,15, which is the most accurate technique for this task16. When we combine all the features mentioned, no published model complies with all of them (Table 1)8,9,10,11,12,13,14,15: using a survival model, bootstrapping, analysis of events-per-variable, general population of patients with BC, discrimination considering censoring and calibration using smooth curves. Our model outperforms existing models in statistical methodology (development and validation). Finally, we stress that the outcome of our model is cancer-specific mortality, which is the most important consideration when establishing the prognosis of the disease.

The first factor in our points system was age, where risk of mortality increases with increasing patient age. This is consistent with the scientific literature, as our population contained few patients under 35 years5. The second factor was history of BC, which increases the risk of mortality. As previously mentioned (Variables and measurements), a second BC has been shown to worsen the prognosis24, 25. As expected, the stage and grade were associated with higher mortality, consistent with other predictive models and a large number of studies (Table 1)5, 8,9,10,11,12,13,14,15, 20, 21. Finally, we found that multicentricity indicated a greater likelihood of death. This is consistent with other studies, which found that this was an independent prognostic factor30.

Our points system can be applied quickly in routine clinical practice to any patient with BC using simple parameters (using the points system integrated into the mobile application). With this we can easily determine the prognosis and inform the patient. We encourage other authors to validate our points system in other populations, and to employ the methodology followed here to develop predictive models in patients with BC, as well as with other types of cancer or other diseases. Although our model adjusts our data well, it could nevertheless be over-estimating; accordingly, its calibration and discrimination need to be confirmed with other datasets in other areas, and which include patients of different races and ethnicities. Finally, another interesting line of research concerns comparing the prognostic power of our model with that of other models already published (Table 1). This, though, would require cohorts with large sample sizes including at least 100 events and 100 non-events16.

Conclusion

A points system for predicting mortality due to BC at 5 and 10 years has been constructed and internally validated. A mobile application in Android has been designed and is available for use. To use our forecasting system in other populations, external validation studies are needed.