Introduction

Type 1 diabetes is a chronic autoimmune disease that can affect the cardiac and peripheral circulation. Later in life, many patients with diabetes develop major complications related to the circulation. Such major outcomes include CHD, stroke, end-stage renal failure, amputations and blindness [13], which considerably decrease their quality of life [4]. It has been found that patients with type 1 diabetes have a four- to eightfold increased risk of CHD compared with those without diabetes [5, 6].

An important lesson learned from the DCTT was that intensive glycaemic control aiming at near-normal glycaemic levels reduces the risk of all the diabetic complications including retinopathy, nephropathy and cardiovascular diseases [79]. Thus, intensive glycaemic control should be initiated as early as possible in the course of diabetes and not after 5–10 years, when the first clinically detectable signs of complications such as retinopathy or nephropathy are already present [10].

Absolute risk predictions in individual patients with type 1 diabetes are important to allow a timely identification of patients with a high risk of major outcomes in order to enable strategies to prevent the development of such complications and to reduce healthcare costs. Furthermore, prognostic models have an important role in informing patients and in selecting high-risk populations for randomised controlled trials.

Prognostic research in type 1 diabetes has so far focused on CHD as a major outcome, disregarding stroke, end-stage renal failure, amputations and blindness [11, 12]. However, from a patient perspective, it is relevant to predict all major outcomes since any of these outcomes will lead to a restricted quality of life in relatively young patients. More importantly, the presence of nephropathy is associated with a substantially increased risk of premature death [13].

We here describe a prognostic model based on easily accessible patient and clinical characteristics for the composite of the major outcomes and death, which would allow a straightforward application of the model in practice. This model may help to identify patients with type 1 diabetes at high risk of major outcome events. We use the large prospective cohort from the EURODIAB Prospective Complications Study (PCS) to develop the model. The model’s performance was then tested in three different cohorts from North America and Europe.

Methods

Patients with type 1 diabetes

Data from the EURODIAB PCS were used to develop the prognostic model (development set). EURODIAB PCS was designed to study risk factors for complications in patients with type 1 diabetes. The study included these patients from 31 centres in 16 European countries (see ESM List of Investigators). Ethics committee approval was obtained at each centre, and all participants provided written informed consent. Inclusion and predictor measurements occurred between 1989 and 1991. Full details of this study have previously been described [1, 14]. The major outcomes examined for the prognostic model were major CHD, stroke, end-stage renal failure, amputation, blindness and death. We selected 1,973 study participants with no such complications at inclusion (between 1989 and 1991) and who were followed up to 1997–1999. The first occurrence of any of the major outcomes was used for analysis.

We used three prospective cohorts including patients with type 1 diabetes to test the model’s performance: the Epidemiology of Diabetes Complications (EDC) study, the Finnish Diabetic Nephropathy (FinnDiane) study and the Coronary Artery Calcification in Type 1 Diabetes (CACTI) study (Table 1). The EDC study is a prospective investigation of factors that lead to the development of complications among individuals with childhood-onset type 1 diabetes diagnosed or seen within 1 year of diagnosis at the Children’s Hospital of Pittsburgh, USA [15]. Participant inclusion occurred between 1986 and 1988. Since participants were re-examined biennially, two groups could be analysed: (1) participants without severe comorbidity at baseline (EDC original, n = 554); and (2) participants without severe comorbidity at a later examination (1996–1998, EDC recent, n = 314). The second selection allowed an evaluation of the prognostic model in a group of more recently treated participants. Note that the participants from the EDC recent cohort are also members of the EDC original cohort. FinnDiane is an ongoing, prospective, nationwide, multicentre study with the aim of identifying genetic and clinical risk factors for diabetic nephropathy in patients with type 1 diabetes [16]. Participants were included between 1994 and 2009. The CACTI study is a prospective cohort study of individuals with type 1 diabetes without baseline CHD that was designed to assess risk factors associated with the development and progression of subclinical CHD [17]. Participants were enrolled between 2000 and 2002. All three studies are ongoing. For the present analysis, we selected participants who had no major outcomes at inclusion. Follow-up measurements were taken until 7–9 years after inclusion to assess the occurrence of major outcomes.

Table 1 Characteristics of patients with type 1 diabetes in the development and validation sets

Major outcomes

Major outcomes included major CHD, stroke, end-stage renal failure, amputations, blindness and all-cause death. Major CHD comprised fatal CHD, non-fatal myocardial infarction (coded according to ICD-9 410-414; www.icd9data.com/2007/Volume1) and major Q waves on Minnesota-coded ECGs (codes 1.1, 1.2) [18]. Stroke (ICD-9 430-438) included fatal and non-fatal stroke. End-stage renal failure comprised renal dialysis or renal transplantation as assessed from hospital records. Amputation was ascertained by self-report and/or physician examination. Blindness was ascertained by a self-report of physician-diagnosed blindness and by visual acuity testing using the Snellen chart [19]. Data on death was collected from death certificates. The time of occurrence of the major outcomes was assessed as precisely as possible from hospital records and death certificates. Assessments of the major outcomes were similar for the three validation cohorts [1517].

Prognostic factors for model development

Based on the literature and clinical expertise, the following characteristics were considered to be potential prognostic factors for the defined outcomes: age, sex, duration of diabetes, HbA1c level, WHR, BMI, smoking, systolic and diastolic BP, antihypertensive medication, albumin/creatinine ratio, fasting triacylglycerol and HDL-, non-HDL- and LDL-cholesterol.

Demographic variables, duration of diabetes, medication and smoking status were collected using standard questionnaires in the EURODIAB PCS study. Height, weight and waist and hip circumferences were measured in a standardised way to calculate the BMI and WHR. Systolic and diastolic BP were measured using a random zero sphygmomanometer and the mean of two measurements was calculated. Two 24 h urine samples were analysed to determine mean urinary albumin and creatinine concentrations; albumin/creatinine ratios were calculated. Fasting blood samples were taken to measure HbA1c and lipids. HbA1c was measured with an enzyme immunoassay using a monoclonal antibody. The HbA1c values obtained were converted to DCCT values [20] and International Federation of Clinical Chemistry and Laboratory Medicine (mmol/mol) values in line with the EASD recommendations [21]. Lipids (fasting triacylglycerols, total cholesterol and HDL-cholesterol) were measured with standard enzymatic methods. Non-HDL-cholesterol was defined as the difference between the total and HDL-cholesterol, and as representing cholesterol carried on all of the potentially proatherogenic apoprotein B-containing particles. LDL-cholesterol was calculated from Friedewald’s formula if the triacylglycerol level was <4 mmol/l [22]. Measurements of prognostic factors in the three validation cohorts were performed using similar methods [1517].

Statistical analysis

Missing values occurred for most prognostic factors, varying from 2 to 92 missing values among the 1,973 EURODIAB PCS participants. For the variables LDL-cholesterol and triacylglycerols, more values (n = 673) were missing. We imputed missing values with multiple imputation techniques using predictive mean matching, allowing all the observed values to be analysed (the aregImpute algorithm in R software; http://cran.us.r-project.org/) [23].

To develop the prognostic model, we used Weibull regression analysis to estimate univariate and multivariable regression coefficients, and HR with 95% CIs for each prognostic factor. The functional form between continuous prognostic factors and the occurrence of major outcomes was explored with restricted cubic splines.

A full multivariable model was fitted that included all candidate prognostic factors with chosen transformations. The number of prognostic factors was reduced with backward stepwise selection. Variables with weak associations (p > 0.3) were deleted from the model. This analytical strategy aims to limit overfitting of a model [24]. Therefore, the backward selection procedure uses a liberal p value (0.3 in this study), which results in the inclusion of relatively weaker prognostic factors in the model at the cost of the possible selection of a variable without predictive value. Such a model can perform well with new participants [25]. We did not explore multiplicity (interaction terms) because of the relatively small sample size.

Internal validity was studied in 100 bootstrap samples to assess possible optimism. The regression coefficients in the final model were multiplied by a shrinkage factor, which was estimated with the bootstrapping procedure [24]. Shrinking the regression coefficients to zero reduces overconfidence in the predicted probabilities.

External validity was assessed in the EDC (original and recent), FinnDiane and CACTI cohorts. Follow-up in the first EDC selection was truncated in order to analyse discrimination for a median follow-up time of 8 years, which is similar to the follow-up time of the development set. Discrimination was assessed with the Harrell’s concordance statistic (C-statistic) [26]. Discrimination was assessed graphically using Kaplan–Meier plots for three risk groups (high, intermediate and low risk). Calibration plots were made to compare observed with predicted risks at external validation.

Finally, a score chart was made based on the regression coefficients in the final model. Scores were calculated as the products of regression coefficients and prognostic factor values and rounded to integers. The sum scores were then related to 3, 5 and 7 year risks of major outcomes.

Results

Development of the prognostic model

Major outcomes occurred within 7 years of follow-up in 95 of 1,973 participants in the EURODIAB PCS. The participants had a mean age of 30.3 years (SD 8.3) and the median duration of diabetes was 11.5 years (Table 1).

Table 2 shows the univariate associations of the possible prognostic factors and first incident major outcomes. All the variables were positively associated with incident major outcomes except for HDL-cholesterol level, which showed an inverse association. The final prognostic model included age, HbA1c, WHR, albumin/creatinine ratio and HDL-cholesterol level (Table 3). The albumin/creatinine ratio was log-transformed whereas all other continuous prognostic factors were included linearly. The strongest factors were age (Wald statistic = 5.3) and albumin/creatinine ratio (Wald = 4.4). The regression coefficients in the final model were multiplied by the estimated shrinkage factor of 0.93 (see the ESM Results for the final regression formula). The C-statistic was 0.74 at internal validation.

Table 2 Occurrence of major outcomes for candidate prognostic factors in EURODIAB PCS
Table 3 Multivariable associations of the selected prognostic factors with the occurrence of major outcomes

A score chart (Fig. 1, ESM Model) was made to facilitate the calculation of the absolute risk of major outcomes for an individual patient with type 1 diabetes. Values for continuous prognostic factors are given with small steps. A sum score for an individual patient consists of the sum of all scores. The lower part of ESM Model shows the predicted 3, 5 and 7 year risks that correspond to the sum scores.

Fig. 1
figure 1

(a) Tool to predict 3, 5 and 7 year absolute risks of major outcomes in type 1 diabetes. Values can be given for each prognostic factor. These values and hypothetical scores for a hypothetical patient have been filled in. (b) Graph with 3, 5 and 7 year risks of major outcomes in patients with type 1 diabetes. The vertical dashed line indicates a sum score of 20 for the hypothetical patient on the x-axis, and a 3 year risk of major outcomes of 6% (solid line), a 5 year risk of 10% (dotted line) and a 7 year risk of 15% (dashed line) on the y-axis. To convert values for HbA1c (glycated haemoglobin) in DCCT % into mmol/mol, subtract 2.15 and multiply by 10.929

We found that many participants had intermediate scores of 16–20. We therefore distinguished three (absolute) risk groups that were based on the sum scores: a low-risk group with a score between 8 and 15; an intermediate-risk with a score between 16 and 20; and a high-risk group with a score of 21 or more. The risk groups showed a 7 year risk of 1%, 4% and 11%. The high-risk group contained 457 (23%) participants (Fig. 2a). The highest two risk groups (sum scores 16+) together contained 94% (n = 50 + 39 = 89) of all the participants who developed major outcomes.

Fig. 2
figure 2

Kaplan–Meier estimates for the risk of major outcomes. Patients are categorised based on their total score (see Fig. 1): dotted line, 8–15, low risk; dashed line, 16–20, intermediate risk; solid line, ≥21, high risk. (a) EURODIAB PCS, (b) EDC original, (c) EDC recent, (d) FinnDiane, (e) CACTI. (f) The observed number of participants (n) and number of events (Ev.) per score group. %, percentage of participants per score group divided by total number of participants in each cohort

External validation of the prognostic model

Most participant characteristics from the external validation studies were comparable with the EURODIAB participants (Table 1). The mean ages for the EURODIAB and original EDC cohorts were similar, with means of 30.3 and 28.0. The recent EDC had a population that was 8 years older than that in the original EDC as the predictor values were measured 8 years later. FinnDiane and CACTI included older participants than EURODIAB. The median duration of diabetes differed between the studies, from 11.5 years in EURODIAB to 26.4 in the EDC recent cohort. The occurrence of major outcomes within 7 years was relatively high in the EDC original (14%), EDC recent (20%) and FinnDiane (11%) compared with the EURODIAB (4%) and CACTI (7%) populations. The composition of major outcomes differed between various cohorts.

The discriminative ability of the prognostic model was adequate in all cohorts, with C-statistics of 0.79 in EDC original, 0.74 in EDC recent and 0.73 in CACTI compared with a C-statistic of 0.74 in the development set. The discriminative ability tended to be even higher in the FinnDiane cohort than in the other cohorts, with a C-statistic of 0.82. The relatively high discriminative ability in the FinnDiane was explained by greater heterogeneity in the prognostic factor values. The SD of the linear prognostic factor (the sumproduct of risk estimates and regression coefficients) was 1.01 for FinnDiane compared with 0.81 for EURODIAB. The majority of participants with major outcomes were categorised in one of the two groups with the highest risk: 92/98 (94%) for EDC original, 82/84 (98%) for EDC recent, 306/315 (98%) for FinnDiane and 40/42 (95%) for CACTI.

The observed risks were much higher for the original and recent EDC participants than for participants from the other cohorts. Kaplan–Meier estimates at 7 years were 15.9% (EDC original), 21.5% (EDC recent), 10.3% (FinnDiane) and 6.9% (CACTI). As a consequence, the mean predicted risks were too low for the EDC cohorts (5.2% and 7.2%) and for FinnDiane (7.1%). The same pattern is shown for the three risk score groups. For instance, the risk at 7 years was 1% (low risk), 4% (medium risk) and 11% (high risk) for the EURODIAB participants, with 5%, 7% and 34% for the original EDC and 7%, 10% and 34% for the recent EDC. High-risk participants from FinnDiane had also higher observed risks than predicted (21% observed vs 15% predicted at 7 years) (Fig. 2; to obtain risk estimates, read the y-axis for each Kaplan–Meier curve at the 7 year point on the x-axis).

The underprediction is further shown in the calibration plots (Fig. 3a, c, e). Quintiles of risk estimates corresponded to the Kaplan–Meier estimates only for CACTI (Fig. 3g). To provide well-calibrated risk estimates, a small model adjustment was necessary for participants from EDC and FinnDiane (Fig. 3b, d, f). The intercept of the Weibull model was adjusted for the EDC recent population to an intercept of 9.1 rather than 10.2; for participants from FinnDiane, the intercept became 9.8.

Fig. 3
figure 3

Calibration plots for observed and predicted risk at external validation for EDC, EDC recent, FinnDiane and CACTI. Triangles represent the observed frequencies by quintile of predicted risk. Calibration of the model was poor for EDC (a, c) and FinnDiane (e). Adjustment of the model intercept (b, d, f) markedly improved the agreement between the observed and predicted risks. (a) EDC original, (b) EDC original adjusted for intercept, (c) EDC recent, (d) EDC recent adjusted for intercept, (e) FinnDiane, (f) FinnDiane adjusted for intercept, (g) CACTI. K–M, Kaplan–Meier

Discussion

This paper describes the development of a prognostic model for the risk of major outcomes (CHD, stroke, end-stage renal failure, amputations, blindness and death) in participants with type 1 diabetes. The prognostic model is based on common and easily measured prognostic factors, which will allow for a translation to clinical care. The model showed good discriminative ability in external validation cohorts. We found a systematic difference between the predicted and observed complications in participants from the EDC and FinnDiane populations, but the miscalibrations disappeared after a small adjustment of the model.

Our prognostic model included age, HbA1c level, WHR, albumin/creatinine ratio and HDL-cholesterol level. From aetiological research in patients with type 1 diabetes, most of these are known to be associated with CHD and other complications. HbA1c level and albumin/creatinine ratio were shown to be strong prognostic factors for CHD, renal failure and death in type 1 diabetes [27]. A number of prognostic models for the major outcome CHD have been evaluated [2832], but only a few specifically for type 1 diabetes [12]. A prognostic model for CHD in participants with type 1 diabetes from the EDC study [12] included HDL-cholesterol level, micro- or macroalbuminuria, duration of diabetes and white blood cell count as the strongest prognostic factors in men, and non-HDL-cholesterol level, WHR, duration of diabetes, systolic BP and use of antihypertensive medication as the strongest prognostic factors in women. In our prognostic model, age was selected rather than duration of diabetes. The high correlation between age and duration of diabetes in the EDC (Pearson correlation = 0.85) indicates that the two variables can include similar information. This correlation was lower in the EURODIAB (0.43), with age being the stronger prognostic factor.

Other known prognostic factors, such as BP or the use of antihypertensive medication, LDL-cholesterol and smoking were not included in our model because of weak additional effects. The exclusion of some of these prognostic factors (BP, LDL-cholesterol, smoking and family history) from our model does not mean these should not be measured or controlled. Instead, it implies that these factors are less relevant for the prediction of major outcomes, particularly when other prognostic factors are already being considered. For instance, hypertension was correlated with age and AER. Older participants with a higher AER have a relatively high BP. With age and AER already in the model, the added value of hypertension or BP was limited. In EURODIAB, fibrinogen level was available but its added predictive value was negligible (the C-statistic with and without fibrinogen was 0.74). The minor additive value of predicting cardiovascular diseases with inflammatory markers such as fibrinogen and C-reactive protein was also recently demonstrated by the Emerging Risk Factors collaboration [33]. We did not examine sex-specific prognostic models since the number of participants with major outcomes in the development set was relatively low and this would result in unstable risk estimates.

Our development cohort included participants from 16 European countries. The relatively heterogeneous cohort allowed the development of a model that generalised well to participants from other cohorts from Northern America and Europe. The FinnDiane and CACTI cohorts in particular allowed the model to be validated in more recently treated patients with type 1 diabetes. Participants from the EDC and FinnDiane cohorts experienced major outcomes within 7 years more often than the participants from EURODIAB and CACTI. This could not completely be explained by higher values for the prognostic factors in the model. Hence the risk estimates may require regular updating according to specific population characteristics, such as calendar year or local organisation of diabetes management [34, 35]. For the EDC and FinnDiane populations, a simple intercept adjustment suffices.

Predicting major outcomes permits the establishment of a risk profile for individual patients with type 1 diabetes. Physicians may consider active intervention in the identified high-risk patients. Such interventions could include intensifying the insulin regimen and cardiovascular risk management. We stress that our prognostic model is particularly able to identify patients with type 1 diabetes at the highest risk of complications. Clinicians should nevertheless focus on an aggressive lowering of all known prognostic factors. Furthermore, physicians can use the risk assessments to inform patients on lifestyle and diet. Prior research has indicated benefits of diet and physical activity even in intensively treated patients with type 1 diabetes [36, 37]. The risk assessment also permits a more efficient design and analysis of randomised controlled trials on the effectiveness of new interventions in patients with type 1 diabetes [38]. Patients with a very good or poor prognosis may be excluded; we can perform a covariate adjustment of an intervention effect to increase the statistical power [39].

Our study has several limitations. Relatively low numbers of participants had major outcomes as most of the participants were young. Using the composite of CHD, stroke, renal failure, amputations, blindness and death, more participants experienced one of the outcomes, which increased the statistical power [40]. The trade-off of using a composite endpoint is that the predictor effects of different outcomes are averaged [40]. The outcomes of diabetes that we combined are related to the circulatory system, essentially vascular outcomes. It has been shown that microvascular outcomes predict macrovascular outcomes, indicating that the two types of outcome are related [18, 41], with a possible common underlying pathophysiology [42, 43]. The division of the major outcomes into micro- and macrovascular-specific outcomes for prognostic models was demonstrated recently in type 2 diabetes patients [44]. The prognostic factors and risk estimates varied somewhat across types of complication. Unfortunately, the much younger age of the type 1 diabetes patients with fewer events during the first 7 years of follow-up did not allow a separate examination of the major outcomes.

Furthermore, participants in most studies were included between 1988 and 2002, except for FinnDiane, which had a more recent inclusion period (enrolment up to 2009). Even though evaluation in the FinnDiane cohort confirmed the validity of the risk estimates for more recent times, we cannot exclude the possibility that the frequency of major outcomes is different today. Furthermore, we could not examine the generalisability of the prognostic model to non-white populations since such cohorts were not available.

In conclusion, a prognostic model is now available to assess the absolute risk of major outcomes in patients with type 1 diabetes. The prognostic model may be useful for providing individual risk estimates for major outcomes. The risk estimates can guide surveillance recommendations, inform patients and allow the efficient design and analysis of clinical trials.