Introduction

Cardiovascular disease (CVD) is a major cause of morbidity and mortality in people with diabetes. Prediction of CVD risk among people with diabetes is important not only to tailor clinical care but also to stratify higher risk patients in clinical trials to maximise trial power [1]. Biomarkers may help improve prediction, but whilst there are many reports on associations of single biomarkers with CVD risk in diabetes, there are few joint assessments of large numbers of biomarkers. Thus, there is little consensus on which subset of biomarkers in the literature is most informative for CVD risk prediction in diabetes.

The SUrrogate markers for Micro- and Macro-vascular hard endpoints for Innovative diabetes Tools (SUMMIT) consortium, funded by the European Union Innovative Medicines Initiative, is a collaboration involving 19 academic centres and six European Federation of Pharmaceutical Industries and Associations (EFPIA) partners across Europe to identify and characterise biomarkers for complications of diabetes [2]. We measured a panel of 42 candidate biomarkers with existing evidence for association with CVD risk in 1,123 incident CVD cases and 1,187 controls with type 2 diabetes from five European cohorts. The aims of the analysis were to select biomarkers predictive of CVD and assess the added value of the biomarkers beyond Framingham and clinical factors to prediction of CVD.

Methods

Data and sample sources

This study was a nested case–control design matched for sex and age. Cases and controls were identified within five cohorts: Genetics of Diabetes Audit and Research Tayside Study (Go-DARTS), Scania Diabetes Registry (SDR), MONItoring of trends and determinants in CArdiovascular disease/Cooperative Health Research in the Region of Augsburg (MONICA/KORA) study, Carotid Intima Media Thickness [IMT] and IMT-Progression as Predictors of Vascular Events in a High Risk European Population (IMPROVE) study and Stockholm 60-year Old Study [3–7] (electronic supplementary material [ESM] Table 1). For each cohort, participants with type 2 diabetes and no clinical history of CVD when the blood samples were taken were eligible for inclusion. A small number (n = 65) of participants who were prediabetic (i.e. had non-diabetic hyperglycaemia at the time of sampling and subsequently progressed to type 2 diabetes) were also eligible. Incident major CVD cases were defined as having acute CHD or an ischaemic stroke event subsequent to the baseline sample being taken. Detailed criteria are included in the ESM (Methods: Data and sample source).

Controls were free of CVD at the end of follow-up and matched to cases within each cohort by sex and were stratum-matched for age at the time of blood sampling (within 5 years). Overall 2,318 individuals were sampled, of whom eight had insufficient sample volume, leaving serum samples from 1,123 cases and 1,187 controls.

Biomarker selection process and laboratory analysis

Biomarker selection was based on existing evidence from the literature (incorporating a text mining process [8] and a manual literature review) and the availability of reliable validated assays to measure biomarker concentrations in small volumes of serum. We prioritised biomarkers of cardiac damage (high-sensitivity troponin T [hsTnT] and N-terminal pro-B-type natriuretic peptide [NT-proBNP]), renal disease (cystatin C, fetuin A), glucose-mediated damage in diabetes (soluble receptor for AGE [sRAGE]), matrix remodelling (matrix metalloproteinases and osteopontin) and coagulation (factor VII), as well as a lipoprotein of particular interest in diabetes (apolipoprotein C-III [apoCIII]) [9, 10], and a large set of immune- and inflammation-related proteins including interleukins and interferons. We measured 33 protein biomarkers (ESM Table 2) by multiplex immunoassay on the Human DiscoveryMAP customised panel using the Luminex 100/200 instrument by RBM (Myriad Rules Based Medicine, Austin, TX, USA) [11] and standard commercially available ELISAs and kits for nine biomarkers. The included biomarkers are listed in ESM Table 2 along with details of assays and assay metrics. For most biomarkers the inter-assay CV was <10% with the exception of apoCIII (16%), cathepsin S (17%), granulocyte-macrophage colony-stimulating factor (23%) and TNF-α (18%). (For more details on laboratory methods see ESM Methods: Biomarker selection process and laboratory analysis and ESM Table 2).

Statistical methods

Choice of covariates in the baseline model

Clinical covariates included in the models for biomarker selection were selected based upon their inclusion in existing CVD risk engines [12, 13] or known associations with CVD in type 2 diabetes and their availability for all cohorts. We also included baseline medication data in case any biomarker measurements were affected by drug use. All clinical covariates were measured at or close to the time of sampling. The chosen covariates were age, sex, smoking, systolic and diastolic blood pressure, LDL-cholesterol (LDL-C), HDL-cholesterol (HDL-C), triacylglycerol, diabetes duration, HbA1C, BMI, height, estimated GFR (eGFR) calculated using the Modification of Diet in Renal Disease 4-variable (MDRD4) equation, cohort, and current medication (including antihypertensive agents, aspirin, lipid-lowering agents and insulin).

Pre-processing and imputation

All continuous variables were Gaussianised using the LambertW package version 0.5 in R (http://cran.r-project.org/web/packages/LambertW) [14, 15]. Missing values of biomarkers and covariates were imputed using a sparse iterative regression model. For six biomarkers, >96% of participants had below detectable or missing levels and these were not analysed further (ESM Methods: Pre-processing and imputation). The impact of imputation on the analyses was explored.

Selection of predictive sets of biomarkers

We applied two complementary approaches for selecting which subsets of biomarkers contributed most to prediction: forward selection using logistic regression, and a top-down method based on logistic regression with an L1 (least absolute shrinkage and selection operator [LASSO]) regularisation penalty [16]. See ESM for details (Methods: Selection of predictive sets of biomarkers).

Predictive performance evaluation

We evaluated predictive performance of models by computing the area under the receiver operating characteristic curve (AUROC) on 50 folds of test data, where the test folds were not used either for model fitting or for biomarker selection. We used 30 inner folds to iteratively select biomarkers (forward selection) or to learn the penalty parameter (top-down approach). This nested cross-validation procedure provides an unbiased estimate of the predictive performance of the considered models. We assessed four models, two including only clinical covariates and two including clinical covariates and biomarkers. The first model considered is based on the Framingham Risk Score and was limited to the clinical factors in that score along with the relevant interaction terms for variables that have stratum-specific coefficients in that equation; the main effects were age, sex, total cholesterol, HDL-C, systolic blood pressure, antihypertensive treatment and current smoking, and interaction terms (sex × other variables, systolic blood pressure × antihypertensive treatment) were also allowed to enter the model [12]. We also considered a model including an extended set of clinical factors that included clinical measures beyond the Framingham Risk Score factors, including important diabetes measures such as diabetes duration, HbA1c, eGFR and insulin use. To the extended clinical covariate model we then added selected biomarkers from either the forward selection or the top-down selection models.

Net reclassification improvement

To assess the clinical relevance of the increment in predictive performance and for consistency with other reports on biomarker performance we estimated the net reclassification improvement (NRI) with a classification threshold of 10% risk at 5 years. Calculation of the NRI from a nested case–control study requires the reconstruction of the original cohort from the cases and controls that were sampled from this cohort [17]. This reconstruction was done by weighting the observations on cases and controls so that the proportion of cases equated to the risk at 5 years computed from the annual event rate, and by using the logistic regression model from the nested case–control study to calculate the predictive probability of disease for individuals in this reconstructed cohort. The event rates for the Go-DARTS cohort were used as a proxy for the rates in all cohorts.

Results

The final dataset consisted of 36 biomarkers in 2,310 individuals (1,123 cases, of which 755 were acute CHD and 368 strokes, and 1,187 controls). Median time to event was 3.2 years (interquartile range [IQR] 1.5–4.9) for cases and median follow-up for controls was 6.5 years (IQR 3.9–7.9). Baseline clinical data are summarised in Table 1.

Table 1 Demographics and clinical characteristics at baseline
Full size table

Forward selection of biomarkers using logistic regression

The distributions of biomarkers in cases and controls are summarised in ESM Table 3. After adjustment for all clinical covariates, 16 biomarkers showed association with CVD at p < 0.05 (shown in the first two data columns in Table 2). The first iteration of the forward selection process fits a cross-validated logistic regression model for each biomarker singly, and evidence for association independently of all clinical covariate data is given by the magnitude of the increment in validation log-likelihood and by the false discovery rate (FDR)—the lower the FDR the stronger the association (Table 2). The strongest associations with CVD were for NT-proBNP, hsTnT, IL-6, apoCIII, cystatin C and IL-15 (p < 0.001, and with increments in test log-likelihood ≥5 natural log units and FDR <0.1 in the cross-validated forward selection models). Of these biomarkers, other than apoCIII, levels were higher in cases than controls.

Table 2 Single biomarker logistic regressions on the full dataset with results of first iteration of the nested k-fold cross-validated forward selection model across outer folds
Full size table

Many of the biomarkers studied showed strong correlations with each other (ESM Fig. 1). Table 3 shows the selection of biomarkers in the forward selection nested procedure and their rank in the models (after all iterations) summarised over the 50 outer test/training folds. Six biomarkers were selected in 100% of the 50 outer training folds as improving prediction of CVD beyond clinical covariates. The ORs for the final model including these six biomarkers are also shown in Table 3.

Table 3 Biomarkers selected by forward selection and a simple logistic regression model adjusted for all covariates and biomarkers selected at least once by forward selection
Full size table

Five of the six selected biomarkers were those that showed the strongest associations examined singly. Cystatin C, which showed a strong univariate association, was not retained by the forward regression due to its strong correlation with other biomarkers including NT-proBNP. Conversely, sRAGE was retained in the forward selection although its association with CVD before adjustment for the other biomarkers was weak. The biomarkers selected by forward selection were also retained as the best predictive biomarkers by the top-down method (data not shown). We used forward selection to identify which clinical covariates, beyond those included in the Framingham Risk Score and the already selected biomarkers, were important predictors of CVD. Three additional clinical covariates were selected: (in order of selection in the forward selection model) insulin therapy status, eGFR and HbA1c. See ESM (Results: Forward selection of biomarkers using logistic regression) for evaluation of the impact of imputation, Gaussianisation and the deletion of outliers on the biomarker selections.

Predictive value of models evaluated using AUROC

The impact of adding clinical covariates and biomarkers to the Framingham Risk Score model is shown in Table 4. The addition of the extended clinical covariates to the Framingham Risk Score model significantly increased the test log-likelihood as did addition of the selected biomarkers. There was also a significant impact of the addition of the biomarkers to the full clinical covariate model. The Framingham Risk Score model had the poorest predictive performance (AUROC = 0.59). However, in an age and sex stratum matched design the predictive performance attributable to the age and sex covariates is constrained by the matched design so the AUROCs for all the models will be less than if age and sex are not matched. The AUROCs were 0.66 for models including clinical covariates, only 0.72 for models with addition of biomarkers chosen by forward selection, and 0.71 for models learned from all biomarkers by the top-down approach. The greatest increase in AUROC was seen on addition of the first biomarker, which was always NT-proBNP (ESM Fig. 2).

Table 4 Predictive performance for models assessed by AUROC
Full size table

NRI

By use of a 5 year CVD risk threshold of 10% to define high risk, adding the six biomarkers chosen by forward selection to a model based on clinical covariates only resulted in 3.67% of cases and 4.40% of non-cases being reclassified with respect to this threshold, giving a NRI of 8.07%.

Discussion

In this study of 42 potential CVD biomarkers we confirmed associations with CVD independently of clinical covariates for 16 biomarkers, and use of these biomarkers modestly improved the prediction of CVD beyond that obtained by clinical covariates with an increase in the AUROC from 0.66 to 0.72. Based on the conventional decision threshold of a 5-year CVD risk of 10%, we estimated that 8% of individuals would be reclassified. This improvement was generated by six biomarkers: NT-proBNP, hsTnT, IL-6, IL-15, apoCIII and sRAGE.

Our analysis has confirmed previously reported associations as well as novel associations with CVD in diabetes. There is increasing data on the importance of NT-proBNP [18] and hsTnT [19] for CVD prediction in the general population. Our data confirm their importance as predictive biomarkers in diabetes independently of eGFR and the other biomarkers. ApoCIII inhibits the catabolism of triacyglycerol-rich lipoproteins, partly through inhibition of their hepatic uptake and partly through inhibition of lipoprotein lipase activity. ApoCIII non-specific gene deletion in animal models is associated with premature atherosclerosis [20], and most studies suggest a likely pro-atherogenic role [21]. ApoCIII is known to be an important link between glycaemia and dyslipidaemia [22] as its expression is induced by high glucose levels and is responsive to insulin. However, whilst several studies have shown that higher apoCIII content on lipoproteins is associated with their atherogenicity [23, 24], the association of total circulating apoCIII itself with CVD has had less investigation [21]. One of the largest studies is the Hoorn study [25]. In this study, participants who died of CVD had higher baseline levels of apoCIII, and the association was mediated in part by the positive correlation of apoCIII levels with dysglycaemia and triacylglycerol levels [25]. In another analysis of two sex-specific cohorts, after adjustment for triacylglycerol levels, no association was found between total plasma apoCIII levels and incident CVD [24]. Our finding of an inverse association with CVD that became even more apparent on adjustment for HDL-C and triacylglycerol levels is therefore unexpected. As the direction of association with apoCIII was unexpected, we confirmed that apoCIII levels correlated positively with triacylglycerol levels (r = 0.25) but had little correlation with LDL-C (r = −0.02) or HDL-C (r = 0.06). The intra- and inter-assay CVs were quite high at 14.56% and 15.84%, respectively (ESM Table 2), but we confirmed that the lower apoCIII levels in cases than controls was found in all participating cohorts (data not shown). Thus, we are confident that the assay performed as expected and that this is a true finding that warrants further study. We note that our analysis is in non-fasting frozen samples; however, apoCIII has been shown to be unaltered by fasting [26] and by freezing [27]. One small study in patients with type 2 diabetes also found a lower level of apoCIII in cases than controls of CVD [27]. In view of the unexpected association in our study and the elevated inter-run CV for the assay, we re-ran all analyses excluding apoCIII and found that the selection of the other five biomarkers was not affected, although the total AUROC was somewhat lower (data not shown).

IL-15 is a proinflammatory cytokine for which there is substantial cell and animal model evidence for a role in atherosclerosis [28, 29], but its characteristics as a predictive biomarker for CVD in diabetes have not been reported previously. IL-6 has been characterised as a risk factor for CVD in several cohorts [30] and we confirm its importance here. In a subanalysis of the MONICA/KORA samples that also included C-reactive protein (CRP) measurement, the association between IL-6 and CVD was attenuated but the association of IL-15 with CVD remained (data not shown). Thus, it is likely that IL-6 will not add a great amount of information to a model in the presence of CRP. sRAGE has not been studied in many prospective studies of type 2 diabetes; we previously showed that higher levels were associated with an increased risk of CHD but little difference in stroke risk in clinical trials [31]. The inverse association between sRAGE and CVD found here, which is only apparent once it has been adjusted for eGFR and other biomarkers, is contradictory and requires further exploration in other cohorts.

We have used a candidate biomarker approach in this study rather than a global discovery approach using proteomics or metabolomics platforms. Of the biomarkers included in our study, osteopontin has also been identified as a biomarker in a proteomics study of atherosclerotic plaques and then validated in serum [32]. Other potential biomarkers, such as heat shock protein 27, cathepsin D and transthyretin [33], which have been identified by proteomics of atherosclerotic plaques, or the cytoskeletal protein vinculin from plasma proteomic profiling [34], were not included in our study.

To compare biomarkers we used an approach that is common in the field of machine learning, i.e. N-fold cross-validation, in which the entire dataset from across cohorts is partitioned into N test folds, models are fitted to each of the corresponding training folds and the performance of these models is evaluated by their ability to predict outcome in the test folds. We consider that the advantages of this approach in comparison with model fitting, for example on one half of the available cohorts and performance assessment in the other half are: (1) with N-fold cross-validation, model fitting uses a fraction (1–1/N) of the observations rather than only half the observations so it is more powerful if N is large; (2) evaluation of predictive performance on test data uses all the observations because every observation appears once in a test fold; and (3) all cohorts are represented in each training dataset and in each test dataset, so we can expect that when the existing cohorts are representative of new test patients the results should be more widely generalisable than the conventional approach.

A recent systematic review examined 12 CVD-related models developed for CVD prediction in diabetes and 33 that included diabetes as a factor in the model [35]. Of those developed in patients with diabetes, four examined total CVD risk, with two—one developed in the ADVANCE (Action in Diabetes and Vascular Disease: Preterax and Diamicron Modified Release Controlled Evaluation) study [36] and a Swedish study [37]—reporting the AUROC without any correction for overfitting (0.70 in both). Both of these models included age, sex, diabetes duration and HbA1c. The ADVANCE model also included albuminuria, retinopathy, non-HDL-cholesterol, atrial fibrillation and pulse pressure. The Swedish model included smoking, BMI, lipid-lowering drug use and systolic blood pressure but no measure of renal function. The United Kingdom Prospective Diabetes Study (UKPDS) risk engine for CHD and the Framingham Risk Score for CVD are the models most commonly recommended, but neither includes any measure of renal function, glycaemic control or stage or severity of diabetes. Our analysis shows that inclusion of eGFR, HbA1c and insulin therapy status is also important for CVD prediction in diabetes.

The clinical utility of this work is somewhat limited at present, in that the overall improvement in prediction due to addition of biomarkers is modest over and above clinical covariates. However, this modest improvement may still be of use when designing CVD endpoint trials as another means of enriching trials for high-risk participants. We have also demonstrated that inclusion of diabetes-specific CVD risk factors does improve the prediction of the models and should be considered when assessing patient risk.

A limitation of our study was that all the cohorts included were European and the extent to which our conclusions generalise to other populations remains to be established. A further issue was that low sample volumes precluded study of a more comprehensive set of all candidate biomarkers. This issue is a practical constraint for many epidemiologic cohorts. Larger gains in prediction might be demonstrable through measuring high dimensional metabolomic and lipidomic panels evaluable on small volumes of serum and this approach will be evaluated further. Finally, predictive performance is also a function of assay method: where more sensitive or more accurate assay methods are used these would be expected to improve the performance of the biomarker. Whilst we found that several of the cytokines on the Human DiscoveryMAP panel were undetectable in 70% or more of samples, a high level of missingness is to be expected for these low abundance cytokines when acute inflammation is not present.

In conclusion, six biomarkers—NT-proBNP, hsTnT, IL-6, IL-15, apoCIII and sRAGE—captured most of the predictive information about CVD from a panel of 42 biomarkers. By use of these biomarkers and an extensive set of clinical covariates, prediction was considerably improved compared with using only the covariates included in the Framingham Risk Score. For optimised prediction of CVD in diabetes, novel biomarkers can contribute; however, measures of renal function and diabetes-specific factors, not currently commonly used in risk scores, need to be included.