Advertisement

Diabetologia

, 52:2001 | Cite as

Cardiovascular risk assessment scores for people with diabetes: a systematic review

  • P. Chamnan
  • R. K. Simmons
  • S. J. Sharp
  • S. J. Griffin
  • N. J. WarehamEmail author
Open Access
Review

Abstract

People with type 2 diabetes have an increased risk of cardiovascular disease (CVD). Multivariate cardiovascular risk scores have been used in many countries to identify individuals who are at high risk of CVD. These risk scores include those originally developed in individuals with diabetes and those developed in a general population. This article reviews the published evidence for the performance of CVD risk scores in diabetic patients by: (1) examining the overall rationale for using risk scores; (2) systematically reviewing the literature on available scores; and (3) exploring methodological issues surrounding the development, validation and comparison of risk scores. The predictive performance of cardiovascular risk scores varies substantially between different populations. There is little evidence to suggest that risk scores developed in individuals with diabetes estimate cardiovascular risk more accurately than those developed in the general population. The inconsistency in the methods used in evaluation studies makes it difficult to compare and summarise the predictive ability of risk scores. Overall, CVD risk scores rank individuals reasonably accurately and are therefore useful in the management of diabetes with regard to targeting therapy to patients at highest risk. However, due to the uncertainty in estimation of true risk, care is needed when using scores to communicate absolute CVD risk to individuals.

Keywords

Cardiovascular disease Diabetes Prediction Risk score Systematic review 

Abbreviations

AIC

Akaike’s information criterion

aROC

Area under the receiver operating characteristic curve

BIC

Bayesian information criterion

CVD

Cardiovascular disease

DECODE

Diabetes Epidemiology: Collaborative Analysis of Diagnostic Criteria in Europe

FPG

Fasting plasma glucose

IDI

Integrated discrimination improvement

NRI

Net reclassification improvement

PROCAM

Prospective Cardiovascular Münster

ROC

Receiver operating characteristic curve

SCORE

Systematic Coronary Risk Evaluation

UKPDS

UK Prospective Diabetes Study

Introduction

Type 2 diabetes is a strong risk factor for cardiovascular disease (CVD), with some studies suggesting that it confers an equivalent risk to having had a myocardial infarction [1, 2]. Multifactorial interventions, such as the Steno-2 study, have been effective in reducing the risk of non-fatal [3] and fatal CVD [4] among diabetic patients through therapy targeting hyperglycaemia, hypertension and hypercholesterolaemia. Despite this evidence of effectiveness, many countries use a rationing approach to the prescription of cardiovascular risk reduction treatment, with national guidelines suggesting that patients should have their risk of CVD calculated, to ensure therapy is targeted to patients at highest absolute risk. Multivariate risk scores have, therefore, been used to predict CVD risk in individuals with diabetes.

There are a large number of scores for the general population, but few that are specific to people with diabetes. Whether general population scores can accurately be used in a subgroup of individuals with diabetes is unclear [5, 6]. The most commonly used score is that originally developed in 5,573 men and women participating in the Framingham study in the early 1970s, which, in general, performs well in North America [7], but less well in other populations [7, 8, 9]. We aimed to review the published evidence on performance of CVD risk scores in diabetic populations. First, we examined the overall rationale for using cardiovascular risk scores in patients with diabetes. Second, we provide results from a systematic review of the published literature on CVD risk scores that have been developed or evaluated in individuals with diabetes. Finally, we explored methodological issues surrounding the development, validation and comparison of risk scores.

Why estimate CVD risk in individuals with diabetes?

There are several reasons why it may be important to quantify the risk of developing CVD in patients with diabetes. The clear identification of the rationale for developing a risk score is critical to how its validity is assessed. For example, if the purpose of a score is to rank individuals and groups according to absolute risk for the purpose of targeting therapy to those at greatest risk, then it is the ranking that is important and not necessarily the absolute risk estimate. If, on the other hand, the principal justification is to provide prognostic information or accurate estimation of the likely absolute benefit from a therapeutic intervention, then a precise computation of absolute risk is important. Finally, if the main reason for calculating risk as part of a preventive strategy is to motivate patients to change their behaviour and adhere to medical treatments, it may be important to calculate modifiable risk rather than use a score dominated by fixed variables. In the following sections we describe the origin of CVD risk scores and illustrate how purpose, construction and validation are often disconnected.

CVD risk scores have been widely used in the UK and elsewhere for over 10 years and were introduced at a time when the cost of cholesterol-lowering drugs was an important political issue [10]. National recommendations began appearing in the mid-1990s, when specified annual CVD risk estimates were used to determine thresholds for prescribing therapy. These thresholds were decided on the basis of the number needed to treat to prevent one CHD event, the cost-effectiveness of treatment and, predominantly, the proportion of the population requiring treatment and the total cost of treatment [11]. As such, the threshold for intervention based on the ranking of absolute risk was largely a financial decision. Although results from the Heart Protection Study [12] and the Collaborative Atorvastatin Diabetes Study [13] suggest that statin therapy is effective in patients with diabetes at high CVD risk, irrespective of their initial cholesterol concentrations, not all health systems can afford such a policy. The ranking of absolute risk is clearly important for making collective decisions about therapy, but, beyond statin prescribing, does the calculation of CVD risk aid clinical decision-making?

An estimate of absolute risk can inform the potential for absolute risk reduction, providing patients with an idea of expected benefit from a therapy or intervention. This is important for individual, rather than collective, decision-making. However, further research is needed to understand the process by which the clinician and patient interact once cardiovascular risk has been assessed [9]. While it is assumed that telling individuals their CVD risk is a motivating tool, there is only weak evidence that this assumption is valid. In a systematic review of randomised controlled trials that evaluated the effectiveness of risk-scoring methods, no strong evidence was found that a CVD risk assessment by a clinician improved CVD-related health outcomes [9]. Practitioners need to provide information about what patients can do to reduce this risk; they also need to acknowledge the possibility of false reassurance or a fatalistic response, both of which can in theory lead to an increase in population risk. Qualitative work has demonstrated that very few patients with type 2 diabetes understand the direct link between having diabetes and their CVD risk [14]. Although they are aware of CVD, they are more likely to attribute it to external or immutable factors such as stress and heredity, rather than modifiable risk factors such as high cholesterol, hyperglycaemia and smoking [14]. This phenomenon links in with the idea of using risk scores as preventive or motivating tools. Most of the predictive value of a CVD risk score comes from the inclusion of two unchangeable risk factors, age and sex. It is probably difficult to persuade patients to change their behaviour on the basis of CVD risk scores that are mostly driven by risk factors patients cannot change. Thus risk scores that only incorporate modifiable risk factors are more likely to be useful for preventive strategies in patients with diabetes.

Systematic review of CVD risk assessment tools

Search strategy

A comprehensive literature search for studies of CVD risk assessment tools was performed using MEDLINE, Web of Science and Cochrane Reviews from database inception up to 30 June 2008. The search strategy focused on four key elements: CVD, type 2 diabetes, risk assessment/score/prediction and specific names of known risk scores (see Appendix). We also screened the reference lists of papers identified from the initial electronic search. No language restriction was applied; articles were translated when necessary.

Selection criteria

We included studies reporting CVD risk assessment tools or scores that: (1) were derived from prospective cohort studies or randomised trials; (2) were derived in the general population and evaluated in individuals with diabetes, or developed in a diabetic population; and (3) reported a measure of performance of the risk score for predicting CVD. We restricted the review to studies that reported the following diseases as a primary outcome:
  • Fatal or non-fatal CVD

  • Fatal or non-fatal CHD

  • Fatal or non-fatal cerebrovascular disease or stroke

We excluded studies that derived cardiovascular risk scores for the general population but did not evaluate them in individuals with diabetes. We excluded studies that derived risk prediction tools other than score-type tools, such as those using carotid ultrasonography and myocardial perfusion scintigraphy. If scores and their evaluation were reported in several different papers, we included the score only once by selecting the paper that reported the most information on predictive ability.

Data extraction

Two reviewers (P. Chamnan, R. K. Simmons) independently reviewed the results from the primary search of titles, followed by the abstract and full paper searches (Fig. 1). Where reviewers disagreed, consensus was reached through discussion. The two reviewers used a standardised form to extract data on the performance of the risk scores. This included the name of the risk score and study, the country and setting, details on derivation and validation populations, follow-up for derivation and validation cohorts, definition of diabetes and CVD, risk factors included in the scores, and measures of predictive ability, including discrimination, calibration, sensitivity, specificity, and positive and negative predictive value. We also extracted data from original studies if articles identified through the initial search did not contain information on the development or validation of risk scores (Tables 1 and 2).
Fig. 1

Flow of identification of included studies

Table 1

Summary of the derivation of CVD/CHD risk scores primarily developed in individuals with diabetes

Name of risk score

Derivation population (setting)

Definition of diabetes

Follow-up time

Main outcome

Risk factors included in score

UKPDS risk engine (UKPDS 56) [47]

4,540 men and women aged 25–65 years with diabetes, without history of MI, angina or heart failure (UK)

Newly physician-diagnosed diabetes and FPG >6 mmol/l on two further occasions

Median F/U 10.3 years

Fatal or non-fatal MI, or sudden death (verified by two independent clinical assessors)

Age at diagnosis of DM, sex, ethnicity, smoking, HbA1c, systolic BP, total-cholesterol:HDL-cholesterol ratio, duration of DM

Oxford risk engine (UKPDS risk engine version 3) [48]

3,465 men and women aged 25–65 years with and without diabetes with no known CVD (UK)

Newly physician-diagnosed diabetes and FPG >6 mmol/l on two further occasions

Median F/U 5 years

CVD (first occurrence of fatal or non-fatal MI, sudden cardiac death, other ischaemic heart disease, fatal or non-fatal stroke, or fatal peripheral vascular disease)

Age at diagnosis of DM, sex, ethnicity, smoking, HbA1c, systolic BP, total cholesterol:HDL-cholesterol ratio, duration of DM

Diabetes Audit and Research in Tayside, Scotland (DARTS) [17]

4,569 men and women of any age with diabetes without previous cardiovascular events (Scotland)

Those at any age treated with diet or oral hypoglycaemic agents or those aged >35 years with diagnosis of diabetes (Web-based information system) (Sensitivity/specificity of the ascertainment 97%)

Mean (SD) F/U 4.7 (6.1) years

First major CHD event (fatal or non-fatal acute MI or CHD death)

Age at diagnosis of DM, sex, systolic BP, duration of DM, smoking, total cholesterol, HbA1c, treated hypertension, height

Swedish National Diabetes Register (NDR) [23]

5,823 men and women aged 18–70 years with diabetes and no previous CVD (Sweden)

Diagnosis of diabetes from the Swedish National Diabetes Register

Mean F/U 5.6 years

Fatal or non-fatal CVD (CHD or stroke, including coronary intervention) retrieved by data linkage

Age at diagnosis of DM, sex, smoking, duration of diabetes, BMI, systolic BP, HbA1c, antihypertensive drugs, lipid-lowering drugs

Atherosclerosis Risk in Communities (ARIC) [15]

1,500 men and women aged 45–64 years with diabetes, initially free of CHD (USA) (no validation)

FPG >7.0 mmol/l, non-fasting glucose >11.1 mmol/l, self-reported physician-diagnosed diabetes or pharmacological treatment for diabetes

Median F/U 10.2 years

CHD (validated definite or probable hospitalised MI, definite CHD death, an unrecognised MI defined by ECG readings, or coronary revascularisation)

Age, ethnicity, smoking, systolic BP, use of antihypertensive medication, total cholesterol, HDL; alternative model includes: BMI, WHR, sport activity, Keys score for diet, serum creatinine, serum albumin, WBC, factor VIII, LVH, and IMT

Hong Kong Diabetes Registry [18]

3,521 Chinese men and women with median age of 57 years without a history of CHD or heart failure (Hong Kong)

Type 2 diabetes referred from GP and special clinics, and patients discharged from hospitals

Median F/U 5.59 years

Total CHD (MI or IHD) retrieved from discharge diagnosis

Age, sex, smoking, duration of diabetes, systolic BP, GFR, ACR, non-HDL-cholesterol

UKPDS for stroke (UKPDS 60) [49]

4,549 men and women aged 25–65 years with diabetes, no known stroke (UK)

Newly physician-diagnosed diabetes, and FPG >6 mmol/l on two further occasions

Median F/U 10.5 years

Stroke defined as neurological deficit with symptoms or signs lasting 1 month or more

Age at diagnosis of DM, sex, smoking, atrial fibrillation, systolic BP, total:HDL-cholesterol ratio, duration of DM

Hong Kong Diabetes Registry for stroke [26]

3,668 diabetic patients without history of stroke, referred to the Prince of Wales Hospital (Hong Kong)

Type 2 diabetes referred from GP and special clinics, and patients discharged from hospitals

Median F/U 5.4 years

Fatal and non-fatal haemorrhagic and ischaemic stroke (data retrieved via the Hong Kong hospital authority central computer system)

Age, HbA1c, ACR, history of CHD

ACR, albumin:creatinine ratio; DM, type 2 diabetes mellitus; F/U, follow-up; IHD, ischaemic heart disease; IMT, intima-medial thickness; LVH, left ventricular hypertrophy; MI, myocardial infarction; WBC, white blood cell count

Table 2

Summary of the derivation of CVD/CHD risk scores primarily developed in the general population

Name of risk score

Study population (setting)

Definition of diabetes

Follow-up time

Main outcome

Risk factors included in score

Framingham 1991 [50]

5,573 US white men and women aged 30–74 years with no overt CHD (6% DM)

Treatment with insulin or oral hypoglycaemic drugs, blood glucose >8.3 mmol/l at two clinic visits, or fasting blood glucose >7.8 mmol/l at initial examination of the Offspring Study [51]

12 years

Myocardial infarction, CHD, death from CHD, stroke, CVD, death from CVD

Age, sex, blood pressure, total cholesterol, HDL-cholesterol, smoking, glucose intolerance, left ventricular hypertrophy by echocardiogram

Framingham 1998 [52]

5,345 US white men and women aged 30–74 years with no overt CHD (6% DM)

Treatment with insulin or oral hypoglycaemic drugs, blood glucose >8.3 mmol/l at two clinic visits, or fasting blood glucose >7.8 mmol/l at initial examination of the Offspring Study [51]

12 years

CHD (angina pectoris, recognised and unrecognised MI, coronary insufficiency, CHD death) Hard CHD included total CHD without angina pectoris

Age, sex, diabetes, smoking, blood pressure, total or LDL-cholesterol, HDL-cholesterol

PROCAM score [53]

5,389 German white working men aged 35–65 years with no history of MI, stroke, angina pectoris or baseline ECG signs of IHD (Germany) (16.1% DM)

N/A

10 years

Major coronary event (sudden cardiac death, definite fatal or non-fatal MI on basis of ECG and/or cardiac enzyme changes)

Age, smoking, diabetes, family history of premature MI, systolic blood pressure, HDL-cholesterol, LDL-cholesterol, triacylglycerol

Joint British Societies Risk Chart (JBSRC) [54]

Computerised CHD risk prediction chart based on Framingham risk equations

Not applicable

Not applicable

Coronary heart disease

Age, sex, systolic and diastolic BP, total and HDL-cholesterol, smoking, diabetes, presence of left ventricular hypertrophy from ECG

CardioRisk Manager calculator (CRM), computerised version of Framingham equation [55]

Computerised version of full equations of Framingham risk score to test against a placebo arm of West of Scotland Coronary Prevention Study (WOSCOPS) [56]

Not applicable

Not applicable

CHD or stroke

CHD: age, sex, blood pressure, total cholesterol, HDL-cholesterol, smoking, glucose intolerance, left ventricular hypertrophy by echocardiogram; stroke: age, systolic blood pressure, diabetes, smoking, prior CVD, atrial fibrillation, left ventricular hypertrophy by electrocardiogram, use of antihypertensive medication prior to occurrence of stroke

Myocardial Infarction Population Registry of Girona (REGICOR) [57]

Men and women aged 35–74 in Girona, Spain (data from myocardial infarction population registry of Girona, REGICOR)

N/A

10 years

Fatal or non-fatal AMI, with or without symptoms, or angina

Age, sex, total cholesterol, HDL-cholesterol, blood pressure, smoking, diabetes

SCORE [16]

205,178 European men and women aged 19–80 years with no history of MI (12 European countries)

Not applicable

Varies among cohorts, up to 25 years

CVD mortality (ICD-9 codes 401–414 and 426–443, with the exception of non-atherosclerotic causes of death: 426.7, 429.0, 430.0, 432.1, 437.3, 437.4 and 437.5; ICD-9 codes 798.1 and 798.2 were also classified as CVD deaths)

Age, sex, systolic blood pressure, smoking, total cholesterol or total cholesterol:HDL-cholesterol ratio

DECODE score [58]

25,413 European men and women aged 30–74 years (14 European studies) (4% known DM, 5% screen-detected DM )

Known diabetes and screen-detected diabetes using OGTT

4.8 to more than 10 years

CVD mortality (CVD codes 401–448 of the 8th and 9th ICD revisions, code I10–I79 of the 10th revision)

Age, sex, FPG and 2-h plasma glucose, diabetes, smoking, systolic blood pressure, total cholesterol

Framingham Stroke [59]

5,734 men and women aged 55–84 years with no history of stroke (USA)

Treatment with insulin or oral hypoglycaemic drugs, and fasting blood glucose

10 years

Brain infarction and transient ischaemic attack, cerebral embolus, intracranial haemorrhage

Age, systolic blood pressure, diabetes, smoking, prior CVD, atrial fibrillation, left ventricular hypertrophy by electrocardiogram, use of antihypertensive medication prior to occurrence of stroke

AMI, acute myocardial infarction; DM, type 2 diabetes mellitus; ICD, international classification of disease; IHD, ischaemic heart disease; MI, myocardial infarction; N/A, not available

Results

Our electronic search retrieved 2,113 potentially relevant papers (Fig. 1). After reviewing titles, abstracts, full texts and citation lists, 13 articles reporting the predictive performance of 17 different CVD risk scores met the inclusion criteria. We provide a summary of the derivation of the risk scores in Tables 1 and 2, and a summary describing the evaluation of the scores in Table 3. One paper was translated.
Table 3

Performance of CVD risk scores evaluated in individuals with diabetes

Name of risk score

Validation population

Definition of diabetes

Outcome (n events)

Discrimination

Calibration

Sensitivity and specificity (%)

PPV/NPV (%)

UKPDS risk engine [47]

428 men and women aged 30–74 years with newly diagnosed diabetes without pre-existing CVD, mean F/U 4.2 (0.6) years (GP recruitment in Poole, UK) [21]

Surveillance programme to identify diabetes cases; methods of diagnosis, N/A

CHD determined via hospital and GP notes, death certificates and post-mortem examination record (98)

CHD: aROC 0.670 (0.598–0.742)

Poor calibration for CHD: HL χ 2 17.1 (p = 0.029)

10 year CHD risk >15%: Sen 89.8 (95% CI 82.0–95.0) Spc 30.3 (95% CI 25.4–35.6)

N/A

339 diabetic patients with 10 years of follow-up (Greece) [25]

N/A

CHD determined by coronary angiography (108)

aROC 0.61

N/A

Sen 56; Spc 56 (no data on which risk threshold was used)

PPV 37; NPV 73 (no data on which risk threshold was used)

112 patients with type 2 diabetes with 10 years of follow-up (Spain) [22]

1985 WHO criteria (FPG >7.7 mmol/l or, 2 h OGTT >11.0 mmol/l)

CHD event (19)

N/A

Men: overestimation by 108.8%; women: overestimation by 51.3%

10 year CHD risk >20%: men: Sen 67; Spc 7; women: Sen 71; Spc 59

10 year CHD risk >20%: men: PPV 14; NPV 50; women: PPV 24; NPV 92

 

798 diabetic patients identified via a computerised clinic database, follow-up 10 years [19]

N/A

CHD and CVD determined through computerised clinical records (358 CVD, including 269 CHD events)

CVD: c = 0.74; CHD: c = 0.65

Poor calibration (p < 0.001); observed:predicted events ratio 1.20 and 1.60 for CVD and CHD (r = 0.89 and 0.86 respectively)

N/A

N/A

 

3,546 Chinese men and women with median age of 56 years without a history of CHD or HF, median F/U of 5.59 years (Hong Kong) [18]

Type 2 diabetes referred from GP and special clinics, and those discharged from hospitals

Hard CHD: myocardial infarction and CHD death (170)

Overall c statistic 0.610 (95% CI 0.581–0.639)

Poor calibration: HL χ 2 = 230.1; p < 0.0001

N/A

N/A

Oxford risk engine (UKPDS risk engine version 3) [48]

1,410 men and women aged 40–75 years on the placebo arm of the Collaborative Atorvastatin Diabetes Study (CARDS) (median F/U 3.9 years) [48]

1985 WHO criteria (computerised registers of patients and by opportunistic assessment of individuals attending diabetes clinics)

CVD event (189)

N/A

Underestimation by 10.6%; 189 observed events vs 169 predicted events over 3.9 years

N/A

N/A

Diabetes Audit and Research in Tayside, Scotland (DARTS) [17]

Salford Diabetes Information System, F/U 5 years [17]

Treatment with diet or oral hypoglycaemic agents/age >35 years

CHD determined via hospital episode statistics (N/A)

c statistic = 0.69 (95% CI 0.58–0.78)

Included a graph but no statistical test

N/A

N/A

Swedish National Diabetes Register (NDR) [23]

5,823 men and women aged 18–70 years with diabetes, no previous CVD, F/U 5.6 years (Swedish) [23]

Diagnosis of diabetes from the Swedish National Diabetes Register

Fatal and non-fatal CVD (N/A)

c statistic = 0.69

Good calibration; observed:predicted CVD rate ratio 0.998

N/A

N/A

 

3,068 men and women aged 18–70 years with diabetes, no previous CVD, F/U 4 years (Swedish) [23]

Diagnosis of diabetes from the Swedish National Diabetes Register

Fatal and non-fatal CVD (261)

c statistic = 0.69

Good calibration; observed:predicted CVD rate ratio 0.96

N/A

N/A

Hong Kong Diabetes Registry for CHD [18]

3,546 Chinese men and women with median age of 56 years without CHD or HF, median follow-up of 5.59 years (Hong Kong) [18]

Type 2 diabetes referred from GP and special clinics, and those discharged from hospitals

CHD: myocardial infarction or ischaemic heart disease (170)

Overall aROC 0.704 (95% CI 0.675–0.733) Adjusted aROC 0.737

Good calibration: HL χ 2 = 14.05; p > 0.05

5 year CHD risk >5.2%: Sen 67.6; Spc 68.5

5 year CHD risk >5.2%: PPV 8.6; NPV 98.0

Framingham 1991 [50]

3,898 men and women aged 25–65 years with diabetes without history of MI, angina or stroke (UKPDS) [24]

Newly physician-diagnosed diabetes, and FPG >6 mmol/l on two further occasions

Fatal CVD and fatal CHD (CVD: 7.4%, CHD: 6.3%)

aROC 0.76

32% underestimation

N/A

N/A

112 patients with type 2 diabetes with 10 years of follow-up (Spain) [22]

1985 WHO criteria (FPG >7.7 mmol/l or 2 h OGTT >11.0 mmol/l)

CHD event (19)

N/A

Men: overestimation by 64.8%; women: overestimation by 31.6%

10 year CHD risk >20%: men: Sen 75; Spc 17; women: Sen 71; Spc 67

10 year CHD risk >20%: men: PPV 17; NPV 75; women: PPV 28; NPV 93

428 men and women aged 30–74 years with newly diagnosed diabetes without pre-existing CVD, mean F/U of 4.2 (0.6) years (GP recruitment in Poole, UK) [21]

Surveillance programme to identify diabetes cases, methods of diagnosis, N/A

CVD and CHD determined via hospital and GP notes, death certificates and post-mortem examination record (98)

CVD: aROC 0.673 (0.612–0.734); CHD: aROC 0.657 (0.581–0.732)

Poor calibration for CVD and CHD CVD HL χ 2 = 32.8 (p < 0.001); CHD HL χ 2 = 19.8 (p = 0.011)

10 year CHD risk >15%: Sen 85.7 (95% CI 77.8–99.5); Spc 33.0 (95% CI 30.7–34.7)

N/A

938 men and women with type 1 and type 2 diabetes, F/U 4 years (Cardiff, UK) [20]

Type 1 and type 2 diabetes identified through the Cardiff Diabetes Database; methods of diagnosis, N/A

CHD via primary or subsidiary diagnosis in hospital record linkage (172)

Men: aROC 0.64; women: aROC 0.66

N/A

10 year CHD risk >30%: men: Sen 34%, women: Sen25%; 10 year CHD risk >20%: men: Sen 75%; women: Sen 58%

10 year CHD risk >30%: men: PPV 26%, NPV 83%; women: PPV 33%, NPV 86%; 10 year CHD risk >20%: men: PPV 24%, NPV 88%; women: PPV 24%, NPV 88%

Framingham 1998 [52]

112 patients with type 2 diabetes with 10 years of follow-up (Spain) [22]

1985 WHO criteria (FPG >7.7 mmol/l or 2 h OGTT >11.0 mmol/l)

CHD event (19)

N/A

Men: overestimation by 97.8%; women: overestimation by 38.2%

10 year CHD risk >20%: men: Sen 75; Spc 11; women: Sen 71; Spc 69

10 year CHD risk >20%: men: PPV 16; NPV 67; women: PPV 29; NPV 93

339 diabetic patients, follow-up of 10 years (Greece) [25]

N/A

CHD determined using coronary angiography (108)

aROC 0.65

N/A

Sen 55; Spc 65 (no data on which risk threshold was used)

PPV 43; NPV 75 (no data on which risk threshold was used)

PROCAM score [53]

798 diabetic patients identified via computerised clinic database with 10 years of follow-up (UK) [19]

N/A

CHD and CVD determined through computerised clinical records (358 CVD, including 269 CHD events)

CVD: c = 0.67; CHD: c = 0.76

Poor calibration (p < 0.001); observed:predicted events ratio 2.79 and 2.05 for CVD and CHD (r = 0.79 and 0.81 respectively)

N/A

N/A

Joint British Societies Risk Chart (JBSRC) [54]

798 diabetic patients identified via computerised clinic database with 10 years of follow-up (UK) [19]

N/A

CHD and CVD determined through computerised clinical records (358 CVD, including 269 CHD events)

CVD: c = 0.80; CHD: c = 0.77

Poor calibration (p < 0.001)

N/A

At CVD risk >30%: PPV 85%; at CHD risk >30%: PPV 66%

CardioRisk Manager calculator (CRM) [55]

798 diabetic patients identified via a computerised clinic database with 10 years of follow-up (UK) [19]

N/A

CHD and CVD determined through computerised clinical records (358 CVD, including 269 CHD events)

CVD: c = 0.76; CHD: c = 0.73

Poor calibration (p < 0.001) observed:predicted events ratio 2.30 and 1.74 for CVD and CHD (r = 0.98 and 0.97 respectively)

N/A

N/A

Myocardial Infarction Population Registry of Girona (REGICOR) [57]

112 patients with type 2 diabetes with 10 years of follow-up (Spain) [22]

1985 WHO criteria (FPG >7.7 mmol/l or 2 h OGTT >11.0 mmol/l)

CHD event (19)

N/A

Men: underestimation by 17.6%; women: underestimation by 34.2%

10 year CHD risk >20%: men: Sen 25; Spc 67; women: Sen 0; Spc 100

10 year CHD risk >20%: men: PPV 14; NPV 80; women: PPV –; NPV 85

SCORE [16]

3,898 men and women aged 25–65 years with diabetes, without history of MI, angina or stroke within 1 year (UKPDS) [24]

Newly physician-diagnosed diabetes and FPG >6 mmol/l on two further occasions

Fatal CVD and fatal CHD (CVD: 7.4%, CHD: 6.3%)

aROC 0.77

18% overestimation

N/A

N/A

DECODE score [58]

3,898 men and women aged 25–65 years with diabetes, without history of MI, angina or stroke within 1 year (UKPDS) [24]

Newly physician-diagnosed diabetes and FPG >6 mmol/l on two further occasions

Fatal CVD (CVD: 7.4%, CHD: 6.3%)

aROC 0.67

11% underestimation

N/A

N/A

UKPDS for stroke [49]

1,370 patients aged >30 years of the Wisconsin Epidemiologic Study of Diabetic Retinopathy cohort, follow-up 8.3 years (USA) [49]

N/A

Fatal stroke (197)

N/A

Underestimation of fatal stroke by 12.5%

N/A

N/A

 

3,541 Chinese diabetic patients without history of stroke (Hong Kong) [26]

Physician-diagnosed diabetes (GP and special clinics, and those discharged from hospital)

Stroke: hospital discharge diagnosis confirmed by physicians (182)

Unadjusted aROC 0.588 (95% CI 0.549–0.626)

N/A

N/A

N/A

Hong Kong Diabetes Registry for stroke [26]

3,541 Chinese diabetic patients without history of stroke, median follow-up of 5.37 years (Hong Kong) [26]

Physician-diagnosed diabetes referred from GP and special clinics, and those discharged from hospital

Stroke: hospital discharge diagnosis confirmed by physicians (182)

Adjusted aROC 0.770 for haemorrhagic stroke; 0.785 for ischaemic stroke

N/A

5 year stroke risk >6.1%: Sen 65.7; Spc 74.9

5 year stroke risk >6.1%: PPV 11.6

Framingham stroke risk [59]

178 men and women with diabetes, with or without metabolic syndrome (Spain) [60]

FPG and OGTT based on WHO criteria

Stroke (9)

N/A

Predicted 10 year risk varied from 17.3 to 19.1%; observed 5 year cumulative incidence varied from 3.5 to 5.8

N/A

N/A

F/U, follow-up; HF, heart failure; HL χ 2, Hosmer–Lemeshow χ 2; MI, myocardial infarction; N/A, not available; NPV, negative predictive value; PPV, positive predictive value; Sen, sensitivity; Spc, specificity

Development of risk scores

Out of 17 different risk scores, 15 were developed in predominantly white populations (USA and Europe) and two were developed in Chinese populations (Hong Kong). Cohort size ranged from 1,500 [15] to 205,178 individuals [16] and follow-up time from 4.7 [17] to 25 years [16]. The age of men and women included in development cohorts ranged from 18 to 84 years; patients with previous CVD were usually excluded. Eight risk scores were originally developed in a cohort of individuals with diabetes (Table 1), while the other nine were developed in a general population and subsequently evaluated in a cohort of individuals with diabetes (Table 2). The majority of risk scores (n = 10) provided estimates of risk for CHD. Two risk scores estimated risk of non-fatal or fatal CVD outcomes, while two risk scores predicted CVD deaths. The remaining three scores estimated risk of non-fatal or fatal stroke. The majority of risk scores incorporated classic CVD risk factors, such as age, sex, smoking, blood pressure and total cholesterol. Most risk scores derived from the general population contained a dichotomous variable for diabetes (yes/no) (n = 8) and did not take account of diabetes-specific risk factors, such as duration of diabetes or glycaemia. Conversely, risk scores developed in individuals with diabetes often included age at diagnosis, duration of diabetes and/or a measure of glycaemic control, such as HbA1c or fasting plasma glucose (FPG).

Evaluation of risk scores

Sixteen risk scores were evaluated in 13 different validation cohorts. Only the Atherosclerosis Risk in Communities risk score was not evaluated in an external validation cohort. The majority of validation cohorts were based in Europe (n = 10) and varied in size from 112 to 5,823 individuals with diabetes (Table 3). The 10-year cumulative incidence of CVD varied considerably from 5% in a Chinese cohort [18] to 45% in a British cohort [19]. The age of individuals ranged from 18 to 75 years at baseline and median follow-up time from 4 to 10 years. All studies validated risk scores in patients with type 2 diabetes, including one study that evaluated a risk score in type 1 and type 2 diabetes [20]. Individuals with diabetes were recruited from the general population (n = 7) or specialist diabetes clinics (n = 6) and were identified through (1) computerised databases or registries in five cohorts; (2) clinical records in seven cohorts; and (3) from the placebo arm of a trial in one cohort. Only three studies included a clear definition of diabetes, e.g. whether individuals were diagnosed by FPG or an OGTT according to WHO criteria (Table 3). Two articles failed to include any information on how individuals with diabetes were identified or diagnosed. Similarly, four articles did not include a clear definition of which CVD endpoint was used, with definitions varying between studies, e.g. clinical diagnosis of fatal and non-fatal CVD, CHD determined using coronary angiography or fatal CVD. Clinical records were used to retrieve data on CVD endpoints (n = 11) in most validation cohorts and these were confirmed by expert reviewers in two studies. Diagnosis of CVD was made using coronary angiography in one validation study in Greece (Table 3). The majority of validation studies compared the performance of different CVD risk scores in a single population (n = 7). Among these, four studies compared the predictive ability of risk scores developed in individuals with diabetes and those developed in a general population.

Performance of the risk scores

Table 3 summarises the measures of predictive ability assessed in each validation study. Few studies reported complete measures of predictive performance, including discrimination, calibration and global model fit. The majority of studies reported a measure of discrimination (area under the receiver operating characteristic curve [aROC], also known as a c-statistic). Risk scores predicting CVD or CHD outcomes showed moderate to good discriminatory power in validation cohorts (aROC range 0.61 to 0.80). Similarly, risk scores predicting stroke reported aROCs from 0.59 to 0.79.

Three risk scores were evaluated using a Hosmer–Lemeshow χ 2 statistic as a measure of calibration in two validation cohorts, while eight studies reported either predicted and observed event rates, predicted to observed rate ratio or whether the scores over- or underestimated CVD risk. Two validation studies did not report any measure of calibration. The UK Prospective Diabetes Study (UKPDS) risk engine showed poor calibration in both British [19, 21] and non-British validation cohorts [18, 22], while the Swedish National Diabetes Register [23] and Hong Kong Diabetes Registry [18] reported good calibration in Swedish and Chinese populations respectively. Most risk scores developed in the general population underestimated CVD risk in diabetic patients. We found underestimations ranging from 11% for fatal CVD risk using the Diabetes Epidemiology: Collaborative Analysis of Diagnostic Criteria in Europe (DECODE) score [24] to 64% for fatal and non-fatal CVD risk using the Prospective Cardiovascular Münster (PROCAM) score [19]. Coleman et al. validated the Framingham, Systematic Coronary Risk Evaluation (SCORE) and DECODE risk engines in a population of 3,898 individuals with diabetes and showed that they did not provide reliable fatal CVD and CHD risk estimates [24]. Conversely, in populations with low CVD risk, such as Mediterranean countries, most of the risk scores overestimated CVD risk, even those developed in individuals without diabetes [22].

Studies comparing the predictive ability of a CVD risk score developed in a diabetic cohort and a score developed in the general population reported inconsistent results. In a study of 339 patients with diabetes, Protopsaltis et al. showed that the Framingham risk equations were more accurate than the UKPDS risk engine for predicting coronary artery disease risk (aROC 0.65 and 0.61 respectively) [25]. With similar sensitivity, Framingham risk equations had higher specificity and positive and negative predictive values than the UKPDS risk engine (65%, 43% and 75% vs 56%, 37% and 73%, respectively). However, the study population was small and the diagnosis of coronary artery disease was established by means of coronary angiography, which includes individuals with sub-clinical CHD, an endpoint not included in the UKPDS definition. By contrast, in a British community-based cohort of 428 individuals with newly diagnosed type 2 diabetes [21], the Framingham risk equations appeared to underestimate cardiovascular and coronary disease events by 33% and 32%. The UKPDS risk engine had lower levels of underestimation at 13%, suggesting that this diabetes-specific risk score performed better than a risk score developed in the general population. However, both the Framingham equations and UKPDS risk engine showed modest discriminatory ability and poor calibration (aROC of 0.66 and 0.67, Hosmer–Lemeshow χ 2 of 19.8 [p = 0.011] and 17.1 [p = 0.029] respectively), making it difficult to reach any firm conclusions.

Sensitivity, specificity and both positive and negative predictive values varied considerably between risk scores and validation cohorts, although most studies reported moderate to good sensitivity and specificity (Table 2). Risk scores for CHD and stroke developed in the Hong Kong Diabetes Register had good sensitivity and specificity when tested in Chinese validation cohorts [18, 26], while the Framingham risk score and the UKPDS risk engine also had good sensitivity, but relatively low specificity when tested in European populations [21, 22].

Discussion

This systematic review has shown that the predictive ability of CVD risk scores, which were developed mainly for White populations, varies considerably between different populations. There is little evidence to suggest that using risk scores developed in individuals with diabetes will help to estimate CVD risk among diabetic patients more accurately than use of those developed in the general population. The inconsistency in methods used to evaluate CVD risk scores makes it difficult to compare or summarise the predictive ability of different risk scores.

Our review supports previous research showing that CVD risk scores developed in the general population are likely to underestimate CVD risk in individuals with diabetes [9]. In theory, one way of addressing this underestimation would be to use risk prediction tools solely derived from diabetic populations [10]. However, our results suggest that diabetes-specific risk engines need to be validated in other populations before they are widely adopted and can replace Framingham-based methods of risk assessment. The issue of predicting risk for people with diabetes is likely to be increasingly complex, as a greater proportion of patients are treated with CVD risk factor modifying therapy. The development of risk prediction scores in populations with long-standing and treated diabetes, possibly incorporating information about the degree of glycaemic control, would be an important issue for future research. At the same time, the prediction of absolute future CVD risk in a clinical situation is likely to be based on single baseline measures of cardiovascular risk factors and not on any time-averaged measures collected during repeat visits. While the latter has a role in assessing the aetiological association between risk factors and outcome, repeated measures have limited utility in the practical clinical situation of risk prediction.

Most diabetes-specific risk scores incorporate measures of glycaemic control such as FPG and HbA1c, while most scores developed in the general population include a binary variable indicating whether an individual has diabetes or not. Hence, an important question in the general population is whether adding measures of glycaemic control improves the predictive ability of existing risk scores incorporating traditional CVD risk factors. In populations of people with diabetes, studies comparing the predictive performance of the Framingham risk score and the UKPDS risk engine have shown conflicting results [21, 25]. Part of the uncertainty about the additional predictive contribution of hyperglycaemia in this context is explained by the fact that, in comparison with age, cholesterol and smoking, hyperglycaemia is a relatively weak CVD risk factor, thus making it harder to show significantly improved prediction [27]. The differences between studies could also be explained by changes in the distribution of risk factors for CVD and their treatment over time, which could differ within and between populations. This has implications for the generalisability of a questionnaire, even within the population in which it was developed, since by necessity the development stage of a risk score has to take place in a different temporal period to the practical application of the score in a clinical setting. It is conceivable that temporal trends in risk factors and the way they are treated could have major impacts on the predictive accuracy of risk scores that were developed using historical data but are applied to predict what may happen in the future. Differences between populations in the underlying distributions of risk factors and their treatment would limit the potential generalisability of risk scores to populations other than that in which they were derived.

It is also important to recognise that the choice of validation population will have an influence on the performance of a risk score in estimating CVD risk. Studies that develop risk scores in one half of a cohort and then validate them in the other half are likely to report better predictive abilities. This is true of scores developed from the Hong Kong Diabetes Registry [18, 26] and the Swedish National Diabetes Register [23]. Conversely, validating risk scores in different populations and ethnic groups is likely to result in relatively poorer prediction. For example, the UKPDS risk engine had moderate discrimination and poor calibration when evaluated in a Chinese diabetic population [18]. This underlines the fact that the accuracy of a risk score largely relies on the background risk of a specific population to which it is applied. It may be more useful to develop or recalibrate population-specific risk prediction tools, rather than trying to find a universal risk score that will work in all populations.

This review used a comprehensive search without language restrictions. To the best of our knowledge, it is the first systematic review to assess the ability of risk scores to estimate cardiovascular risk in individuals with diabetes. We extracted data on widely used measures of predictive accuracy, including discrimination, calibration, sensitivity, specificity and positive and negative predictive values. Apart from a difference in the choice of validation population, each study had different inclusion criteria, follow-up, ascertainment methods, definition of diabetes and CVD endpoints. This makes it difficult to compare the predictive ability between different risk scores and between validation populations. The performance of CVD risk scores varied considerably, and there is no conclusive evidence of a difference in the predictive performance of risk scores developed in individuals with diabetes and of those developed in the general population. A variety of statistical approaches was used to describe and compare the predictive performance of the different risk scores; our review suggests that a more systematic and standardised approach is needed.

Methodological issues

The estimation of CVD risk is a dynamic research field and there are a number of unresolved methodological issues concerning the development, validation and comparison of risk scores. It is clear from the systematic review that a number of statistical techniques can be employed to assess the performance of predictive models. However, these measures are not uniformly calculated across all studies and it is sometimes difficult to obtain a comprehensive picture of the use of a risk score without taking all these measures into account. Risk scores are frequently derived using a logistic regression model, which models the log odds of having the event during the specified follow-up period as a function of the other variables. The estimated coefficients from the logistic regression model are then used to define the risk score. Although this risk estimate is continuous, many of the methods of comparing prediction assume that a threshold will be used to categorise people into two groups on the basis of the risk score.

Measures of discrimination

Discrimination is the ability of the prediction model to correctly separate individuals into those who will and those who will not have the event of interest. The commonly used receiver operating characteristic curve (ROC) is a plot of sensitivity on the vertical axis against (1−specificity) on the horizontal axis for every possible cut-off value of the continuous risk score. The aROC is the area under the ROC curve and is equal to the probability that a randomly selected individual with the event has a higher value of the risk score than a randomly selected individual without the event. A test that was totally uninformative would have a c value of 0.5, while a perfect test would have a c value of 1. The discrimination of two possible models can be compared by comparing the values of their aROCs. Net reclassification improvement (NRI) [28] is used to compare two models A and B, which share all risk factors except for one new marker included in model B. After classification of the predicted probabilities from the two models into risk categories, the NRI is the proportion of times model B correctly moves an individual with the event into a higher risk category or an individual without the event into a lower risk category, minus the proportion of times the new model incorrectly moves an individual with the event into a lower risk category or an individual without the event into a higher risk category. Integrated discrimination improvement (IDI) [28] is calculated by subtracting, for model A and B, the average probability of the event in the individuals without the event from the average probability of the event in the individuals with the event, and then calculating the difference in these two quantities. The values of NRI and IDI are both between 0 and 1, or 0 and 100%. The higher the values, the greater the improvement in performance of the new model. It is possible to test whether each statistic is significantly different from 0.

Calibration of risk scores

Calibration is the extent to which predicted risk from a model equals observed risk in the data. The Hosmer–Lemeshow Goodness of fit test is calculated by ordering the predicted probabilities of the event into, say, ten near equal sized groups. The observed and expected numbers of individuals with the event within each group are then compared using a χ 2 statistic (with nine degrees of freedom if there are ten groups). A statistically significant result implies that the model may be poorly calibrated.

Global measures of model fit

The likelihood ratio statistic is a global measure of model fit. The Bayesian information criterion (BIC) and Akaike’s information criterion (AIC) are two measures that combine both fit (as measured by the likelihood ratio statistic) and model complexity, in terms of number of variables (BIC and AIC) and also (for BIC) number of observations. These measures can be used in the same dataset for model selection; comparing two models, the model with the lower value of either BIC or AIC would be preferred. However, as Ware [29] has demonstrated, a new marker could be an important risk factor that significantly improves the fit of the model, but it may have almost no impact on model discrimination as measured by aROC.

CVD risk scores are often compared using the aROC, with a significant increase in the aROC being taken as evidence that the discrimination of a risk score has improved. More recently, Cook and others [30, 31] have argued that the aROC is an insensitive measure based purely on ranks and that we should also calculate a measure of calibration (e.g. the Hosmer–Lemeshow statistic) and global fit (e.g. BIC) in order to comprehensively assess the utility of a risk score. They argue that the critical issue for clinical application is ‘the proportion of patients reclassified using a new risk algorithm and whether the magnitude of this reclassification is large enough to alter physician behaviour with regard to prevention’ [31, 32]. Pepe et al. [33] recently suggested calculating the NRI and IDI measures when comparing two scores, in order to assess whether any reclassification was in the right or wrong direction.

An example from our previous research illustrates some of these analytical approaches [34]. Using data from EPIC-Norfolk, a UK population-based prospective cohort [35], we computed two novel risk scores by fitting Cox proportional hazards regression models with CHD as the outcome. In model A, we used the original Framingham risk score variables, i.e. age, total cholesterol, HDL-cholesterol, systolic blood pressure, smoking status and diabetes. In model B, we replaced the diabetes variable with HbA1c to examine whether the addition of this continuous variable improved the prediction of CHD. While the results on discrimination indicated that model B was better than model A at distinguishing between individuals with and without the disease, results from the NRI statistic suggested that this difference did not significantly improve reclassification. This conflicting information highlights the challenges of comparing CVD risk scores and demonstrates that despite statistical advances, a number of questions remain unanswered.

Most of the risk scores in this systematic review were developed using models for censored survival time data, such as the Cox proportional hazards model or the accelerated failure time model. However, the most common method for deriving the aROC is based on a logistic regression model, which ignores the time-dependent nature of the data. New approaches to assessing the performance of a risk score estimated from survival data in the presence of censoring have been proposed [36, 37]. These new performance measures have been shown to be unbiased, unlike the aROC from logistic regression, which tends to be underestimated and have a larger standard deviation [37].

Conclusions

The computation of CVD risk is likely to remain an important part of the process of prioritising therapy for individuals and populations. The degree to which such risk scores can be improved is questionable, as attempts to add novel risk factors to existing CVD scores and thereby improve their predictive ability have not been very successful [38, 39], largely because a risk factor must be very strongly associated with a disorder for it to be useful for prediction [40]. Few novel biomarkers have demonstrated an ability to predict risk over and above information available from global assessment tools such as Framingham [38, 41]. Furthermore, there is rarely evidence that reductions in any of these novel markers will lower cardiovascular risk [42]. Genetic risk information currently adds little to prediction, but may become increasingly important in the future. However, there is little evidence that the provision of genetic risk information is associated with behaviour change [43].

Thus while there is clearly some scope for improving prediction, it is likely that improvements will be marginal. Indeed, it may be more profitable to focus on ensuring that tools currently available for risk prediction are applied more broadly and routinely throughout clinical practice in order to address the gap between the promise of CVD prevention and its reality [39]. When attempting to reduce CVD risk, the precision of the instrument may be less important than how it is used. As such, there is still a need for further research into provider and patient perceptions of CVD risk. There is considerable uncertainty on how best to present risk information and on whether the presentation of risk information is associated with lifestyle change or the degree of medication adherence [44, 45, 46]. The downsides of presenting risk information also need to be considered. For example, it is possible that showing how little CVD risk is reduced by lowering blood glucose alone may discourage people from adhering to hypoglycaemic medication.

Cardiovascular risk scores are useful tools in the management of individuals with diabetes, particularly when the score has been developed in a population of similar individuals. Scores that rank risk well are appropriate for identifying those at highest risk, to whom therapy can then be targeted. Conversely, the process of predicting risk accurately in order to provide prognostic information is better aided by risk scores that accurately quantify absolute risk. Finally, we see a potentially important role for scores capable of quantifying that element of risk that is modifiable, a strategy that could help motivate patients to change. While improvement in the predictive ability of risk scores might still be obtained, the public health utility of their application depends on a far wider range of issues.

Notes

Acknowledgements

There was no funding for this study. The authors would like to thank I. Kuhn (Cambridge University Medical Library) for her assistance with the literature search.

Duality of interest

The authors declare that there is no duality of interest associated with this manuscript.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

References

  1. 1.
    Haffner SM, Lehto S, Ronnemaa T, Pyorala K, Laakso M (1998) Mortality from coronary heart disease in subjects with type 2 diabetes and in nondiabetic subjects with and without prior myocardial infarction. N Engl J Med 339:229–234PubMedCrossRefGoogle Scholar
  2. 2.
    Juutilainen A, Lehto S, Ronnemaa T, Pyorala K, Laakso M (2005) Type 2 diabetes as a “coronary heart disease equivalent”: an 18-year prospective population-based study in Finnish subjects. Diabetes Care 28:2901–2907PubMedCrossRefGoogle Scholar
  3. 3.
    Gaede P, Vedel P, Larsen N, Jensen GV, Parving HH, Pedersen O (2003) Multifactorial intervention and cardiovascular disease in patients with type 2 diabetes. N Engl J Med 348:383–393PubMedCrossRefGoogle Scholar
  4. 4.
    Gaede P, Lund-Andersen H, Parving HH, Pedersen O (2008) Effect of a multifactorial intervention on mortality in type 2 diabetes. N Engl J Med 358:580–591PubMedCrossRefGoogle Scholar
  5. 5.
    Stevens RJ, Coleman RL, Holman RR (2005) Framingham risk equations underestimate coronary heart disease risk in diabetes. Diabet Med 22:228PubMedCrossRefGoogle Scholar
  6. 6.
    Yeo WW, Yeo KR (2001) Predicting CHD risk in patients with diabetes mellitus. Diabet Med 18:341–344PubMedCrossRefGoogle Scholar
  7. 7.
    Eichler K, Puhan MA, Steurer J, Bachmann LM (2007) Prediction of first coronary events with the Framingham score: a systematic review. Am Heart J 153: 722–731, 731.e1–e8Google Scholar
  8. 8.
    Brindle P, Emberson J, Lampe F et al (2003) Predictive accuracy of the Framingham coronary risk score in British men: prospective cohort study. BMJ 327:1267PubMedCrossRefGoogle Scholar
  9. 9.
    Brindle P, Beswick A, Fahey T, Ebrahim S (2006) Accuracy and impact of risk assessment in the primary prevention of cardiovascular disease: a systematic review. Heart 92:1752–1759PubMedCrossRefGoogle Scholar
  10. 10.
    Winocour PH, Fisher M (2003) Prediction of cardiovascular risk in people with diabetes. Diabet Med 20:515–527PubMedCrossRefGoogle Scholar
  11. 11.
    Ul Haq I, Ramsay LE, Pickin DM, Yeo WW, Jackson PR, Payne JN (1996) Lipid-lowering for prevention of coronary heart disease: what policy now? Clin Sci (Lond) 91:399–413Google Scholar
  12. 12.
    Collins R, Armitage J, Parish S, Sleigh P, Peto R (2003) MRC/BHF Heart Protection Study of cholesterol-lowering with simvastatin in 5963 people with diabetes: a randomised placebo-controlled trial. Lancet 361:2005–2016PubMedCrossRefGoogle Scholar
  13. 13.
    Colhoun HM, Betteridge DJ, Durrington PN et al (2004) Primary prevention of cardiovascular disease with atorvastatin in type 2 diabetes in the Collaborative Atorvastatin Diabetes Study (CARDS): multicentre randomised placebo-controlled trial. Lancet 364:685–696PubMedCrossRefGoogle Scholar
  14. 14.
    Carroll C, Naylor E, Marsden P, Dornan T (2003) How do people with type 2 diabetes perceive and respond to cardiovascular risk? Diabet Med 20:355–360PubMedCrossRefGoogle Scholar
  15. 15.
    Folsom AR, Chambless LE, Duncan BB, Gilbert AC, Pankow JS (2003) Prediction of coronary heart disease in middle-aged adults with diabetes. Diabetes Care 26:2777–2784PubMedCrossRefGoogle Scholar
  16. 16.
    Conroy RM, Pyorala K, Fitzgerald AP et al (2003) Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 24:987–1003PubMedCrossRefGoogle Scholar
  17. 17.
    Donnan PT, Donnelly L, New JP, Morris AD (2006) Derivation and validation of a prediction score for major coronary heart disease events in a U.K. type 2 diabetic population. Diabetes Care 29:1231–1236PubMedCrossRefGoogle Scholar
  18. 18.
    Yang X, So WY, Kong AP et al (2008) Development and validation of a total coronary heart disease risk score in type 2 diabetes mellitus. Am J Cardiol 101:596–601PubMedCrossRefGoogle Scholar
  19. 19.
    Stephens JW, Ambler G, Vallance P, Betteridge DJ, Humphries SE, Hurel SJ (2004) Cardiovascular risk and diabetes. Are the methods of risk prediction satisfactory? Eur J Cardiovasc Prev Rehabil 11:521–528PubMedCrossRefGoogle Scholar
  20. 20.
    McEwan P, Williams JE, Griffiths JD et al (2004) Evaluating the performance of the Framingham risk equations in a population with diabetes. Diabet Med 21:318–323PubMedCrossRefGoogle Scholar
  21. 21.
    Guzder RN, Gatling W, Mullee MA, Mehta RL, Byrne CD (2005) Prognostic value of the Framingham cardiovascular risk equation and the UKPDS risk engine for coronary heart disease in newly diagnosed type 2 diabetes: results from a United Kingdom study. Diabet Med 22:554–562PubMedCrossRefGoogle Scholar
  22. 22.
    Jimeno Mollet J, Molist Brunet N, Franch Nadal J, Serrano Borraz V, Serrano Barragan L, Gracia Gimenez R (2005) Variability in the calculation of coronary risk in type-2 diabetes mellitus. Aten Primaria 35:30–36 (article in Spanish)PubMedCrossRefGoogle Scholar
  23. 23.
    Cederholm J, Eeg-Olofsson K, Eliasson B, Zethelius B, Nilsson PM, Gudbjornsdottir S (2008) Risk prediction of cardiovascular disease in type 2 diabetes: a risk equation from the Swedish National Diabetes Register. Diabetes Care 31:2038–2043PubMedCrossRefGoogle Scholar
  24. 24.
    Coleman RL, Stevens RJ, Retnakaran R, Holman RR (2007) Framingham, SCORE, and DECODE risk equations do not provide reliable cardiovascular risk estimates in type 2 diabetes. Diabetes Care 30:1292–1293PubMedCrossRefGoogle Scholar
  25. 25.
    Protopsaltis ID, Konstantinopoulos PA, Kamaratos AV, Melidonis AI (2004) Comparative study of prognostic value for coronary disease risk between the U.K. prospective diabetes study and Framingham models. Diabetes Care 27:277–278PubMedCrossRefGoogle Scholar
  26. 26.
    Yang X, So WY, Kong AP et al (2007) Development and validation of stroke risk equation for Hong Kong Chinese patients with type 2 diabetes: the Hong Kong Diabetes Registry. Diabetes Care 30:65–70PubMedCrossRefGoogle Scholar
  27. 27.
    Raccah D (2006) Importance of blood glucose management in the multifactorial approach of absolute cardiovascular risk in type 2 diabetes: the lessons from the Steno 2 study. Diabetes Metab 32(Spec No 2):2S48–2S51PubMedGoogle Scholar
  28. 28.
    Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS (2008) Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 27:157–172PubMedCrossRefGoogle Scholar
  29. 29.
    Ware JH (2006) The limitations of risk factors as prognostic tools. N Engl J Med 355:2615–2617PubMedCrossRefGoogle Scholar
  30. 30.
    Cook NR (2007) Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 115:928–935PubMedCrossRefGoogle Scholar
  31. 31.
    Ridker PM, Buring JE, Rifai N, Cook NR (2007) Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 297:611–619PubMedCrossRefGoogle Scholar
  32. 32.
    Cook NR, Buring JE, Ridker PM (2006) The effect of including C-reactive protein in cardiovascular risk prediction models for women. Ann Intern Med 145:21–29PubMedGoogle Scholar
  33. 33.
    Pepe MS, Feng Z, Gu JW (2008) Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine. Stat Med 27:173–181 doi: 10.1002/sim.2929 PubMedCrossRefGoogle Scholar
  34. 34.
    Simmons RK, Sharp S, Boekholdt SM et al (2008) Evaluation of the Framingham risk score in the European Prospective Investigation of Cancer-Norfolk cohort: does adding glycated hemoglobin improve the prediction of coronary heart disease events? Arch Intern Med 168:1209–1216PubMedCrossRefGoogle Scholar
  35. 35.
    Day N, Oakes S, Luben R et al (1999) EPIC-Norfolk: study design and characteristics of the cohort. European Prospective Investigation of Cancer. Br J Cancer 80(Suppl 1):95–103PubMedGoogle Scholar
  36. 36.
    Pencina MJ, D'Agostino RB (2004) Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med 23:2109–2123PubMedCrossRefGoogle Scholar
  37. 37.
    Chambless LE, Diao G (2006) Estimation of time-dependent area under the ROC curve for long-term risk prediction. Stat Med 25:3474–3486PubMedCrossRefGoogle Scholar
  38. 38.
    Folsom AR, Chambless LE, Ballantyne CM et al (2006) An assessment of incremental coronary risk prediction using C-reactive protein and other novel risk markers: the atherosclerosis risk in communities study. Arch Intern Med 166:1368–1373PubMedCrossRefGoogle Scholar
  39. 39.
    Lloyd-Jones DM, Tian L (2006) Predicting cardiovascular risk: so what do we do now? Arch Intern Med 166:1342–1344PubMedCrossRefGoogle Scholar
  40. 40.
    Wald NJ, Hackshaw AK, Frost CD (1999) When can a risk factor be used as a worthwhile screening test? BMJ 319:1562–1565PubMedGoogle Scholar
  41. 41.
    Wang TJ, Gona P, Larson MG et al (2006) Multiple biomarkers for the prediction of first major cardiovascular events and death. N Engl J Med 355:2631–2639PubMedCrossRefGoogle Scholar
  42. 42.
    Ridker PM, Brown NJ, Vaughan DE, Harrison DG, Mehta JL (2004) Established and emerging plasma biomarkers in the prediction of first atherothrombotic events. Circulation 109:IV6–IV19PubMedGoogle Scholar
  43. 43.
    Marteau TM, Weinman J (2006) Self-regulation and the behavioural response to DNA risk information: a theoretical analysis and framework for future research. Soc Sci Med 62:1360–1368PubMedCrossRefGoogle Scholar
  44. 44.
    Hardeman W, Sutton S, Griffin S et al (2005) A causal modelling approach to the development of theory-based behaviour change programmes for trial evaluation. Health Educ Res 20:676–687PubMedCrossRefGoogle Scholar
  45. 45.
    Kinmonth AL, Wareham NJ, Hardeman W et al (2008) Efficacy of a theory-based behavioural intervention to increase physical activity in an at-risk group in primary care (ProActive UK): a randomised trial. Lancet 371:41–48PubMedCrossRefGoogle Scholar
  46. 46.
    Price HC, Tucker L, Griffin SJ, Holman RR (2008) The impact of individualised cardiovascular disease (CVD) risk estimates and lifestyle advice on physical activity in individuals at high risk of CVD: a pilot 2 × 2 factorial understanding risk trial. Cardiovasc Diabetol 7:21PubMedCrossRefGoogle Scholar
  47. 47.
    Stevens RJ, Kothari V, Adler AI, Stratton IM (2001) The UKPDS risk engine: a model for the risk of coronary heart disease in type II diabetes (UKPDS 56). Clin Sci (Lond) 101:671–679CrossRefGoogle Scholar
  48. 48.
    Simmons RK, Coleman RL, Price HC et al (2009) Performance of the UK prospective diabetes study risk engine and the Framingham risk equations in estimating cardiovascular disease in the EPIC-Norfolk cohort. Diabetes Care 32:708–713PubMedCrossRefGoogle Scholar
  49. 49.
    Kothari V, Stevens RJ, Adler AI et al (2002) UKPDS 60: risk of stroke in type 2 diabetes estimated by the UK Prospective Diabetes Study risk engine. Stroke 33:1776–1781PubMedCrossRefGoogle Scholar
  50. 50.
    Anderson KM, Odell PM, Wilson PW, Kannel WB (1991) Cardiovascular disease risk profiles. Am Heart J 121:293–298PubMedCrossRefGoogle Scholar
  51. 51.
    Kannel WB, Feinleib M, McNamara PM, Garrison RJ, Castelli WP (1979) An investigation of coronary heart disease in families: the Framingham Offspring Study. Am J Epidemiol 110:281–290PubMedGoogle Scholar
  52. 52.
    Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB (1998) Prediction of coronary heart disease using risk factor categories. Circulation 97:1837–1847PubMedGoogle Scholar
  53. 53.
    Assmann G, Cullen P, Schulte H (2002) Simple scoring scheme for calculating the risk of acute coronary events based on the 10-year follow-up of the prospective cardiovascular Munster (PROCAM) study. Circulation 105:310–315PubMedCrossRefGoogle Scholar
  54. 54.
    (1998) Joint British recommendations on prevention of coronary heart disease in clinical practice. British Cardiac Society, British Hyperlipidaemia Association, British Hypertension Society, endorsed by the British Diabetic Association. Heart 80(Suppl 2):1–29Google Scholar
  55. 55.
    Hingorani AD, Vallance P (1999) A simple computer program for guiding management of cardiovascular risk factors and prescribing. BMJ 318:101–105PubMedGoogle Scholar
  56. 56.
    WOSCOPS Study Group (1992) A coronary primary prevention study of Scottish men aged 45–64 years: trial design. The West of Scotland Coronary Prevention Study Group. J Clin Epidemiol 45:849–860CrossRefGoogle Scholar
  57. 57.
    Marrugat J, Solanas P, D'Agostino R et al (2003) Coronary risk estimation in Spain using a calibrated Framingham function. Rev Esp Cardiol 56:253–261 (article in Spanish)PubMedCrossRefGoogle Scholar
  58. 58.
    Balkau B, Hu G, Qiao Q, Tuomilehto J, Borch-Johnsen K, Pyorala K (2004) Prediction of the risk of cardiovascular mortality using a score that includes glucose as a risk factor. The DECODE Study. Diabetologia 47:2118–2128PubMedCrossRefGoogle Scholar
  59. 59.
    D’Agostino RB, Wolf PA, Belanger AJ, Kannel WB (1994) Stroke risk profile: adjustment for antihypertensive medication. The Framingham Study. Stroke 25:40–43PubMedGoogle Scholar
  60. 60.
    Costa B, Cabre JJ, Martin F, Pinol JL, Basora J, Blade J (2005) The Framingham function overestimates stroke risk for diabetes and metabolic syndrome among Spanish population. Aten Primaria 35:392–398 (article in Spanish)PubMedCrossRefGoogle Scholar

Copyright information

© The Author(s) 2009

Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Authors and Affiliations

  • P. Chamnan
    • 1
  • R. K. Simmons
    • 1
  • S. J. Sharp
    • 1
  • S. J. Griffin
    • 1
  • N. J. Wareham
    • 1
    Email author
  1. 1.MRC Epidemiology Unit, Institute of Metabolic ScienceAddenbrooke’s HospitalCambridgeUK

Personalised recommendations