Introduction

To estimate atherosclerotic cardiovascular disease (ASCVD) risk, the 2013 American College of Cardiology (ACC)/American Heart Association (AHA) guidelines [1] recommended the use of pooled cohort equations (PCEs) derived from four major population-based cohort studies in the United States in the 1990s [2]. However, these equations overestimated the ASCVD risk in three major United States cohorts [3]. Two external validation cohorts, multi ethnic study and atherosclerosis (MESA) [4] and reasons for geographic and racial differences in stroke (REGARDS) [5], observed a similar risk overestimation.

The use of statin therapy for patients whose 10-year ASCVD risk exceeded 5% or 7.5% is recommended by 2013ACC/AHA guidelines [1]. However, this approach has proved controversial, particularly for primary prevention in adults whose low-density lipoprotein cholesterol (LDL-C) level is between 70 and 189 mg/dL but who are not diagnosed with ASCVD or diabetes. The guidelines recommend these patients receive statin treatment; however, it has been reported that their predicted ASCVD risk by the PCEs was systematically higher than the observed risk in the Bioimage study [6], although not in the REGARDS study [5].

The aim of the present study was to conduct an external validation of PCEs among patients eligible for statin therapy using data from the systolic blood pressure intervention trial (SPRINT) [7].

Main text

Methods

We conducted a secondary analysis of SPRINT data to evaluate the calibration and discriminatory ability of PCEs. SPRINT was a randomized, controlled, open-label trial sponsored by the National Heart, Lung, and Blood Institute (NHLBI) in a formal data repository that was intended to facilitate data sharing from clinical trials and observational studies (https://biolincc.nhlbi.nih.gov/home/). Patients were enrolled into SPRINT from November 2010 to March 2013 at 102 clinical sites in the United States, including Puerto Rico. The trial was approved by institutional review boards at participating study sites; all patients provided written informed consent [7]. In the analysis of SPRINT data, we received ethical approval from the institutional review board of the Hyogo Prefectural Amagasaki General Medical Center (No. 28-100).

Study population

The SPRINT trial included patients who met the following inclusion criteria: age at least 50 years; a systolic blood pressure of 130–180 mmHg; no history of diabetes or stroke; and at least one cardiovascular risk factor (clinical or subclinical cardiovascular disease except stroke or chronic kidney disease), or a 10-year risk of cardiovascular disease ≥ 15% based on the Framingham risk score, or age ≥ 75 years [7]. From these patients, we selected patients who had been recommended to begin taking statins according to the ACC/AHA guidelines [1]. These met the following criteria: age, 40–79 years; no clinical ASCVD events; no statin use at baseline; and LDL-C in the range 70–190 mg/dL (or, if the LDL-C level was unavailable, a non-high-density lipoprotein cholesterol level in the range 100–219 mg/dL). We used the primary SPRINT outcomes (cardiovascular death, myocardial infarction, and stroke) but we excluded heart failure. We applied the PCEs to calculate the predicted 10-year ASCVD risk for each patient. Because the SPRINT trial had not yet completed 10 years of follow-up, we calculated the observed and predicted ASCVD incidences at 5 years. We performed calculations with up to 5 years of follow-up using a baseline hazard adjusted from a 10-year to a 5-year rate, as previously described. [5].

Statistical analysis

The calibration of the PCEs was determined using the Hosmer–Lemeshow test and a calibration plot. The discriminatory ability was calculated according to the area under the receiver operating characteristic curve. The 95% confidence interval was calculated by using the bootstrap method. All statistical analyses were performed by using Stata® ver. 13.0 (Stata Corp, College Station, TX, USA) and R 3.3.2 (R foundation for Statistical Computing). Two-sided p-values of < 0.05 were considered statistically significant. Our data analysis used complete-case analysis.

Results

Among the 9361 SPRINT patients, 4057 were included in the final analysis (Fig. 1). Of these, 2467 (60.8%) were males and 1603 (39.5%) were African-Americans. The median predicted 10-year ASCVD risks were 17.2% in men and 12.3% in women. Patients were categorized into four groups based on their 10-year predicted ASCVD risk: < 5%, 5% to less than 7.5%, 7.5% to less than 10%, and ≥ 10% (Table 1). A total of 3126 (77%) patients had a predicted ASCVD risk > 10%. Patients with higher 10-year predicted ASCVD risk were older, and this group included higher percentages of African Americans and men.

Fig. 1
figure 1

Flowchart of patients who met inclusion criteria for the study

Full size image
Table 1 Baseline characteristics in predicted 10-y ASCVD risk (n = 4057)
Full size table

There were 133 primary events (including 23 cardiovascular deaths) observed over a median follow-up of 3.3 years, whereas 304 events were predicted by the PCEs. The PCEs overestimated the risk of cardiovascular events by 128.6%. The calibration plots demonstrated consistent overestimation of the predicted probabilities (Hosmer–Lemeshow test, p < 0.001) according to linear correlation analysis (Fig. 2). The PCEs were only moderately discriminatory toward the ASCVD prediction, with an area under the curve of 0.65 (95% confidence interval: 0.60–0.69).

Fig. 2
figure 2

Comparison of observed versus predicted risk

Full size image

Discussion

Our study showed that the PCEs had moderate discriminatory ability, poor calibration, and consistently overestimated the ASCVD risks among patients for whom the ACC/AHA guidelines recommended statin initiation in the SPRINT population. Furthermore, this difference was greater in the patient group with higher predicted ASCVD risk because the observed risk did not necessarily match the predicted risk. In most external validation cohorts in the United States, the PCEs appeared to overestimate the observed risk [8]. However, among the patients considered for statin initiation, whether PCEs were appropriately calibrated for ASCVD events or not is controversial. Indeed, the PCEs showed good calibration in an analysis of the REGARDS study [5] but showed poor calibration in the Bioimage study [6]. Rana et al. [9] reported the PCEs substantially overestimated ASCVD risk in a large multi-ethnic “real world” population. Similarly, our results demonstrated that the PCEs overestimated ASCVD risk in the SPRINT population. This indicates that PCEs overestimates regardless choice of the study population; risk overestimation may be explained by changes in ASCVD rates over time. PCEs derived from the 1990s population data do not reflect the lower current ASCVD rates. Other reasons suggested for the overestimation include more contemporary use of statin therapy, increasing use of revascularization therapies, and under ascertainment of outcomes in selected cohorts [8, 10, 11]. In fact, it was reported the measuring accurate outcomes by comprehensive surveillance reduced overestimation in a Multiethnic Cohort from the Women’s Health Initiative [12]. Consequently, many patients would be inappropriately prescribed statin therapy because of overestimation. Recently, Yadlowsky et al. [13] showed the ‘revised PCEs’ reduced overestimation of need for statins among contemporary populations. Accurate estimation of risk is essential to effectively balance the therapeutic risk and benefit of statins.

Conclusion

The PCEs overestimated ASCVD risk in patients for whom statin therapy is recommended in the SPRINT population. Risk prediction equations may need to be updated according to changing patterns of risk in contemporary society.

Limitation

A limitation of this study was the shorter follow-up period, because the SPRINT study was stopped early despite the planned follow-up of 5 years. Although, the SPRINT was a blood pressure-lowering study for high CVD-risk patients, PCEs was calculated the future ASCVD risk using only baseline data, and blood pressure change over time is not considered. Further, since SPRINT study has not been examined for the use of statins or antiplatelet therapy during follow-up, cardiovascular risk may have been modified by the provision of risk-reducing therapies.