The aim of this study was to compare the QRISKII, an electronic health data-based risk score, to the Framingham Risk Score (FRS) and atherosclerotic cardiovascular disease (ASCVD) score. Risk estimates were calculated for a cohort of 8783 patients, and the patients were followed up from November 29, 2012, through June 1, 2015, for a cardiovascular disease (CVD) event. During follow-up, 246 men and 247 women had a CVD event. Cohen’s kappa statistic for the comparison of the QRISKII and FRS was 0.22 for men and 0.23 for women, with the QRISKII classifying more patients in the higher-risk groups. The QRISKII and ASCVD were more similar with kappa statistics of 0.49 for men and 0.51 for women. The QRISKII shows increased discrimination with area under the curve (AUC) statistics of 0.65 and 0.71, respectively, compared to the FRS (0.59 and 0.66) and ASCVD (0.63 and 0.69). These results demonstrate that incorporating additional data from the electronic health record (EHR) may improve CVD risk stratification.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Atherosclerotic cardiovascular disease
Body mass index
Coronary heart disease
Chronic kidney disease
Electronic health record
Health Information Technology for Economic and Clinical Health
Framingham Risk Score
Rochester Epidemiology Project
United States of America
Go, A. S., Mozaffarian, D., Roger, V. L., Benjamin, E. J., Berry, J. D., Blaha, M. J., Dai, S., Ford, E. S., Fox, C. S., Franco, S., Fullerton, H. J., Gillespie, C., Hailpern, S. M., Heit, J. A., Howard, V. J., et al. (2014). Heart disease and stroke statistics—2014 update: a report from the American Heart Association. Circulation, 129(3), e28–e292.
Wilson, P. W., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998). Prediction of coronary heart disease using risk factor categories. Circulation, 97(18), 1837–1847.
D’Agostino, R. B., Sr., Grundy, S., Sullivan, L. M., & Wilson, P. (2001). Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA, 286(2), 180–187.
D’Agostino, R. B., Sr., Vasan, R. S., Pencina, M. J., Wolf, P. A., Cobain, M., Massaro, J. M., & Kannel, W. B. (2008). General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation, 117(6), 743–753.
Goff, D. C., Jr., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., Robinson, J. G., Schwartz, J. S., Shero, S. T., Smith, S. C., Jr., Sorlie, P., et al. (2014). 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation, 129(25 Suppl 2), S49–S73.
Hippisley-Cox, J., Coupland, C., Robson, J., & Brindle, P. (2010). Derivation, validation, and evaluation of a new QRISK model to estimate lifetime risk of cardiovascular disease: cohort study using QResearch database. BMJ, 341(c6624).
Hippisley-Cox, J., Coupland, C., Vinogradova, Y., Robson, J., May, M., & Brindle, P. (2007). Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ, 335(7611), 136.
Zhao, D., Liu, J., Xie, W., & Qi, Y. (2015). Cardiovascular risk assessment: a global perspective. Nature Reviews Cardiology, 12(5), 301–311.
Hippisley-Cox, J., Coupland, C., Vinogradova, Y., Robson, J., Minhas, R., Sheikh, A., & Brindle, P. (2008). Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ, 336(7659), 1475–1482.
Blumenthal, D., & Tavenner, M. (2010). The “meaningful use” regulation for electronic health records. New England Journal of Medicine, 363(6), 501–504.
Bielinski, S. J., Pathak, J., Carrell, D. S., Takahashi, P. Y., Olson, J. E., Larson, N. B., Liu, H., Sohn, S., Wells, Q. S., Denny, J. C., Rasmussen-Torvik, L. J., Pacheco, J. A., Jackson, K. L., Lesnick, T. G., Gullerud, R. E., et al. (2015). A robust e-Epidemiology Tool in Phenotyping Heart Failure with Differentiation for Preserved and Reduced Ejection Fraction: the Electronic Medical Records and Genomics (eMERGE) Network. Journal of Cardiovascular Translational Research, 8(8), 475–483.
Krishnamoorthy, P., Gupta, D., Chatterjee, S., Huston, J., & Ryan, J. J. (2014). A review of the role of electronic health record in genomic research. Journal of Cardiovascular Translational Research, 7(8), 692–700.
Rasmussen, L. V. (2014). The electronic health record for translational research. Journal of Cardiovascular Translational Research, 7(6), 607–614.
Kho, A. N., Pacheco, J. A., Peissig, P. L., Rasmussen, L., Newton, K. M., Weston, N., Crane, P. K., Pathak, J., Chute, C. G., Bielinski, S. J., Kullo, I. J., Li, R., Manolio, T. A., Chisholm, R. L., & Denny, J. C. (2011). Electronic medical records for genetic research: results of the eMERGE consortium. Science Translational Medicine, 3(79), 79re71.
Newton, K. M., Peissig, P. L., Kho, A. N., Bielinski, S. J., Berg, R. L., Choudhary, V., Basford, M., Chute, C. G., Kullo, I. J., Li, R., Pacheco, J. A., Rasmussen, L. V., Spangler, L., & Denny, J. C. (2013). Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network. Journal of the American Medical Informatics Association, 20(e1), e147–e154.
Roger, V. L. (2015). Of the importance of motherhood and apple pie: what big data can learn from small data. Circulation. Cardiovascular Quality and Outcomes, 8(4), 329–331.
Olson, J. E., Ryu, E., Johnson, K. J., Koenig, B. A., Maschke, K. J., Morrisette, J. A., Liebow, M., Takahashi, P. Y., Fredericksen, Z. S., Sharma, R. G., Anderson, K. S., Hathcock, M. A., Carnahan, J. A., Pathak, J., Lindor, N. M., et al. (2013). The Mayo Clinic Biobank: a building block for individualized medicine. Mayo Clinic Proceedings, 88(9), 952–962.
Kho, A. N., Hayes, M. G., Rasmussen-Torvik, L., Pacheco, J. A., Thompson, W. K., Armstrong, L. L., Denny, J. C., Peissig, P. L., Miller, A. W., Wei, W. Q., Bielinski, S. J., Chute, C. G., Leibson, C. L., Jarvik, G. P., Crosslin, D. R., et al. (2012). Use of diverse electronic medical record systems to identify genetic risk for type 2 diabetes within a genome-wide association study. Journal of the American Medical Informatics Association, 19(2), 212–218.
Juhn, Y. J., Beebe, T. J., Finnie, D. M., Sloan, J., Wheeler, P. H., Yawn, B., & Williams, A. R. (2011). Development and initial testing of a new socioeconomic status measure based on housing data. Journal of Urban Health, 88(5), 933–944.
Carletta, J. (1996). Assessing agreement on classification tasks: the kappa statistic. Computational Linguistics, 22(2), 249–254.
National Institute for Health and Care Excellence. Cardiovascular disease: risk assessment and reduction, including lipid modification. NICE Guidelines [CG181]. Published 18 July 2014. http://www.nice.org.uk/guidance/cg181. Accessed 13 Nov 2015.
Collins, G. S., & Altman, D. G. (2010), An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ, 340(c2442).
Hippisley-Cox, J., Coupland, C., Vinogradova, Y., Robson, J., & Brindle, P. (2008). Performance of the QRISK cardiovascular risk prediction algorithm in an independent UK sample of patients from general practice: a validation study. Heart, 94(1), 34–39.
Muntner, P., Colantonio, L. D., Cushman, M., Goff, D. C., Jr., Howard, G., Howard, V. J., Kissela, B., Levitan, E. B., Lloyd-Jones, D. M., & Safford, M. M. (2014). Validation of the atherosclerotic cardiovascular disease pooled cohort risk equations. JAMA, 311(14), 1406–1415.
Mosca, L., Benjamin, E. J., Berra, K., Bezanson, J. L., Dolor, R. J., Lloyd-Jones, D. M., Newby, L. K., Pina, I. L., Roger, V. L., Shaw, L. J., Zhao, D., Beckie, T. M., Bushnell, C., D’Armiento, J., Kris-Etherton, P. M., et al. (2011). Effectiveness-based guidelines for the prevention of cardiovascular disease in women—2011 update: a guideline from the American Heart Association. Journal of the American College of Cardiology, 57(12), 1404–1423.
McCormick, N., Lacaille, D., Bhole, V., & Avina-Zubieta, J. A. (2014). Validity of myocardial infarction diagnoses in administrative databases: a systematic review. PLoS ONE, 9(3), e92286.
McCormick, N., Bhole, V., Lacaille, D., & Avina-Zubieta, J. A. (2015). Validity of diagnostic codes for acute stroke in administrative databases: a systematic review. PLoS ONE, 10(8), e0135834.
This study was made possible by using the resources of the Rochester Epidemiology Project, which is supported by the National Institute on Aging of the National Institutes of Health under award number R01AG034676. The Mayo Clinic Biobank is supported by the Mayo Clinic Center for Individualized Medicine.
All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000. All research procedures were approved by the Institutional Review Committee of the Mayo Clinic.
Conflict of Interest
The authors declare that they have no competing interests.
Human and Animal Rights and Informed Consent
No animal studies were carried out by the authors for this article. The participants provided written and informed consent for the general research.
Editor-in-Chief Jennifer L. Hall oversaw the review of this article
The content of this study is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Electronic supplementary material
Below is the link to the electronic supplementary material.
(PDF 106 kb)
About this article
Cite this article
Pike, M.M., Decker, P.A., Larson, N.B. et al. Improvement in Cardiovascular Risk Prediction with Electronic Health Records. J. of Cardiovasc. Trans. Res. 9, 214–222 (2016). https://doi.org/10.1007/s12265-016-9687-z