Improvement in Cardiovascular Risk Prediction with Electronic Health Records

  • Mindy M. Pike
  • Paul A. Decker
  • Nicholas B. Larson
  • Jennifer L. St. Sauver
  • Paul Y. Takahashi
  • Véronique L. Roger
  • Walter A. Rocca
  • Virginia M. Miller
  • Janet E. Olson
  • Jyotishman Pathak
  • Suzette J. BielinskiEmail author
Original Article


The aim of this study was to compare the QRISKII, an electronic health data-based risk score, to the Framingham Risk Score (FRS) and atherosclerotic cardiovascular disease (ASCVD) score. Risk estimates were calculated for a cohort of 8783 patients, and the patients were followed up from November 29, 2012, through June 1, 2015, for a cardiovascular disease (CVD) event. During follow-up, 246 men and 247 women had a CVD event. Cohen’s kappa statistic for the comparison of the QRISKII and FRS was 0.22 for men and 0.23 for women, with the QRISKII classifying more patients in the higher-risk groups. The QRISKII and ASCVD were more similar with kappa statistics of 0.49 for men and 0.51 for women. The QRISKII shows increased discrimination with area under the curve (AUC) statistics of 0.65 and 0.71, respectively, compared to the FRS (0.59 and 0.66) and ASCVD (0.63 and 0.69). These results demonstrate that incorporating additional data from the electronic health record (EHR) may improve CVD risk stratification.


Cardiovascular QRISK Framingham risk score ASCVD Biobank 



Atrial fibrillation


Atherosclerotic cardiovascular disease


Body mass index


Cardiovascular disease


Coronary heart disease


Chronic kidney disease


Electronic health record


Health Information Technology for Economic and Clinical Health


Housing data


Framingham Risk Score


High-density lipoprotein


Myocardial infarction


Rheumatoid arthritis


Rochester Epidemiology Project


United Kingdom


United States of America



This study was made possible by using the resources of the Rochester Epidemiology Project, which is supported by the National Institute on Aging of the National Institutes of Health under award number R01AG034676. The Mayo Clinic Biobank is supported by the Mayo Clinic Center for Individualized Medicine.

Compliance with Ethical Standards

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000. All research procedures were approved by the Institutional Review Committee of the Mayo Clinic.

Conflict of Interest

The authors declare that they have no competing interests.

Human and Animal Rights and Informed Consent

No animal studies were carried out by the authors for this article. The participants provided written and informed consent for the general research.

Supplementary material

12265_2016_9687_MOESM1_ESM.pdf (106 kb)
ESM 1 (PDF 106 kb)


  1. 1.
    Go, A. S., Mozaffarian, D., Roger, V. L., Benjamin, E. J., Berry, J. D., Blaha, M. J., Dai, S., Ford, E. S., Fox, C. S., Franco, S., Fullerton, H. J., Gillespie, C., Hailpern, S. M., Heit, J. A., Howard, V. J., et al. (2014). Heart disease and stroke statistics—2014 update: a report from the American Heart Association. Circulation, 129(3), e28–e292.CrossRefPubMedGoogle Scholar
  2. 2.
    Wilson, P. W., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998). Prediction of coronary heart disease using risk factor categories. Circulation, 97(18), 1837–1847.CrossRefPubMedGoogle Scholar
  3. 3.
    D’Agostino, R. B., Sr., Grundy, S., Sullivan, L. M., & Wilson, P. (2001). Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA, 286(2), 180–187.CrossRefPubMedGoogle Scholar
  4. 4.
    D’Agostino, R. B., Sr., Vasan, R. S., Pencina, M. J., Wolf, P. A., Cobain, M., Massaro, J. M., & Kannel, W. B. (2008). General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation, 117(6), 743–753.CrossRefPubMedGoogle Scholar
  5. 5.
    Goff, D. C., Jr., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., Robinson, J. G., Schwartz, J. S., Shero, S. T., Smith, S. C., Jr., Sorlie, P., et al. (2014). 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation, 129(25 Suppl 2), S49–S73.CrossRefPubMedGoogle Scholar
  6. 6.
    Hippisley-Cox, J., Coupland, C., Robson, J., & Brindle, P. (2010). Derivation, validation, and evaluation of a new QRISK model to estimate lifetime risk of cardiovascular disease: cohort study using QResearch database. BMJ, 341(c6624).Google Scholar
  7. 7.
    Hippisley-Cox, J., Coupland, C., Vinogradova, Y., Robson, J., May, M., & Brindle, P. (2007). Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ, 335(7611), 136.CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Zhao, D., Liu, J., Xie, W., & Qi, Y. (2015). Cardiovascular risk assessment: a global perspective. Nature Reviews Cardiology, 12(5), 301–311.CrossRefPubMedGoogle Scholar
  9. 9.
    Hippisley-Cox, J., Coupland, C., Vinogradova, Y., Robson, J., Minhas, R., Sheikh, A., & Brindle, P. (2008). Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ, 336(7659), 1475–1482.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Blumenthal, D., & Tavenner, M. (2010). The “meaningful use” regulation for electronic health records. New England Journal of Medicine, 363(6), 501–504.CrossRefPubMedGoogle Scholar
  11. 11.
    Bielinski, S. J., Pathak, J., Carrell, D. S., Takahashi, P. Y., Olson, J. E., Larson, N. B., Liu, H., Sohn, S., Wells, Q. S., Denny, J. C., Rasmussen-Torvik, L. J., Pacheco, J. A., Jackson, K. L., Lesnick, T. G., Gullerud, R. E., et al. (2015). A robust e-Epidemiology Tool in Phenotyping Heart Failure with Differentiation for Preserved and Reduced Ejection Fraction: the Electronic Medical Records and Genomics (eMERGE) Network. Journal of Cardiovascular Translational Research, 8(8), 475–483.CrossRefPubMedGoogle Scholar
  12. 12.
    Krishnamoorthy, P., Gupta, D., Chatterjee, S., Huston, J., & Ryan, J. J. (2014). A review of the role of electronic health record in genomic research. Journal of Cardiovascular Translational Research, 7(8), 692–700.CrossRefPubMedGoogle Scholar
  13. 13.
    Rasmussen, L. V. (2014). The electronic health record for translational research. Journal of Cardiovascular Translational Research, 7(6), 607–614.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Kho, A. N., Pacheco, J. A., Peissig, P. L., Rasmussen, L., Newton, K. M., Weston, N., Crane, P. K., Pathak, J., Chute, C. G., Bielinski, S. J., Kullo, I. J., Li, R., Manolio, T. A., Chisholm, R. L., & Denny, J. C. (2011). Electronic medical records for genetic research: results of the eMERGE consortium. Science Translational Medicine, 3(79), 79re71.CrossRefGoogle Scholar
  15. 15.
    Newton, K. M., Peissig, P. L., Kho, A. N., Bielinski, S. J., Berg, R. L., Choudhary, V., Basford, M., Chute, C. G., Kullo, I. J., Li, R., Pacheco, J. A., Rasmussen, L. V., Spangler, L., & Denny, J. C. (2013). Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network. Journal of the American Medical Informatics Association, 20(e1), e147–e154.CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Roger, V. L. (2015). Of the importance of motherhood and apple pie: what big data can learn from small data. Circulation. Cardiovascular Quality and Outcomes, 8(4), 329–331.CrossRefPubMedGoogle Scholar
  17. 17.
    Olson, J. E., Ryu, E., Johnson, K. J., Koenig, B. A., Maschke, K. J., Morrisette, J. A., Liebow, M., Takahashi, P. Y., Fredericksen, Z. S., Sharma, R. G., Anderson, K. S., Hathcock, M. A., Carnahan, J. A., Pathak, J., Lindor, N. M., et al. (2013). The Mayo Clinic Biobank: a building block for individualized medicine. Mayo Clinic Proceedings, 88(9), 952–962.Google Scholar
  18. 18.
    Kho, A. N., Hayes, M. G., Rasmussen-Torvik, L., Pacheco, J. A., Thompson, W. K., Armstrong, L. L., Denny, J. C., Peissig, P. L., Miller, A. W., Wei, W. Q., Bielinski, S. J., Chute, C. G., Leibson, C. L., Jarvik, G. P., Crosslin, D. R., et al. (2012). Use of diverse electronic medical record systems to identify genetic risk for type 2 diabetes within a genome-wide association study. Journal of the American Medical Informatics Association, 19(2), 212–218.CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Juhn, Y. J., Beebe, T. J., Finnie, D. M., Sloan, J., Wheeler, P. H., Yawn, B., & Williams, A. R. (2011). Development and initial testing of a new socioeconomic status measure based on housing data. Journal of Urban Health, 88(5), 933–944.CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Carletta, J. (1996). Assessing agreement on classification tasks: the kappa statistic. Computational Linguistics, 22(2), 249–254.Google Scholar
  21. 21.
    National Institute for Health and Care Excellence. Cardiovascular disease: risk assessment and reduction, including lipid modification. NICE Guidelines [CG181]. Published 18 July 2014. Accessed 13 Nov 2015.
  22. 22.
    Collins, G. S., & Altman, D. G. (2010), An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ, 340(c2442).Google Scholar
  23. 23.
    Hippisley-Cox, J., Coupland, C., Vinogradova, Y., Robson, J., & Brindle, P. (2008). Performance of the QRISK cardiovascular risk prediction algorithm in an independent UK sample of patients from general practice: a validation study. Heart, 94(1), 34–39.CrossRefPubMedGoogle Scholar
  24. 24.
    Muntner, P., Colantonio, L. D., Cushman, M., Goff, D. C., Jr., Howard, G., Howard, V. J., Kissela, B., Levitan, E. B., Lloyd-Jones, D. M., & Safford, M. M. (2014). Validation of the atherosclerotic cardiovascular disease pooled cohort risk equations. JAMA, 311(14), 1406–1415.CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Mosca, L., Benjamin, E. J., Berra, K., Bezanson, J. L., Dolor, R. J., Lloyd-Jones, D. M., Newby, L. K., Pina, I. L., Roger, V. L., Shaw, L. J., Zhao, D., Beckie, T. M., Bushnell, C., D’Armiento, J., Kris-Etherton, P. M., et al. (2011). Effectiveness-based guidelines for the prevention of cardiovascular disease in women—2011 update: a guideline from the American Heart Association. Journal of the American College of Cardiology, 57(12), 1404–1423.CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    McCormick, N., Lacaille, D., Bhole, V., & Avina-Zubieta, J. A. (2014). Validity of myocardial infarction diagnoses in administrative databases: a systematic review. PLoS ONE, 9(3), e92286.CrossRefPubMedPubMedCentralGoogle Scholar
  27. 27.
    McCormick, N., Bhole, V., Lacaille, D., & Avina-Zubieta, J. A. (2015). Validity of diagnostic codes for acute stroke in administrative databases: a systematic review. PLoS ONE, 10(8), e0135834.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Mindy M. Pike
    • 1
  • Paul A. Decker
    • 1
  • Nicholas B. Larson
    • 1
  • Jennifer L. St. Sauver
    • 1
    • 2
  • Paul Y. Takahashi
    • 3
  • Véronique L. Roger
    • 1
    • 4
  • Walter A. Rocca
    • 1
    • 5
  • Virginia M. Miller
    • 6
    • 7
  • Janet E. Olson
    • 1
  • Jyotishman Pathak
    • 8
  • Suzette J. Bielinski
    • 1
    Email author
  1. 1.Department of Health Sciences ResearchMayo ClinicRochesterUSA
  2. 2.Robert D. and Patricia E. Kern Center for the Science of Health Care DeliveryMayo ClinicRochesterUSA
  3. 3.Department of MedicineMayo ClinicRochesterUSA
  4. 4.Division of Cardiovascular Diseases in the Department of Internal MedicineMayo ClinicRochesterUSA
  5. 5.Department of NeurologyMayo ClinicRochesterUSA
  6. 6.Department of SurgeryMayo ClinicRochesterUSA
  7. 7.Department of Physiology and Biomedical EngineeringMayo ClinicRochesterUSA
  8. 8.Department of Healthcare Policy and ResearchWeill Cornell Medical CollegeNew YorkUSA

Personalised recommendations