Avoid common mistakes on your manuscript.
One of the most important findings to come out of the landmark Framingham Heart Study was the development of tools to predict patients’ risk of developing coronary artery disease (CHD).1 By analyzing data across multiple CHD risk factor categories derived from this prospective cohort, it became possible to predict patients’ 10-year risk for CHD; the prediction model was based on a set of parameter estimates for each risk factor and combinations of risk factors.2 Wilson et al. recognized if physicians were to use these tools for clinical decision-making that they needed to be quick and easy to use. In 1998, they published the sex-specific Framingham Risk Scoring (FRS) system.3 FRS involved converting the parameter estimates from the original prediction model into whole numbers (points) that could then be summed and converted into a 10-year risk. Scoring algorithms using these points are available in print, online and through hand-held electronic devices. Use of FRS point scoring was promulgated in the Third Adult Treatment Panel (ATP III) recommendations for treatment of high cholesterol by the National Cholesterol Education Program.4 FRS arguably represents the most widely used validated-tool for personalized clinical decision making.
In a provocative study, Gordon et al. compared the risk classification of FRS point scoring with the original Framingham prediction model using risk factor prevalence information derived from the National Health and Nutrition Survey (NHANES) 2001–2006.5 They observed significant discrepancies between the two methods. Relative to the original model, point counting reclassifies a large percentage of patients into risk categories that could potentially alter treatment eligibility based on ATP III recommendations; notably, use of the counting system instead of the original parameter estimates resulted in 5% of patients being “under-treated” and 10% being “over-treated” (assuming the original parameter estimates represent the gold standard). While the authors do not compare the actual performance of these tools, they argue that it is implausible that a simplified version in which parameter estimates are rounded to the nearest whole number would outperform the original, more sophisticated version. They further suggest that widespread use of electronic devices including those loaded with FRS software make it feasible to use the original Framingham model to estimate risk, essentially rendering point scoring a relic of the pre-electronic age.
Are their findings valid? Consistent with ATP III guidelines, Gordon et al. excluded those with low risk and known CHD (or CHD equivalent) disease from the analysis, thus sharpening the contrast between the two models by focusing on those in the intermediate risk range. Patients taking lipid-lowering therapy were excluded because FRS was not designed to assess risk among persons receiving drug treatment. However, this exclusion creates a non-representative sample. Potentially, persons who were treated based on use of FRS point scoring were excluded from further analysis. The impact of this bias is hard to quantify, but given that only 11% of the study sample were affected, it seems unlikely that this exclusion would significantly alter their findings, particularly among those in the intermediate risk categories where drug treatment is less frequent. Gordon et al. confined their sample to those with two or more major risk factors. Had they included all untreated adults with complete data and compared the original FRS model to point scoring using for the ATP III two-step procedure, even greater discrepancies may have been noted as suggested by an analysis of earlier NHANES.6 Thus, on balance it appears that the authors’ conclusion regarding significant discrepancies in CHD risk or “misclassification” is justified.
Are their recommendations sound? Do these observed discrepancies affect patient outcomes? Is it time to incorporate the original Framingham prediction model into electronic versions of FRS? Studies have suggested that the proper use of assessment tools improves outcomes. In the absence of formal risk assessment, physicians often misjudge CHD risk.7 A meta-analysis showed that CHD risk assessment improves patient outcomes and reduces harm.8 Nonetheless, Wilson et al.’s hope that most physicians would actually use FRS point scoring has not been realized. In fact, most physicians do not use any type of FRS.9,10 Furthermore, even when CHD risk scores are appended to patients’ charts, most physicians do not alter their clinical decision-making.11 In the context of current medical practice, discrepancies between these models may have relatively little impact on patient outcomes because physicians typically do not use FRS point counting or FRS-based on more precise parameter estimates.
But, medical practice is at the cusp of an electronic transformation aided by rapidly evolving technology, recent federal standards, and financial incentives for “meaningful use” of electronic health records (EHRs).12 The recent appointment of health care improvement maven Donald Berwick to head the Centers for Medicare and Medicaid Services may also herald the acceleration of the use of technology to improve quality. Adoption of EHRs by community physicians is rapidly approaching the tipping point. By 2009, 44% of office physicians in the US reported use of some type of EHR.13 EHRs make it technically feasible to automatically generate CHD risk assessments and identify patients in need of interventions.14,15 Increasingly, EHR data will also be used to monitor (and potentially improve) physician performance. These same data will also be fodder for malpractice claims. Physicians who choose to deviate from embedded clinical decision support systems that include risk assessment and corresponding specific clinical recommendations will be expected to document their reasons for doing so.
In this rapidly evolving environment for medical practice, simple point scoring systems may have seen their day. As Gordon et al. suggest, EHRs and portable electronic devices enable risk tools that require complex computations. EHRs could also facilitate tailoring risk assessment-based social and other novel risk factors. For example, consideration of SES improves risk calibration, particularly for low socioeconomic status persons whose risk is underestimated by FRS.16 Recent health reform legislation strengthening requirements for collection of patient sociodemographic information coupled with increasing use of area-based socioeconomic measures will facilitate consideration of a range of CHD risk factors when personalizing risk.
The above scenario is not farfetched. The UK, perhaps a decade or so ahead of the US in EHR adoption in primary care, provides a glimpse of future practice in the US.17 SES is incorporated into the most widely used risk assessment tool, QRISK2.18 Moreover, widespread use of EHRs in the UK has facilitated establishment of prospective patient cohorts that have yielded improved cardiovascular risk tools such as QRISK2. These cohorts, often comprising more than a million patients, and unimaginable in the pre-electronic age, have made it possible to model CHD risk for relatively small ethnic minority groups.19 These huge samples derived from hundreds of networked practices, also facilitate individualized estimates of the benefits and harms associated with statin treatment.20 Establishment of similar comparative effectiveness research cohorts in the US will enable far more precisely personalized risk assessment and treatment guidelines than has been heretofore possible. Thus, Gordon et al.’s suggestion to begin using the best available FRS prediction models with currently available electronic devices seems timely.
REFERENCES
Gordon T, Kannel WB. Multiple risk functions for predicting coronary heart disease: the concept, accuracy, and application. Am Heart J. 1982;103:1031–9.
Anderson KM, Wilson PW, Odell PM, Kannel WB. An updated coronary risk profile. A statement for health professionals. Circulation. 1991;83:356–62.
Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.
Expert Panel on Detection and Treatment of High Blood Cholesterol. Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP). JAMA. 2001;285:2486–97.
Gordon WJ, Polansky JM, Boscardin WJ, Fung KZ, Steinman MA. Coronary risk assessment by point-based vs. equation-based Framingham models: Significant implications for clinical care. JGIM. 2010; doi:10.1007/s11606-010-1454-2.
Persell SD, Lloyd-Jones DM, Baker DW. National Cholesterol Education Program risk assessment and potential for risk misclassification. Prev Med. 2006;43:368–71.
Pignone M, Phillips CJ, Elasy TA, Fernandez A. Physicians' ability to predict the risk of coronary heart disease. BMC Health Serv Res. 2003;3:13.
Sheridan SL, Crespo E. Does the routine use of global coronary heart disease risk scores translate into clinical benefits or harms? A systematic review of the literature. BMC Health Serv Res. 2008;8:60.
Sposito AC, Ramires JA, Jukema JW, et al. Physicians' attitudes and adherence to use of risk scores for primary prevention of cardiovascular disease: cross-sectional survey in three world regions. Curr Med Res Opin. 2009;25:1171–8.
Eaton CB, Galliher JM, McBride PE, Bonham AJ, Kappus JA, Hickner J. Family physician's knowledge, beliefs, and self-reported practice patterns regarding hyperlipidemia: a National Research Network (NRN) survey. J Am Board Fam Med: JABFM. 2006;19:46–53.
Jacobson TA, Gutkin SW, Harper CR. Effects of a global risk educational tool on primary coronary prevention: the Atherosclerosis Assessment Via Total Risk (AVIATOR) study. Curr Med Res Opin. 2006;22:1065–73.
Buntin MB, Jain SH, Blumenthal D. Health information technology: laying the infrastructure for national health reform. Health Aff. 2010;29:1214–9.
Hsiao C-J, Beatty PC, Hing ES, Woodwell DA, Rechtsteiner EA, Sisk JE. Electronic Medical Record/Electronic Health Record Use by Office-based Physicians: United States, 2008 and Preliminary 2009 . Available from: National Center for Health Statistics, Centers for Disease Control and Prevention (http://www.cdc.gov/nchs/data/hestat/emr_ehr/emr_ehr.pdf, accessed July 14, 2010).
Persell SD, Dunne AP, Lloyd-Jones DM, Baker DW. Electronic health record-based cardiac risk assessment and identification of unmet preventive needs. Med Care. 2009;47:418–24.
Wells S, Furness S, Rafter N, et al. Integrated electronic decision support increases cardiovascular disease risk assessment four fold in routine primary care practice. Eur J Cardiovasc Prev Rehab. 2008;15:173–8.
Fiscella K, Tancredi D, Franks P. Adding socioeconomic status to Framingham scoring to reduce disparities in coronary risk assessment. Am Heart J. 2009;157:988–94.
Schade CP, Sullivan FM. de LS, Madeley J. e-Prescribing, efficiency, quality: lessons from the computerization of UK family practice. J Am Med Inform Assoc. 2006;13:470–5.
Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007;335:136.
Hippisley-Cox J, Coupland C, Vinogradova Y, et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ. 2008;336:1475–82.
Hippisley-Cox J, Coupland C. Unintended effects of statins in men and women in England and Wales: population based cohort study using the QResearch database. BMJ. 2010;340:c2197.
Conflict of Interest
Neither author reports any.
Funding
The National Heart Lung and Blood Institute (1R01 HL081066-01A2).
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Fiscella, K., Franks, P. Assessing Coronary Risk Assessment: What's Next?. J GEN INTERN MED 25, 1140–1141 (2010). https://doi.org/10.1007/s11606-010-1469-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-010-1469-8