Skip to main content

Artificial neural network-based model enhances risk stratification and reduces non-invasive cardiac stress imaging compared to Diamond–Forrester and Morise risk assessment models: A prospective study

Journal of Nuclear Cardiology volume 25, pages 1601–1609 (2018)Cite this article

Abstract

Background

Coronary artery disease (CAD) accounts for more than half of all cardiovascular events. Stress testing remains the cornerstone for non-invasive assessment of patients with possible or known CAD. Clinical utilization reviews show that most patients presenting for evaluation of stable CAD by stress testing are categorized as low risk prior to the test. Attempts to enhance risk stratification of individuals who are sent for stress testing seem to be more in need today. The present study compares artificial neural networks (ANN)-based prediction models to the other risk models being used in practice (the Diamond–Forrester and the Morise models).

Methods

In our study, we prospectively recruited patients who were 19 years of age or older, and were being evaluated for coronary artery disease with imaging-based stress tests. For ANN, the network architecture employed a systematic method, where the number of neurons is changed incrementally, and bootstrapping was performed to evaluate the accuracy of the models.

Results

We prospectively enrolled 486 patients. The mean age of patients undergoing stress test was 55.2 ± 11.2 years, 35% were women, and 12% had a positive stress test for ischemic heart disease. When compared to Diamond–Forrester and Morise risk models, the ANN model for predicting ischemia provided higher discriminatory power (DP)(1.61), had a negative predictive value of 98%, Sensitivity 91% [81%-97%], Specificity 65% [60%-79%], positive predictive value 26%, and a potential 59% reduction of non-invasive imaging.

Conclusion

The ANN models improved risk stratification when compared to the other risk scores (Diamond–Forrester and Morise) with a 98% negative predictive value and a significant potential reduction in non-invasive imaging tests.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

References

  1. Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Borden WB, et al. Heart disease and stroke statistics—2013 update: A report from the American Heart Association. Circulation 2013;127:e6-245.

    Article  Google Scholar 

  2. Chest Pain of Recent Onset. Assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin. London: National Clinical Guideline Centre for Acute and Chronic Conditions; 2010.

    Google Scholar 

  3. Demir OM, Alfakih K, Plein S. Current international guidelines for the investigation of patients with suspected coronary artery disease. Eur Heart J Cardiovasc Imaging 2014;15:1422-4.

    Article  Google Scholar 

  4. Miller TD, Askew JW, Anavekar NS. Noninvasive stress testing for coronary artery disease. Heart Fail Clin 2016;12:65-82.

    Article  Google Scholar 

  5. Cubukcu A, Murray I, Anderson S. What’s the risk? Assessment of patients with stable chest pain. Echo Res Pract 2015;2:41-8.

    Article  Google Scholar 

  6. Cotarlan V, Ho D, Pineda J, Qureshi A, Shirani J. Impact of clinical predictors and routine coronary artery disease testing on outcome of patients admitted to chest pain decision unit. Clin Cardiol 2014;37:146-51.

    Article  Google Scholar 

  7. Pope JH, Aufderheide TP, Ruthazer R, Woolard RH, Feldman JA, Beshansky JR, et al. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med 2000;342:1163-70.

    CAS  Article  Google Scholar 

  8. Herren KR, Mackway-Jones K. Emergency management of cardiac chest pain: A review. Emerg Med J 2001;18:6-10.

    CAS  Article  Google Scholar 

  9. Chou R. Cardiac screening with electrocardiography, stress echocardiography, or myocardial perfusion imaging: Advice for high-value care from the American College of Physicians. Ann Intern Med 2015;162:438-47.

    Article  Google Scholar 

  10. Freeman RV, Eagle KA, Bates ER, Werns SW, Kline-Rogers E, Karavite D, et al. Comparison of artificial neural networks with logistic regression in prediction of in-hospital death after percutaneous transluminal coronary angioplasty. Am Heart J 2000;140:511-20.

    CAS  Article  Google Scholar 

  11. Isma’eel HA, Cremer PC, Khalaf S, Almedawar MM, Elhajj IH, Sakr GE, et al. Artificial neural network modeling enhances risk stratification and can reduce downstream testing for patients with suspected acute coronary syndromes, negative cardiac biomarkers, and normal ECGs. Int J Cardiovasc Imaging 2015;32:687-96.

    Article  Google Scholar 

  12. Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary-artery disease. N Engl J Med 1979;300:1350-8.

    CAS  Article  Google Scholar 

  13. Morise AP, Detrano R, Bobbio M, Diamond GA. Development and validation of a logistic regression-derived algorithm for estimating the incremental probability of coronary artery disease before and after exercise testing. J Am Coll Cardiol 1992;20:1187-96.

    CAS  Article  Google Scholar 

  14. Genders TSS, Steyerberg EW, Alkadhi H, Leschka S, Desbiolles L, Nieman K, et al. A clinical prediction rule for the diagnosis of coronary artery disease: validation, updating, and extension. Eur Heart J 2011;32:1316-30.

    Article  Google Scholar 

  15. Fox K, Garcia MA, Ardissino D, Buszman P, Camici PG, Crea F, et al. Guidelines on the management of stable angina pectoris. Executive summary. Rev esp Cardiol 2006;59:919-70.

    Article  Google Scholar 

  16. Hagan MT, Demuth HB, Beale MH. Neural network design. Boston: PWS Publishing Co.; 1996.

    Google Scholar 

  17. Scales LE. Introduction to non-linear optimization. London: MacMillan, City and Guilds; 1985.

    Book  Google Scholar 

  18. CaEP Clopper. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934;4:404-13.

    Google Scholar 

  19. Blakeley DD, Oddone EZ, Hasselblad V, Simel DL, Matchar DB. Noninvasive carotid artery testing. A meta-analytic review. Ann Intern Med 1995;122:360-7.

    CAS  Article  Google Scholar 

  20. Altman DG, Bland JM. Diagnostic tests 2: Predictive values. BMJ 1994;309:102.

    CAS  Article  Google Scholar 

  21. Isma’eel H, Sakr, G. ANN risk model to predict ischemia from stress imaging. Université Saint-Joseph. 2016. https://cimti.usj.edu.lb/files/VMP/StressImaging/index.html.

  22. Isma’eel HA, Sakr GE, Habib RH, Almedawar MM, Zgheib NK, Elhajj IH. Improved accuracy of anticoagulant dose prediction using a pharmacogenetic and artificial neural network-based method. Eur J Clin Pharmacol 2014;70:265-73.

    Article  Google Scholar 

  23. Wise ES, Hocking KM, Brophy CM. Prediction of in-hospital mortality after ruptured abdominal aortic aneurysm repair using an artificial neural network. J Vasc Surg 2015;62:8-15.

    Article  Google Scholar 

  24. Purwanto E, Eswaran C, Logeswaran R, Abdul Rahman AR. Prediction models for early risk detection of cardiovascular event. J Med Syst 2012;36:521-31.

    CAS  Article  Google Scholar 

  25. Diamond GA, Staniloff HM, Forrester JS, Pollock BH, Swan HJ. Computer-assisted diagnosis in the noninvasive evaluation of patients with suspected coronary artery disease. J Am Coll Cardiol 1983;1:444-55.

    CAS  Article  Google Scholar 

  26. Rozanski A, Gransar H, Shaw LJ, Kim J, Miranda-Peats L, Wong ND, et al. Impact of coronary artery calcium scanning on coronary risk factors and downstream testing the EISNER (Early Identification of Subclinical Atherosclerosis by Noninvasive Imaging Research) prospective randomized trial. J Am Coll Cardiol 2011;57:1622-32.

    CAS  Article  Google Scholar 

  27. Cremer PC, Khalaf S, Agarwal S, Mayer-Sabik E, Ellis SG, Menon V, et al. Myocardial perfusion imaging in emergency department patients with negative cardiac biomarkers: Yield for detecting ischemia, short-term events, and impact of downstream revascularization on mortality. Circ Cardiovasc Imaging 2014;7:912-9.

    Article  Google Scholar 

  28. Cremer PC, Mentias A, Newton D, Menon V, Wazni O, Tchou PJ, et al. Low yield of myocardial perfusion imaging in asymptomatic patients with atrial fibrillation. JAMA Intern Med 2015;175:1854-5.

    Article  Google Scholar 

  29. Nawar EW, Niska RW, Xu J. National Hospital Ambulatory Medical Care Survey: 2005 emergency department summary. Adv Data 2005;2007:1-32.

    Google Scholar 

  30. Isma’eel HA, Serhan M, Sakr GE, Lamaa N, Garabedian T, Elhajj I, et al. Diamond-Forrester and Morise risk models perform poorly in predicting obstructive coronary disease in Middle Eastern Cohort. Int J Cardiol 2016;203:803-5.

    Article  Google Scholar 

Download references

Disclosure

The authors have indicated that they have no financial conflict of interest.

Author information

Author notes

    Authors and Affiliations

    1. Division of Cardiology, Department of Internal Medicine, American University of Beirut, PO-BOX 11-0236, Riad el Solh, Beirut, 11072020, Lebanon

      Hussain A. Isma’eel MD, Mustapha Serhan MD, Nader Lamaa MD, Ayman Hakim MD & Antoine B. Abchee MD

    2. Vascular Medicine Program, American University of Beirut Medical Center, Beirut, Lebanon

      Hussain A. Isma’eel MD, Imad Elhajj PhD & Antoine B. Abchee MD

    3. École Superieurd’Ing. de Beirut (ESIB), St Joseph University, Beirut, Lebanon

      George E. Sakr PhD

    4. Department of Cardiovascular Medicine, Cleveland Clinic Foundation, Cleveland, OH, USA

      Paul C. Cremer MD & Wael A. Jaber MD

    5. Department of Internal Medicine, Saint Elizabeth’s Medical Center, Boston, MA, USA

      Torkom Garabedian MD

    6. Department of Electrical & Computer Engineering, American University of Beirut, PO-BOX 11-023, Riad el Solh, Beirut, 11072020, Lebanon

      Imad Elhajj PhD

    Authors
    1. Hussain A. Isma’eel MD
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. George E. Sakr PhD
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Mustapha Serhan MD
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Nader Lamaa MD
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Ayman Hakim MD
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Paul C. Cremer MD
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Wael A. Jaber MD
      View author publications

      You can also search for this author in PubMed Google Scholar

    8. Torkom Garabedian MD
      View author publications

      You can also search for this author in PubMed Google Scholar

    9. Imad Elhajj PhD
      View author publications

      You can also search for this author in PubMed Google Scholar

    10. Antoine B. Abchee MD
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Hussain A. Isma’eel MD.

    Additional information

    The authors of this article have provided a PowerPoint file, available for download at SpringerLink, which summarizes the contents of the paper and is free for re-use at meetings and presentations. Search for the article DOI on https://www.SpringerLink.com.

    Hussain A. Isma’eel and George E. Sakr—co-authors with equal contribution.

    Electronic supplementary material

    Below is the link to the electronic supplementary material.

    Supplementary material 1 (PPTX 253 kb)

    Appendix: Discretization of Morise and DF scores from pretest probabilities to positive/negative outcomes

    Appendix: Discretization of Morise and DF scores from pretest probabilities to positive/negative outcomes

    Table 4 allows us to compute Morise score.14,15 The Morise score represents a pretest probability of stenosis. We picked a cutoff point for the total score and assumed that if the total score is higher than this cutoff then the result is positive, otherwise, it is negative. This cutoff point was varied over all possible values, which yielded the ROC curve below. Similar to our group’s previous work,30 the cutoff point that yielded the highest DP was then used (Table 5).

    Table 4 Morise score calculation: how to calculate the pretest score and assigned risk groups
    Full size table
    Table 5 Method of transforming Diamond–Forrester risk score into a numerical probability
    Full size table

    The same was applied to DF.14

    $$ \begin{aligned} & {\text{Score}} = 0.04\,*\,{\text{age}} + 1.34\,*\,{\text{Male}} + 1.91\,*\,{\text{Typical}} + 0.64\,*\,{\text{Atypical}} - 4.37 \\ & {\text{Probability}} = \exp ({\text{score}})/(1 + \exp ({\text{score}})) \\ \end{aligned} $$

    A cutoff probability was chosen and we assumed that if the current probability is higher than the cutoff then the result is positive otherwise it is negative. This cutoff varied between 0 and 1 with a step of 0.01 which led to the ROC below (Figures 6, 7). The area under the curve for Morise score is 0.59 and the area under the curve for DF is 0.55 compared to 0.7 by ANN (P < .01).

    Figure 6
    figure 6

    AUROC for Morise

    Full size image
    Figure 7
    figure 7

    AUROC for DF

    Full size image

    Rights and permissions

    Reprints and Permissions

    About this article

    Verify currency and authenticity via CrossMark

    Cite this article

    Isma’eel, H.A., Sakr, G.E., Serhan, M. et al. Artificial neural network-based model enhances risk stratification and reduces non-invasive cardiac stress imaging compared to Diamond–Forrester and Morise risk assessment models: A prospective study. J. Nucl. Cardiol. 25, 1601–1609 (2018). https://doi.org/10.1007/s12350-017-0823-1

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s12350-017-0823-1

    Keywords

    • Artificial neural networks (ANN)
    • Diamond–Forrester score
    • Morise score
    • stress echocardiography
    • nuclear stress test