Skip to main content

Artificial neural network-based model enhances risk stratification and reduces non-invasive cardiac stress imaging compared to Diamond–Forrester and Morise risk assessment models: A prospective study



Coronary artery disease (CAD) accounts for more than half of all cardiovascular events. Stress testing remains the cornerstone for non-invasive assessment of patients with possible or known CAD. Clinical utilization reviews show that most patients presenting for evaluation of stable CAD by stress testing are categorized as low risk prior to the test. Attempts to enhance risk stratification of individuals who are sent for stress testing seem to be more in need today. The present study compares artificial neural networks (ANN)-based prediction models to the other risk models being used in practice (the Diamond–Forrester and the Morise models).


In our study, we prospectively recruited patients who were 19 years of age or older, and were being evaluated for coronary artery disease with imaging-based stress tests. For ANN, the network architecture employed a systematic method, where the number of neurons is changed incrementally, and bootstrapping was performed to evaluate the accuracy of the models.


We prospectively enrolled 486 patients. The mean age of patients undergoing stress test was 55.2 ± 11.2 years, 35% were women, and 12% had a positive stress test for ischemic heart disease. When compared to Diamond–Forrester and Morise risk models, the ANN model for predicting ischemia provided higher discriminatory power (DP)(1.61), had a negative predictive value of 98%, Sensitivity 91% [81%-97%], Specificity 65% [60%-79%], positive predictive value 26%, and a potential 59% reduction of non-invasive imaging.


The ANN models improved risk stratification when compared to the other risk scores (Diamond–Forrester and Morise) with a 98% negative predictive value and a significant potential reduction in non-invasive imaging tests.

This is a preview of subscription content, access via your institution.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5


  1. Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Borden WB, et al. Heart disease and stroke statistics—2013 update: A report from the American Heart Association. Circulation 2013;127:e6-245.

    Article  Google Scholar 

  2. Chest Pain of Recent Onset. Assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin. London: National Clinical Guideline Centre for Acute and Chronic Conditions; 2010.

    Google Scholar 

  3. Demir OM, Alfakih K, Plein S. Current international guidelines for the investigation of patients with suspected coronary artery disease. Eur Heart J Cardiovasc Imaging 2014;15:1422-4.

    Article  Google Scholar 

  4. Miller TD, Askew JW, Anavekar NS. Noninvasive stress testing for coronary artery disease. Heart Fail Clin 2016;12:65-82.

    Article  Google Scholar 

  5. Cubukcu A, Murray I, Anderson S. What’s the risk? Assessment of patients with stable chest pain. Echo Res Pract 2015;2:41-8.

    Article  Google Scholar 

  6. Cotarlan V, Ho D, Pineda J, Qureshi A, Shirani J. Impact of clinical predictors and routine coronary artery disease testing on outcome of patients admitted to chest pain decision unit. Clin Cardiol 2014;37:146-51.

    Article  Google Scholar 

  7. Pope JH, Aufderheide TP, Ruthazer R, Woolard RH, Feldman JA, Beshansky JR, et al. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med 2000;342:1163-70.

    CAS  Article  Google Scholar 

  8. Herren KR, Mackway-Jones K. Emergency management of cardiac chest pain: A review. Emerg Med J 2001;18:6-10.

    CAS  Article  Google Scholar 

  9. Chou R. Cardiac screening with electrocardiography, stress echocardiography, or myocardial perfusion imaging: Advice for high-value care from the American College of Physicians. Ann Intern Med 2015;162:438-47.

    Article  Google Scholar 

  10. Freeman RV, Eagle KA, Bates ER, Werns SW, Kline-Rogers E, Karavite D, et al. Comparison of artificial neural networks with logistic regression in prediction of in-hospital death after percutaneous transluminal coronary angioplasty. Am Heart J 2000;140:511-20.

    CAS  Article  Google Scholar 

  11. Isma’eel HA, Cremer PC, Khalaf S, Almedawar MM, Elhajj IH, Sakr GE, et al. Artificial neural network modeling enhances risk stratification and can reduce downstream testing for patients with suspected acute coronary syndromes, negative cardiac biomarkers, and normal ECGs. Int J Cardiovasc Imaging 2015;32:687-96.

    Article  Google Scholar 

  12. Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary-artery disease. N Engl J Med 1979;300:1350-8.

    CAS  Article  Google Scholar 

  13. Morise AP, Detrano R, Bobbio M, Diamond GA. Development and validation of a logistic regression-derived algorithm for estimating the incremental probability of coronary artery disease before and after exercise testing. J Am Coll Cardiol 1992;20:1187-96.

    CAS  Article  Google Scholar 

  14. Genders TSS, Steyerberg EW, Alkadhi H, Leschka S, Desbiolles L, Nieman K, et al. A clinical prediction rule for the diagnosis of coronary artery disease: validation, updating, and extension. Eur Heart J 2011;32:1316-30.

    Article  Google Scholar 

  15. Fox K, Garcia MA, Ardissino D, Buszman P, Camici PG, Crea F, et al. Guidelines on the management of stable angina pectoris. Executive summary. Rev esp Cardiol 2006;59:919-70.

    Article  Google Scholar 

  16. Hagan MT, Demuth HB, Beale MH. Neural network design. Boston: PWS Publishing Co.; 1996.

    Google Scholar 

  17. Scales LE. Introduction to non-linear optimization. London: MacMillan, City and Guilds; 1985.

    Book  Google Scholar 

  18. CaEP Clopper. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934;4:404-13.

    Google Scholar 

  19. Blakeley DD, Oddone EZ, Hasselblad V, Simel DL, Matchar DB. Noninvasive carotid artery testing. A meta-analytic review. Ann Intern Med 1995;122:360-7.

    CAS  Article  Google Scholar 

  20. Altman DG, Bland JM. Diagnostic tests 2: Predictive values. BMJ 1994;309:102.

    CAS  Article  Google Scholar 

  21. Isma’eel H, Sakr, G. ANN risk model to predict ischemia from stress imaging. Université Saint-Joseph. 2016.

  22. Isma’eel HA, Sakr GE, Habib RH, Almedawar MM, Zgheib NK, Elhajj IH. Improved accuracy of anticoagulant dose prediction using a pharmacogenetic and artificial neural network-based method. Eur J Clin Pharmacol 2014;70:265-73.

    Article  Google Scholar 

  23. Wise ES, Hocking KM, Brophy CM. Prediction of in-hospital mortality after ruptured abdominal aortic aneurysm repair using an artificial neural network. J Vasc Surg 2015;62:8-15.

    Article  Google Scholar 

  24. Purwanto E, Eswaran C, Logeswaran R, Abdul Rahman AR. Prediction models for early risk detection of cardiovascular event. J Med Syst 2012;36:521-31.

    CAS  Article  Google Scholar 

  25. Diamond GA, Staniloff HM, Forrester JS, Pollock BH, Swan HJ. Computer-assisted diagnosis in the noninvasive evaluation of patients with suspected coronary artery disease. J Am Coll Cardiol 1983;1:444-55.

    CAS  Article  Google Scholar 

  26. Rozanski A, Gransar H, Shaw LJ, Kim J, Miranda-Peats L, Wong ND, et al. Impact of coronary artery calcium scanning on coronary risk factors and downstream testing the EISNER (Early Identification of Subclinical Atherosclerosis by Noninvasive Imaging Research) prospective randomized trial. J Am Coll Cardiol 2011;57:1622-32.

    CAS  Article  Google Scholar 

  27. Cremer PC, Khalaf S, Agarwal S, Mayer-Sabik E, Ellis SG, Menon V, et al. Myocardial perfusion imaging in emergency department patients with negative cardiac biomarkers: Yield for detecting ischemia, short-term events, and impact of downstream revascularization on mortality. Circ Cardiovasc Imaging 2014;7:912-9.

    Article  Google Scholar 

  28. Cremer PC, Mentias A, Newton D, Menon V, Wazni O, Tchou PJ, et al. Low yield of myocardial perfusion imaging in asymptomatic patients with atrial fibrillation. JAMA Intern Med 2015;175:1854-5.

    Article  Google Scholar 

  29. Nawar EW, Niska RW, Xu J. National Hospital Ambulatory Medical Care Survey: 2005 emergency department summary. Adv Data 2005;2007:1-32.

    Google Scholar 

  30. Isma’eel HA, Serhan M, Sakr GE, Lamaa N, Garabedian T, Elhajj I, et al. Diamond-Forrester and Morise risk models perform poorly in predicting obstructive coronary disease in Middle Eastern Cohort. Int J Cardiol 2016;203:803-5.

    Article  Google Scholar 

Download references


The authors have indicated that they have no financial conflict of interest.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Hussain A. Isma’eel MD.

Additional information

The authors of this article have provided a PowerPoint file, available for download at SpringerLink, which summarizes the contents of the paper and is free for re-use at meetings and presentations. Search for the article DOI on

Hussain A. Isma’eel and George E. Sakr—co-authors with equal contribution.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PPTX 253 kb)

Appendix: Discretization of Morise and DF scores from pretest probabilities to positive/negative outcomes

Appendix: Discretization of Morise and DF scores from pretest probabilities to positive/negative outcomes

Table 4 allows us to compute Morise score.14,15 The Morise score represents a pretest probability of stenosis. We picked a cutoff point for the total score and assumed that if the total score is higher than this cutoff then the result is positive, otherwise, it is negative. This cutoff point was varied over all possible values, which yielded the ROC curve below. Similar to our group’s previous work,30 the cutoff point that yielded the highest DP was then used (Table 5).

Table 4 Morise score calculation: how to calculate the pretest score and assigned risk groups
Table 5 Method of transforming Diamond–Forrester risk score into a numerical probability

The same was applied to DF.14

$$ \begin{aligned} & {\text{Score}} = 0.04\,*\,{\text{age}} + 1.34\,*\,{\text{Male}} + 1.91\,*\,{\text{Typical}} + 0.64\,*\,{\text{Atypical}} - 4.37 \\ & {\text{Probability}} = \exp ({\text{score}})/(1 + \exp ({\text{score}})) \\ \end{aligned} $$

A cutoff probability was chosen and we assumed that if the current probability is higher than the cutoff then the result is positive otherwise it is negative. This cutoff varied between 0 and 1 with a step of 0.01 which led to the ROC below (Figures 6, 7). The area under the curve for Morise score is 0.59 and the area under the curve for DF is 0.55 compared to 0.7 by ANN (P < .01).

Figure 6
figure 6

AUROC for Morise

Figure 7
figure 7


Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Isma’eel, H.A., Sakr, G.E., Serhan, M. et al. Artificial neural network-based model enhances risk stratification and reduces non-invasive cardiac stress imaging compared to Diamond–Forrester and Morise risk assessment models: A prospective study. J. Nucl. Cardiol. 25, 1601–1609 (2018).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • Artificial neural networks (ANN)
  • Diamond–Forrester score
  • Morise score
  • stress echocardiography
  • nuclear stress test