Abstract
Machine learning (ML)-based algorithms for cardiovascular disease (CVD) risk assessment have shown promise in clinical decisions. However, they usually predict binary events using only conventional risk factors. Our overall goal was to develop the “multiclass machine learning (MCML)-based algorithms” (labelled as AtheroEdge 3.0ML) and assess whether considering carotid ultrasound imaging fused with conventional risk factors can provide better CVD/stroke risk prediction than conventional CVD risk calculators (CCVRC). Carotid ultrasound and coronary angiography were performed on 500 participants. Stenosis in the coronary arteries was used to assign participants a coronary angiographic score (CAS). CVD/stroke risk was determined using three types of MCML algorithms: (i) support vector machine (SVM), (ii) random forest (RF), and (iii) extreme gradient boost (XGBoost). The performance of CVD risk assessment using MCML and CCVRC (such as Framingham Risk Score, the Systematic Coronary Risk Evaluation score, and the Atherosclerotic CVD) was evaluated on test patients against the CAS as the gold standard for each class using the area-under-the-curve (AUC) and classification accuracy. The mean percentage improvement in AUC and the mean absolute improvement in accuracy over CCVRC using 90% training and 10% testing protocol (labelled as K10) were ~ 105% and ~ 28%, respectively. Of all the three MCML systems, RF showed the best performance. Further, carotid image phenotypes showed the most effective clinical feature in AtheroEdge 3.0ML performance. The AtheroEdge 3.0ML using carotid imaging are reliable, accurate, and superior to traditional CVD risk scoring methods for predicting the CVD/stroke risk due to coronary artery disease.
Similar content being viewed by others
References
W. H. Organization (2017) Cardiovascular diseases (CVDs): key facts by WHO May 2016. https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds)
Naghavi M, Libby P, Falk E, Casscells SW, Litovsky S, Rumberger J et al (2003) From vulnerable plaque to vulnerable patient: a call for new definitions and risk assessment strategies: Part I. Circulation 108:1664–1672
Hansson GK, Libby P, Tabas I (2015) Inflammation and plaque vulnerability. J Intern Med 278:483–493
Libby P (2003) Vascular biology of atherosclerosis: overview and state of the art. Am J Cardiol 91:3–6
Anderson TJ, Grégoire J, Pearson GJ, Barry AR, Couture P, Dawes M et al (2016) 2016 Canadian Cardiovascular Society Guidelines for the Management of Dyslipidemia for the prevention of cardiovascular disease in the adult. Can J Cardiol 32:1263–1282
Anderson TJ, Grégoire J, Hegele RA, Couture P, Mancini GJ, McPherson R et al (2013) 2012 update of the Canadian Cardiovascular Society guidelines for the diagnosis and treatment of dyslipidemia for the prevention of cardiovascular disease in the adult. Can J Cardiol 29:151–167
Conroy R, Pyörälä K, Fitzgerald AE, Sans S, Menotti A, De Backer G et al (2003) Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 24:987–1003
Goff DC, Lloyd-Jones DM, Bennett G, Coady S, D’agostino RB, Gibbons R et al (2014) 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 63:2935–2959
NICE (2014) Cardiovascular disease: risk assessment and reduction, including lipid modification. National Institute for Health and Care Excellence, London. https://www.nice.org.uk/guidance/cg181. Accessed 1 Jan 2020
Cooper A, O’Flynn N, G. Guideline Development (2008) Risk assessment and lipid modification for primary and secondary prevention of cardiovascular disease: summary of NICE guidance. BMJ (Clin Res Ed) 336:1246–1248
Goldstein BA, Navar AM, Carter RE (2016) Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges. Eur Heart J 38:1805–1814
Jamthikar A, Gupta D, Khanna NN, Araki T, Saba L, Nicolaides A et al (2019) A special report on changing trends in preventive stroke/cardiovascular risk assessment via B-mode ultrasonography. Curr Atheroscler Rep 21:25
Jamthikar A, Gupta D, Saba L, Khanna NN, Araki T, Viskovic K et al (2020) Cardiovascular/stroke risk predictive calculators: a comparison between statistical and machine learning models. Cardiovasc Diagn Ther 10:919–938
Jamthikar A, Gupta D, Khanna NN, Saba L, Laird JR, Suri JS (2020) Cardiovascular/stroke risk prevention: a new machine learning framework integrating carotid ultrasound image-based phenotypes and its harmonics with conventional risk factors. Indian Heart J. https://doi.org/10.1016/j.ihj.2020.06.004
Jamthikar A, Gupta D, Khanna NN, Saba L, Araki T, Viskovic K et al (2019) A low-cost machine learning-based cardiovascular/stroke risk assessment system: integration of conventional factors with image phenotypes. Cardiovasc Diagn Ther 9:420–430
Viswanathan V, Jamthikar A, Gupta D, Shanu N, Puvvula A, Khanna N et al (2020) Low-cost preventive screening using carotid ultrasound in patients with diabetes. Front Biosci (Landmark Ed) 25:1132
Khanna NN, Jamthikar AD, Gupta D, Piga M, Saba L, Carcassi C et al (2019) Rheumatoid arthritis: atherosclerosis imaging and cardiovascular risk assessment using machine and deep learning-based tissue characterization. Curr Atheroscler Rep 21:7
Saba L, Jamthikar A, Gupta D, Khanna NN, Viskovic K, Suri HS et al (2019) Global perspective on carotid intima-media thickness and plaque: should the current measurement guidelines be revisited? Int Angiol J Int Union Angiol 38:451–465
Porcu M, Mannelli L, Melis M, Suri JS, Gerosa C, Cerrone G et al (2020) Carotid plaque imaging profiling in subjects with risk factors (diabetes and hypertension). Cardiovasc Diagn Ther 10:1005
Ikeda N, Gupta A, Dey N, Bose S, Shafique S, Arak T et al (2015) Improved correlation between carotid and coronary atherosclerosis SYNTAX score using automated ultrasound carotid bulb plaque IMT measurement. Ultrasound Med Biol 41:1247–1262
Mach F, Baigent C, Catapano AL, Koskinas KC, Casula M, Badimon L et al (2019) 2019 ESC/EAS Guidelines for the management of dyslipidaemias: lipid modification to reduce cardiovascular risk: the Task Force for the management of dyslipidaemias of the European Society of Cardiology (ESC) and European Atherosclerosis Society (EAS). Eur Heart J 41:111–188
Knuuti J, Wijns W, Saraste A, Capodanno D, Barbato E, Funck-Brentano C et al (2019) 2019 ESC Guidelines for the diagnosis and management of chronic coronary syndromes: the Task Force for the diagnosis and management of chronic coronary syndromes of the European Society of Cardiology (ESC). Eur Heart J 41:407–477
Viswanathan V, Jamthikar AD, Gupta D, Puvvula A, Khanna NN, Saba L et al (2020) Integration of eGFR biomarker in image-based CV/stroke risk calculator: a south Asian-Indian diabetes cohort with moderate chronic kidney disease. Int Angiol 39:290–306
Khanna NN, Jamthikar AD, Gupta D, Nicolaides A, Araki T, Saba L et al (2019) Performance evaluation of 10-year ultrasound image-based stroke/cardiovascular (CV) risk calculator by comparing against ten conventional CV risk calculators: a diabetic study. Comput Biol Med 105:125–143
Araki T, Jain PK, Suri HS, Londhe ND, Ikeda N, El-Baz A et al (2017) Stroke risk stratification and its validation using ultrasonic echolucent carotid wall plaque morphology: a machine learning paradigm. Comput Biol Med 80:77–96
Alaa AM, Bolton T, Di Angelantonio E, Rudd JH, van Der Schaar M (2019) Cardiovascular disease risk prediction using automated machine learning: a prospective study of 423,604 UK Biobank participants. PLoS One 14:e0213653
Kakadiaris IA, Vrigkas M, Yen AA, Kuznetsova T, Budoff M, Naghavi M (2018) Machine learning outperforms ACC/AHA CVD risk calculator in MESA. J Am Heart Assoc 7:e009476
Weng SF, Reps J, Kai J, Garibaldi JM, Qureshi N (2017) Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLoS Osne 12:e0174944
Suri JS (2011) Imaging based symptomatic classification and cardiovascular stroke risk score estimation, Google Patents ed
Li R, Liu W, Lin Y, Zhao H, Zhang C (2017) An ensemble multilabel classification for disease risk prediction. J Healthc Eng 2017:1–10
Kumar PR, Priya M (2014) Classification of atherosclerotic and non-atherosclerotic individuals using multiclass support vector machine. Technol Health Care 22:583–595
Tandel GS, Balestrieri A, Jujaray T, Khanna NN, Saba L, Suri JS (2020) Multiclass magnetic resonance imaging brain tumor classification using artificial intelligence paradigm. Comput Biol Med 122:103804
Ambale-Venkatesh B, Wu CO, Liu K, Hundley W, McClelland RL, Gomes AS et al (2017) Cardiovascular event prediction by machine learning: the multi-ethnic study of atherosclerosis. Circ Res. https://doi.org/10.1161/CIRCRESAHA.117.311312
Mantella LE, Colledanchise KN, Hétu M-F, Feinstein SB, Abunassar J, Johri AM (2019) Carotid intraplaque neovascularization predicts coronary artery disease and cardiovascular events. Eur Heart J Cardiovasc Imaging 20:1239–1247
Colledanchise KN, Mantella LE, Bullen M, Hétu M-F, Abunassar JG, Johri AM (2020) Combined femoral and carotid plaque burden identifies obstructive coronary artery disease in women. J Am Soc Echocardiogr 33:90–100
Herr JE, Hétu M-F, Li TY, Ewart P, Johri AM (2019) Presence of calcium-like tissue composition in carotid plaque is indicative of significant coronary artery disease in high-risk patients. J Am Soc Echocardiogr 32:633–642
Roger VL, Go AS, Lloyd-Jones DM, Adams RJ, Berry JD, Brown TM et al (2011) Heart disease and stroke statistics—2011 update: a report from the American Heart Association. Circulation 123:e18–e209
Khanna NN, Jamthikar AD, Araki T, Gupta D, Piga M, Saba L et al (2019) Nonlinear model for the carotid artery disease 10-year risk prediction by fusing conventional cardiovascular factors to carotid ultrasound image phenotypes: a Japanese diabetes cohort study. Echocardiography 36:345–361
Khanna NN, Jamthikar AD, Gupta D, Araki T, Piga M, Saba L et al (2019) Effect of carotid image-based phenotypes on cardiovascular risk calculator: AECRS1. 0. Med Biol Eng Comput 57:1553–1566
Johri AM, Calnan CM, Matangi MF, MacHaalany J, Hétu M-F (2016) Focused vascular ultrasound for the assessment of atherosclerosis: a proof-of-concept study. J Am Soc Echocardiogr 29:842–849
Johri AM, Chitty DW, Matangi M, Malik P, Mousavi P, Day A et al (2013) Can carotid bulb plaque assessment rule out significant coronary artery disease? A comparison of plaque quantification by two-and three-dimensional ultrasound. J Am Soc Echocardiogr 26:86–95
Touboul P-J, Hennerici M, Meairs S, Adams H, Amarenco P, Bornstein N et al (2012) Mannheim carotid intima-media thickness and plaque consensus (2004–2006–2011). Cerebrovasc Dis 34:290–296
Deyama J, Nakamura T, Takishima I, Fujioka D, Kawabata K-I, Obata J-E et al (2013) Contrast-enhanced ultrasound imaging of carotid plaque neovascularization is useful for identifying high-risk patients with coronary artery disease. Circ J 77:1499–1507
Members TF, Montalescot G, Sechtem U, Achenbach S, Andreotti F, Arden C et al (2013) 2013 ESC guidelines on the management of stable coronary artery disease: the Task Force on the management of stable coronary artery disease of the European Society of Cardiology. Eur Heart J 34:2949–3003
Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357
Unnikrishnan P, Kumar DK, Poosapadi Arjunan S, Kumar H, Mitchell P, Kawasaki R (2016) Development of health parameter model for risk prediction of CVD using SVM. Comput Math Methods Med 2016:1–7
Acharya UR, Mookiah MRK, Sree SV, Afonso D, Sanches J, Shafique S et al (2013) Atherosclerotic plaque tissue characterization in 2D ultrasound longitudinal carotid scans for automated classification: a paradigm for stroke risk assessment. Med Biol Eng Comput 51:513–523
Motwani M, Dey D, Berman DS, Germano G, Achenbach S, Al-Mallah MH et al (2017) Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis. Eur Heart J 38:500–507
Acharya UR, Sree SV, Molinari F, Saba L, Nicolaides A, Suri JS (2015) An automated technique for carotid far wall classification using grayscale features and wall thickness variability. J Clin Ultrasound 43:302–311
Maniruzzaman M, Rahman MJ, Al-MehediHasan M, Suri HS, Abedin MM, El-Baz A et al (2018) Accurate diabetes risk stratification using machine learning: role of missing value and outliers. J Med Syst 42:92
Saba L, Dey N, Ashour AS, Samanta S, Nath SS, Chakraborty S et al (2016) Automated stratification of liver disease in ultrasound: an online accurate feature classification paradigm. Comput Methods Programs Biomed 130:118–134
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297
Ho TK (1995) Random decision forests. In: Proceedings of the third international conference on document analysis and recognition, 1995, pp 278–282
Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp 785–794
Chen T, He T, Benesty M, Khotilovich V, Tang Y (2015) Xgboost: extreme gradient boosting. R package version 0.4-2, pp 1–4
Maniruzzaman M, Kumar N, Abedin MM, Islam MS, Suri HS, El-Baz AS et al (2017) Comparative approaches for classification of diabetes mellitus data: machine learning paradigm. Comput Methods Programs Biomed 152:23–34
Suri JS, Kathuria C, Molinari F (2010) Atherosclerosis disease management. Springer Science & Business Media, Berlin
Sanches JM, Laine AF, Suri JS (2012) Ultrasound imaging. Springer, New York
Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons C, Catapano AL et al (2016) 2016 European Guidelines on cardiovascular disease prevention in clinical practice: the Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts) Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Eur Heart J 37:2315–2381
D’agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM et al (2008) General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 117:743–753
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830
Jeong B, Cho H, Kim J, Kwon SK, Hong S, Lee C et al (2020) Comparison between statistical models and machine learning methods on classification for highly imbalanced multiclass kidney data. Diagnostics 10:415
Maniruzzaman M, Rahman MJ, Ahammed B, Abedin MM, Suri HS, Biswas M et al (2019) Statistical characterization and classification of colon microarray gene expression data using multiple machine learning paradigms. Comput Methods Programs Biomed 176:173–193
Dimitriadis SI, Liparas D (2018) How random is the random forest? Random forest algorithm on the service of structural imaging biomarkers for Alzheimer’s disease: from Alzheimer’s disease neuroimaging initiative (ADNI) database. Neural Regen Res 13:962–970
Marchese Robinson RL, Palczewska A, Palczewski J, Kidley N (2017) Comparison of the predictive performance and interpretability of random forest and linear models on benchmark data sets. J Chem Inf Model 57:1773–1792
Srivastava SK, Singh SK, Suri JS (2019) Effect of incremental feature enrichment on healthcare text classification system: a machine learning paradigm. Comput Methods Programs Biomed 172:35–51
Scariano SM, Davenport JM (1987) The effects of violations of independence assumptions in the one-way ANOVA. Am Stat 41:123–129
Chen X, Zhao P-L, Zhang J (2002) A note on ANOVA assumptions and robust analysis for a cross-over study. Stat Med 21:1377–1386
Zarkogianni K, Athanasiou M, Thanopoulou AC, Nikita KS (2018) Comparison of machine learning approaches toward assessing the risk of developing cardiovascular disease as a long-term diabetes complication. IEEE J Biomed Health Inform 22:1637–1647
Viswanathan V, Jamthikar AD, Gupta D, Puvvula A, Khanna NN, Saba L et al (2020) Does the carotid bulb offer a better 10-year CVD/stroke risk assessment compared to the common carotid artery? A 1516 ultrasound scan study. Angiology. https://doi.org/10.1177/0003319720941730
Acharya UR, Sree SV, Krishnan MMR, Krishnananda N, Ranjan S, Umesh P et al (2013) Automated classification of patients with coronary artery disease using grayscale features from left ventricle echocardiographic images. Comput Methods Programs Biomed 112:624–632
Banchhor SK, Londhe ND, Araki T, Saba L, Radeva P, Khanna NN et al (2018) Calcium detection, its quantification, and grayscale morphology-based risk stratification using machine learning in multimodality big data coronary and carotid scans: a review. Comput Biol Med 101:184–198
Shrivastava VK, Londhe ND, Sonawane RS, Suri JS (2016) Computer-aided diagnosis of psoriasis skin images with HOS, texture and color features: a first comparative study of its kind. Comput Methods Programs Biomed 126:98–109
Acharya UR, Sree SV, Ribeiro R, Krishnamurthi G, Marinho RT, Sanches J et al (2012) Data mining framework for fatty liver disease classification in ultrasound: a hybrid feature extraction paradigm. Med Phys 39:4255–4264
Acharya UR, Swapna G, Sree SV, Molinari F, Gupta S, Bardales RH et al (2014) A review on ultrasound-based thyroid cancer tissue characterization and automated classification. Technol Cancer Res Treat 13:289–301
Ikeda N, Araki T, Sugi K, Nakamura M, Deidda M, Molinari F et al (2014) Ankle-brachial index and its link to automated carotid ultrasound measurement of intima-media thickness variability in 500 Japanese coronary artery disease patients. Curr Atheroscler Rep 16:393
Molinari F, Meiburger KM, Saba L, Acharya UR, Famiglietti L, Georgiou N et al (2014) Automated carotid IMT measurement and its validation in low contrast ultrasound database of 885 patient Indian population epidemiological study: results of AtheroEdge® software. In: Saba L, Miguel Sanches J, Mendes Pedro L, Suri JS (eds) Multi-modality atherosclerosis imaging and diagnosis, ed. Springer, New York, pp 209–219
Ikeda N, Dey N, Sharma A, Gupta A, Bose S, Acharjee S et al (2017) Automated segmental-IMT measurement in thin/thick plaque with bulb presence in carotid ultrasound from multiple scanners: stroke risk assessment. Comput Methods Programs Biomed 141:73–81
Saba L, Molinari F, Meiburger KM, Acharya UR, Nicolaides A, Suri JS (2013) Inter- and intra-observer variability analysis of completely automated cIMT measurement software (AtheroEdge) and its benchmarking against commercial ultrasound scanner and expert Readers. Comput Biol Med 43:1261–1272
Puvvula A, Jamthikar AD, Gupta D, Khanna NN, Porcu M, Saba L et al (2020) Morphological carotid plaque area is associated with glomerular filtration rate: a study of South Asian Indian patients with diabetes and chronic kidney disease. Angiology 71:520–535. https://doi.org/10.1177/0003319720910660
Saba L, Banchhor SK, Araki T, Viskovic K, Londhe ND, Laird JR et al (2018) Intra-and inter-operator reproducibility of automated cloud-based carotid lumen diameter ultrasound measurement. Indian Heart J 70:649–664
Saba L, Than JC, Noor NM, Rijal OM, Kassim RM, Yunus A et al (2016) Inter-observer variability analysis of automatic lung delineation in normal and disease patients. J Med Syst 40:142
Saba L, Biswas M, Kuppili V, Godia EC, Suri HS, Edla DR et al (2019) The present and future of deep learning in radiology. Eur J Radiol 114:14–24
Saba L, Biswas M, Suri HS, Viskovic K, Laird JR, Cuadrado-Godia E et al (2019) Ultrasound-based carotid stenosis measurement and risk stratification in diabetic cohort: a deep learning paradigm. Cardiovasc Diagn Ther 9:439–461
Cuadrado-Godia E, Dwivedi P, Sharma S, Santiago AO, Gonzalez JR, Balcells M et al (2018) Cerebral small vessel disease: a review focusing on pathophysiology, biomarkers, and machine learning strategies. J Stroke 20:302
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Jamthikar, A.D., Gupta, D., Mantella, L.E. et al. Multiclass machine learning vs. conventional calculators for stroke/CVD risk assessment using carotid plaque predictors with coronary angiography scores as gold standard: a 500 participants study. Int J Cardiovasc Imaging 37, 1171–1187 (2021). https://doi.org/10.1007/s10554-020-02099-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10554-020-02099-7