Abstract
Purpose of Review
Bias in artificial intelligence (AI) models can result in unintended consequences. In cardiovascular imaging, biased AI models used in clinical practice can negatively affect patient outcomes. Biased AI models result from decisions made when training and evaluating a model. This paper is a comprehensive guide for AI development teams to understand assumptions in datasets and chosen metrics for outcome/ground truth, and how this translates to real-world performance for cardiovascular disease (CVD).
Recent Findings
CVDs are the number one cause of mortality worldwide; however, the prevalence, burden, and outcomes of CVD vary across gender and race. Several biomarkers are also shown to vary among different populations and ethnic/racial groups. Inequalities in clinical trial inclusion, clinical presentation, diagnosis, and treatment are preserved in health data that is ultimately used to train AI algorithms, leading to potential biases in model performance. Despite the notion that AI models themselves are biased, AI can also help to mitigate bias (e.g., bias auditing tools).
Summary
In this review paper, we describe in detail implicit and explicit biases in the care of cardiovascular disease that may be present in existing datasets but are not obvious to model developers. We review disparities in CVD outcomes across different genders and race groups, differences in treatment of historically marginalized groups, and disparities in clinical trials for various cardiovascular diseases and outcomes. Thereafter, we summarize some CVD AI literature that shows bias in CVD AI as well as approaches that AI is being used to mitigate CVD bias.
Similar content being viewed by others
References
Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance
Virani SS, Alonso A, Aparicio HJ, Benjamin EJ, Bittencourt MS, Callaway CW, Carson AP, Chamberlain AM, Cheng S, Delling FN, et al. Heart Disease and Stroke Statistics-2021 update: a report from the American Heart Association. Circulation. 2021;143:e254–743. https://doi.org/10.1161/CIR.0000000000000950.
Mensah GA, Roth GA, Fuster V. The global burden of cardiovascular diseases and risk factors: 2020 and beyond. J Am Coll Cardiol. 2019;74:2529–32. https://doi.org/10.1016/j.jacc.2019.10.009.
Zhu S, Gilbert M, Chetty I, Siddiqui F. The 2021 landscape of FDA-approved artificial intelligence/machine learning-enabled medical devices: an analysis of the characteristics and intended use. Int J Med Inform. 2022;165:104828. https://doi.org/10.1016/j.ijmedinf.2022.104828.
Hanneman K, Playford D, Dey D, van Assen M, Mastrodicasa D, Cook TS, et al. Value creation through artificial intelligence and cardiovascular imaging: a scientific statement from the American Heart Association. Circulation. 2024;149:e296–e311. https://doi.org/10.1161/CIR.0000000000001202.
Tat E, Bhatt DL, Rabbat MG. Addressing bias: artificial intelligence in cardiovascular medicine. Lancet Digit Health. 2020;2:e635–6. https://doi.org/10.1016/S2589-7500(20)30249-1.
Zhang K, Khosravi B, Vahdati S, Faghani S, Nugen F, Rassoulinejad-Mousavi SM, Moassefi M, Jagtap JMM, Singh Y, Rouzrokh P, et al. Mitigating bias in radiology machine learning: 2. model development. Radiol Artif Intell. 2022;4:e220010. https://doi.org/10.1148/ryai.220010.
Rouzrokh P, Khosravi B, Faghani S, Moassefi M, Vera Garcia DV, Singh Y, Zhang K, Conte GM, Erickson BJ. Mitigating bias in radiology machine learning: 1. data handling. Radiol Artif Intell. 2022;4:e210290. https://doi.org/10.1148/ryai.210290.
Faghani S, Khosravi B, Zhang K, Moassefi M, Jagtap JM, Nugen F, Vahdati S, Kuanar SP, Rassoulinejad-Mousavi SM, Singh Y, et al. Mitigating bias in radiology machine learning: 3. performance metrics. Radiol Artif Intell. 2022;4:e220061. https://doi.org/10.1148/ryai.220061.
Chong LR, Tsai KT, Lee LL, Foo SG, Chang PC. Artificial intelligence predictive analytics in the management of outpatient MRI appointment no-shows. AJR Am J Roentgenol. 2020;215:1155–62. https://doi.org/10.2214/AJR.19.22594.
Winkel DJ, Suryanarayana VR, Ali AM, Gorich J, Buss SJ, Mendoza A, Schwemmer C, Sharma P, Schoepf UJ, Rapaka S. Deep learning for vessel-specific coronary artery calcium scoring: validation on a multi-centre dataset. Eur Heart J Cardiovasc Imaging. 2022;23:846–54. https://doi.org/10.1093/ehjci/jeab119.
Tzolos E, Williams MC, McElhinney P, Lin A, Grodecki K, Flores Tomasino G, Cadet S, Kwiecinski J, Doris M, Adamson PD, et al. Pericoronary adipose tissue attenuation, low-attenuation plaque burden, and 5-year risk of myocardial infarction. JACC Cardiovasc Imaging. 2022. https://doi.org/10.1016/j.jcmg.2022.02.004.
Griffin WF, Choi AD, Riess JS, Marques H, Chang HJ, Choi JH, Doh JH, Her AY, Koo BK, Nam CW, et al. AI evaluation of stenosis on coronary CT angiography, comparison with quantitative coronary angiography and fractional flow reserve: a CREDENCE trial substudy. JACC Cardiovasc Imaging. 2022. https://doi.org/10.1016/j.jcmg.2021.10.020.
Bhuva AN, Bai W, Lau C, Davies RH, Ye Y, Bulluck H, McAlindon E, Culotta V, Swoboda PP, Captur G, et al. A multicenter, scan-rescan, human and machine learning CMR study to test generalizability and precision in imaging biomarker analysis. Circ Cardiovasc Imaging. 2019;12:e009214. https://doi.org/10.1161/CIRCIMAGING.119.009214.
Motwani M, Dey D, Berman DS, Germano G, Achenbach S, Al-Mallah MH, Andreini D, Budoff MJ, Cademartiri F, Callister TQ, et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis. Eur Heart J. 2017;38:500–7. https://doi.org/10.1093/eurheartj/ehw188.
van Rosendael AR, Maliakal G, Kolli KK, Beecy A, Al’Aref SJ, Dwivedi A, Singh G, Panday M, Kumar A, Ma X, et al. Maximization of the usage of coronary CTA derived plaque information using a machine learning based algorithm to improve risk stratification; insights from the CONFIRM registry. J Cardiovasc Comput Tomogr. 2018;12:204–9. https://doi.org/10.1016/j.jcct.2018.04.011.
Churchwell K, Elkind MSV, Benjamin RM, Carson AP, Chang EK, Lawrence W, Mills A, Odom TM, Rodriguez CJ, Rodriguez F, et al. Call to action: structural racism as a fundamental driver of health disparities: a presidential advisory from the American Heart Association. Circulation. 2020;142:E454–68. https://doi.org/10.1161/Cir.0000000000000936.
Javed Z, Haisum Maqsood M, Yahya T, Amin Z, Acquah I, Valero-Elizondo J, Andrieni J, Dubey P, Jackson RK, Daffin MA, et al. Race, racism, and cardiovascular health: applying a social determinants of health framework to racial/ethnic disparities in cardiovascular disease. Circ Cardiovasc Qual Outcomes. 2022;15: e007917. https://doi.org/10.1161/CIRCOUTCOMES.121.007917.
Kyalwazi AN, Loccoh EC, Brewer LC, Ofili EO, Xu JM, Song Y, Maddoxe KEJ, Yeh RW, Wadhera RK. Disparities in cardiovascular mortality between Black and White adults in the United States, 1999 to 2019. Circulation. 2022;146:211–28. https://doi.org/10.1161/Circulationaha.122.060199.
Centers for Disease Control and Prevention. CDC health disparities and inequalities report — United States, 2013. MMWR. 2013;62(Suppl 3):157–60.
Glynn P, Lloyd-Jones DM, Feinstein MJ, Carnethon M, Khan SS. Disparities in cardiovascular mortality related to heart failure in the United States. J Am Coll Cardiol. 2019;73(18):2354–5. https://doi.org/10.1016/j.jacc.2019.02.042.
Mazimba S, Peterson PN. JAHA spotlight on racial and ethnic disparities in cardiovascular disease. J Am Heart Assoc. 2021;10(17):e023650. https://doi.org/10.1161/JAHA.121.023650.
Hannan EL, Racz MJ, Walford G, Jacobs AK, Stamato NJ, Gesten F, Berger PB, Sharma S, King SB. Disparities in the use of drug-eluting coronary stents by race, ethnicity, payer, and hospital. Can J Cardiol. 2016;32: 987.e25 https://doi.org/10.1016/j.cjca.2016.01.012
Fang J, Yang QH, Ayala C, Loustalot F. Disparities in access to care among US adults with self-reported hypertension. Am J Hypertens. 2014;27:1377–86. https://doi.org/10.1093/ajh/hpu061.
Sullivan S, Hammadah M, Wilmot K, Ramadan R, Pearce BD, Shah A, Kaseer B, Gafeer MM, Lima BB, Kim JH, et al. Young women with coronary artery disease exhibit higher concentrations of interleukin-6 at baseline and in response to mental stress. J Am Heart Assoc. 2018;7:e010329. https://doi.org/10.1161/JAHA.118.010329.
Kershaw KN, Lewis TT, Diez Roux AV, Jenny NS, Liu K, Penedo FJ, et al. Self-reported experiences of discrimination and inflammation among men and women: the multi-ethnic study of atherosclerosis. Health Psychol. 2016;35(4):343–50. https://doi.org/10.1037/hea0000331.
Liaudat CC, Vaucher P, De Francesco T, Jaunin-Stalder N, Herzig L, Verdon F, Favrat B, Locatelli I, Clair C. Sex/gender bias in the management of chest pain in ambulatory care. Womens Health. 2018;14:1745506518805641 https://doi.org/10.1177/1745506518805641 (Artn)
Shahian DM, Jacobs JP, Badhwar V, Kurlansky PA, Furnary AP, Cleveland JC, Lobdell KW, Vassileva C, von Ballmoos MCW, Thourani VH, et al. The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 1-background, design considerations, and model development. Ann Thorac Surg. 2018;105:1411–8. https://doi.org/10.1016/j.athoracsur.2018.03.002.
Sambola A, Del Blanco BG, Kunadian V, Vogel B, Chieffo A, Vidal M, Ratcovich H, Botti G, Wilkinson C, Mehran R. Sex-based differences in percutaneous coronary intervention outcomes in patients with ischaemic heart disease. Eur Cardiol. 2023;18:e06. https://doi.org/10.15420/ecr.2022.24.
Daugherty SL, Magid DJ. Do sex differences exist in patient preferences for cardiovascular testing? Ann Emerg Med. 2011;57:561–2. https://doi.org/10.1016/j.annemergmed.2011.01.010.
Lee P, Le Saux M, Siegel R, Goyal M, Chen C, Ma Y, Meltzer AC. Racial and ethnic disparities in the management of acute pain in US emergency departments: meta-analysis and systematic review. Am J Emerg Med. 2019;37:1770–7. https://doi.org/10.1016/j.ajem.2019.06.014.
Hsia RY, Sarkar N, Shen YC. Impact of ambulance diversion: Black patients with acute myocardial infarction had higher mortality than Whites. Health Aff (Millwood). 2017;36:1070–7. https://doi.org/10.1377/hlthaff.2016.0925.
Graham G. Racial and ethnic differences in acute coronary syndrome and myocardial infarction within the United States: from demographics to outcomes. Clin Cardiol. 2016;39:299–306. https://doi.org/10.1002/clc.22524.
National Healthcare Quality and Disparities Report. Content last reviewed July 2023. Rockville: Agency for Healthcare Research and Quality; 2019. https://www.ahrq.gov/research/findings/nhqrdr/nhqdr19/index.html.
Popescu I, Huckfeldt P, Pane JD, Escarce JJ. Contributions of geography and nongeographic factors to the White-Black gap in hospital quality for coronary heart disease: a decomposition analysis. J Am Heart Assoc. 2019;8: e011964 https://doi.org/10.1161/JAHA.119.011964 (ARTN)
Johnson A. Understanding why Black patients have worse coronary heart disease outcomes: does the answer lie in knowing where patients seek care? J Am Heart Assoc. 2019;8:e014706. https://doi.org/10.1161/jaha.119.014706.
Norris CM, Yip CYY, Nerenberg KA, Clavel MA, Pacheco C, Foulds HJA, Hardy M, Gonsalves CA, Jaffer S, Parry M, et al. State of the science in women’s cardiovascular disease: a Canadian perspective on the influence of sex and gender. J Am Heart Assoc. 2020;9:e015634. https://doi.org/10.1161/JAHA.119.015634.
Geller SE, Koch AR, Roesch P, Filut A, Hallgren E, Carnes M. The more things change, the more they stay the same: a study to evaluate compliance with inclusion and assessment of women and minorities in randomized controlled trials. Acad Med. 2018;93:630–5. https://doi.org/10.1097/Acm.0000000000002027.
Jin X, Chandramouli C, Allocco B, Gong E, Lam CSP, Yan LL. Women’s participation in cardiovascular clinical trials from 2010 to 2017. Circulation. 2020;141(7):540–8. https://doi.org/10.1161/Circulationaha.119.043594.
Lolic M, Araojo R, Okeke M, Temple RUS. racial and ethnic participation in global clinical trials by therapeutic areas. J Clin Pharm Ther. 2021;46:1576–81. https://doi.org/10.1111/jcpt.13532.
Lewsey SC, Breathett K. Racial and ethnic disparities in heart failure: current state and future directions. Curr Opin Cardiol. 2021;36(3):320–8. https://doi.org/10.1097/Hco.0000000000000855.
Ford C, Xie CX, Low A, Rajakariar K, Koshy AN, Sajeev JK, et al. Comparison of 2 smart watch algorithms for detection of atrial fibrillation and the benefit of clinician interpretation: SMART WARS Study. JACC Clin Electrophysiol. 2022;8(6):782–91. https://doi.org/10.1016/j.jacep.2022.02.013.
Bent B, Goldstein BA, Kibbe WA, Dunn JP. Investigating sources of inaccuracy in wearable optical heart rate sensors. NPJ Digit Med. 2020;3:18. https://doi.org/10.1038/s41746-020-0226-6.
Colvonen PJ. Response to: investigating sources of inaccuracy in wearable optical heart rate sensors. Npj Digit Med. 2021;4:38 https://doi.org/10.1038/s41746-021-00408-5 (ARTN)
Breathett K, Yee E, Pool N, Hebdon M, Crist JD, Yee RH, Knapp SM, Solola S, Luy L, Herrera-Theut K, et al. Association of gender and race with allocation of advanced heart failure therapies.
• Gichoya JW, Banerjee I, Bhimireddy AR, Burns JL, Celi LA, Chen LC, Correa R, Dullerud N, Ghassemi M, Huang SC, et al. AI recognition of patient race in medical imaging: a modelling study. Lancet Digit Health. 2022;4:e406–14. https://doi.org/10.1016/S2589-7500(22)00063-2. (This study shows that race can be detected from unknown signals in medical images, potentially biasing AI evaluations).
McClelland RL, Chung HJ, Detrano R, Post W, Kronmal RA. Distribution of coronary artery calcium by race, gender, and age - results from the multi-ethnic study of atherosclerosis (MESA). Circulation. 2006;113:30–7. https://doi.org/10.1161/Circulationaha.105.580696.
Burrell S, MacDonald A. Artifacts and pitfalls in myocardial perfusion imaging. J Nucl Med Technol. 2006;34:193–211(quiz 212–194)
Taqueti VR, Dorbala S, Wolinsky D, Abbott B, Heller GV, Bateman TM, Mieres JH, Phillips LM, Wenger NK, Shaw LJ. Myocardial perfusion imaging in women for the evaluation of stable ischemic heart disease-state-of-the-evidence and clinical recommendations.
Aggarwal NR, Bond RM, Mieres JH. The role of imaging in women with ischemic heart disease.
Betancur J, Otaki Y, Motwani M, Fish MB, Lemley M, Dey D, Gransar H, Tamarappoo B, Germano G, Sharir T, et al. Prognostic value of combined clinical and myocardial perfusion imaging data using machine learning. JACC Cardiovasc Imaging. 2018;11:1000–9. https://doi.org/10.1016/j.jcmg.2017.07.024.
•• Puyol-Anton E, Ruijsink B, Mariscal Harana J, Piechnik SK, Neubauer S, Petersen SE, Razavi R, Chowienczyk P, King AP. Fairness in cardiac magnetic resonance imaging: assessing sex and racial bias in deep learning-based segmentation. Front Cardiovasc Med. 2022;9:859310. https://doi.org/10.3389/fcvm.2022.859310. (This study shows that AI applications, based on the training data, can have decreased performance in underrepresented populations).
Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366:447–53. https://doi.org/10.1126/science.aax2342.
Stray J. The AI learns to lie to please you: preventing biased feedback loops in machine-assisted intelligence analysis. Analytics. 2023;2:350–8.
Lenert MC, Matheny ME, Walsh CG. Prognostic models will be victims of their own success, unless. J Am Med Inform Assn. 2019;26:1645–50. https://doi.org/10.1093/jamia/ocz145.
Drukker K, Chen W, Gichoya J, Gruszauskas N, Kalpathy-Cramer J, Koyejo S, Myers K, Sa RC, Sahiner B, Whitney H, et al. Toward fairness in artificial intelligence for medical image analysis: identification and mitigation of potential biases in the roadmap from data collection to model deployment. J Med Imaging (Bellingham). 2023;10:061104. https://doi.org/10.1117/1.JMI.10.6.061104.
• Miller RJH, Singh A, Otaki Y, Tamarappoo BK, Kavanagh P, Parekh T, Hu LH, Gransar H, Sharir T, Einstein AJ, et al. Mitigating bias in deep learning for diagnosis of coronary artery disease from myocardial perfusion SPECT images. Eur J Nucl Med Mol Imaging. 2023;50:387–97. https://doi.org/10.1007/s00259-022-05972-w. (This study demonstrated that AI can also be used to mitigate bias in imaging studies).
Chen Y, Clayton EW, Novak LL, Anders S, Malin B. Human-centered design to address biases in artificial intelligence. J Med Internet Res. 2023;25:e43251. https://doi.org/10.2196/43251. (ARTN)
Vyas DA, Eisenstein LG, Jones DS. Hidden in plain sight - reconsidering the use of race correction in clinical algorithms. N Engl J Med. 2020;383:874–82. https://doi.org/10.1056/NEJMms2004740.
Suri JS, Bhagawati M, Paul S, Protogeron A, Sfikakis PP, Kitas GD, Khanna NN, Ruzsa Z, Sharma AM, Saxena S, et al. Understanding the bias in machine learning systems for cardiovascular disease risk assessment: the first of its kind review. Comput Biol Med. 2022;142:105204. https://doi.org/10.1016/j.compbiomed.2021.105204.
Calaprice-Whitty D, Galil K, Salloum W, Zariv A, Jimenez B. Improving clinical trial participant prescreening with artificial intelligence (AI): a comparison of the results of ai-assisted vs standard methods in 3 oncology trials. Ther Innov Regul Sci. 2020;54:69–74. https://doi.org/10.1007/s43441-019-00030-4.
Administration USFaD. Proposed regulatory framework for modifications to artificial intelligence / machine learning (AI/ML)-based software as a medical device (SaMD) In: https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/SoftwareasaMedicalDevice/UCM635052.pdf.; 2019.
Geis JR, Brady AP, Wu CC, Spencer J, Ranschaert E, Jaremko JL, Langer SG, Borondy Kitts A, Birch J, Shields WF, et al. Ethics of artificial intelligence in radiology: summary of the joint European and North American multisociety statement. Radiology. 2019;293:436–40. https://doi.org/10.1148/radiol.2019191586.
Perez MV, Mahaffey KW, Hedlin H, Rumsfeld JS, Garcia A, Ferris T, Balasubramanian V, Russo AM, Rajmane A, Cheung L, et al. Large-scale assessment of a smartwatch to identify atrial fibrillation. N Engl J Med. 2019;381:1909–17. https://doi.org/10.1056/NEJMoa1901183.
Funding
Marly van Assen receives research funding from Siemens Healthineers. Judy Gichoya is a 2022 Robert Wood Johnson Foundation Harold Amos Medical Faculty Development Program Award and declares support from RSNA Health Disparities grant (#EIHD2204), Lacuna Fund, and NIH (NIBIB) MIDRC grant under contracts 75N92020C00008 and 75N92020C00021.
Author information
Authors and Affiliations
Contributions
MvA, GG, and JG wrote the main manuscript text; AB, JN, and HT provided information and references on additional topics. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing Interests
Marly van Assen receives research funding from Siemens Healthineers. Judy Gichoya is a 2022 Robert Wood Johnson Foundation Harold Amos Medical Faculty Development Program Award and declares support from RSNA Health Disparities grant (#EIHD2204), Lacuna Fund, and NIH (NIBIB) MIDRC grant under contracts 75N92020C00008 and 75N92020C00021.
Human and Animal Rights and Informed Consent
This article does not contain any studies with human or animal subjects performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
van Assen, M., Beecy, A., Gershon, G. et al. Implications of Bias in Artificial Intelligence: Considerations for Cardiovascular Imaging. Curr Atheroscler Rep 26, 91–102 (2024). https://doi.org/10.1007/s11883-024-01190-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11883-024-01190-x