Abstract
Health Artificial Intelligence (AI) has the potential to improve health care, but at the same time, raises many ethical challenges. Within the field of health AI ethics, the solutions to the questions posed by ethical issues such as informed consent, bias, safety, transparency, patient privacy, and allocation are complex and difficult to navigate. The increasing amount of data, market forces, and changing landscape of health care suggest that medical students may be faced with a workplace in which understanding how to safely and effectively interact with health AIs will be essential. Here we argue that there is a need to teach health AI ethics in medical schools. Real events in health AI already pose ethical challenges to the medical community. We discuss key ethical issues requiring medical school education and suggest that case studies based on recent real-life examples are useful tools to teach the ethical issues raised by health AIs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adamson, A. S., & Smith, A. (2018). Machine learning and health care disparities in dermatology. JAMA Dermatology, 154, 1247–1248.
AMA (American Medical Association). (2018). Report of the Council on Long Range Planning and Development. Retrieved from https://www.ama-assn.org/system/files/2018-11/a18-clrpd-reports.pdf
AMA (American Medical Association). (2019). AMA Adopt Policy, Integrate Augmented Intelligence in Physician Training. Retrieved from https://www.ama-assn.org/press-center/press-releases/ama-adopt-policy-integrate-augmented-intelligence-physician-training
Babic, B., Gerke, S., Evgeniou, T., & Cohen, I. G. (2019). Algorithms on regulatory lockdown in medicine. Science, 366, 1202–1204.
Babic, B., Cohen, I. G., Evgeniou, T., Gerke, S., & Trichakis, N. (2020). Can AI fairly decide Who gets an organ transplant? HBR. Retrieved from https://hbr.org/2020/12/can-ai-fairly-decide-who-gets-an-organ-transplant.
Beauchamp, T. L., & Childress, J. F. (2012). Principles of biomedical ethics. Oxford University Press.
Becker, J. (2020). Insufficient Protections for Health Data Privacy: Lessons from Dinerstein v. Google. Retrieved from https://blog.petrieflom.law.harvard.edu/2020/09/28/dinerstein-google-health-data-privacy
Cohen, I. G. (2020). Informed consent and medical artificial intelligence: What to tell the patient? Georgetown Law Journal, 108, 1425–1469.
Cohen, I. G., & Mello, M. M. (2019). Big data, big tech, and protecting patient privacy. JAMA, 322, 1141–1142.
Daniel, G., Sharma, I., Silcox, C., & Wright, M. B. (2019). Current State and Near-Term Priorities for AI-Enabled Diagnostic Support Software in Health Care. Retrieved from https://healthpolicy.duke.edu/sites/default/files/atoms/files/dukemargolisaienableddxss.pdf.
Digital Diagnostics (2020). Retrieved from https://dxs.ai
Dinerstein v. Google. (2019). No. 1:19-cv-04311.
FDA (U.S. Food and Drug Administration). (2017). K163253. Retrieved from https://www.accessdata.fda.gov/cdrh_docs/pdf16/K163253.pdf
FDA (U.S. Food and Drug Administration). (2018a). De Novo Classification Request for IDx-DR. Retrieved from https://www.accessdata.fda.gov/cdrh_docs/reviews/DEN180001.pdf
FDA (U.S. Food and Drug Administration. (2018b). FDA Permits Marketing of Artificial Intelligence Algorithm for Aiding Providers in Detecting Wrist Fractures. Retrieved from: https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-algorithm-aiding-providers-detecting-wrist-fractures
FDA (U.S. Food and Drug Administration). (2018c). FDA Permits Marketing of Clinical Decision Support Software for Alerting Providers of a Potential Stroke in Patients. Retrieved from: https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-clinical-decision-support-software-alerting-providers-potential-stroke
FDA (U.S. Food and Drug Administration). (2018d). Step 3: Pathway to Approval. Retrieved from https://www.fda.gov/patients/device-development-process/step-3-pathway-approval
FDA (U.S. Food and Drug Administration). (2019a). K190815. Retrieved from https://www.accessdata.fda.gov/cdrh_docs/pdf19/K190815.pdf
FDA (U.S. Food and Drug Administration). (2019b). Developing a Software Precertification Program: A Working Model, v1.0 – 2019. Retrieved from https://www.fda.gov/media/119722/download
FDA (U.S. Food and Drug Administration). (2019c). Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD). Retrieved from https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf
FDA (U.S. Food and Drug Administration). (2021). Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. https://www.fda.gov/media/145022/download.
Fieser, J. (1999). Metaethics, normative ethics, and applied ethics: Contemporary and historical readings. Wadsworth Publishing.
Gerke, S., Minssen, T., Yu, H., & Cohen, I. G. (2019). Ethical and legal issues of ingestible electronic sensors. Nature Electronics, 2, 329–234.
Gerke, S., Babic, B., Evgeniou, T., & Cohen, I. G. (2020a). The need for a system view to regulate artificial intelligence/machine learning-based software as medical device. NPJ Digital Medicine, 3, 53.
Gerke, S., Yeung, S., & Cohen, I. G. (2020b). Ethical and legal aspects of ambient intelligence in hospital. JAMA, 323, 601–602.
Gerke, S., Minssen, T., & Cohen, I. G. (2020c). Ethical and legal challenges of artificial intelligence-driven health care. In A. Bohr & K. Memarzadeh (Eds.), Artificial intelligence in healthcare (pp. 295–336). Elsevier Inc.
Ho, A. (2019). Deep ethical learning: Taking the interplay of human and artificial intelligence seriously. Hastings Center Report, 49, 36–39.
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. (2018). Artificial intelligence in radiology. Nature Reviews Cancer, 18, 500–510.
Huber, M., Knottnerus, J. A., Green, L., van der Horst, H., Jadad, A. R., Kromhout, D., Leonard, B., Lorig, K., Loureiro, M. I., van der Meer, J. W., Schnabel, P., Smith, R., van Weel, C., & Smid, H. (2011). How should we define health? BMJ, 343, d4163.
IMDRF (International Medical Device Regulators Forum). (2013). Software as a Medical Device (SaMD): Key Definitions. Retrieved from http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf
Kolachalama, V. B., & Garg, P. S. (2018). Machine learning and medical education. NPJ Digital Medicine, 1, 54.
Kundu, M., Nasipuri, M., & Basu, D. K. (2000). Knowledge-based ECG interpretation: A critical review. Pattern Recognition, 33, 351–373.
Lecher, C. (2018). What Happens When an Algorithm Cuts Your Health Care. The Verge. Retrieved from https://www.theverge.com/2018/3/21/17144260/healthcare-medicaid-algorithm-arkansas-cerebral-palsy
Lipton, Z.C. (2016). The Myths of Model Interpretability. ICML Workshop on Human Interpretability in Machine Learning. Retrieved from https://arxiv.org/abs/1606.03490
London, A. J. (2019). Artificial intelligence and black-box medical decisions: accuracy versus explainability. Hastings Center Report, 49, 15–21.
Markets and Markets. (2020). Artificial Intelligence in Healthcare Market with Covid-19 Impact Analysis by Offering (Hardware, Software, Services), Technology (Machine Learning, NLP, Context-Aware Computing, Computer Vision), End-Use Application, End User and Region - Global Forecast to 2026. Retrieved from https://www.marketsandmarkets.com/Market-Reports/artificial-intelligence-healthcare-market-54679303.html
McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the Dartmouth summer research project on artificial intelligence, August 31, 1955. AI Magazine, 27, 12–14.
Motwani, M., Dey, D., Berman, D. S., Germano, G., Achenbach, S., Al-Mallah, M. H., Andreini, D., Budoff, M. J., Cademartiri, F., Callister, T. Q., Chang, H. J., Chinnaiyan, K., Chow, B. J. W., Cury, R. C., Delago, A., Gomez, M., Gransar, H., Hadamitzky, M., Hausleiter, J., … Slomka, P. J. (2017). Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: A 5-year multicentre prospective registry analysis. European Heart Journal, 38, 500–507.
Murphy, K. P. (2012). Machine learning: a probabilistic perspective. MIT Press.
Nabi, J. (2018). How bioethics can shape artificial intelligence and machine learning. Hastings Center Report., 48, 10–13.
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366, 447–453.
Olsen, L. A., Aisner, D., & McGinnis, J. M. (2007). The learning healthcare system: Workshop summary. National Academies Press.
Paranjape, K., Schinkel, M., Panday, R. N., Car, J., & Nanayakkara, P. (2019). Introducing artificial intelligence training in medical education. JMIR Medical Education, 5, e16048.
Price, W. N., & Cohen, I. G. (2019). Privacy in the age of medical big data. Nature Medicine, 25, 37–43.
Price, W. N., Gerke, S., & Cohen, I. G. (2019). Potential liability for physicians using artificial intelligence. JAMA, 322, 1765–1766.
Price, W. N., Gerke S., & Cohen, I. G. (2021). How much can potential jurors tell us about liability for medical artificial intelligence? Journal of Nuclear Medicine, 62, 15–16.
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., Sundberg, P., Yee, H., Zhang, K., Zhang, Y., Flores, G., Duggan, G. E., Irvine, J., Le, Q., Litsch, K., … Dean, J. (2018). Scalable and accurate deep learning with electronic health records. NPJ Digital Medicine, 1, 18.
Reinsel, D., Gantz, J., & Rydning, J. (2018). The Digitization of the World From Edge to Core. Retrieved from https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf.
Ross, C. (2021). As the FDA Clears a Flood of AI Tools, Missing Data Raise Troubling Questions on Safety and Fairness. Retrieved from https://www.statnews.com/2021/02/03/fda-clearances-artificial-intelligence-data.
Shouval, R., Hadanny, A., Shlomo, N., Iakobishvili, Z., Unger, R., Zahger, D., Alcalai, R., Atar, S., Gottlieb, S., Matetzky, S., Goldenberg, I., & Beigel, R. (2017). Machine learning for prediction of 30-day mortality after ST elevation myocardial infraction: An Acute Coronary Syndrome Israeli Survey data mining study. International Journal of Cardiology, 246, 7–13.
The Economist. (2017). The World’s Most Valuable Resource is No Longer Oil, But Data. Retrieved from https://www.economist.com/leaders/2017/05/06/the-worlds-most-valuable-resource-is-no-longer-oil-but-data
U.S. Federal Food, Drug, and Cosmetic Act, as amended through P.L. 116–260, enacted December 27, 2020, s. 520(o)(1).
Vane, J. R., & Botting, R. M. (2003). The mechanism of action of aspirin. Thrombosis Research, 110, 255–258.
Villines, Z. (2019). What is Acral Lentiginous Melanoma? MedicalNewsToday. Retrieved from https://www.medicalnewstoday.com/articles/320223
Wartman, S. A., & Combs, C. D. (2018). Medical education must move from the information age to the age of artificial intelligence. Academic Medicine, 93, 1107–1109.
Wartman, S. A., & Combs, C. D. (2019). Reimagining medical education in the age of AI. AMA Journal of Ethics, 21, 146–152.
Weng, S. F., Vaz, L., Qureshi, N., & Kai, J. (2019). Prediction of premature all-cause mortality: A prospective general population cohort study comparing machine-learning and standard epidemiological approaches. PLoS ONE, 14, 3.
Wiens, J., Price, W. N., & Sjoding, M. W. (2020). Diagnosing bias in data-driven algorithms for healthcare. Nature Medicine, 26, 25–26.
Wood, M. (2017). UChicago Medicine Collaborates with Google to Use Machine Learning for Better Health Care. Retrieved from https://www.uchicagomedicine.org/forefront/research-and-discoveries-articles/2017/may/uchicago-medicine-collaborates-with-google-to-use-machine-learning-for-better-health-care
Yu, K. H., Beam, A. L., & Kohane, I. S. (2018). Artificial intelligence in healthcare. Nature Biomedical Engineering, 2, 719–731.
Acknowledgements
We thank I. Glenn Cohen for valuable comments on an earlier version of this manuscript.
Funding
GK was supported by the 2019 Ontario Medical Student Association (OMSA) Medical Student Education Research Grant. S.G. was supported by a grant from the Collaborative Research Program for Biomedical Innovation Law, a scientifically independent collaborative research program supported by a Novo Nordisk Foundation Grant (NNF17SA0027784). S.G. received funding from the German Federal Ministry of Education and Research (BMBF), from April 1, 2016, to March 31, 2018, outside the submitted work.
Author information
Authors and Affiliations
Contributions
GK and SG contributed equally to the analysis and drafting of the paper. SG supervised the work.
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Katznelson, G., Gerke, S. The need for health AI ethics in medical school education. Adv in Health Sci Educ 26, 1447–1458 (2021). https://doi.org/10.1007/s10459-021-10040-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-021-10040-3