Abstract
Objectives
To evaluate the potential of a novel system using outlier detection screening algorithms and to identify medication related risks in an inpatient setting.
Methods
In the first phase of the study, we evaluated the transferability of models refined at another medical center using a different electronic medical record system (EMR) on 3 years of historical data (2017–2019), extracted from the local EMR system. Following the retrospective analysis, the system’s models were fine-tuned to the specific local practice patterns. In the second, prospective phase of the study, the system was fully integrated in the local EMR and after a short run-in period was activated live. All alerts generated by the system, in both phases, were analyzed by a clinical team of physicians and pharmacists for accuracy and clinical relevance.
Results
In the retrospective phase of the study, 226,804 medical orders were analyzed, generating a total of 2731 alerts (1.2% of medical orders). Of the alerts analyzed, 69% were clinically relevant alerts and 31% were false alerts. In the prospective phase of the study, 399 alerts were generated by the system (1.6% of medical orders). The vast majority of the alerts (72%) were considered clinically relevant, and 41% of the alerts caused a change in prescriber behavior (i.e. cancel/modify the medical order).
Conclusion
In an inpatient setting of a 600 bed computerized decision support system (CDSS) -naïve medical center, the system generated accurate and clinically valid alerts with low alert burden enabling physicians to improve daily medical practice.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of supporting data
All data is included in this manuscript.
References
Velo GP, Minuz P. Medication errors: prescribing faults and prescription errors. Br J Clin Pharmacol. 2009;67:624–8.
Weetman T, Aronson J, Maxwell S. How to reduce prescribing errors. Lancet. 2009:374:1945.
Gandhi TK, Weingart SN, Seger AC, et al. Outpatient prescribing errors and the impact of computerized prescribing. J Gen Intern Med. 2005;20:837–41.
Hoffman S, Podgurski A. Drug-drug interaction alerts: emphasizing the evidence. J Health Law Policy. 2012;5:2012–22.
Devine EB, Hansen RN, Wilson-Norton JL, et al. The impact of computerized provider order entry on medication errors in a multispecialty group practice. J Am Med Inform Assoc. 2010. 17:78–84.
Jani YH, Ghaleb MA, Marks SD, et al. Electronic prescribing reduced prescribing errors in a pediatric renal outpatient clinic. J Pediatr. 2008;152:214–8.
Kaushal R, Kern LM, Barron Y, et al. Electronic prescribing improves medication safety in community-based office practices. J Gen Intern Med. 2010.25:530–6.
Aronson JK. Medication errors: what they are, how they happen, and how to avoid them. QJM. 2009;102:513–21.
McCoy AB, Thomas EJ, Krousel-Wood M, et al. Clinical decision support alert appropriateness: a review and proposal for improvement. J Am Med Inform Assoc. 2017;24:281–87.
Rozenblum R, Rodriguez-Monguio R, Volk LA, et al. Using a machine learning system to identify and prevent medication prescribing errors: A clinical and cost analysis evaluation. Jt Comm J Qual Patient Saf. 2020;46:3–10.
Segal G, Segev A, Brom A, et al. Reducing drug prescription errors and adverse drug events by application of a probabilistic, machine-learning based clinical decision support system in an inpatient setting. J Am Med Inform Assoc. 2019;26:1560–65.
Adelman JS, Kalkut GE, Schechter CB, et al. Understanding and preventing wrong-patient electronic orders: a randomized controlled trial. J Am Med Inform Assoc. 2013;20:305–10.
Leviatan I, Oberman B, Zimlichman E, et al. Associations of physicians’ prescribing experience, work hours, and workload with prescription errors. J Am Med Inform Assoc. 2021;28:1074–80.
Syrowatka A, Song W, Amato MG, et al. Key use cases for artificial intelligence to reduce the frequency of adverse drug events: a scoping review.Lancet Digit Health. 2022;4:e137–48.
Schiff GD, Volk LA, Volodarskaya M, et al. Screening for medication errors using an outlier detection system. J Am Med Inform Assoc. 2017;24:281–87.
Straichman YZ, Kurnik D, Matok I, et al.Prescriber response to computerized drug alerts for electronic prescriptions among hospitalized patients. Int J Med Inform. 2017;107:70–75.
Carroll AE. Averting Alert Fatigue to Prevent Adverse Drug Reactions. JAMA. 2019;322:601.
Acknowledgements
Not applicable.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for profit sectors.
Author information
Authors and Affiliations
Contributions
• All authors made substantial contributions to all of the following:
(1) The conception and design of the study: Milena Tocut, Galit Mor, Gisele Zandman-Goddard, Gideon Y. Stein.
(2) Acquisition of data, or analysis and interpretation of data: Milena Tocut, Mayan Moalem, Israel Feinberg.
(3) Drafting the article or revising it critically for important intellectual content: Galit Mor Naor, Milena Tocut, Anat Engel, Gisele Zandman-Goddard, Gideon Y. Stein.
(4) Final approval of the version to be submitted: approved by all authors.
Corresponding author
Ethics declarations
Ethical Approval and Consent to participate
– according to the Declaration of Helsinki Wolfson medical center Nr. 0049-20-WOMC, this article was retrospective therefore no patient consent was needed.
Human and Animal Ethics
Not applicable.
Consent for publication
All authors approve this manuscript for publication.
Conflict of Interest
GYS reports personal fees from MedAware Ltd. during the conduct of the study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix 1 - Causes of False Alerts Produced by the System, in Phase #1
Appendix 1 - Causes of False Alerts Produced by the System, in Phase #1
-
1.
Model transferability issues: MedAware’s system is based on artificial intelligence algorithms that use medical records data from the patient to create a model for each drug (using data such as patient’s age, sex, drug indications and contraindications, etc.). When the system detects an irregularity between the treatment prescribed to the patient and his clinical profile, it creates an alert. These models are fine-tuned per each medical institution, using historical data which represents local practices. In the retrospective phase we evaluated the system’s performance prior to fine-tuning of the models.
-
2.
Mapping issues: MedAware’s system maps the hospital’s drug catalog. The medication mapping is done semi-automatically. If an error was made in the mapping of a medication, a false alert will be created by the system.
-
3.
Data issues: any error in the transfer of data or its decipherment by MedAware software. In order to build a clinical profile of a patient, the system receives data from the local EMR including diagnoses, instructions, demographic details, blood tests, vital signs (blood pressure, pulse). In some cases, the data is not transmitted in full due to a glitch. In other cases, the data is transmitted properly, but is not received properly by the system. There are several places from where data is extracted from the local EMR. At times, this data will be extracted only from one source and not from all the sources available
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Naor, G.M., Tocut, M., Moalem, M. et al. Screening for Medication Errors and Adverse Events Using Outlier Detection Screening Algorithms in an Inpatient Setting. J Med Syst 46, 88 (2022). https://doi.org/10.1007/s10916-022-01864-6
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-022-01864-6