Drug Safety

, Volume 34, Issue 3, pp 233–242 | Cite as

A Computerized Adverse Drug Event Alerting System Using Clinical Rules

A Retrospective and Prospective Comparison with Conventional Medication Surveillance in the Netherlands
  • Mirjam K. Rommers
  • Irene M. Teepe-Twiss
  • Henk-Jan Guchelaar
Original Research Article


Background: Adverse drug events (ADEs) are an important problem in hospital practice. Computerized physician order entry (CPOE) and clinical decision support systems (CDSS) are useful tools in the prevention of ADEs. In the Netherlands there are some basic CDSS within CPOE systems, but there is not much experience with sophisticated systems. We have recently developed a more advanced CDSS, a computerized adverse drug event alerting system (ADEAS).

Objective: The aim of the study was to compare the newly developed ADEAS, which uses a set of clinical rules, with the conventional medication surveillance, a basic CDSS within a CPOE, to assess its additional value in detecting patients with a potential ADE.

Setting: Leiden University Medical Center (LUMC), a university hospital in Leiden, the Netherlands.

Design: Two studies were carried out; one retrospective and one prospective.The retrospective comparison of ADEAS with conventional medicationsurveillance was conducted on all patients admitted to the hospital (exceptintensive care unit patients) during a 1-month period (15 November-15 December2006). A prospective comparison of both systems was performedduring a 6-month period (May–October 2007) on one general internal medicineward.

Measurements: The endpoint was the total number of alerts and content of alerts generated by both methods. In the prospective study we also focused on the number of unique alerts and interventions by the hospital pharmacist following the alerts.

Results: In the retrospective study, ADEAS generated 2010 alerts compared with 2322 generated by the conventional method. In the prospective study, 248 and 177 alerts were generated by ADEAS and the conventional method, respectively. The number of unique alerts was 85 (of which 72 were considered true positive alerts) and 136, respectively. The hospital pharmacist made 14 (19.4%) interventions following a true positive alert with ADEAS and 5 (3.7%) with the conventional method. The contents of alerts generated by ADEAS were different to the safety alerts generated by conventional medication surveillance. The conventional medication surveillance generated safety alerts regarding drug-drug interactions and drug-overdosing. ADEAS generated alerts regarding declined renal function or other laboratory abnormalities and absence of essential concurrent medication.

Conclusions: Compared with our conventional medication surveillance, the computerized alert system ADEAS selected different patients at risk for an ADE. This makes ADEAS in our hospital of additional value to the hospital pharmacist as a suitable tool in reducing the number of preventable ADEs.


Positive Predictive Value Clinical Decision Support System Hospital Information System Computerize Physician Order Entry Hospital Pharmacist 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This study is financially supported by the Dutch Society of Hospital Pharmacists (NVZA), the Dutch Society of Hospital Physicians (OMS) and the Dutch Ministry of Health, Welfare and Sports. The authors have no conflicts of interest to declare that are directly relevant to the content of this study.


  1. 1.
    Krahenbuhl-Melcher A, Schlienger R, Lampert M, et al. Drug-related problems in hospitals: a review of the recent literature. Drug Saf 2007; 30(5): 379–407PubMedCrossRefGoogle Scholar
  2. 2.
    Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003 Jun; 163(12): 1409–16PubMedCrossRefGoogle Scholar
  3. 3.
    Rommers MK, Teepe-Twiss IM, Guchelaar HJ. Preventing adverse drug events in hospital practice: an overview. Pharmacoepidemiol Drug Saf 2007 Oct; 16(10): 1129–35PubMedCrossRefGoogle Scholar
  4. 4.
    Classen DC, Pestotnik SL, Evans RS, et al. Computerized surveillance of adverse drug events in hospital patients. JAMA 1991 Nov; 266(20): 2847–51PubMedCrossRefGoogle Scholar
  5. 5.
    Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998 May; 5(3): 305–14PubMedCrossRefGoogle Scholar
  6. 6.
    Raschke RA, Gollihare B, Wunderlich TA, et al. A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital. JAMA 1998 Oct; 280(15): 1317–20PubMedCrossRefGoogle Scholar
  7. 7.
    Silverman JB, Stapinski CD, Huber C, et al. Computer-based system for preventing adverse drug events. Am J Health Syst Pharm 2004 Aug; 61(15): 1599–603PubMedGoogle Scholar
  8. 8.
    Ammenwerth E, Schnell-Inderst P, Machan C, et al. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008 Sep; 15(5): 585–600PubMedCrossRefGoogle Scholar
  9. 9.
    Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006 May; 144(10): 742–52PubMedGoogle Scholar
  10. 10.
    Eslami S, de Keizer NF, bu-Hanna A. The impact of computerized physician medication order entry in hospitalized patients: a systematic review. Int J Med Inform 2008 Jun; 77(6): 365–76PubMedCrossRefGoogle Scholar
  11. 11.
    Shamliyan TA, Duval S, Du J, et al. Just what the doctor ordered: review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res 2008 Feb; 43 (1 Pt 1): 32–53PubMedCrossRefGoogle Scholar
  12. 12.
    Wolfstadt JI, Gurwitz JH, Field TS, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008 Apr; 23(4): 451–8PubMedCrossRefGoogle Scholar
  13. 13.
    Kalmeijer MD, Holtzer W, van Dongen R, et al. Implementation of a computerized physician medication order entry system at the Academic Medical Centre in Amsterdam. Pharm World Sci 2003 Jun; 25(3): 88–93PubMedCrossRefGoogle Scholar
  14. 14.
    Guchelaar HJ, Kalmeijer MD. The potential role of computerisation and information technology in improving prescribing in hospitals. Pharm World Sci 2003 Jun; 25(3): 83–7PubMedCrossRefGoogle Scholar
  15. 15.
    van der Sijs H, Aarts J, Vulto A, et al. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006 Mar; 13(2): 138–47PubMedCrossRefGoogle Scholar
  16. 16.
    van Doormaal JE, van den Bemt PM, Zaal RJ, et al. The influence that electronic prescribing has on medication errors and preventable adverse drug events: an interrupted time-series study. J Am Med Inform Assoc 2009 Nov; 16(6): 816–25PubMedCrossRefGoogle Scholar
  17. 17.
    Rommers MK, Zegers MH, De Clercq PA, et al. Development of a computerized adverse drug event alerting system (ADEAS) to identify patients at risk for an adverse drug event. Qual Saf Health Care 2010 Dec; 19(6): e35PubMedCrossRefGoogle Scholar
  18. 18.
    van Doormaal JE, Rommers MK, Kosterink JGW, et al. Comparison of methods for identifying patients at risk of medication-related harm. Qual Saf Health Care 2010 Dec; 19(6): e26PubMedCrossRefGoogle Scholar
  19. 19.
    Goldberg DE, Baardsgaard G, Johnson MT, et al. Computer-based program for identifying medication orders requiring dosage modification based on renal function. Am J Hosp Pharm 1991 Sep; 48(9): 1965–9PubMedGoogle Scholar
  20. 20.
    Kane-Gill SL, Dasta JF, Schneider PJ, et al. Monitoring abnormal laboratory values as antecedents to drug-induced injury. J Trauma 2005 Dec; 59(6): 1457–62PubMedCrossRefGoogle Scholar
  21. 21.
    Peterson JP, Colucci VJ, Schiff SE. Using serum creatinine concentrations to screen for inappropriate dosage of renally eliminated drugs. Am J Hosp Pharm 1991 Sep; 48(9): 1962–4PubMedGoogle Scholar
  22. 22.
    Schiff GD, Aggarwal HC, Kumar S, et al. Prescribing potassium despite hyperkalemia: medication errors uncovered by linking laboratory and pharmacy information systems. Am J Med 2000 Oct; 109(6): 494–7PubMedCrossRefGoogle Scholar
  23. 23.
    Schiff GD, Klass D, Peterson J, et al. Linking laboratory and pharmacy: opportunities for reducing errors and improving care. Arch Intern Med 2003 Apr; 163(8): 893–900PubMedCrossRefGoogle Scholar
  24. 24.
    van der Sijs H, Aarts J, van GT, et al. Turning off frequently overridden drug alerts: limited opportunities for doing it safely. J Am Med Inform Assoc 2008 Jul; 15(4): 439–48PubMedCrossRefGoogle Scholar
  25. 25.
    Jha AK, Laguette J, Seger A, et al. Can surveillance systems identify and avert adverse drug events? A prospective evaluation of a commercial application. J Am Med Inform Assoc 2008 Sep; 15(5): 647–53PubMedCrossRefGoogle Scholar
  26. 26.
    Kilbridge PM, Alexander L, Ahmad A. Implementation of a system for computerized adverse drug event surveillance and intervention at an academic medical center. J Clin Outcomes Manag 2006; 13(2): 94–100Google Scholar
  27. 27.
    Kilbridge PM, Campbell UC, Cozart HB, et al. Automated surveillance for adverse drug events at a community hospital and an academic medical center. J Am Med Inform Assoc 2006 Jul; 13(4): 372–7PubMedCrossRefGoogle Scholar
  28. 28.
    Seger AC, Jha AK, Bates DW. Adverse drug event detection in a community hospital utilising computerised medication and laboratory data. Drug Saf 2007; 30(9): 817–24PubMedCrossRefGoogle Scholar
  29. 29.
    Kaboli PJ, Hoth AB, McClimon BJ, et al. Clinical pharmacists and inpatient medical care: a systematic review. Arch Intern Med 2006 May; 166(9): 955–64PubMedCrossRefGoogle Scholar

Copyright information

© Adis Data Information BV 2011

Authors and Affiliations

  • Mirjam K. Rommers
    • 1
  • Irene M. Teepe-Twiss
    • 1
  • Henk-Jan Guchelaar
    • 1
  1. 1.PharmD, Department of Clinical Pharmacy & ToxicologyLeiden University Medical CenterLeidenthe Netherlands

Personalised recommendations