Journal of General Internal Medicine

, Volume 34, Issue 2, pp 285–292 | Cite as

Screening for Adverse Drug Events: a Randomized Trial of Automated Calls Coupled with Phone-Based Pharmacist Counseling

  • Gordon D. SchiffEmail author
  • Elissa Klinger
  • Alejandra Salazar
  • Jeffrey Medoff
  • Mary G. Amato
  • E. John Orav
  • Shimon Shaykevich
  • Enrique V. Seoane
  • Lake Walsh
  • Theresa E. Fuller
  • Patricia C. Dykes
  • David W. Bates
  • Jennifer S. Haas
Original Research



Medication adverse events are important and common yet are often not identified by clinicians. We evaluated an automated telephone surveillance system coupled with transfer to a live pharmacist to screen potentially drug-related symptoms after newly starting medications for four common primary care conditions: hypertension, diabetes, depression, and insomnia.


Cluster randomized trial with automated calls to eligible patients at 1 and 4 months after starting target drugs from intervention primary care clinics compared to propensity-matched patients from control clinics. Primary and secondary outcomes were physician documentation of any adverse effects associated with newly prescribed target medication, and whether the medication was discontinued and, if yes, whether the reason for stopping was an adverse effect.


Of 4876 eligible intervention clinic patients who were contacted using automated calls, 776 (15.1%) responded and participated in the automated call. Based on positive symptom responses or request to speak to a pharmacist, 320 patients were transferred to the pharmacist and discussed 1021 potentially drug-related symptoms. Of these, 188 (18.5%) were assessed as probably and 479 (47.1%) as possibly related to the medication. Compared to a propensity-matched cohort of control clinic patients, intervention patients were significantly more likely to have adverse effects documented in the medical record by a physician (277 vs. 164 adverse effects, p < 0.0001, and 177 vs. 122 patients discontinued with documented adverse effects, p < 0.0001).


Systematic automated telephone outreach monitoring coupled with real-time phone referral to a pharmacist identified a substantial number of previously unidentified potentially drug-related symptoms, many of which were validated as probably or possibly related to the drug by the pharmacist or their physicians. Multiple challenges were encountered using the interactive voice response (IVR) automated calling system, suggesting that other approaches may need to be considered and evaluated.

Trial Registration NCT02087293



The team would like to acknowledge Japneet Kwatra, SM for her research assistance.


This research was supported by the Agency for Healthcare Research and Quality (AHRQ), H.I.T Center for Education and Research on Therapeutics (CERT) grant number 1U19HS021094-01, part of AHRQ’s CERT funding mechanism.

Compliance with Ethical Standards

The research protocol was approved by the PHS Institutional Review Board and registered at (NCT NCT02087293).


The content is solely the responsibility of the authors and does not necessarily represent the official views of AHRQ.

Conflict of Interest

Dr. Schiff previously received grant compensation to evaluate the Medaware software. Dr. Bates consults for EarlySense, which makes patient safety monitoring systems. He receives cash compensation from CDI (Negev), Ltd., which is a not-for-profit incubator for health IT startups. He receives equity from ValeraHealth which makes software to help patients with chronic diseases. He receives equity from Clew which makes software to support clinical decision-making in intensive care. He receives equity from MDClone which takes clinical data and produces deidentified versions of it. Dr. Schiff’s and Dr. Bates’ financial interests have been reviewed by Brigham and Women’s Hospital and Partners HealthCare in accordance with their institutional policies.

The remaining authors declare no potential conflicts of interest.

Supplementary material

11606_2018_4672_MOESM1_ESM.pdf (76 kb)
ESM 1 (PDF 76 kb)


  1. 1.
    CDC. National Center for Health Statistics: Therapeutic Drug Use. 2015; Accessed 12/5/2016.
  2. 2.
    Kantor ED, Rehm CD, Haas JS, Chan AT, Giovannucci EL. Trends in prescription drug use among adults in the united states from 1999-2012. JAMA 2015;314(17):1818–1830.Google Scholar
  3. 3.
    Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med 2003;348(16):1556–1564.Google Scholar
  4. 4.
    Taché SV, Sönnichsen A, Ashcroft DM. Prevalence of adverse drug events in ambulatory care: a systematic review. Ann Pharmacother 2011;45(7–8):977–989.Google Scholar
  5. 5.
    Budnitz DS, Shehab N, Kegler SR, Richards CL. Medication use leading to emergency department visits for adverse drug events in older adults. Ann Intern Med 2007;147(11):755–765.Google Scholar
  6. 6.
    Budnitz DS, Lovegrove MC, Shehab N, Richards CL. Emergency hospitalizations for adverse drug events in older Americans. N Engl J Med 2011;365(21):2002–2012.Google Scholar
  7. 7.
    Shehab N, Lovegrove MC, Geller AI, Rose KO, Weidle NJ, Budnitz DS. US emergency department visits for outpatient adverse drug events, 2013-2014. JAMA 2016;316(20):2115–2125.Google Scholar
  8. 8.
    Morrato EH. The Case for Funding Our National Drug Safety Surveillance System. In: LWW; 2017.Google Scholar
  9. 9.
    Classen DC, Resar R, Griffin F, et al. ‘Global trigger tool’shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff 2011;30(4):581–589.Google Scholar
  10. 10.
    Bourgeois FT, Shannon MW, Valim C, Mandl KD. Adverse drug events in the outpatient setting: an 11-year national analysis. Pharmacoepidemiol Drug Saf 2010;19(9):901–910.Google Scholar
  11. 11.
    Weissman JS, Schneider EC, Weingart SN, et al. Comparing patient-reported hospital adverse events with medical record review: do patients know something that hospitals do not? Ann Intern Med 2008;149(2):100–108.Google Scholar
  12. 12.
    Berner ES, Ray MN, Panjamapirom A, et al. Exploration of an automated approach for receiving patient feedback after outpatient acute care visits. J Gen Intern Med 2014;29(8):1105–1112.Google Scholar
  13. 13.
    Berner ES, Burkhardt JH, Panjamapirom A, Ray MN. Cost implications of human and automated follow-up in ambulatory care. Am J Manag Care. 2014;20(11 Spec No. 17):SP531–540.Google Scholar
  14. 14.
    Willig JH, Krawitz M, Panjamapirom A, et al. Closing the feedback loop: an interactive voice response system to provide follow-up and feedback in primary care settings. J Med Syst 2013;37(2):9905.Google Scholar
  15. 15.
    Auger C, Forster AJ, Oake N, Tamblyn R. Usability of a computerised drug monitoring programme to detect adverse drug events and non-compliance in outpatient ambulatory care. BMJ Qual Saf 2013;22(4):306–316.Google Scholar
  16. 16.
    Houser SH, Ray MN, Maisiak R, et al. Telephone follow-up in primary care: can interactive voice response calls work? Stud Health Technol Inform 2013;192:112–116.Google Scholar
  17. 17.
    Haas JS, Iyer A, Orav EJ, Schiff GD, Bates DW. Participation in an ambulatory e-pharmacovigilance system. Pharmacoepidemiol Drug Saf 2010;19(9):961–969.Google Scholar
  18. 18.
    Haas JS, Amato M, Marinacci L, Orav EJ, Schiff GD, Bates DW. Do package inserts reflect symptoms experienced in practice?: assessment using an automated phone pharmacovigilance system with varenicline and zolpidem in a primary care setting. Drug Saf 2012;35(8):623–628.Google Scholar
  19. 19.
    Mahida S, Seoane E, Klinger E, et al. Cost of reporting possible adverse Drug reactions in medical Outpatients using a Telephonic interactive Voice response system. Value Health 2015;18(3):A257.Google Scholar
  20. 20.
    Rehan HS, Chopra D, Kakkar AK. Physician’s guide to pharmacovigilance: Terminology and causality assessment. Eur J Intern Med 2009;20(1):3–8.Google Scholar
  21. 21.
    Farcas A, Bojita M. Adverse drug reactions in clinical practice: a causality assessment of a case of drug-induced pancreatitis. J Gastrointestin Liver Dis 2009;18(3):353–358.Google Scholar
  22. 22.
    Bates DW, Evans RS, Murff H, Stetson PD, Pizziferri L, Hripcsak G. Detecting adverse events using information technology. J Am Med Inform Assoc 2003;10(2):115–128.Google Scholar
  23. 23.
    Forster AJ, Erlanger TE, Jennings A, et al. Effectiveness of a computerized drug-monitoring program to detect and prevent adverse drug events and medication non-adherence in outpatient ambulatory care: study protocol of a randomized controlled trial. Trials 2015;16(1):1.Google Scholar
  24. 24.
    Forster AJ, Auger C. Using information technology to improve the monitoring of outpatient prescribing. JAMA Intern Med 2013;173(5):382–384.Google Scholar
  25. 25.
    Falconi M, Johnston S, Hogg W. A scoping review to explore the suitability of interactive voice response to conduct automated performance measurement of the patient’s experience in primary care. Prim Health Care Res Dev 2016;17(3):209–225.Google Scholar
  26. 26.
    Boulos MNK, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online 2011;10(1):1.Google Scholar
  27. 27.
    Link MW, Murphy J, Schober MF, et al. Mobile technologies for conducting, augmenting and potentially replacing surveys: report of the AAPOR task force on emerging technologies in public opinion research. Deerfield: AAPOR 2014.Google Scholar

Copyright information

© Society of General Internal Medicine 2018

Authors and Affiliations

  • Gordon D. Schiff
    • 1
    • 2
    Email author
  • Elissa Klinger
    • 1
    • 3
  • Alejandra Salazar
    • 1
  • Jeffrey Medoff
    • 1
  • Mary G. Amato
    • 4
  • E. John Orav
    • 1
    • 2
  • Shimon Shaykevich
    • 1
  • Enrique V. Seoane
    • 5
  • Lake Walsh
    • 1
  • Theresa E. Fuller
    • 1
    • 6
  • Patricia C. Dykes
    • 1
    • 2
  • David W. Bates
    • 1
    • 2
  • Jennifer S. Haas
    • 1
    • 2
  1. 1.Division of General Internal Medicine and Primary Care Brigham and Women’s HospitalBostonUSA
  2. 2.Harvard Medical SchoolBostonUSA
  3. 3.Penn MedicineUniversity of Pennsylvania Health SystemPhiladelphiaUSA
  4. 4.Massachusetts College of Pharmacy and Health Sciences UniversityBostonUSA
  5. 5.School of PharmacyChapman UniversityOrangeUSA
  6. 6.Northeastern UniversityBostonUSA

Personalised recommendations