Skip to main content
SpringerLink
Log in

Towards Automating Adverse Event Review: A Prediction Model for Case Report Utility

  • Original Research Article
  • Published:
Drug Safety Aims and scope Submit manuscript

Abstract

Introduction

The rapidly expanding size of the Food and Drug Administration’s (FDA) Adverse Event Reporting System database requires modernized pharmacovigilance practices. Techniques to systematically identify high utility individual case safety reports (ICSRs) will support safety signal management.

Objectives

The aim of this study was to develop and validate a model predictive of an ICSR’s pharmacovigilance utility (PVU).

Methods

PVU was operationalized as an ICSR’s inclusion in an FDA-authored pharmacovigilance review’s case series supporting a recommendation to modify product labeling. Multivariable logistic regression models were used to examine the association between PVU and ICSR features. The best performing model was selected for bootstrapping validation. As a sensitivity analysis, we evaluated the model’s performance across subgroups of safety issues.

Results

We identified 10,381 ICSRs evaluated in 69 pharmacovigilance reviews, of which 2115 ICSRs were included in a case series. The strongest predictors of ICSR inclusion were reporting of a designated medical event (odds ratio (OR) 1.93, 95% CI 1.54–2.43) and positive dechallenge (OR 1.67, 95% CI 1.50–1.87). The strongest predictors of ICSR exclusion were death reported as the only outcome (OR 2.72, 95% CI 1.76–4.35), more than three suspect products (OR 2.69, 95% CI 2.23–3.24), and > 15 preferred terms reported (OR 2.69, 95% CI 1.90–3.82). The validated model showed modest discriminative ability (C-statistic of 0.71). Our sensitivity analysis demonstrated heterogeneity in model performance by safety issue (C-statistic range 0.58–0.74).

Conclusions

Our model demonstrated the feasibility of developing a tool predictive of ICSR utility. The model’s modest discriminative ability highlights opportunities for further enhancement and suggests algorithms tailored to safety issues may be beneficial.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Serious outcomes are defined by US regulations (CFRs 314.80, 600.80) and include death, life-threatening, hospitalization, disability, congenital anomaly, and other serious outcomes.

  2. Expedited and non-expedited ICSRs are reports that pharmaceutical manufacturers are required to submit by regulation (CFRs 314.80, 600.80).

References

  1. Lester J, Neyarapally GA, Lipowski E, Graham CF, Hall M, Dal Pan G. Evaluation of FDA safety-related drug label changes in 2010. Pharmacoepidemiol Drug Saf. 2013;22(3):302–5.

    Article  Google Scholar 

  2. Ishiguro C, Hall M, Neyarapally GA, Dal Pan G. Post-market drug safety evidence sources: an analysis of FDA drug safety communications. Pharmacoepidemiol Drug Saf. 2012;21(10):1134–6.

    Article  Google Scholar 

  3. FDA Adverse Event Reporting System (FAERS) Public Dashboard. https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/ucm070093.htm. Accessed 6 Dec 2019.

  4. Duggirala, HJ, Tonning JM, Smith E, Bright RA, Baker JD, Ball R, et al. Data mining at FDA. http://www.fda.gov/downloads/ScienceResearch/DataMiningatFDA/UCM443675.pdf. Accessed 28 Nov 2019.

  5. Bergvall T, Noren GN, Lindquist M. vigiGrade: a tool to identify well-documented individual case reports and highlight systematic data quality issues. Drug Saf. 2014;37(1):65–77.

    Article  Google Scholar 

  6. Guidance for Industry. Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment, US Food and Drug Administration; 2005. http://www.fda.gov/downloads/regulatoryinformation/guidances/ucm126834.pdf. Accessed 28 Nov 2019.

  7. FDA Drug Safety Guidances. https://www.fda.gov/drugs/guidance-compliance-regulatory-information/guidances-drugs. Accessed 28 Nov 2019.

  8. Moore TJ, Furberg CD, Mattison DR, Cohen MR. Completeness of serious adverse drug event reports received by the US Food and Drug Administration in 2014. Pharmacoepidemiol Drug Saf. 2016;25(6):713–8.

    Article  Google Scholar 

  9. Oosterhuis I, Rolfes L, Ekhart C, Muller-Hansma A, Harmark L. First experiences with a tool to measure the level of clinical information present in adverse drug reaction reports. Expert Opin Drug Saf. 2018;17(2):111–5.

    Article  CAS  Google Scholar 

  10. Han L, Ball R, Pamer CA, Altman RB, Proestel S. Development of an automated assessment tool for MedWatch reports in the FDA adverse event reporting system. J Am Med Inform Assoc. 2017;24(5):913–20.

    Article  Google Scholar 

  11. CIOMS Working Group VIII. Practical aspects of signal detection in pharmacovigilance. Geneva: Report of CIOMS Working Group VIII; 2010.

    Google Scholar 

  12. Pinnow E, Amr S, Bentzen SM, Brajovic S, Hungerford L, St George DM, et al. Postmarket safety outcomes for new molecular entity (NME) drugs approved by the food and drug administration between 2002 and 2014. Clin Pharmacol Ther. 2018;104(2):390–400.

    Article  Google Scholar 

  13. Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc. 1983;78:316–31.

    Article  Google Scholar 

  14. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001;54(8):774–81.

    Article  CAS  Google Scholar 

  15. Postmarketing reporting of adverse drug experiences. 21 CFR 314.80.

  16. Gummin DD, Mowry JB, Spyker DA, Brooks DE, Fraser MO, Banner W. 2016 Annual Report of the American Association of Poison Control Centers’ National Poison Data System (NPDS): 34th Annual Report. Clin Toxicol (Phila). 2017;55(10):1072–252.

    Article  Google Scholar 

  17. Wunnava S, Qin X, Kakar T, Socrates V, Wallace A, Rundensteiner E. Towards transforming FDA adverse event narratives into actionable structured data for improved pharmacovigilance. In: 2017 Proceedings of the symposium on applied computing, pp 777–782. https://doi.org/10.1145/3019612.3022875.

  18. Schotland P, Racz R, Jackson D, Levin R, Strauss DG, Burkhart K. Target-adverse event profiles to augment pharmacovigilance: a pilot study with six new molecular entities. CPT Pharmacomet Syst Pharmacol. 2018;7(12):809–17.

    Article  CAS  Google Scholar 

  19. Caster O, Juhlin K, Watson S, Noren GN. Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence aspects in vigiRank. Drug Saf. 2014;37(8):617–28.

    Article  CAS  Google Scholar 

  20. Scholl JHG, van Hunsel F, Hak E, van Puijenbroek EP. A prediction model-based algorithm for computer-assisted database screening of adverse drug reactions in the Netherlands. Pharmacoepidemiol Drug Saf. 2018;27(2):199–205.

    Article  Google Scholar 

  21. Caster O, Sandberg L, Bergvall T, Watson S, Noren GN. vigiRank for statistical signal detection in pharmacovigilance: first results from prospective real-world use. Pharmacoepidemiol Drug Saf. 2017;26(8):1006–10.

    Article  CAS  Google Scholar 

  22. Dal Pan GJ. Ongoing challenges in pharmacovigilance. Drug Saf. 2014;37(1):1–8.

    Article  Google Scholar 

  23. PRAC Strategy on Measuring the Impact of Pharmacovigilance Activities. European Medicines Agency. November 30, 2017. http://www.ema.europa.eu/docs/en_GB/document_library/Other/2016/01/WC500199756.pdf. Accessed 18 Apr 2018.

  24. Meyboom RH, Hekster YA, Egberts AC, Gribnau FW, Edwards IR. Causal or casual? The role of causality assessment in pharmacovigilance. Drug Saf. 1997;17(6):374–89.

    Article  CAS  Google Scholar 

  25. Agbabiaka TB, Savovic J, Ernst E. Methods for causality assessment of adverse drug reactions: a systematic review. Drug Saf. 2008;31(1):21–37.

    Article  Google Scholar 

  26. Noren GN. The power of the case narrative—can it be brought to bear on duplicate detection? Drug Saf. 2017;40(7):543–6.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA

    Monica A. Muñoz, Gerald J. Dal Pan & Cindy M. Kortepeter

  2. Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, FL, USA

    Monica A. Muñoz, Yu-Jung Jenny Wei, Hong Xiao & Almut G. Winterstein

  3. Department of Pharmacy Practice and Science, College of Pharmacy, University of Kentucky, Lexington, KY, USA

    Chris Delcher

  4. Center for Drug Evaluation and Safety (CoDES), University of Florida, Gainesville, FL, USA

    Yu-Jung Jenny Wei & Almut G. Winterstein

  5. Department of Epidemiology, College of Medicine and College of Public Health and Health Professions, University of Florida, Gainesville, FL, USA

    Almut G. Winterstein

Authors
  1. Monica A. Muñoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerald J. Dal Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu-Jung Jenny Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chris Delcher
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hong Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cindy M. Kortepeter
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Almut G. Winterstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Monica A. Muñoz.

Ethics declarations

Funding

No funding was used for the preparation of this manuscript.

Conflict of interest

Monica A. Muñoz, Gerald J. Dal Pan, Jenny Wei, Chris Delcher, Hong Xiao, Cindy M. Kortepeter, and Almut G. Winterstein declare that they have no conflict of interest.

Ethical approval

This study was approved by both the US FDA and University of Florida’s Institutional Review and Boards.

Additional information

The views expressed are those of the authors and do not necessarily represent the position of, nor imply endorsement from, the US Food and Drug Administration or the US Government.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 44 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Muñoz, M.A., Dal Pan, G.J., Wei, YJ.J. et al. Towards Automating Adverse Event Review: A Prediction Model for Case Report Utility. Drug Saf 43, 329–338 (2020). https://doi.org/10.1007/s40264-019-00897-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40264-019-00897-0

Search

Navigation