Drug Safety

, Volume 39, Issue 6, pp 561–575 | Cite as

A Pharmacovigilance Signaling System Based on FDA Regulatory Action and Post-Marketing Adverse Event Reports

  • Keith B. HoffmanEmail author
  • Mo Dimbil
  • Nicholas P. Tatonetti
  • Robert F. Kyle
Original Research Article



Many serious drug adverse events (AEs) only manifest well after regulatory approval. Therefore, the development of signaling methods to use with post-approval AE databases appears vital to comprehensively assess real-world drug safety. However, with millions of potential drug–AE pairs to analyze, the issue of focus is daunting.


Our objective was to develop a signaling platform that focuses on AEs with historically demonstrated regulatory interest and to analyze such AEs with a disproportional reporting method that offers broad signal detection and acceptable false-positive rates.


We analyzed over 1500 US FDA regulatory actions (safety communications and drug label changes) from 2008 to 2015 to construct a list of eligible signal AEs. The FDA Adverse Event Reporting System (FAERS) was used to evaluate disproportional reporting rates, constrained by minimum case counts and confidence interval limits, of these selected AEs for 109 training drugs. This step led to 45 AEs that appeared to have a low likelihood of being added to a label by FDA, so they were removed from the signal eligible list. We measured disproportional reporting for the final group of eligible AEs on a test group of 29 drugs that were not used in either the eligible list construction or the training steps.


In a group of 29 test drugs, our model reduced the number of potential drug–AE signals from 41,834 to 97 and predicted 73 % of individual drug label changes. The model also predicted at least one AE–drug pair label change in 66 % of all the label changes for the test drugs.


By concentrating on AE types with already demonstrated interest to FDA, we constructed a signaling system that provided focus regarding drug–AE pairs and suitable accuracy with regard to the issuance of FDA labeling changes. We suggest that focus on historical regulatory actions may increase the utility of pharmacovigilance signaling systems.


Progressive Multifocal Leukoencephalopathy Reporting Odds Ratio Label Change Drug Pair National Drug File Reference Terminology 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors are indebted to Colin B. Erdman and Dingguo Chen for their expert analytic assistance that enabled this study to be undertaken. We also thank Brian M. Overstreet for early conceptual input regarding these methods. MedDRA®, the Medical Dictionary for Regulatory Activities, terminology is the international medical terminology developed under the auspices of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH).

Compliance with Ethical Standards

Conflicts of interest

Keith B. Hoffman, Mo Dimbil, and Robert F. Kyle have all declared employment- and stock-related conflicts of interests in their declaration forms related to Advera Health Analytics, Inc. (AHA). Nicholas P. Tatonetti has declared a stock-related conflict of interest in his declaration forms related to AHA. Keith B. Hoffman, Mo Dimbil, Nicholas P. Tatonetti, and Robert F. Kyle have no other conflicts of interest that are directly relevant to the content of this manuscript.


This study, and the preparation of this manuscript, was funded solely in the form of salaries (KBH, MD, and RFK) paid by AHA. No specific funds were allocated for this study.

Author contributions

Keith B. Hoffman, Mo Dimbil, and Robert F. Kyle conceived of the study, analyzed and interpreted data, and approved the final submitted manuscript. Keith B. Hoffman drafted the final submitted manuscript. Nicholas P. Tatonetti made suggestions for data interpretation and approved the final submitted manuscript.

Supplementary material

40264_2016_409_MOESM1_ESM.docx (44 kb)
Supplementary material 1 (DOCX 43 kb)


  1. 1.
    Ahmad SR. Adverse drug event monitoring at the Food and Drug Administration. J Gen Int Med. 2003;18(1):57–60.CrossRefGoogle Scholar
  2. 2.
    FDA. Follow-up to the November 2009 early communication about an ongoing safety review of sibutramine, marketed as Meridia. 2010 [online]. Accessed 3 Feb 2016.
  3. 3.
    FDA. Safety Information: Vioxx (rofecoxib). 2002 [online]. Accessed 3 Feb 2016.
  4. 4.
    Charatan F. Bayer decides to withdraw cholesterol lowering drug. BMJ. 2001;323(7309):359.CrossRefGoogle Scholar
  5. 5.
    Tang E, Ravaud P, Riveros C, Perrodeau E, Dechartres A. Comparison of serious adverse events posted at and published in corresponding journal articles. BMC Med. 2015;13:189.CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.CrossRefPubMedGoogle Scholar
  7. 7.
    Naci H, Ioannidis JP. How good is “evidence” from clinical studies of drug effects and why might such evidence fail in the prediction of the clinical utility of drugs? Ann Rev Pharmacol Toxicol. 2015;55:169–89.CrossRefGoogle Scholar
  8. 8.
    Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2012;12:MR000033.PubMedGoogle Scholar
  9. 9.
    Eyding D, Lelgemann M, Grouven U, Harter M, Kromp M, Kaiser T, et al. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ. 2010;341:c4737.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine–selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ. 2003;326(7400):1171–3.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Cowley AJ, Skene A, Stainer K, Hampton JR. The effect of lorcainide on arrhythmias and survival in patients with acute myocardial infarction: an example of publication bias. Int J Cardiol. 1993;40(2):161–6.CrossRefPubMedGoogle Scholar
  12. 12.
    Hemminki E. Study of information submitted by drug companies to licensing authorities. BMJ. 1980;280(6217):833–6.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Le Noury J, Nardo JM, Healy D, Jureidini J, Raven M, Tufanaru C, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ. 2015;351:h4320.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Ma P, Marinovic I, Karaca-Mandic P. Drug manufacturers’ delayed disclosure of serious and unexpected adverse events to the US Food and Drug Administration. JAMA Intern Med. 2015;175(9):1565–6.CrossRefPubMedGoogle Scholar
  15. 15.
    Gruber S, van der Laan MJ. An application of targeted maximum likelihood estimation to the meta-analysis of safety data. Biometrics. 2013;69(1):254–62.CrossRefPubMedGoogle Scholar
  16. 16.
    Cole LW, Kesselheim JC, Kesselheim AS. Ethical issues in new drug prescribing. J Bioeth Inq. 2012;9(1):77–83.CrossRefPubMedGoogle Scholar
  17. 17.
    FDA. Adverse Event Reporting System (FAERS) (formerly AERS). 2015 [online]. Accessed 3 Feb 2016.
  18. 18.
    European Medicines Agency: EudraVigilance 2015 [online]. Accessed 3 Feb 2016.
  19. 19.
    Uppsala Monitoring Center: VigiBase, the WHO Global ICSR Database System 2015 [online]. Accessed 3 Feb 2016.
  20. 20.
    FDA. FAERS Quarterly Data Files Documentation. 2015 [online]. Accessed 3 Feb 2016.
  21. 21.
    MedDRA. Medical Dictionary for Regulatory Activities and the Maintenance and Support Services 2015 [online]. Accessed 3 Feb 2016.
  22. 22.
    Hoffman KB, Overstreet BM, Doraiswamy PM. A drug safety ePlatform for physicians, pharmacists and consumers based on post-marketing adverse events. Drugs Ther Stud. 2013;3(e4):15–19.Google Scholar
  23. 23.
    FDA. Pharmacological Class: National Drug File Reference Terminology. 2013 [online]. Accessed 3 Feb 2016.
  24. 24.
    Peters L, Kapusnik-Uner JE, Nguyen T, Bodenreider O. An approximate matching method for clinical drug names. AMIA Annu Symp Proc. 2011;2011:1117–26.PubMedPubMedCentralGoogle Scholar
  25. 25.
    Bate A, Evans SJ. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiol Drug Saf. 2009;18(6):427–36.CrossRefPubMedGoogle Scholar
  26. 26.
    EudraVigilance. Expert Working Group: Important Medical Event Terms (IME) list. 2015 [online]. Accessed 3 Feb 2016.
  27. 27.
    FDA. Drug Safety Communications 2015 [online]. Accessed 3 Feb 2016.
  28. 28.
    FDA. Designated Medical Events List. Center for Drug Evaluation and Research. 2015.Google Scholar
  29. 29.
    FDA. Drug Safety Labeling Changes 2015 [online]. Accessed 3 Feb 2016.
  30. 30.
    Hu N, Huang L, Tiwari RC. Signal detection in FDA AERS database using Dirichlet process. Stat Med. 2015;34(19):2725–42.CrossRefPubMedGoogle Scholar
  31. 31.
    Sakaeda T, Kadoyama K, Minami K, Okuno Y. Commonality of drug-associated adverse events detected by 4 commonly used data mining algorithms. Int J Med Sci. 2014;11(5):461–5.CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Grigoriev I, zu Castell W, Tsvetkov P, Antonov AV. AERS spider: an online interactive tool to mine statistical associations in Adverse Event Reporting System. Pharmacoepidemiol Drug Saf. 2014;23(8):795–801.PubMedGoogle Scholar
  33. 33.
    Sakaeda T, Tamon A, Kadoyama K, Okuno Y. Data mining of the public version of the FDA Adverse Event Reporting System. Int J Med Sci. 2013;10(7):796–803.CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Hochberg AM, Hauben M, Pearson RK, O’Hara DJ, Reisinger SJ, Goldsmith DI, et al. An evaluation of three signal-detection algorithms using a highly inclusive reference event database. Drug Saf. 2009;32(6):509–25.CrossRefPubMedGoogle Scholar
  35. 35.
    Almenoff JS, LaCroix KK, Yuen NA, Fram D, DuMouchel W. Comparative performance of two quantitative safety signalling methods: implications for use in a pharmacovigilance department. Drug Saf. 2006;29(10):875–87.CrossRefPubMedGoogle Scholar
  36. 36.
    Hauben M, Reich L. Safety related drug-labelling changes: findings from two data mining algorithms. Drug Saf. 2004;27(10):735–44.CrossRefPubMedGoogle Scholar
  37. 37.
    Szarfman A, Machado SG, O’Neill RT. Use of screening algorithms and computer systems to efficiently signal higher-than-expected combinations of drugs and events in the US FDA’s spontaneous reports database. Drug Saf. 2002;25(6):381–92.CrossRefPubMedGoogle Scholar
  38. 38.
    Edwards R, Faich G, Tilson H. Points to consider: the roles of surveillance and epidemiology in advancing drug safety. Pharmacoepidemiol Drug Saf. 2005;14(9):665–7.CrossRefPubMedGoogle Scholar
  39. 39.
    Tatonetti NP, Ye PP, Daneshjou R, Altman RB. Data-driven prediction of drug effects and interactions. Sci Transl Med. 2012;4(125):125ra31.CrossRefPubMedPubMedCentralGoogle Scholar
  40. 40.
    Auerbach M, Kane RC. Caution in making inferences from FDA’s Adverse Event Reporting System. Am J Health Syst Pharm. 2012;69(11):922–3.CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Keith B. Hoffman
    • 1
    Email author
  • Mo Dimbil
    • 1
  • Nicholas P. Tatonetti
    • 2
  • Robert F. Kyle
    • 1
  1. 1.Advera Health Analytics, Inc.Santa RosaUSA
  2. 2.Department of Biomedical InformaticsColumbia UniversityNew YorkUSA

Personalised recommendations