Skip to main content
SpringerLink
Log in

Sources of Safety Data and Statistical Strategies for Design and Analysis: Postmarket Surveillance

  • Special Section: Sources of Safety Data and Statistical Strategies for Design and Analysis
  • Published:
Therapeutic Innovation & Regulatory Science Aims and scope Submit manuscript

Abstract

Background

Safety data are continuously evaluated throughout the life cycle of a medical product to accurately assess and characterize the risks associated with the product. The knowledge about a medical product’s safety profile continually evolves as safety data accumulate.

Methods

This paper discusses data sources and analysis considerations for safety signal detection after a medical product is approved for marketing. This manuscript is the second in a series of papers from the American Statistical Association Biopharmaceutical Section Safety Working Group.

Results

We share our recommendations for the statistical and graphical methodologies necessary to appropriately analyze, report, and interpret safety outcomes, and we discuss the advantages and disadvantages of safety data obtained from passive postmarketing surveillance systems compared to other sources.

Conclusions

Signal detection has traditionally relied on spontaneous reporting databases that have been available worldwide for decades. However, current regulatory guidelines and ease of reporting have increased the size of these databases exponentially over the last few years. With such large databases, data-mining tools using disproportionality analysis and helpful graphics are often used to detect potential signals. Although the data sources have many limitations, analyses of these data have been successful at identifying safety signals postmarketing. Experience analyzing these dynamic data is useful in understanding the potential and limitations of analyses with new data sources such as social media, claims, or electronic medical records data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Furberg CD, Pitt B. Withdrawal of cerivastatin from the world market. Curr Control Trials Cardiovasc Med. 2001;2:205–207.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Staffa JA, Chang J, Green L. Cerivastatin and reports of fatal rhabdomyolysis. N Engl J Med. 2002;346:539–540.

    Article  PubMed  Google Scholar 

  3. Duggirala H, Tonning JM, Smith E, et al. Data mining at the food and drug administration. J Am Med Inform Assoc. 2016;23:428–434.

    Article  PubMed  Google Scholar 

  4. Food and Drug Administration. FDA-FAERS—quarterly report. https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/ucm082196.htm#QuarterlyReports. Accessed April 2017.

  5. Health and Human Services. VAERS Data. https://vaers.hhs.gov/data/data. Accessed April 2017.

  6. Medical Dictionary for Regulatory Activities. MedDRA home page. https://www.meddra.org/. Accessed April 2017.

  7. Food and Drug Administration. Food and Drug Administration—individual case safety reports. https://www.fda.gov/ForIndustry/DataStandards/IndividualCaseSafetyReports/default.htm. Accessed April 2017.

  8. Food and Drug Administration. OpenFDA public website. https://open.fda.gov/research/. Accessed April 2017.

  9. Food and Drug Administration. FDA, post-market drug and biologic evaluation. https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/ucm204091.htm. Accessed April 2017.

  10. Food and Drug Administration. FDA post-market safety information for patients and providers. https://www.fda.gov/Drugs/DrugSafety/PostmarketDrugSafetyInformationforPatientsandProviders/default.htm. Accessed April 2017.

  11. European Medicines Agency. EudraVigilance. http://www.ema.europa.eu/ema/index.jsp?%20curl=pages/regulation/general/general_content_000679.jsp&mid=WC0b01ac05800250b5. Accessed April 2017.

  12. European Center for Disease Prevention and Control. Vaccine adverse event surveillance and communication. http://vaesco.net/vaesco.html. Accessed April 2017.

  13. World Health Organization. WHO: VigiBase. https://www.who-umc.org/vigibase/vigibase/. Accessed April 2017.

  14. Food and Drug Administration. Public Workshop: The Sentinel Post-Licensure Rapid Immunization Safety Monitoring (PRISM) System. Silver Spring, MD: US FDA; 2016.

    Google Scholar 

  15. Kulldorff M, Fang ZX, Walsh SJ. A tree-based scan statistic for database disease surveillance. Biometrics. 2003;59:323–331.

    Article  PubMed  Google Scholar 

  16. Coloma PM, Trifirò G, Patadia V, Sturkenboom M. Postmarketing safety surveillance: where does signal detection using electronic healthcare records fit into the big picture? Drug Saf. 2013;36:183–197.

    Article  PubMed  Google Scholar 

  17. Patadia VK, Coloma P, Schuemie MJ, et al.; EU-ADR consortium. Using real-world healthcare data for pharmacovigilance signal detection—the experience of the EU-ADR project. Expert Rev Clin Pharmacol. 2015;8:95–102.

    Article  PubMed  CAS  Google Scholar 

  18. Schuemie MJ, Coloma PM, Straatman H, et al. Using electronic health care records for drug safety signal detection: a comparative evaluation of statistical methods. Med Care. 2012;50:890–897.

    Article  PubMed  Google Scholar 

  19. Evans SJ, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf. 2001;10:483–486.

    Article  CAS  PubMed  Google Scholar 

  20. Rothman KJ, Lanes S, Sacks ST. The reporting odds ratio and its advantages over the proportional reporting ratio. Pharmacoepidemiol Drug Saf. 2004;13:519–523.

    Article  PubMed  Google Scholar 

  21. Bate A, Lindquist M, Edwards IR, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol. 1998;54:315–321.

    Article  CAS  PubMed  Google Scholar 

  22. Huang L, Zalkikar J, Tiwari RC. A likelihood ratio test based method for signal detection with application to FDA’s drug safety data. J Am Stat Assoc. 2011;106:1230–1241.

    Article  CAS  Google Scholar 

  23. Huang L, Zalkikar J, Tiwari RC. Likelihood ratio test-based method for signal detection in drug classes using FDA’s AERS database. J Biopharm Stat. 2013;23:178–200.

    Article  PubMed  Google Scholar 

  24. Huang L, Zheng D, Zalkikar J, Tiwari R, et al. Zero-inflated Poisson model based likelihood ratio test for drug safety signal detection. Stat Methods Med Res. 2017;26:471–488.

    Article  PubMed  Google Scholar 

  25. Zhao Y, Yi M, Tiwari RC. Extended likelihood ratio test-based methods for signal detection in a drug class with application to FDA’s adverse event reporting system database [published online ahead of print]. Stat Methods Med Res. 2016 May 2.

  26. Nam K, Henderson NC2, Rohan P3, Woo EJ3, Russek-Cohen E. Logistic regression likelihood ratio test analysis for detecting signals of adverse events in post-market safety surveillance. J Biopharm Stat. 2017:1–19.

  27. Bate A. Bayesian confidence propagation neural network. Drug Saf. 2007;30:623–625.

    Article  PubMed  Google Scholar 

  28. Bate A, Edwards IR. Data mining in spontaneous reports. Basic Clin Pharmacol Toxicol. 2006;98:324–330.

    Article  CAS  PubMed  Google Scholar 

  29. Bate A, Evans SJ. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiol Drug Saf. 2009;18:427–436.

    Article  CAS  PubMed  Google Scholar 

  30. Bate A, Lindquist M, Edwards IR, Orre R. A data mining approach for signal detection and analysis. Drug Saf. 2002;25:393–397.

    Article  CAS  PubMed  Google Scholar 

  31. Orre R, Bate A, Norén GN, Swahn E, Arnborg S, Edwards IR. A Bayesian recurrent neural network for unsupervised pattern recognition in large incomplete data sets. Int J Neural Syst. 2005;15:207–222.

    Article  PubMed  Google Scholar 

  32. Orre R, Lansner A, Bate A, Lindquist M. Bayesian neural networks with confidence estimations applied to data mining. Comput Stat Data Anal. 2000;34:473–493.

    Article  Google Scholar 

  33. Almenoff JS, DuMouchel W, Kindman LA, Yang X, Fram D. Disproportionality analysis using empirical Bayes data mining: a tool for the evaluation of drug interactions in the post-marketing setting. Pharmacoepidemiol Drug Saf. 2003. 12:517–21.

    Article  PubMed  Google Scholar 

  34. Almenoff JS, DuMouchel W, Kindman LA, Yang X, Fram D. Novel statistical tools for monitoring the safety of marketed drugs. Clin Pharmacol Ther. 2007;82:157–166.

    Article  CAS  PubMed  Google Scholar 

  35. Curtis JR, Cheng H, Delzell E, et al. Adaptation of Bayesian data mining algorithms to longitudinal claims data: coxib safety as an example. Med Care. 2008;46:969–975.

    Article  PubMed  PubMed Central  Google Scholar 

  36. DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat. 1999;53:177–190.

    Google Scholar 

  37. Kulldorff M, Dashevsky I, Avery TR, et al. Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiol Drug Saf. 2013;22:517–523.

    Article  PubMed  Google Scholar 

  38. Kulldorff M, Davis RL, Kolczak M, Lewis E, Lieu T, Platt R. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Seq Anal Design Methods Appl. 2011;30:58–78.

    Google Scholar 

  39. Silva IR, Kulldorff M. Continuous versus group sequential analysis for post-market drug and vaccine safety surveillance. Biometrics. 2015;71:851–858.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Li L, Kulldorff M. A conditional maximized sequential probability ratio test for pharmacovigilance. Stat Med. 2010;29:284–295.

    Article  PubMed  Google Scholar 

  41. Cook AJ, Wellman RD, Nelson JC, Jackson LA, Tiwari RC. Group sequential method for observational data by using generalized estimating equations: application to Vaccine Safety Datalink. J Royal Stat Soc C Appl Stat. 2015;64:319–338.

    Article  Google Scholar 

  42. Chan KA, Hauben M. Signal detection in pharmacovigilance: empirical evaluation of data mining tools. Pharmacoepidemiol Drug Saf. 2005;14:597–579.

    Article  PubMed  Google Scholar 

  43. Zhao SS, Cook A, Jackson L, Nelson N. Statistical performance of group sequential methods for observational post-licensure medical product safety surveillance: a simulation study. Stat Interface. 2012;5:381–390.

    Article  Google Scholar 

  44. Cook AJ, Tiwari RC, Wellman RD, et al. Statistical approaches to group sequential monitoring of postmarket safety surveillance data: current state of the art for use in the Mini-Sentinel pilot. Pharmacoepidemiol Drug Saf. 2012;21(suppl 1):72–81.

    Article  PubMed  Google Scholar 

  45. Huang L, Zalkikar J, Tiwari R. Likelihood ratio based tests for longitudinal drug safety data. Stat Med. 2014;33:2408–2424.

    Article  PubMed  Google Scholar 

  46. Nelson JC, Cook AJ, Yu O, Zhao S, Jackson LA, Psaty BM. Methods for observational post-licensure medical product safety surveillance. Stat Methods Med Res. 2015;24:177–193.

    Article  PubMed  Google Scholar 

  47. Banks D, Woo EJ, Burwen DR, Perucci P, Braun MM, Ball R. Comparing data mining methods on the VAERS database. Pharmacoepidemiol Drug Saf. 2005;14:601–609.

    Article  PubMed  Google Scholar 

  48. Food and Drug Administration. FDA Adverse Event Reporting System (FAERS): latest quarterly data files, October-December 2016. https://www.fda.gov/drugs/guidancecomplianceregulatoryinformation/surveillance/adversedrugeffects/ucm082193.htm. Accessed April 2017.

  49. Food and Drug Administration. Manufacturer and User Facility Device Experience Database (MAUDE). https://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/PostmarketRequirements/ReportingAdverseEvents/ucm127891.htm. Accessed April 2017.

  50. Food and Drug Administration. MedSun: Medical Product Safety Network. https://www.fda.gov/MedicalDevices/Safety/MedSunMedicalProductSafetyNetwork/default.htm. Accessed April 2017.

  51. European Commission. EUDAMED: European Database on Medical Devices. http://ec.europa.eu/idabc/en/document/2256/5637.html. Accessed April 2017.

  52. Goldman S.A., et al., A MedWatch Continuing Education Article. Poster presented at: Center for Drug Evaluation and Research Staff College, Food and Drug Administration. 1996.

  53. Botsis T, Buttolph T, Nguyen MD, Winiecki S, Woo EJ, Ball R. Vaccine adverse event text mining system for extracting features from vaccine safety reports. J Am Med Inform Assoc. 2012;19:1011–1018.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Moore TJ, Furberg CD. Electronic health data for postmarket surveillance: a vision not realized. Drug Saf. 2015;38:601–610.

    Article  PubMed  Google Scholar 

  55. Heinrich J. Adverse drug events: Substantial problem but magnitude uncertain. Testimony before the Committee on Health, Education, Labor, and Pensions, U.S. Senate, February 1, 2000.

  56. Dimbil M, Erdman CB, Dmakas A, Kyle RF. Adverse Drug Event Reporting Rates: Comparing FAERS to Clinical Trials. Paper presented at: AMPC Annual Meeting. 2017.

  57. Sharrar RG, Dieck GS. Monitoring product safety in the postmarketing environment. Ther Adv Drug Saf. 2013;4:211–219.

    Article  PubMed  PubMed Central  Google Scholar 

  58. de Boer A. When to publish measures of disproportionality derived from spontaneous reporting databases? Br J Clin Pharmacol. 2011;72:909–911.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Montastruc JL, Sommet A, Bagheri H, Lapeyre-Mestre M. Benefits and strengths of the disproportionality analysis for identification of adverse drug reactions in a pharmacovigilance database. Br J Clin Pharmacol. 2011;72:905–908.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Sakaeda T, Kadoyama K, Minami K, Okuno Y. Commonality of drug-associated adverse events detected by 4 commonly used data mining algorithms. Int J Med Sci. 2014;11:461.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Fang H, Su Z, Wang Y, et al. Exploring the FDA Adverse event reporting system to generate hypotheses for monitoring of disease characteristics. Clin Pharmacol Ther. 2014;95:496–498.

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Food and Drug Administration, Center for Drug Evaluation and Research, Office of Biostatistics, WO Building 21, Silver Spring, MD, 20903, USA

    Rima Izem PhD & Yueqin Zhao PhD

  2. SanchezKam, LLC, Alexandria, VA, USA

    Matilde Sanchez-Kam PhD

  3. Amgen, Inc, Thousand Oaks, CA, USA

    Haijun Ma PhD

  4. SAS institute, Inc, JMP Life Sciences, Cary, NC, USA

    Richard Zink PhD

Authors
  1. Rima Izem PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matilde Sanchez-Kam PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haijun Ma PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Richard Zink PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yueqin Zhao PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rima Izem PhD.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Izem, R., Sanchez-Kam, M., Ma, H. et al. Sources of Safety Data and Statistical Strategies for Design and Analysis: Postmarket Surveillance. Ther Innov Regul Sci 52, 159–169 (2018). https://doi.org/10.1177/2168479017741112

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1177/2168479017741112

Keywords

Search

Navigation