Skip to main content
SpringerLink
Log in

Adverse Drug Reaction Risk Measures: A Comparison of Estimates from Drug Surveillance and Randomised Trials

  • Original Research Article
  • Published:
Pharmaceutical Medicine Aims and scope Submit manuscript

Abstract

Background

Most drug regulatory agencies and pharmaceutical companies hold databases of spontaneous reports of suspected adverse drug reactions (ADRs). Detection systems for ADR signals have been created by specialists to analyse such reports, based on the concept of disproportionality, in order to support safety decision making. However, these measures are often misinterpreted by non-specialists in pharmacovigilance.

Objectives

Our aim was to assess agreement between estimates of risk from spontaneous reports of suspected ADRs and estimates of risks of ADRs from randomised controlled trials (RCTs).

Methods

From 150 drugs randomly selected from the US Food and Drug Administration’s Adverse Event Reporting System (FAERS), we identified drugs where FAERS provided reporting odds ratios (RORs) and corresponding systematic reviews from the Cochrane database gave (pooled) odds ratios (ORs) for the same drugs and adverse reactions. We assessed agreement between (ln) RORs and (ln) ORs using the Pearson correlation coefficient and the Bland–Altman agreement method, and performed sensitivity analyses.

Results

We identified 6 drugs and 125 ADRs. Overall, there was a weak correlation (r = 0.20) between RORs (FAERS) and ORs (RCTs). However, we observed a stronger correlation (r = 0.78) between RORs and ORs for one drug (roflumilast) that received market approval relatively recently (2011).

Conclusions

Spontaneous reporting of suspected ADRs is an important tool for regulatory agencies and pharmaceutical companies in making decisions and detecting drug safety signals. Although there was moderate-to-strong agreement between ADR risk estimates from drug surveillance and RCTs for one drug, this study illustrates the current recommendations not to use disproportionality measures as valid proxies for risk estimates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. The FDA Adverse Event Reporting System FAERS. Available from: http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/default.htm. Accessed 29 May 2019.

  2. The Yellow Card scheme. Available from: https://yellowcard.mhra.gov.uk/the-yellow-card-scheme/. Accessed 29 May 2019.

  3. Abanmy NO, Al-Quait NA, Alami AH, Al-Juhani MH, Al-Aqeel S. The utilization of Arabic online drug information among adults in Saudi Arabia. Saudi Pharm J. 2012;20(4):317–21.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Chandler RE, Juhlin K, Fransson J, Caster O, Edwards IR, Noren GN. Current safety concerns with human papillomavirus vaccine: a cluster analysis of reports in VigiBase(R). Drug Saf. 2016. https://doi.org/10.1007/s40264-016-0456-3.

    Article  PubMed Central  Google Scholar 

  5. Wong CK, Marshall NS, Grunstein RR, Ho SS, Fois RA, Hibbs DE, et al. Spontaneous adverse event reports associated with zolpidem in the United States 2003–2012. J Clin Sleep Med. 2016;13(2):223–34.

    Article  Google Scholar 

  6. Lehman HP, Chen J, Gould AL, Kassekert R, Beninger PR, Carney R, et al. An evaluation of computer-aided disproportionality analysis for post-marketing signal detection. Clin Pharmacol Ther. 2007;82(2):173–80.

    Article  CAS  PubMed  Google Scholar 

  7. Flaig T, Douros A, Bronder E, Klimpel A, Kreutz R, Garbe E. Tocilizumab-induced pancreatitis: case report and review of data from the FDA adverse event reporting system. J Clin Pharm Ther. 2016;41(6):718–21.

    Article  CAS  PubMed  Google Scholar 

  8. Evans SJ, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf. 2001;10(6):483–6.

    Article  CAS  PubMed  Google Scholar 

  9. Rothman KJ, Lanes S, Sacks ST. The reporting odds ratio and its advantages over the proportional reporting ratio. Pharmacoepidemiol Drug Saf. 2004;13(8):519–23.

    Article  PubMed  Google Scholar 

  10. Harpaz R, DuMouchel W, LePendu P, Bauer-Mehren A, Ryan P, Shah NH. Performance of pharmacovigilance signal-detection algorithms for the FDA adverse event reporting system. Clin Pharmacol Ther. 2013;93(6):539–46.

    Article  CAS  PubMed  Google Scholar 

  11. Coloma PM, Avillach P, Salvo F, Schuemie MJ, Ferrajolo C, Pariente A, et al. A reference standard for evaluation of methods for drug safety signal detection using electronic healthcare record databases. Drug Saf. 2013;36(1):13–23.

    Article  CAS  PubMed  Google Scholar 

  12. Candore G, Juhlin K, Manlik K, Thakrar B, Quarcoo N, Seabroke S, et al. Comparison of statistical signal detection methods within and across spontaneous reporting databases. Drug Saf. 2015;38(6):577–87.

    Article  CAS  PubMed  Google Scholar 

  13. Hauben M, Bate A. Decision support methods for the detection of adverse events in post-marketing data. Drug Discov Today. 2009;14(7–8):343–57.

    Article  CAS  PubMed  Google Scholar 

  14. Rawlins MD. Spontaneous reporting of adverse drug reactions. II: uses. Br J Clin Pharmacol. 1988;26(1):7–11.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Bate A, Edwards IR. Data mining in spontaneous reports. Basic Clin Pharmacol Toxicol. 2006;98(3):324–30.

    Article  CAS  PubMed  Google Scholar 

  16. Wisniewski AF, Bate A, Bousquet C, Brueckner A, Candore G, Juhlin K, et al. Good signal detection practices: evidence from IMI PROTECT. Drug Saf. 2016;39(6):469–90.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Cheng YJ, Nie XY, Chen XM, Lin XX, Tang K, Zeng WT, et al. The role of macrolide antibiotics in increasing cardiovascular risk. J Am Coll Cardiol. 2015;66(20):2173–84.

    Article  CAS  PubMed  Google Scholar 

  18. Raschi E, Salvo F, Poluzzi E, De Ponti F. Safety meta-analysis: a call for appropriate use of disproportionality measures from spontaneous reporting systems. J Am Coll Cardiol. 2016;67(18):2193.

    Article  PubMed  Google Scholar 

  19. Macia-Martinez MA, de Abajo FJ, Roberts G, Slattery J, Thakrar B, Wisniewski AF. An empirical approach to explore the relationship between measures of disproportionate reporting and relative risks from analytical studies. Drug Saf. 2016;39(1):29–43.

    Article  CAS  PubMed  Google Scholar 

  20. Moore N, Thiessard F, Begaud B. The history of disproportionality measures (reporting odds ratio, proportional reporting rates) in spontaneous reporting of adverse drug reactions. Pharmacoepidemiol Drug Saf. 2005;14(4):285–6.

    Article  PubMed  Google Scholar 

  21. Bate A, Evans SJ. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiol Drug Saf. 2009;18(6):427–36.

    Article  CAS  PubMed  Google Scholar 

  22. Pariente A, Avillach P, Salvo F, Thiessard F, Miremont-Salame G, Fourrier-Reglat A, et al. Effect of competition bias in safety signal generation: analysis of a research database of spontaneous reports in France. Drug Saf. 2012;35(10):855–64.

    Article  PubMed  Google Scholar 

  23. FDA adverse event reporting system (FAERS).

  24. The cochrane database of systematic reviews (CDSR).

  25. The cochrane methods of adverse effects group. Available from: http://aem.cochrane.org/. Accessed 29 May 2019.

  26. Advera health analytics: real world data, analytics and insights for healthcare decision makers. Available from: http://www.adverahealth.com/. Accessed 29 May 2019.

  27. The MedDRA [Website]. Available from: http://www.meddra.org/. Accessed 29 May 2019.

  28. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10.

    Article  CAS  PubMed  Google Scholar 

  29. Chong J, Leung B, Poole P. Phosphodiesterase 4 inhibitors for chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2013;11:CD002309.

    Google Scholar 

  30. Yu T, Fain K, Boyd CM, Singh S, Weiss CO, Li T, et al. Benefits and harms of roflumilast in moderate to severe COPD. Thorax. 2014;69(7):616–22.

    Article  PubMed  Google Scholar 

  31. Evans SJ. What is the plural of a ‘yellow’ anecdote? Drug Saf. 2016;39(1):1–3.

    Article  PubMed  Google Scholar 

  32. van Puijenbroek EP, Bate A, Leufkens HG, Lindquist M, Orre R, Egberts AC. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol Drug Saf. 2002;11(1):3–10.

    Article  CAS  PubMed  Google Scholar 

  33. Golder S, Loke YK, Bland M. Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med. 2011;8(5):e1001026.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Zorzela L, Golder S, Liu Y, Pilkington K, Hartling L, Joffe A, et al. Quality of reporting in systematic reviews of adverse events: systematic review. BMJ. 2014;348:f7668.

    Article  PubMed  PubMed Central  Google Scholar 

  35. World Health Organisation. Vigibase. Available from: https://www.who-umc.org/vigibase/vigibase/. Accessed 29 May 2019.

  36. Gartlehner G, Nussbaumer B, Gaynes BN, Forneris CA, Morgan LC, Kaminski-Hartenthaler A, et al. Second-generation antidepressants for preventing seasonal affective disorder in adults. Cochrane Database Syst Rev. 2015;11:CD011268.

    Google Scholar 

  37. van Echteld I, Wechalekar MD, Schlesinger N, Buchbinder R, Aletaha D. Colchicine for acute gout. Cochrane Database Syst Rev. 2014;8:CD006190.

    Google Scholar 

  38. Caddy C, Amit BH, McCloud TL, Rendell JM, Furukawa TA, McShane R, et al. Ketamine and other glutamate receptor modulators for depression in adults. Cochrane Database Syst Rev. 2015;9:CD011612.

    Google Scholar 

  39. Nussbaum A, Stroup TS. Paliperidone for schizophrenia. Cochrane Database Syst Rev. 2008;2:CD006369.

    Google Scholar 

  40. Nussbaum AM, Stroup TS. Paliperidone palmitate for schizophrenia. Cochrane Database Syst Rev. 2012;6:CD008296.

    Google Scholar 

  41. Schwarz C, Volz A, Li C, Leucht S. Valproate for schizophrenia. Cochrane Database Syst Rev. 2008;3:CD004028.

    Google Scholar 

  42. Lonergan E, Luxenberg J. Valproate preparations for agitation in dementia. Cochrane Database Syst Rev. 2009;3:CD003945.

    Google Scholar 

  43. Huband N, Ferriter M, Nathan R, Jones H. Antiepileptics for aggression and associated impulsivity. Cochrane Database Syst Rev. 2010;2:003499.

    Google Scholar 

Download references

Acknowledgements

We would like to thank Prof. Stephen Evans for his valuable contribution to this study.

Author information

Author notes

  1. Tsung Yu

    Present address: Department of Public Health, National Cheng Kung University, Tainan, Taiwan

Authors and Affiliations

  1. Department of Epidemiology, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Hirschengraben 84, 8001, Zurich, Switzerland

    Raphaelle Beau-Lejdstrom, Sarah Crook, Alessandra Spanu, Tsung Yu & Milo A. Puhan

Authors
  1. Raphaelle Beau-Lejdstrom
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarah Crook
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alessandra Spanu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tsung Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Milo A. Puhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raphaelle Beau-Lejdstrom.

Ethics declarations

Funding

No third-party funding was received for the submitted work. AdveraHealth© was contracted by the University of Zurich to provide disproportionality measures on the FAERS database.

Conflict of interest

SC, AS, TY and MAP report no conflicts of interest. RBL has been working as a consultant in Epidemiology for Novartis and Hoffmann-La Roche.

Ethical Approval

No ethics approval or informed consent was required for this study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beau-Lejdstrom, R., Crook, S., Spanu, A. et al. Adverse Drug Reaction Risk Measures: A Comparison of Estimates from Drug Surveillance and Randomised Trials. Pharm Med 33, 331–339 (2019). https://doi.org/10.1007/s40290-019-00287-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40290-019-00287-y

Search

Navigation