Abstract
Introduction: With increasing volumes of postmarketing safety surveillance data, data mining algorithms (DMAs) have been developed to search large spontaneous reporting system (SRS) databases for disproportional statistical dependencies between drugs and events. A crucial question is the proper deployment of such techniques within the universe of methods historically used for signal detection. One question of interest is comparative performance of algorithms based on simple forms of disproportionality analysis versus those incorporating Bayesian modelling. A potential benefit of Bayesian methods is a reduced volume of signals, including false-positive signals.
Objective: To compare performance of two well described DMAs (proportional reporting ratios [PRRs] and an empirical Bayesian algorithm known as multi-item gamma Poisson shrinker [MGPS]) using commonly recommended thresholds on a diverse data set of adverse events that triggered drug labelling changes.
Methods: PRRs and MGPS were retrospectively applied to a diverse sample of drug-event combinations (DECs) identified on a government Internet site for a 7-month period. Metrics for this comparative analysis included the number and proportion of these DECs that generated signals of disproportionate reporting with PRRs, MGPS, both or neither method, differential timing of signal generation between the two methods, and clinical nature of events that generated signals with only one, both or neither method.
Results: There were 136 relevant DECs that triggered safety-related labelling changes for 39 drugs during a 7-month period. PRRs generated a signal of disproportionate reporting with almost twice as many DECs as MGPS (77 vs 40). No DECs were flagged by MGPS only. PRRs highlighted DECs in advance of MGPS (1–15 years) and a label change (1–30 years). For 59 DECs, there was no signal with either DMA. DECs generating signals of disproportionate reporting with only PRRs were both medically serious and non-serious.
Discussion/conclusion: In most instances in which a DEC generated a signal of disproportionate reporting with both DMAs (almost twice as many with PRRs), the signal was generated using PRRs in advance of MGPS. No medically important events were signalled only by MGPS. It is likely that the incremental utility of DMAs are highly situation-dependent. It is clear, however, that the volume of signals generated by itself is an inadequate criterion for comparison and that clinical nature of signalled events and differential timing of signals needs to be considered. Accepting commonly recommended threshold criteria for DMAs examined in this study as universal benchmarks for signal detection is not justified.
Similar content being viewed by others
References
Bate A, Lindquist M, Edwards I, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol 1998; 54: 315–21
DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat 1999; 53(3): 170–90
Szarfman A, Machado SG, O’Neill R. Use of screening algorithms and computer systems to efficiently signal higher-than expected combinations of drugs and events in the US FDA’s spontaneous reports database. Drug Saf 2002; 25(6): 381–92
Evans SJW, Waller P, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001; 10: 483–6
Van Puijenbroek E, Diemont W, van Groothest K. Application of quantitative signal detection in the Dutch spontaneous reporting system for adverse drug reactions. Drug Saf 2003; 26(5): 293–301
Van der Heijden P, van Puijenbroek E, van Buuren S, et al. On the assessment of adverse drug reactions from spontaneous reporting systems: the influence of under-reporting on odds ratios. Stat Med 2002; 21: 2027–44
Kennedy D, Goldman S, Lillie R. Spontaneous reporting systems in the United States. In: Strom B, editor. Pharmacoepidemiology. 3rd ed. New York: Wiley, 2000: 151–74
Wiholm B, Olsson S, Moore N, et al. Spontaneous reporting systems outside the US. In: Strom B, editor. Pharmacoepidemiology. 3rd ed. New York: Wiley, 2000: 175–92
Reintjes R, de Boer A, van Pelt W. Simpson’s paradox: an example from hospital epidemiology. Epidemiology 2000; 11(1): 81–3
Hauben M. A brief primer on automated signal detection. Ann Pharmacother 2002; 37(7/8): 1117–26
Lindquist M, Stahl M, Bate A. A retrospective evaluation of a data mining approach to aid finding new adverse drug reaction signals in the WHO international database. Drug Saf 2000; 23(6): 533–42
Gould AL. Practical pharmacovigilance analysis strategies. Pharmacoepidemiol Drug Saf 2003; 12(7): 559–74
Van Puijenbroek E, Bate A, Leufkens H, et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol Drug Saf 2002; 11(1): 3–10
Medwatch: the FDA safety information and adverse drug event reporting program [online]. Available from URL: http://www.fda.gov/medwatch/index.html [Accessed 2004 Jul 12]
Center for Drug Evaluation and Research. Drug information: Adverse Event Reporting System (AERS) [online]. Available from URL: http://www.fda.gov/cder/aers/default.htm [Accessed 2004 Jul 12]
Hauben M. Trimethoprim-induced hyperkalemia: lessons in data mining. Br J Clin Pharmacol. D0I:10.1111/j.1365-2125.2004.02153.x [online]. Available from URL: http://www.blackwell-synergy.com/ [Accessed 2004 Jul 12]
Hauben M, Reich L. Drug-induced pancreatitis: lessons in data mining. Br J Clin Pharmacol. In press
Acknowledgements
We would like to express our sincere thanks to our intern, Miss Stephanie Chung, for her help in preparation for this paper.
No sources of funding were used to assist in conducting this study. The authors have no conflicts of interest that are directly relevant to the content of this study.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hauben, M., Reich, L. Safety Related Drug-Labelling Changes. Drug-Safety 27, 735–744 (2004). https://doi.org/10.2165/00002018-200427100-00004
Published:
Issue Date:
DOI: https://doi.org/10.2165/00002018-200427100-00004