Number Needed to Detect
- 63 Downloads
- 1 Citations
Abstract
Data mining algorithms are increasingly being used to support the process of signal detection and evaluation in pharmacovigilance. Published data mining exercises formulated within a screening paradigm typically calculate classical performance indicators such as sensitivity, specificity, predictive value and receiver operator characteristic curves. Extrapolating signal detection performance from these isolated data mining exercises to performance in real-world pharmacovigilance scenarios is complicated by numerous factors and some published exercises may promote an inappropriate and exclusive focus on only one aspect of performance. In this article, we discuss a variation on positive predictive value that we call the ‘number needed to detect’ that provides a simple and intuitive screening metric that might usefully supplement the usual presentations of data mining performance. We use a series of figures to demonstrate the nature and application of this metric, and selected adaptive variations. Even with simple and intuitive metrics, precisely quantifying the performance of contemporary data mining algorithms in pharmacovigilance is complicated by the complexity of the phenomena under surveillance and the manner in which the data are recorded in spontaneous reporting systems.
Keywords
Data Mining Algorithm Spontaneous Reporting System Credible Signal Classifier Efficiency Disproportionate ReportingNotes
Acknowledgements
The views expressed in this paper are those of the authors and not necessarily the official view of the European Medicines Agency. No sources of funding were used to assist in the preparation of this review. The authors have no conflicts of interest that are directly relevant to the content of this review. The authors would like to thank the following individuals who generously shared their time in reviewing early versions of this manuscript and providing thoughtful insights: Alan Hochberg, Jeffrey Aronson, David Goldsmith, David Madigan and Panos Tsintis.
References
- 1.Bate A, Lindquist M, Edwards IR, et al. A bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol 1998; 54: 315–21PubMedCrossRefGoogle Scholar
- 2.DuMouchel W. Bayesian data mining in large frequency tables with an application to the FDA spontaneous reporting system. Am Stat 1999; 53 (3): 177–202Google Scholar
- 3.Moore N. Thiessard F, Begaud B. The history of disproportionality measures (reporting odds ratio, proportional reporting rates) in spontaneous reporting of adverse drug reactions. Pharmacoepidemiol Drug Saf 2005; 14: 285–86PubMedCrossRefGoogle Scholar
- 4.Hauben M, Madigan D, Gerrits C, et al. The role of data mining in pharmacovigilance. Expert Opin Drug Saf 2005; 4 (5): 929–48PubMedCrossRefGoogle Scholar
- 5.Hauben M. Signal detection in the pharmaceutical industry: integrating clinical and computational approaches. Drug Saf 2007; 30 (7): 627–30PubMedCrossRefGoogle Scholar
- 6.Hauben M, Reich L, Chung S. Post-marketing surveillance of potentially fatal reactions to oncology drugs: potential utility of two signal detection algorithms. Eur J Clin Pharmacol 2004; 60 (10): 747–50PubMedCrossRefGoogle Scholar
- 7.Hauben M, Reich L. Communication of findings in pharmacovigilance: use of the term “signal” and the need for precision in its use. Eur J Clin Pharmacol 2005; 61 (5–6): 479–80PubMedCrossRefGoogle Scholar
- 8.Hauben M, Reich L. Reply: the evaluation of data mining methods for the simultaneous and systematic detection of safety signals in large databases: lessons to be learned. Response to letter by Levine, et al. Br J Clin Pharmacol 2006; 61: 115–7CrossRefGoogle Scholar
- 9.Hauben M, Aronson JK. Gold standards in pharmacovigilance-the use of definitive anecdotal reports of adverse drug reactions as pure gold and high-grade ore. Drug Saf 2007; 30: 645–55PubMedCrossRefGoogle Scholar
- 10.Henegar C, Bousquet C, Lillo-Le Louët A, et al. A knowledge based approach for automated signal generation in pharmacovigilance. Proceedings of the 11th World Congress on Medical Informatics; 2004 Sep 7–11; San Francisco (CA) [online]. Available from URL: http://cmbi.bjmu.edu.cn/news/report/2004/medinfo2004/pdffiles/papers/4696Henegar.pdf [Accessed 2008 Jan 28]Google Scholar
- 11.Hauben M, Van Puijenbroek EP, Gerrits C, et al. Data mining in pharmacovigilance: lessons from phantom ships. Eur J Clin Pharmacol 2006; 62 (11): 967–70PubMedCrossRefGoogle Scholar