Number Needed to Detect

Abstract

Data mining algorithms are increasingly being used to support the process of signal detection and evaluation in pharmacovigilance. Published data mining exercises formulated within a screening paradigm typically calculate classical performance indicators such as sensitivity, specificity, predictive value and receiver operator characteristic curves. Extrapolating signal detection performance from these isolated data mining exercises to performance in real-world pharmacovigilance scenarios is complicated by numerous factors and some published exercises may promote an inappropriate and exclusive focus on only one aspect of performance. In this article, we discuss a variation on positive predictive value that we call the ‘number needed to detect’ that provides a simple and intuitive screening metric that might usefully supplement the usual presentations of data mining performance. We use a series of figures to demonstrate the nature and application of this metric, and selected adaptive variations. Even with simple and intuitive metrics, precisely quantifying the performance of contemporary data mining algorithms in pharmacovigilance is complicated by the complexity of the phenomena under surveillance and the manner in which the data are recorded in spontaneous reporting systems.