Measuring Signal Detection Performance: Can We Trust Negative Controls and Do We Need Them?

Once a drug is introduced into clinical practice pharmaceutical manufacturers and regulatory agencies invest considerable time and effort in scrutinising data from treated patients to identify previously unknown adverse reactions. There are a number of methods to do this but none have perfect accuracy and all inaccuracies come with penalties. False-positive signals waste time and effort in further investigation and false negatives mean that doctors and patients remain unaware of the problem and may be unable to identify the appropriate actions to prevent or treat an adverse event. To choose the best methods to monitor safety it is important to quantify the performance of all possible methods.

One of the challenges in assessing whether a signal detection method works is that our knowledge of which adverse event may be caused by any drug is incomplete. Thus, the ‘reference standard’ against which to test is not clear. The Observational Medical Outcomes...