Drug Safety

, Volume 26, Issue 3, pp 159–186 | Cite as

Quantitative Methods in Pharmacovigilance

Focus on Signal Detection
  • Manfred Hauben
  • Xiaofeng Zhou
Review Article


Pharmacovigilance serves to detect previously unrecognised adverse events associated with the use of medicines. The simplest method for detecting signals of such events is crude inspection of lists of spontaneously reported drug-event combinations. Quantitative and automated numerator-based methods such as Bayesian data mining can supplement or supplant these methods. The theoretical basis and limitations of these methods should be understood by drug safety professionals, and automated methods should not be automatically accepted. Published evaluations of these techniques are mainly limited to large regulatory databases, and performance characteristics may differ in smaller safety databases of drug developers. Head-to-head comparisons of the major techniques have not been published. Regardless of previous statistical training, pharmacovigilance practitioners should understand how these methods work. The mathematical basis of these techniques should not obscure the numerous confounders and biases inherent in the data. This article seeks to make automated signal detection methods transparent to drug safety professionals of various backgrounds. This is accomplished by first providing a brief overview of the evolution of signal detection followed by a series of sections devoted to the methods with the greatest utilisation and evidentiary support: proportional reporting rations, the Bayesian Confidence Propagation Neural Network and empirical Bayes screening. Sophisticated yet intuitive explanations are provided for each method, supported by figures in which the underlying statistical concepts are explored. Finally the strengths, limitations, pitfalls and outstanding unresolved issues are discussed. Pharmacovigilance specialists should not be intimidated by the mathematics. Understanding the theoretical basis of these methods should enhance the effective assessment and possible implementation of these techniques by drug safety professionals.


Signal Score Information Component Spontaneous Reporting System Proportional Reporting Ratio Medicine Control Agency 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Dedicated to the memory of my father Richard S. Hauben, MD.

We are grateful for the thoughtful insights and encouragement of Dr Ana Szarfman of the FDA and Andrew Bate of the WHO Collaborating Centre for International Drug Monitoring.


  1. 1.
    Meyboom RHB, Egberts ACG, Edwards IR, et al. Principles of signal detection in pharmacovigilance. Drug Saf 1997; 16(6): 355–65PubMedCrossRefGoogle Scholar
  2. 2.
    Waller PC, Lee EH. Responding to drug safety issues. Pharmacoepidemiol Drug Saf 1999; 8: 535–52PubMedCrossRefGoogle Scholar
  3. 3.
    Lindquist M, Stahl M, Bate A, et al. A retrospective evaluation of a data mining approach to aid finding new adverse drug reaction signals in the WHO database. Drug Saf 2000; 23(6): 533–42PubMedCrossRefGoogle Scholar
  4. 4.
    Kaufman DW, Rosenberg L, Mitchell A. Signal generation and clarification: use of case-control data. Pharmacoepidemiol Drug Saf 2001; 10: 197–203PubMedCrossRefGoogle Scholar
  5. 5.
    Evans SJW. Pharmacovigilance: a science or fielding emergencies? Stat Med 2000; 19: 3199–209PubMedCrossRefGoogle Scholar
  6. 6.
    Lindquist M, Edwards IR, Bate A, et al. From association to alert: a revised approach to international signal analysis. Pharmacoepidemiol Drug Saf 1999; 8: S15–25PubMedCrossRefGoogle Scholar
  7. 7.
    Evans SJW, Waller P, Davis S. Proportional reporting ratios: the use of epidemiological methods for signal detection [abstract]. Pharmacoepidemiol Drug Saf 1998; 7Suppl. 2: S102Google Scholar
  8. 8.
    Evans SJW, Waller PC, Davis S. Use of proportional reporting ratios for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001; 10: 483–6PubMedCrossRefGoogle Scholar
  9. 9.
    Heeley EL, Wilton LV, Shakir SAW. The use of proportional reporting rations for signal generation in prescription event monitoring [abstract]. Pharmacoepidemiol Drug Saf 2002; 10(S1): S48Google Scholar
  10. 10.
    Kilgour-Christie J, Czarnecki A, Simmons V. Proportional reporting ratio calculations from a pharmaceutical company database [abstract]. Pharmacoepidemiol Drug Saf 2001; 10: S159Google Scholar
  11. 11.
    DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat 1999; 53(3): 177–90Google Scholar
  12. 12.
    O’Neill RT, Szarfman A. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system [discussion]. Am Stat 1999; 53(3): 190–5Google Scholar
  13. 13.
    Louis TA, Shen W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system [discussion]. Am Stat 1999; 53(3): 196–8Google Scholar
  14. 14.
    Madigan D. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system [discussion]. Am Stat 1999; 53(3): 198–200Google Scholar
  15. 15.
    DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system [reply]. Am Stat 1999; 53(3): 201–2Google Scholar
  16. 16.
    Praus M, Schindel F, Fescharek R, et al. Alert systems for postmarketing surveillance of adverse drug reactions. Stat Med 1993; 12: 2383–93PubMedCrossRefGoogle Scholar
  17. 17.
    Shore DL, Quade D. A surveillance system based on a short memory scheme. Stat Med 1989; 8: 311–22PubMedCrossRefGoogle Scholar
  18. 18.
    Norwood PK, Sampson AR. A statistical methodology for postmarketing surveillance of adverse drug reaction reports. Stat Med 1988; 7: 1023–30PubMedCrossRefGoogle Scholar
  19. 19.
    Parker RA. Analysis of surveillance data with Poisson regression: a case study. Stat Med 1989; 8: 285–94PubMedCrossRefGoogle Scholar
  20. 20.
    Tubert P, Begaud B, Haramburu F, et al. Spontaneous reporting: how many cases are required to trigger a warning? Br J Clin Pharmacol 1991; 32: 407–8PubMedCrossRefGoogle Scholar
  21. 21.
    Schroeder DR. Detecting a rare adverse drug reaction using spontaneous reports [statistics]. Reg Anesth Pain Med 1998; 2: 183–9Google Scholar
  22. 22.
    Purcell P, Barty S. Statistical techniques for signal generation: the Australian experience. Drug Saf 2002; 25(6): 415–21PubMedCrossRefGoogle Scholar
  23. 23.
    Heeley E, Wilton L, Shakir A. Automated signal generation in prescription event monitoring. Drug Saf 2002; 25(6): 423–32PubMedCrossRefGoogle Scholar
  24. 24.
    Evans S. Statistical and epidemiological principles in detecting signals of adverse drug reactions. Drug Information Association 38th Annual Meeting; 2002 Jun 16-20: Chicago (IL)Google Scholar
  25. 25.
    Code of Federal Regulations (CFR). Parts 310, 314, and 600 [online]. Available from URL: [Accessed 2003 Jan 28]Google Scholar
  26. 26.
    Clark JA, Berk RH, Klincewitz SL. Calculation of the probability of multiplicities in two cell-occupancy models: implications for spontaneous reporting systems. Drug Inf J 1999; 33: 1195–203CrossRefGoogle Scholar
  27. 27.
    Begaud B, Moride Y, Tubert-Bitter P, et al. False positives in spontaneous reporting: should we worry about them? Br J Clin Pharmacol 1994; 38: 401–4PubMedCrossRefGoogle Scholar
  28. 28.
    Kwong K, Sammon J, Hornbuckle K, et al. A novel method for detecting potential drug-drug interactions using post-marketing adverse event database [abstract]. Pharmacoepidemiol Drug Saf 2001; 10(S1): S39Google Scholar
  29. 29.
    O’Neill RT, Szarfman A. Some US Food and Drug Administration perspectives on data mining for pediatric safety assessment. Curr Ther Res Clin Exp 2001; 62: 650–63CrossRefGoogle Scholar
  30. 30.
    A new strategy for detection of signals in the WHO database 2002. Uppsala Reports 2001 Nov; 17: 8–9Google Scholar
  31. 31.
    Szarfman A, Machado GS, O’Neill RT. Screening algorithms and computer systems to efficiently signal higher-than expected combinations of drugs and events in FDA’s spontaneous reports database. Drug Saf 2002; 25(6): 381–92PubMedCrossRefGoogle Scholar
  32. 32.
    Thakrar BT, Blesch KS, Sacks ST, et al. On technical methods of signal detection [abstract]. Pharmacoepidemiol Drug Saf 2001; 10(S1): S98Google Scholar
  33. 33.
    Van Puijenbroek E, Bate A, Leufkens H, et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol Drug Saf 2002; 11: 3–10PubMedCrossRefGoogle Scholar
  34. 34.
    Coulter DM, Bate A, Meyboom R, et al. Antipsychotic drugs and heart muscle disorder in international pharmacovigilance: data mining study. BMJ 2001; 232: 1207–9CrossRefGoogle Scholar
  35. 35.
    Bate A, Lindquist M, Edwards IR. Pattern recognition using a recurrent neural network and its application to the WHO database [abstract]. 17th International Society for Pharmacoepidemiology; 2001 Aug 23-26; Toronto, CanadaGoogle Scholar
  36. 36.
    DuMouchel W, Pregibon D. Empirical Bayes screening for multi-item associations. Proc. KDD. San Diego (CA): ACM Press, 2001Google Scholar

Copyright information

© Adis International Limited 2003

Authors and Affiliations

  1. 1.Safety Evaluation and Epidemiology, Pfizer Inc.New YorkUSA
  2. 2.New York University School of MedicineNew YorkUSA
  3. 3.New York Medical CollegeValhallaUSA
  4. 4.Clinical Safety and Risk Management, Worldwide Regulatory Affairs, Pfizer Inc.Ann ArborUSA

Personalised recommendations