Drug Safety

, Volume 40, Issue 8, pp 703–713 | Cite as

Detecting Signals of Disproportionate Reporting from Singapore’s Spontaneous Adverse Event Reporting System: An Application of the Sequential Probability Ratio Test

  • Cheng Leng ChanEmail author
  • Sowmya Rudrappa
  • Pei San Ang
  • Shu Chuen Li
  • Stephen J. W. Evans
Original Research Article



The ability to detect safety concerns from spontaneous adverse drug reaction reports in a timely and efficient manner remains important in public health.


This paper explores the behaviour of the Sequential Probability Ratio Test (SPRT) and ability to detect signals of disproportionate reporting (SDRs) in the Singapore context.


We used SPRT with a combination of two hypothesised relative risks (hRRs) of 2 and 4.1 to detect signals of both common and rare adverse events in our small database. We compared SPRT with other methods in terms of number of signals detected and whether labelled adverse drug reactions were detected or the reaction terms were considered serious. The other methods used were reporting odds ratio (ROR), Bayesian Confidence Propagation Neural Network (BCPNN) and Gamma Poisson Shrinker (GPS).


The SPRT produced 2187 signals in common with all methods, 268 unique signals, and 70 signals in common with at least one other method, and did not produce signals in 178 cases where two other methods detected them, and there were 403 signals unique to one of the other methods. In terms of sensitivity, ROR performed better than other methods, but the SPRT method found more new signals. The performances of the methods were similar for negative predictive value and specificity.


Using a combination of hRRs for SPRT could be a useful screening tool for regulatory agencies, and more detailed investigation of the medical utility of the system is merited.


Compliance with Ethical Standards

Ethical approval

All data used in this study are anonymised, and ethical approval was not required.


The SAPhIRE (Surveillance and Pharmacogenomics Initiative for Adverse Drug Reactions) Project was funded by a Strategic Positioning Funds grant from the Biomedical Research Council of the Agency for Science, Technology and Research of Singapore (SPF2014/001).

Conflict of interest

Cheng Leng Chan, Sowmya Rudrappa, Pei San Ang, Shu Chuen Li and Stephen J.W. Evans have no conflicts of interest that are directly relevant to the content of this study.


  1. 1.
    Waller PC, Evans SJW. A model for the future conduct of pharmacovigilance. Pharmacoepidemiol Drug Saf. 2003;12(1):17–29. doi: 10.1002/pds.773.CrossRefPubMedGoogle Scholar
  2. 2.
    Waller P. An introduction to pharmacovigilance. UK: Wiley-Blackwell; 2010.Google Scholar
  3. 3.
    Koh Y LA, Tan L, Ang PS, Tan SH, Toh D, Chan CL. Pharmacovigilance in Singapore—harnessing IT and genomics to detect safety signals. Regul Aff J-Pharma. 2012;(Oct issue):13–5. Accessed 15 Mar 2017.
  4. 4.
    Hauben M, Bate A. Decision support methods for the detection of adverse events in post-marketing data. Drug Discov Today. 2009;14(7–8):343–57. doi: 10.1016/j.drudis.2008.12.012.CrossRefPubMedGoogle Scholar
  5. 5.
    Zink RC, Huang Q, Zhang L-Y, Bao W-J. Statistical and graphical approaches for disproportionality analysis of spontaneously-reported adverse events in pharmacovigilance. Chin J Nat Med. 2013;11(3):314–20. doi: 10.1016/S1875-5364(13)60035-7.CrossRefPubMedGoogle Scholar
  6. 6.
    Evans SJW, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf. 2001;10(6):483–6. doi: 10.1002/pds.677.CrossRefPubMedGoogle Scholar
  7. 7.
    Bate A, Evans SJW. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiol Drug Saf. 2009;18(6):427–36. doi: 10.1002/pds.1742.CrossRefPubMedGoogle Scholar
  8. 8.
    Egberts AC, Meyboom RH, van Puijenbroek EP. Use of measures of disproportionality in pharmacovigilance: three Dutch examples. Drug Saf. 2002;25(6):453–8. doi: 10.2165/00002018-200225060-00010.CrossRefPubMedGoogle Scholar
  9. 9.
    Bate A, Lindquist M, Edwards IR, Olsson S, Orre R, Lansner A, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol. 1998;54(4):315–21. doi: 10.1007/s002280050466.CrossRefPubMedGoogle Scholar
  10. 10.
    Dumouchel W. Bayesian data mining in large frequency tables, with an application to the FDA Spontaneous Reporting System. Am Stat. 1999;53(3):177–90. doi: 10.1080/00031305.1999.10474456.CrossRefGoogle Scholar
  11. 11.
    Szarfman A, Machado SG, O’Neill RT. Use of screening algorithms and computer systems to efficiently signal higher-than-expected combinations of drugs and events in the US FDA’s Spontaneous Reports Database. Drug Saf. 2002;25(6):381–92. doi: 10.2165/00002018-200225060-00001.CrossRefPubMedGoogle Scholar
  12. 12.
    Gould AL. Detecting potential safety issues in clinical trials by Bayesian screening. Biom J. 2008;50(5):837–51. doi: 10.1002/bimj.200710469.CrossRefPubMedGoogle Scholar
  13. 13.
    Candore G, Juhlin K, Manlik K, Thakrar B, Quarcoo N, Seabroke S, et al. Comparison of statistical signal detection methods within and across spontaneous reporting databases. Drug Saf. 2015;38(6):577–87. doi: 10.1007/s40264-015-0289-5.CrossRefPubMedGoogle Scholar
  14. 14.
    Wald A. Sequential tests of statistical hypotheses. Ann Math Stat. 1945;16(2):117–86.CrossRefGoogle Scholar
  15. 15.
    Wald A. Sequential analysis. New York: Wiley; 1947.Google Scholar
  16. 16.
    Evans S. Sequential probability ratio tests applied to public health problems. Control Clin Trials. 2003;24:67S.Google Scholar
  17. 17.
    Talbot J, Aronson JK. Stephens’ detection and evaluation of adverse drug reactions: principles and practice. 6th ed. Oxford: Wiley-Blackwell; 2012.Google Scholar
  18. 18.
    Leite A, Andrews NJ, Thomas SL. Near real-time vaccine safety surveillance using electronic health records—a systematic review of the application of statistical methods. Pharmacoepidemiol Drug Saf. 2016;25(3):225–37. doi: 10.1002/pds.3966.CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Spiegelhalter D, Grigg O, Kinsman R, Treasure T. Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care. 2003;15(1):7–13.CrossRefGoogle Scholar
  20. 20.
    Roux E, Thiessard F, Fourrier A, Begaud B, Tubert-Bitter P. Evaluation of statistical association measures for the automatic signal generation in pharmacovigilance. IEEE Trans Inf Technol Biomed. 2005;9(4):518–27. doi: 10.1109/TITB.2005.855566A.CrossRefPubMedGoogle Scholar
  21. 21.
    Ang PS, Chen Z, Chan CL, Tai BC. Data mining spontaneous adverse drug event reports for safety signals in Singapore—a comparison of three different disproportionality measures. Expert Opin Drug Saf. 2016;15(5):583–90. doi: 10.1517/14740338.2016.1167184.CrossRefPubMedGoogle Scholar
  22. 22.
    Stahl M, Lindquist M, Edwards IR, Brown EG. Introducing triage logic as a new strategy for the detection of signals in the WHO Drug Monitoring Database. Pharmacoepidemiol Drug Saf. 2004;13(6):355–63. doi: 10.1002/pds.894.CrossRefPubMedGoogle Scholar
  23. 23.
    Micromedex® 2.0 (electronic version). Truven Health Analytics, Greenwood Village, Colorado, USA. Accessed 19 Sept 2015.
  24. 24.
    Safety reporting requirements for human drug and biological products; Federal Register. Department of Health and Human Services. US Food and Drug Administration. 14 March 2003. Accessed 19 Sept 2015.
  25. 25.
    Important Medical Event Terms list (based on MedDRA version 18.1). EudraVigilance Expert Working Group. Accessed 19 Sept 2015.
  26. 26.
    R Development Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. 2014. Accessed 23 Sept 2016.
  27. 27.
    Ahmed I, Poncet A. Package ‘PhViD’. Version:1.0.6. 2013. Accessed 19 Sept 2015.
  28. 28.
    Bisdorff AR. Management of vestibular migraine. Ther Adv Neurol Disord. 2011;4(3):183–91. doi: 10.1177/1756285611401647.CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Ahmed I, Thiessard F, Miremont-Salame G, Haramburu F, Kreft-Jais C, Be’gaud B, et al. Early detection of pharmacovigilance signals with automated methods based on false discovery rates. Drug Saf. 2012;35(6):495–506. doi: 10.2165/11597180-000000000-00000.CrossRefPubMedGoogle Scholar
  30. 30.
    Noren GN, Hopstadius J, Bate A. Shrinkage observed-to-expected ratios for robust and transparent large-scale pattern discovery. Stat Methods Med Res. 2013;22(1):57–69. doi: 10.1177/0962280211403604.CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Cheng Leng Chan
    • 1
    • 2
    Email author
  • Sowmya Rudrappa
    • 1
    • 3
  • Pei San Ang
    • 1
  • Shu Chuen Li
    • 2
  • Stephen J. W. Evans
    • 4
  1. 1.Health Products Regulation GroupHealth Sciences AuthoritySingaporeSingapore
  2. 2.School of Biomedical Sciences and PharmacyThe University of NewcastleCallaghanAustralia
  3. 3.Genome Institute of Singapore (A-Star)SingaporeSingapore
  4. 4.Department of Medical Statistics, London School of Hygiene and Tropical MedicineUniversity of LondonLondonUK

Personalised recommendations