Drug Safety

, Volume 41, Issue 4, pp 377–387 | Cite as

An Automated System Combining Safety Signal Detection and Prioritization from Healthcare Databases: A Pilot Study

  • Mickael Arnaud
  • Bernard Bégaud
  • Frantz Thiessard
  • Quentin Jarrion
  • Julien Bezin
  • Antoine Pariente
  • Francesco SalvoEmail author
Original Research Article



Signal detection from healthcare databases is possible, but is not yet used for routine surveillance of drug safety. One challenge is to develop methods for selecting signals that should be assessed with priority.


The aim of this study was to develop an automated system combining safety signal detection and prioritization from healthcare databases and applicable to drugs used in chronic diseases.


Patients present in the French EGB healthcare database for at least 1 year between 2005 and 2015 were considered. Noninsulin glucose-lowering drugs (NIGLDs) were selected as a case study, and hospitalization data were used to select important medical events (IME). Signal detection was performed quarterly from 2008 to 2015 using sequence symmetry analysis. NIGLD/IME associations were screened if one or more exposed case was identified in the quarter, and three or more exposed cases were identified in the population at the date of screening. Detected signals were prioritized using the Longitudinal-SNIP (L-SNIP) algorithm based on strength (S), novelty (N), and potential impact of signal (I), and pattern of drug use (P). Signals scored in the top 10% were identified as of high priority. A reference set was built based on NIGLD summaries of product characteristics (SPCs) to compute the performance of the developed system.


A total of 815 associations were screened and 241 (29.6%) were detected as signals; among these, 58 (24.1%) were prioritized. The performance for signal detection was sensitivity = 47%; specificity = 80%; positive predictive value (PPV) 33%; negative predictive value = 82%. The use of the L-SNIP algorithm increased the early identification of positive controls, restricted to those mentioned in the SPCs after 2008: PPV = 100% versus PPV = 14% with its non-use. The system revealed a strong new signal with dipeptidylpeptidase-4 inhibitors and venous thromboembolism.


The developed system seems promising for the routine use of healthcare data for safety surveillance of drugs used in chronic diseases.


Compliance with ethical standards

Conflicts of interest

Mickael Arnaud, Bernard Bégaud, Frantz Thiessard, Quentin Jarrion, Julien Bezin, Antoine Pariente, and Francesco Salvo have no conflict of interest directly relevant to the content of this study.


This study is part of the Drugs Systematized Assessment in real-liFe Environment (DRUGS-SAFE) research program that is funded by the French Medicines Agency (Agence Nationale de Sécurité du Médicament et des Produits de Santé, ANSM). The program aims at providing an integrated system allowing the concomitant monitoring of drug use and safety in France. The potential impact of drugs (e.g., non-insulin glucose-lowering drugs), the frailty of populations, and the seriousness of risks drive the research program. This publication represents the views of the authors and does not necessarily represent the opinion of the French Medicines Agency.

Supplementary material

40264_2017_618_MOESM1_ESM.pdf (707 kb)
Supplementary material 1 (PDF 706 kb)


  1. 1.
    Mukherjee D, Nissen SE, Topol EJ. Risk of cardiovascular events associated with selective COX-2 inhibitors. JAMA. 2001;286:954–9.CrossRefPubMedGoogle Scholar
  2. 2.
    Lévesque LE, Brophy JM, Zhang B. The risk for myocardial infarction with cyclooxygenase-2 inhibitors: a population study of elderly adults. Ann Intern Med. 2005;142:481.CrossRefPubMedGoogle Scholar
  3. 3.
    Laheij RJ, Sturkenboom MC, Hassing R-J, Dieleman J, Stricker BH, Jansen JB. Risk of community-acquired pneumonia and use of gastric acid–suppressive drugs. JAMA. 2004;292:1955–60.CrossRefPubMedGoogle Scholar
  4. 4.
    Johnstone J, Nerenberg K, Loeb M. Meta-analysis: proton pump inhibitor use and the risk of community-acquired pneumonia. Aliment Pharmacol Ther. 2010;31:1165–77.CrossRefPubMedGoogle Scholar
  5. 5.
    Yang Y-X, Lewis JD, Epstein S, Metz DC. Long-term proton pump inhibitor therapy and risk of hip fracture. JAMA. 2006;296:2947–53.CrossRefPubMedGoogle Scholar
  6. 6.
    Vestergaard P, Rejnmark L, Mosekilde L. Proton pump inhibitors, histamine H2 receptor antagonists, and other antacid medications and the risk of fracture. Calcif Tissue Int. 2006;79:76–83.CrossRefPubMedGoogle Scholar
  7. 7.
    Targownik LE, Lix LM, Metge CJ, Prior HJ, Leung S, Leslie WD. Use of proton pump inhibitors and risk of osteoporosis-related fractures. Can Med Assoc J. 2008;179:319–26.CrossRefGoogle Scholar
  8. 8.
    Lipscombe LL, Gomes T, Lévesque LE, Hux JE, Juurlink DN, Alter DA. Thiazolidinediones and cardiovascular outcomes in older patients with diabetes. JAMA. 2007;298:2634–43.CrossRefPubMedGoogle Scholar
  9. 9.
    Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med. 2007;356:2457–71.CrossRefPubMedGoogle Scholar
  10. 10.
    Neumann A, Weill A, Ricordeau P, Fagot JP, Alla F, Allemand H. Pioglitazone and risk of bladder cancer among diabetic patients in France: a population-based cohort study. Diabetologia. 2012;55:1953–62.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Tuccori M, Filion KB, Yin H, Yu OH, Platt RW, Azoulay L. Pioglitazone use and risk of bladder cancer: population based cohort study. BMJ. 2016;352:i1541.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Hillaire-Buys D, Faillie J-L. Pioglitazone and the risk of bladder cancer. BMJ. 2012;344:e3500.CrossRefPubMedGoogle Scholar
  13. 13.
    Coloma PM, Schuemie MJ, Trifirò G, Gini R, Herings R, Hippisley-Cox J, et al. Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EU-ADR Project. Pharmacoepidemiol Drug Saf. 2011;20:1–11.CrossRefPubMedGoogle Scholar
  14. 14.
    Wisniewski AFZ, Bate A, Bousquet C, Brueckner A, Candore G, Juhlin K, et al. Good signal detection practices: evidence from IMI PROTECT. Drug Saf. 2016;39:469–90.CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Platt R, Carnahan RM, Brown JS, Chrischilles E, Curtis LH, Hennessy S, et al. The U.S. Food and Drug Administration’s Mini-Sentinel program: status and direction. Pharmacoepidemiol Drug Saf. 2012;21:1–8.PubMedGoogle Scholar
  16. 16.
    Stang PE, Ryan PB, Racoosin JA, Overhage JM, Hartzema AG, Reich C, et al. Advancing the science for active surveillance: rationale and design for the Observational Medical Outcomes Partnership. Ann Intern Med. 2010;153:600–6.CrossRefPubMedGoogle Scholar
  17. 17.
    Andersen M, Bergman U, Choi N-K, Gerhard T, Huang C, Jalbert J, et al. The Asian Pharmacoepidemiology Network (AsPEN): promoting multi-national collaboration for pharmacoepidemiologic research in Asia. Pharmacoepidemiol Drug Saf. 2013;22:700–4.CrossRefPubMedGoogle Scholar
  18. 18.
    Ryan PB, Stang PE, Overhage JM, Suchard MA, Hartzema AG, DuMouchel W, et al. A comparison of the empirical performance of methods for a risk identification system. Drug Saf. 2013;36:143–58.CrossRefGoogle Scholar
  19. 19.
    Schuemie MJ, Gini R, Coloma PM, Straatman H, Herings RMC, Pedersen L, et al. Replication of the OMOP experiment in Europe: evaluating methods for risk identification in electronic health record databases. Drug Saf. 2013;36:159–69.CrossRefGoogle Scholar
  20. 20.
    Pratt N, Andersen M, Bergman U, Choi N-K, Gerhard T, Huang C, et al. Multi-country rapid adverse drug event assessment: the Asian Pharmacoepidemiology Network (AsPEN) antipsychotic and acute hyperglycaemia study. Pharmacoepidemiol Drug Saf. 2013;22:915–24.PubMedGoogle Scholar
  21. 21.
    Kulldorff M, Dashevsky I, Avery TR, Chan AK, Davis RL, Graham D, et al. Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiol Drug Saf. 2013;22:517–23.CrossRefPubMedGoogle Scholar
  22. 22.
    Waller PC, Lee EH. Responding to drug safety issues. Pharmacoepidemiol Drug Saf. 1999;8:535–52.CrossRefPubMedGoogle Scholar
  23. 23.
    Waller P, Heeley E, Moseley J. Impact analysis of signals detected from spontaneous adverse drug reaction reporting data. Drug Saf. 2005;28:843–50.CrossRefPubMedGoogle Scholar
  24. 24.
    Seabroke S, Wise L, Waller P. Development of a novel regulatory pharmacovigilance prioritisation system: an evaluation of its performance at the UK medicines and healthcare products regulatory agency. Drug Saf. 2013;36:1025–32.CrossRefPubMedGoogle Scholar
  25. 25.
    Meyboom RH, Lindquist M, Egberts AC, Edwards IR. Signal selection and follow-up in pharmacovigilance. Drug Saf. 2002;25:459–65.CrossRefPubMedGoogle Scholar
  26. 26.
    Ståhl M, Lindquist M, Edwards IR, Brown EG. Introducing triage logic as a new strategy for the detection of signals in the WHO Drug Monitoring Database. Pharmacoepidemiol Drug Saf. 2004;13:355–63.CrossRefPubMedGoogle Scholar
  27. 27.
    Lindquist M. Use of triage strategies in the WHO signal-detection process. Drug Saf. 2007;30:635–7.CrossRefPubMedGoogle Scholar
  28. 28.
    Caster O, Juhlin K, Watson S, Norén GN. Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence aspects in vigiRank: retrospective evaluation against emerging safety signals. Drug Saf. 2014;37:617–28.CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Van Puijenbroek EP, Van Grootheest K, Diemont WL, Leufkens HG, Egberts AC. Determinants of signal selection in a spontaneous reporting system for adverse drug reactions. Br J Clin Pharmacol. 2001;52:579–86.CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Levitan B, Yee CL, Russo L, Bayney R, Thomas AP, Klincewicz SL. A model for decision support in signal triage. Drug Saf. 2008;31:727–35.CrossRefPubMedGoogle Scholar
  31. 31.
    Jamekornkul C, Chaisumritchoke ST. Developing a signal triage algorithm for Thai national adverse drug reaction database. Thai J Pharm Sci. 2016;40(3):153–7.Google Scholar
  32. 32.
    Salvo F, Raschi E, Moretti U, Chiarolanza A, Fourrier-Réglat A, Moore N, et al. Pharmacological prioritisation of signals of disproportionate reporting: proposal of an algorithm and pilot evaluation. Eur J Clin Pharmacol. 2014;70:617–25.CrossRefPubMedGoogle Scholar
  33. 33.
    European Medicines Agency. Guideline on good pharmacovigilance practices (GVP): module IX—signal management. 2012. Accessed 18 Aug 2017.
  34. 34.
    United States Food and Drug Administration. Classifying significant postmarketing drug safety issues. Fed Regist. 2012. Accessed 16 Nov 2017.
  35. 35.
    Sottosanti L, Ferrazin F. Italian pharmacovigilance system. Manns Pharmacovigil. 3rd ed. Chichester: Wiley; 2014.Google Scholar
  36. 36.
    Coloma PM, Schuemie MJ, Trifirò G, Furlong L, van Mulligen E, Bauer-Mehren A, et al. Drug-induced acute myocardial infarction: identifying “prime suspects” from electronic healthcare records-based surveillance system. PLoS ONE 2013;8:e72148.
  37. 37.
    Bezin J, Duong M, Lassalle R, Droz C, Pariente A, Blin P, et al. The national healthcare system claims databases in France, SNIIRAM and EGB: Powerful tools for pharmacoepidemiology. Pharmacoepidemiol. Drug Saf. 2017. Accessed 2 Jun 2017.
  38. 38.
    European Medicines Agency. Inclusion/exclusion criteria for the “Important medical events” list. 2017. Accessed 16 Nov 2017.
  39. 39.
    Tsiropoulos I, Andersen M, Hallas J. Adverse events with use of antiepileptic drugs: a prescription and event symmetry analysis. Pharmacoepidemiol Drug Saf. 2009;18:483–91.CrossRefPubMedGoogle Scholar
  40. 40.
    Wahab IA, Pratt NL, Wiese MD, Kalisch LM, Roughead EE. The validity of sequence symmetry analysis (SSA) for adverse drug reaction signal detection. Pharmacoepidemiol Drug Saf. 2013;22:496–502.CrossRefPubMedGoogle Scholar
  41. 41.
    Arnaud M, Bégaud B, Thurin N, Moore N, Pariente A, Salvo F. Methods for safety signal detection in healthcare databases: a literature review. Expert Opin Drug Saf. 2017;16:721–32.CrossRefPubMedGoogle Scholar
  42. 42.
    Petri H, De Vet HCW, Naus J, Urquhart J. Prescription sequence analysis: a new and fast method for assessing certain adverse reactions of prescription drugs in large populations. Stat Med. 1988;7:1171–5.CrossRefPubMedGoogle Scholar
  43. 43.
    Hallas J. Evidence of depression provoked by cardiovascular medication: a prescription sequence symmetry analysis. Epidemiology. 1996;7:478–84.CrossRefPubMedGoogle Scholar
  44. 44.
    Coloma PM, Trifirò G, Schuemie MJ, Gini R, Herings R, Hippisley-Cox J, et al. Electronic healthcare databases for active drug safety surveillance: is there enough leverage? Pharmacoepidemiol Drug Saf. 2012;21:611–21.CrossRefPubMedGoogle Scholar
  45. 45.
    Hanczar B, Hua J, Sima C, Weinstein J, Bittner M, Dougherty ER. Small-sample precision of ROC-related estimates. Bioinformatics. 2010;26:822–30.CrossRefPubMedGoogle Scholar
  46. 46.
    Lerch M, Nowicki P, Manlik K, Wirsching G. Statistical signal detection as a routine pharmacovigilance practice: effects of periodicity and resignalling criteria on quality and workload. Drug Saf. 2015;38:1219–31.CrossRefPubMedPubMedCentralGoogle Scholar
  47. 47.
    Avillach P, Coloma PM, Gini R, Schuemie M, Mougin F, Dufour J-C, et al. Harmonization process for the identification of medical events in eight European healthcare databases: the experience from the EU-ADR project. J Am Med Inform Assoc. 2012;20:184–92.CrossRefPubMedPubMedCentralGoogle Scholar
  48. 48.
    Cutrona SL, Toh S, Iyer A, Foy S, Cavagnaro E, Forrow S, et al. Design for validation of acute myocardial infarction cases in Mini-Sentinel. Pharmacoepidemiol Drug Saf. 2012;21:274–81.CrossRefPubMedPubMedCentralGoogle Scholar
  49. 49.
    Trifiro G, Pariente A, Coloma PM, Kors JA, Polimeni G, Miremont-Salamé G, et al. Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor? Pharmacoepidemiol Drug Saf. 2009;18:1176–84.CrossRefPubMedGoogle Scholar
  50. 50.
    Arnaud M, Bezin J, Bégaud B, Pariente A, Salvo F. Trends in the incidence of use of noninsulin glucose-lowering drugs between 2006 and 2013 in France. Fundam Clin Pharmacol. 2017;31(6):663–75.Google Scholar
  51. 51.
    Hauben M, Aronson JK. Defining “signal” and its subtypes in pharmacovigilance based on a systematic review of previous definitions. Drug Saf. 2009;32:99–110.CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2017

Authors and Affiliations

  1. 1.University of Bordeaux, Inserm, Bordeaux Population Health Research Center, Pharmacoepidemiology Team, UMR 1219Bordeaux CedexFrance
  2. 2.CHU de Bordeaux, Pôle de santé publique, Service de pharmacologie médicaleBordeauxFrance
  3. 3.University of Bordeaux, Inserm, Bordeaux Population Health Research Center, ERIAS Team, UMR 1219BordeauxFrance
  4. 4.CHU de Bordeaux, Pôle de santé publique, Service d’Information médicaleBordeauxFrance

Personalised recommendations