Skip to main content

An Automated System Combining Safety Signal Detection and Prioritization from Healthcare Databases: A Pilot Study

Drug Safety volume 41, pages 377–387 (2018)Cite this article

Abstract

Introduction

Signal detection from healthcare databases is possible, but is not yet used for routine surveillance of drug safety. One challenge is to develop methods for selecting signals that should be assessed with priority.

Aim

The aim of this study was to develop an automated system combining safety signal detection and prioritization from healthcare databases and applicable to drugs used in chronic diseases.

Methods

Patients present in the French EGB healthcare database for at least 1 year between 2005 and 2015 were considered. Noninsulin glucose-lowering drugs (NIGLDs) were selected as a case study, and hospitalization data were used to select important medical events (IME). Signal detection was performed quarterly from 2008 to 2015 using sequence symmetry analysis. NIGLD/IME associations were screened if one or more exposed case was identified in the quarter, and three or more exposed cases were identified in the population at the date of screening. Detected signals were prioritized using the Longitudinal-SNIP (L-SNIP) algorithm based on strength (S), novelty (N), and potential impact of signal (I), and pattern of drug use (P). Signals scored in the top 10% were identified as of high priority. A reference set was built based on NIGLD summaries of product characteristics (SPCs) to compute the performance of the developed system.

Results

A total of 815 associations were screened and 241 (29.6%) were detected as signals; among these, 58 (24.1%) were prioritized. The performance for signal detection was sensitivity = 47%; specificity = 80%; positive predictive value (PPV) 33%; negative predictive value = 82%. The use of the L-SNIP algorithm increased the early identification of positive controls, restricted to those mentioned in the SPCs after 2008: PPV = 100% versus PPV = 14% with its non-use. The system revealed a strong new signal with dipeptidylpeptidase-4 inhibitors and venous thromboembolism.

Conclusion

The developed system seems promising for the routine use of healthcare data for safety surveillance of drugs used in chronic diseases.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 49.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

References

  1. Mukherjee D, Nissen SE, Topol EJ. Risk of cardiovascular events associated with selective COX-2 inhibitors. JAMA. 2001;286:954–9.

    CAS  Article  PubMed  Google Scholar 

  2. Lévesque LE, Brophy JM, Zhang B. The risk for myocardial infarction with cyclooxygenase-2 inhibitors: a population study of elderly adults. Ann Intern Med. 2005;142:481.

    Article  PubMed  Google Scholar 

  3. Laheij RJ, Sturkenboom MC, Hassing R-J, Dieleman J, Stricker BH, Jansen JB. Risk of community-acquired pneumonia and use of gastric acid–suppressive drugs. JAMA. 2004;292:1955–60.

    CAS  Article  PubMed  Google Scholar 

  4. Johnstone J, Nerenberg K, Loeb M. Meta-analysis: proton pump inhibitor use and the risk of community-acquired pneumonia. Aliment Pharmacol Ther. 2010;31:1165–77.

    CAS  Article  PubMed  Google Scholar 

  5. Yang Y-X, Lewis JD, Epstein S, Metz DC. Long-term proton pump inhibitor therapy and risk of hip fracture. JAMA. 2006;296:2947–53.

    CAS  Article  PubMed  Google Scholar 

  6. Vestergaard P, Rejnmark L, Mosekilde L. Proton pump inhibitors, histamine H2 receptor antagonists, and other antacid medications and the risk of fracture. Calcif Tissue Int. 2006;79:76–83.

    CAS  Article  PubMed  Google Scholar 

  7. Targownik LE, Lix LM, Metge CJ, Prior HJ, Leung S, Leslie WD. Use of proton pump inhibitors and risk of osteoporosis-related fractures. Can Med Assoc J. 2008;179:319–26.

    Article  Google Scholar 

  8. Lipscombe LL, Gomes T, Lévesque LE, Hux JE, Juurlink DN, Alter DA. Thiazolidinediones and cardiovascular outcomes in older patients with diabetes. JAMA. 2007;298:2634–43.

    CAS  Article  PubMed  Google Scholar 

  9. Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med. 2007;356:2457–71.

    CAS  Article  PubMed  Google Scholar 

  10. Neumann A, Weill A, Ricordeau P, Fagot JP, Alla F, Allemand H. Pioglitazone and risk of bladder cancer among diabetic patients in France: a population-based cohort study. Diabetologia. 2012;55:1953–62.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  11. Tuccori M, Filion KB, Yin H, Yu OH, Platt RW, Azoulay L. Pioglitazone use and risk of bladder cancer: population based cohort study. BMJ. 2016;352:i1541.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Hillaire-Buys D, Faillie J-L. Pioglitazone and the risk of bladder cancer. BMJ. 2012;344:e3500.

    Article  PubMed  Google Scholar 

  13. Coloma PM, Schuemie MJ, Trifirò G, Gini R, Herings R, Hippisley-Cox J, et al. Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EU-ADR Project. Pharmacoepidemiol Drug Saf. 2011;20:1–11.

    Article  PubMed  Google Scholar 

  14. Wisniewski AFZ, Bate A, Bousquet C, Brueckner A, Candore G, Juhlin K, et al. Good signal detection practices: evidence from IMI PROTECT. Drug Saf. 2016;39:469–90.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Platt R, Carnahan RM, Brown JS, Chrischilles E, Curtis LH, Hennessy S, et al. The U.S. Food and Drug Administration’s Mini-Sentinel program: status and direction. Pharmacoepidemiol Drug Saf. 2012;21:1–8.

    PubMed  Google Scholar 

  16. Stang PE, Ryan PB, Racoosin JA, Overhage JM, Hartzema AG, Reich C, et al. Advancing the science for active surveillance: rationale and design for the Observational Medical Outcomes Partnership. Ann Intern Med. 2010;153:600–6.

    Article  PubMed  Google Scholar 

  17. Andersen M, Bergman U, Choi N-K, Gerhard T, Huang C, Jalbert J, et al. The Asian Pharmacoepidemiology Network (AsPEN): promoting multi-national collaboration for pharmacoepidemiologic research in Asia. Pharmacoepidemiol Drug Saf. 2013;22:700–4.

    Article  PubMed  Google Scholar 

  18. Ryan PB, Stang PE, Overhage JM, Suchard MA, Hartzema AG, DuMouchel W, et al. A comparison of the empirical performance of methods for a risk identification system. Drug Saf. 2013;36:143–58.

    Article  Google Scholar 

  19. Schuemie MJ, Gini R, Coloma PM, Straatman H, Herings RMC, Pedersen L, et al. Replication of the OMOP experiment in Europe: evaluating methods for risk identification in electronic health record databases. Drug Saf. 2013;36:159–69.

    Article  Google Scholar 

  20. Pratt N, Andersen M, Bergman U, Choi N-K, Gerhard T, Huang C, et al. Multi-country rapid adverse drug event assessment: the Asian Pharmacoepidemiology Network (AsPEN) antipsychotic and acute hyperglycaemia study. Pharmacoepidemiol Drug Saf. 2013;22:915–24.

    CAS  PubMed  Google Scholar 

  21. Kulldorff M, Dashevsky I, Avery TR, Chan AK, Davis RL, Graham D, et al. Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiol Drug Saf. 2013;22:517–23.

    Article  PubMed  Google Scholar 

  22. Waller PC, Lee EH. Responding to drug safety issues. Pharmacoepidemiol Drug Saf. 1999;8:535–52.

    CAS  Article  PubMed  Google Scholar 

  23. Waller P, Heeley E, Moseley J. Impact analysis of signals detected from spontaneous adverse drug reaction reporting data. Drug Saf. 2005;28:843–50.

    Article  PubMed  Google Scholar 

  24. Seabroke S, Wise L, Waller P. Development of a novel regulatory pharmacovigilance prioritisation system: an evaluation of its performance at the UK medicines and healthcare products regulatory agency. Drug Saf. 2013;36:1025–32.

    Article  PubMed  Google Scholar 

  25. Meyboom RH, Lindquist M, Egberts AC, Edwards IR. Signal selection and follow-up in pharmacovigilance. Drug Saf. 2002;25:459–65.

    Article  PubMed  Google Scholar 

  26. Ståhl M, Lindquist M, Edwards IR, Brown EG. Introducing triage logic as a new strategy for the detection of signals in the WHO Drug Monitoring Database. Pharmacoepidemiol Drug Saf. 2004;13:355–63.

    Article  PubMed  Google Scholar 

  27. Lindquist M. Use of triage strategies in the WHO signal-detection process. Drug Saf. 2007;30:635–7.

    Article  PubMed  Google Scholar 

  28. Caster O, Juhlin K, Watson S, Norén GN. Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence aspects in vigiRank: retrospective evaluation against emerging safety signals. Drug Saf. 2014;37:617–28.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  29. Van Puijenbroek EP, Van Grootheest K, Diemont WL, Leufkens HG, Egberts AC. Determinants of signal selection in a spontaneous reporting system for adverse drug reactions. Br J Clin Pharmacol. 2001;52:579–86.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Levitan B, Yee CL, Russo L, Bayney R, Thomas AP, Klincewicz SL. A model for decision support in signal triage. Drug Saf. 2008;31:727–35.

    Article  PubMed  Google Scholar 

  31. Jamekornkul C, Chaisumritchoke ST. Developing a signal triage algorithm for Thai national adverse drug reaction database. Thai J Pharm Sci. 2016;40(3):153–7.

  32. Salvo F, Raschi E, Moretti U, Chiarolanza A, Fourrier-Réglat A, Moore N, et al. Pharmacological prioritisation of signals of disproportionate reporting: proposal of an algorithm and pilot evaluation. Eur J Clin Pharmacol. 2014;70:617–25.

    CAS  Article  PubMed  Google Scholar 

  33. European Medicines Agency. Guideline on good pharmacovigilance practices (GVP): module IX—signal management. 2012. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129138.pdf. Accessed 18 Aug 2017.

  34. United States Food and Drug Administration. Classifying significant postmarketing drug safety issues. Fed Regist. 2012. https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM295211.pdf. Accessed 16 Nov 2017.

  35. Sottosanti L, Ferrazin F. Italian pharmacovigilance system. Manns Pharmacovigil. 3rd ed. Chichester: Wiley; 2014.

    Google Scholar 

  36. Coloma PM, Schuemie MJ, Trifirò G, Furlong L, van Mulligen E, Bauer-Mehren A, et al. Drug-induced acute myocardial infarction: identifying “prime suspects” from electronic healthcare records-based surveillance system. PLoS ONE 2013;8:e72148. https://doi.org/10.1371/journal.pone.0072148.

  37. Bezin J, Duong M, Lassalle R, Droz C, Pariente A, Blin P, et al. The national healthcare system claims databases in France, SNIIRAM and EGB: Powerful tools for pharmacoepidemiology. Pharmacoepidemiol. Drug Saf. 2017. http://doi.wiley.com/10.1002/pds.4233. Accessed 2 Jun 2017.

  38. European Medicines Agency. Inclusion/exclusion criteria for the “Important medical events” list. 2017. http://www.ema.europa.eu/docs/en_GB/document_library/Other/2016/08/WC500212100.pdf. Accessed 16 Nov 2017.

  39. Tsiropoulos I, Andersen M, Hallas J. Adverse events with use of antiepileptic drugs: a prescription and event symmetry analysis. Pharmacoepidemiol Drug Saf. 2009;18:483–91.

    CAS  Article  PubMed  Google Scholar 

  40. Wahab IA, Pratt NL, Wiese MD, Kalisch LM, Roughead EE. The validity of sequence symmetry analysis (SSA) for adverse drug reaction signal detection. Pharmacoepidemiol Drug Saf. 2013;22:496–502.

    Article  PubMed  Google Scholar 

  41. Arnaud M, Bégaud B, Thurin N, Moore N, Pariente A, Salvo F. Methods for safety signal detection in healthcare databases: a literature review. Expert Opin Drug Saf. 2017;16:721–32.

    Article  PubMed  Google Scholar 

  42. Petri H, De Vet HCW, Naus J, Urquhart J. Prescription sequence analysis: a new and fast method for assessing certain adverse reactions of prescription drugs in large populations. Stat Med. 1988;7:1171–5.

    CAS  Article  PubMed  Google Scholar 

  43. Hallas J. Evidence of depression provoked by cardiovascular medication: a prescription sequence symmetry analysis. Epidemiology. 1996;7:478–84.

    CAS  Article  PubMed  Google Scholar 

  44. Coloma PM, Trifirò G, Schuemie MJ, Gini R, Herings R, Hippisley-Cox J, et al. Electronic healthcare databases for active drug safety surveillance: is there enough leverage? Pharmacoepidemiol Drug Saf. 2012;21:611–21.

    Article  PubMed  Google Scholar 

  45. Hanczar B, Hua J, Sima C, Weinstein J, Bittner M, Dougherty ER. Small-sample precision of ROC-related estimates. Bioinformatics. 2010;26:822–30.

    CAS  Article  PubMed  Google Scholar 

  46. Lerch M, Nowicki P, Manlik K, Wirsching G. Statistical signal detection as a routine pharmacovigilance practice: effects of periodicity and resignalling criteria on quality and workload. Drug Saf. 2015;38:1219–31.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Avillach P, Coloma PM, Gini R, Schuemie M, Mougin F, Dufour J-C, et al. Harmonization process for the identification of medical events in eight European healthcare databases: the experience from the EU-ADR project. J Am Med Inform Assoc. 2012;20:184–92.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Cutrona SL, Toh S, Iyer A, Foy S, Cavagnaro E, Forrow S, et al. Design for validation of acute myocardial infarction cases in Mini-Sentinel. Pharmacoepidemiol Drug Saf. 2012;21:274–81.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Trifiro G, Pariente A, Coloma PM, Kors JA, Polimeni G, Miremont-Salamé G, et al. Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor? Pharmacoepidemiol Drug Saf. 2009;18:1176–84.

    Article  PubMed  Google Scholar 

  50. Arnaud M, Bezin J, Bégaud B, Pariente A, Salvo F. Trends in the incidence of use of noninsulin glucose-lowering drugs between 2006 and 2013 in France. Fundam Clin Pharmacol. 2017;31(6):663–75.

  51. Hauben M, Aronson JK. Defining “signal” and its subtypes in pharmacovigilance based on a systematic review of previous definitions. Drug Saf. 2009;32:99–110.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Bordeaux, Inserm, Bordeaux Population Health Research Center, Pharmacoepidemiology Team, UMR 1219, 146 Rue Léo Saignat, BP 36, 33000, Bordeaux Cedex, France

    Mickael Arnaud, Bernard Bégaud, Quentin Jarrion, Julien Bezin, Antoine Pariente & Francesco Salvo

  2. CHU de Bordeaux, Pôle de santé publique, Service de pharmacologie médicale, 33000, Bordeaux, France

    Bernard Bégaud, Quentin Jarrion, Julien Bezin, Antoine Pariente & Francesco Salvo

  3. University of Bordeaux, Inserm, Bordeaux Population Health Research Center, ERIAS Team, UMR 1219, 33000, Bordeaux, France

    Frantz Thiessard

  4. CHU de Bordeaux, Pôle de santé publique, Service d’Information médicale, 33000, Bordeaux, France

    Frantz Thiessard

Authors
  1. Mickael Arnaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernard Bégaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frantz Thiessard
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Quentin Jarrion
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Julien Bezin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Antoine Pariente
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Francesco Salvo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesco Salvo.

Ethics declarations

Conflicts of interest

Mickael Arnaud, Bernard Bégaud, Frantz Thiessard, Quentin Jarrion, Julien Bezin, Antoine Pariente, and Francesco Salvo have no conflict of interest directly relevant to the content of this study.

Funding

This study is part of the Drugs Systematized Assessment in real-liFe Environment (DRUGS-SAFE) research program that is funded by the French Medicines Agency (Agence Nationale de Sécurité du Médicament et des Produits de Santé, ANSM). The program aims at providing an integrated system allowing the concomitant monitoring of drug use and safety in France. The potential impact of drugs (e.g., non-insulin glucose-lowering drugs), the frailty of populations, and the seriousness of risks drive the research program. This publication represents the views of the authors and does not necessarily represent the opinion of the French Medicines Agency.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 706 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Arnaud, M., Bégaud, B., Thiessard, F. et al. An Automated System Combining Safety Signal Detection and Prioritization from Healthcare Databases: A Pilot Study. Drug Saf 41, 377–387 (2018). https://doi.org/10.1007/s40264-017-0618-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40264-017-0618-y