Drug Safety

, Volume 36, Supplement 1, pp 33–47 | Cite as

Defining a Reference Set to Support Methodological Research in Drug Safety

  • Patrick B. Ryan
  • Martijn J. Schuemie
  • Emily Welebob
  • Jon Duke
  • Sarah Valentine
  • Abraham G. Hartzema
Original Research Article



Methodological research to evaluate the performance of methods requires a benchmark to serve as a referent comparison. In drug safety, the performance of analyses of spontaneous adverse event reporting databases and observational healthcare data, such as administrative claims and electronic health records, has been limited by the lack of such standards.


To establish a reference set of test cases that contain both positive and negative controls, which can serve the basis for methodological research in evaluating methods performance in identifying drug safety issues.

Research Design

Systematic literature review and natural language processing of structured product labeling was performed to identify evidence to support the classification of drugs as either positive controls or negative controls for four outcomes: acute liver injury, acute kidney injury, acute myocardial infarction, and upper gastrointestinal bleeding.


Three-hundred and ninety-nine test cases comprised of 165 positive controls and 234 negative controls were identified across the four outcomes. The majority of positive controls for acute kidney injury and upper gastrointestinal bleeding were supported by randomized clinical trial evidence, while the majority of positive controls for acute liver injury and acute myocardial infarction were only supported based on published case reports. Literature estimates for the positive controls shows substantial variability that limits the ability to establish a reference set with known effect sizes.


A reference set of test cases can be established to facilitate methodological research in drug safety. Creating a sufficient sample of drug-outcome pairs with binary classification of having no effect (negative controls) or having an increased effect (positive controls) is possible and can enable estimation of predictive accuracy through discrimination. Since the magnitude of the positive effects cannot be reliably obtained and the quality of evidence may vary across outcomes, assumptions are required to use the test cases in real data for purposes of measuring bias, mean squared error, or coverage probability.


Acute Myocardial Infarction Acute Kidney Injury Product Label Acute Liver Injury Proportional Reporting Ratio 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The Observational Medical Outcomes Partnership is funded by the Foundation for the National Institutes of Health (FNIH) through generous contributions from the following: Abbott, Amgen Inc., AstraZeneca, Bayer Healthcare Pharmaceuticals, Inc., Biogen Idec, Bristol-Myers Squibb, Eli Lilly & Company, GlaxoSmithKline, Janssen Research and Development, Lundbeck, Inc., Merck & Co., Inc., Novartis Pharmaceuticals Corporation, Pfizer Inc., Pharmaceutical Research Manufacturers of America (PhRMA), Roche, Sanofi-aventis, Schering-Plough Corporation, and Takeda. Drs. Ryan and Schuemie are employees of Janssen Research and Development. Dr. Schuemie received a fellowship from the Office of Medical Policy, Center for Drug Evaluation and Research, Food and Drug Administration. Drs. Duke, Schuemie and Hartzema have previously received funding from FNIH. Emily Welebob and Sarah Valentine have no conflicts of interest to declare.

This article was published in a supplement sponsored by the Foundation for the National Institutes of Health (FNIH). The supplement was guest edited by Stephen J.W. Evans. It was peer reviewed by Olaf H. Klungel who received a small honorarium to cover out-of-pocket expenses. S.J.W.E has received travel funding from the FNIH to travel to the OMOP symposium and received a fee from FNIH for the review of a protocol for OMOP. O.H.K has received funding for the IMI-PROTECT project.from the Innovative Medicines Initiative Joint Undertaking ( under Grant Agreement no 115004, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution.

Supplementary material

40264_2013_97_MOESM1_ESM.xls (168 kb)
Supplementary material 1 (XLS 168 kb)
40264_2013_97_MOESM2_ESM.xlsx (18 kb)
Supplementary material 2 (XLSx 18 kb)


  1. 1.
    FDA. Guidance for industry: good pharmacovigilance practices and pharmacoepidemiologic assessment. US FDA Center for Drug Evaluation and Research and Center for Biologics Evaluation and Research; 2005.Google Scholar
  2. 2.
    Hill AB. The environment and disease: association or causation? Proc R Soc Med. 1965;58:295–300.PubMedGoogle Scholar
  3. 3.
    FDA. The Sentinel Initiative: A National Strategy for Monitoring Medical Product Safety. May 2008 [cited 2012 September 15].
  4. 4.
    Lindquist M, Ståhl M, Bate A, Edwards IR, Meyboom RH. A retrospective evaluation of a data mining approach to aid finding new adverse drug reaction signals in the WHO international database. Drug Safety. 2000;23(6):533–42.PubMedCrossRefGoogle Scholar
  5. 5.
    Hauben M, Reich L. Safety related drug-labelling changes: findings from two data mining algorithms. Drug Safety. 2004;27(10):735–44.PubMedCrossRefGoogle Scholar
  6. 6.
    Hochberg AM, Hauben M, Pearson RK, O’Hara DJ, Reisinger SJ, Goldsmith DI, et al. An evaluation of three signal-detection algorithms using a highly inclusive reference event database. Drug Safety. 2009;32(6):509–25.PubMedCrossRefGoogle Scholar
  7. 7.
    Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol. 2003;158(9):915–20.PubMedCrossRefGoogle Scholar
  8. 8.
    Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–22.PubMedCrossRefGoogle Scholar
  9. 9.
    Woodward M. Epidemiology study design and data analysis. London: Chapman & Hall/CRC; 1999.Google Scholar
  10. 10.
    Whitaker H. The self controlled case series method. BMJ 2008;337:a1069.
  11. 11.
    Norén N, Hopstadius J, Bate A, Star K, Edwards IR. Temporal pattern discovery in longitudinal electronic patient records. Data Min Knowl Discov. 2010;20(3):361–87.CrossRefGoogle Scholar
  12. 12.
    Schuemie MJ. Methods for drug safety signal detection in longitudinal observational databases: LGPS and LEOPARD. Pharmacoepidemiol Drug Saf. 2011;20(3):292–9.PubMedCrossRefGoogle Scholar
  13. 13.
    Ryan PB, Powell GE, Pattishall EN, Beach KJ. Performance of screening multiple observational databases for active drug safety surveillance. Poster presented at the 25 annual meeting of the International Society of Pharmacoepidemiology, Providence, Rhode Island, 16–19 August 2009.Google Scholar
  14. 14.
    Schuemie MJ, Coloma PM, Straatman H, Herings RM, Trifirò G, Matthews JN, et al. Using electronic health care records for drug safety signal detection: a comparative evaluation of statistical methods. Med Care. 2012;50(10):890–7.Google Scholar
  15. 15.
    Coloma PM, Avillach P, Salvo F, Schuemie MJ, Ferrajolo C, Pariente A, et al. A reference standard for evaluation of methods for drug safety signal detection using electronic healthcare record databases. Drug Safety. 2013;36(1):13–23.PubMedCrossRefGoogle Scholar
  16. 16.
    Stang PE, Ryan PB, Racoosin JA, Overhage JM, Hartzema AG, Reich C, et al. Advancing the science for active surveillance: rationale and design for the Observational Medical Outcomes Partnership. Ann Intern Med. 2010;153(9):600–6.PubMedCrossRefGoogle Scholar
  17. 17.
    Ryan PB, Madigan D, Stang PE, Marc Overhage J, Racoosin JA, Hartzema AG. Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership. Stat Med. 2012;31(30):4401–15.PubMedCrossRefGoogle Scholar
  18. 18.
    Wessinger S, Kaplan M, Choi L, Williams M, Lau C, Sharp L, et al. Increased use of selective serotonin reuptake inhibitors in patients admitted with gastrointestinal haemorrhage: a multicentre retrospective analysis. Aliment Pharmacol Ther. 2006;23(7):937–44.PubMedCrossRefGoogle Scholar
  19. 19.
    Trifirò G, Pariente A, Coloma PM, Kors JA, Polimeni G, Miremont-Salame G, et al. Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor? Pharmacoepidemiol Drug Safety. 2009;18(12):1176–84.CrossRefGoogle Scholar
  20. 20.
    Katz AJ, Ryan PB, Racoosin JA, Stang PE. Assessment of case definitions for identifying acute liver injury in large observational databases. Drug Safety. 2013;36(8):651–61.PubMedCrossRefGoogle Scholar
  21. 21.
    Duke J, Friedlin J, Ryan P. A quantitative analysis of adverse events and “overwarning” in drug labeling. Arch Intern Med. 2011;171(10):944–6.PubMedCrossRefGoogle Scholar
  22. 22.
    Duke JD, Friedlin J. ADESSA: a real-time decision support service for delivery of semantically coded adverse drug event data. AMIA Annu Symp Proc. 2010;2010:177–81.PubMedGoogle Scholar
  23. 23.
    Friedlin J, Duke J. Applying natural language processing to extract codify adverse drug reaction in medication labels. 2010 [cited 2013 January 3].
  24. 24.
    Friedlin J, Duke J. Exploration of four outcomes: outcomes and labeling information, in conjunction with other evidence May 2011 [cited 2013 January 3].
  25. 25.
    Tisdale J, Miller D. Drug-induced diseases: prevention, detection, and management. 2nd ed. USA: American Society of Health-System Pharmacists; 2010.Google Scholar
  26. 26.
    Overhage JM, Ryan PB, Schuemie MJ, Stang PE. Desideratum for evidence based epidemiology. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0102-2.
  27. 27.
    Ryan PB, Madigan D. Selecting comparators in active surveillance analyses. 2010 [cited 2013 January 3].
  28. 28.
    Armstrong B. A simple estimator of minimum detectable relative risk, sample size, or power in cohort studies. Am J Epidemiol. 1987;126(2):356–8.PubMedCrossRefGoogle Scholar
  29. 29.
    Nissen SE, Wolski K. Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality. Arch Intern Med. 2010;170(14):1191–201.PubMedCrossRefGoogle Scholar
  30. 30.
    Hansen RA, Gray MD, Fox BI, Hollingsworth JC, Gao J, Zeng P. How well do various health outcome definitions used in observational studies identify cases that are consistent with expert opinion? Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0104-0.
  31. 31.
    de Abajo FJ, Montero D, Madurga M, García Rodríguez LA. Acute and clinically relevant drug-induced liver injury: a population based case-control study. Br J Clin Pharmacol. 2004;58(1):71–80.PubMedCrossRefGoogle Scholar
  32. 32.
    Carson JL, Strom BL, Duff A, Gupta A, Shaw M, Lundin FE, et al. Acute liver disease associated with erythromycins, sulfonamides, and tetracyclines. Ann Intern Med. 1993;119(1):576–83.PubMedCrossRefGoogle Scholar
  33. 33.
    Sabate M, Ibanez L, Perez E, Vidal X, Buti M, Xiol X, et al. Risk of acute liver injury associated with the use of drugs: a multicentre population survey. Aliment Pharmacol Ther. 2007;25(12):1401–9.PubMedCrossRefGoogle Scholar
  34. 34.
    Roumie CL, Choma NN, Kaltenbach L, Mitchel EF Jr, Arbogast PG, Griffin MR. Non-aspirin NSAIDs, cyclooxygenase-2 inhibitors and risk for cardiovascular events-stroke, acute myocardial infarction, and death from coronary heart disease. Pharmacoepidemiol Drug Saf. 2009;18(11):1053–63.PubMedCrossRefGoogle Scholar
  35. 35.
    Varas-Lorenzo C, Castellsague J, Stang MR, Perez-Gutthann S, Aguado J, Rodriguez LA. The use of selective cyclooxygenase-2 inhibitors and the risk of acute myocardial infarction in Saskatchewan, Canada. Pharmacoepidemiol Drug Safety. 2009;18(11):1016–25.CrossRefGoogle Scholar
  36. 36.
    Helin-Salmivaara A, Virtanen A, Vesalainen R, Gronroos JM, Klaukka T, Idanpaan-Heikkila JE, et al. NSAID use and the risk of hospitalization for first myocardial infarction in the general population: a nationwide case-control study from Finland. Eur Heart J. 2006;27(14):1657–63.PubMedCrossRefGoogle Scholar
  37. 37.
    Griffin MR, Yared A, Ray WA. Nonsteroidal antiinflammatory drugs and acute renal failure in elderly persons. Am J Epidemiol. 2000;151(5):488–96.PubMedCrossRefGoogle Scholar
  38. 38.
    Schneider V, Levesque LE, Zhang B, Hutchinson T, Brophy JM. Association of selective and conventional nonsteroidal antiinflammatory drugs with acute renal failure: a population-based, nested case-control analysis. Am J Epidemiol. 2006;164(9):881–9.PubMedCrossRefGoogle Scholar
  39. 39.
    Huerta C, Castellsague J, Varas-Lorenzo C, García Rodríguez LA. Nonsteroidal anti-inflammatory drugs and risk of ARF in the general population. Am J Kidney Dis. 2005;45(3):531–9.PubMedCrossRefGoogle Scholar
  40. 40.
    Loke YK, Trivedi AN, Singh S. Meta-analysis: gastrointestinal bleeding due to interaction between selective serotonin uptake inhibitors and non-steroidal anti-inflammatory drugs. Aliment Pharmacol Ther. 2008;27(1):31–40.PubMedCrossRefGoogle Scholar
  41. 41.
    Duke J, Friedlin J, Li X. Consistency in the safety labeling of bioequivalent medications. Pharmacoepidemiol Drug Saf. 2013;22:294–301.PubMedCrossRefGoogle Scholar
  42. 42.
    Cantor SB, Kattan MW. Determining the area under the ROC curve for a binary diagnostic test. Med Decis Making. 2000;20(4):468–70.PubMedCrossRefGoogle Scholar
  43. 43.
    Coloma PM, Trifirò G, Schuemie MJ, Gini R, Herings R, Hippisley-Cox J, et al. Electronic healthcare databases for active drug safety surveillance: is there enough leverage? Pharmacoepidemiol Drug Safety. 2012;21(6):611–21.CrossRefGoogle Scholar
  44. 44.
    Reich CG, Ryan PB, Suchard MA. The impact of drug and outcome prevalence on the feasibility and performance of analytical methods for a risk identification and analysis system. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0112-0.
  45. 45.
    Schuemie MJ, Madigan D, Ryan PB. Empirical performance of longitudinal gamma poisson shrinker (LGPS) and longitudinal evaluation of observational profiles of adverse events related to drugs (LEOPARD): lessons for developing a risk identification and analysis system. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0107-x.
  46. 46.
    Norén GN, Bergvall T, Ryan PB, Juhlin K, Schuemie MJ, Madigna D. Empirical performance of the calibrated self-controlled cohort analysis within temporal pattern discovery: lessons for developing a risk identification and analysis system. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0095-x.
  47. 47.
    Madigan D, Schuemie MJ, Ryan PB. Empirical performance of the case-control method: lessons for developing a risk identification and analysis system. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0105-z.
  48. 48.
    Ryan PB, Schuemie MJ, Gruber S, Zorych I, Madigan D. Empirical performance of a new user cohort method: lessons for developing a risk identification and analysis system. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0099-6.
  49. 49.
    Suchard MA, Zorych I, Simpson SE, Schuemie MJ, Ryan PB, Madigan D. Empirical performance of the self-controlled case series design: lessons for developing a risk identification and analysis system. Drug Saf. 2013 (in this supplement issue). doi:  10.1007/s40264-013-0100-4.
  50. 50.
    Ryan PB, Schuemie MJ, Madigan D. Empirical performance of a self-controlled cohort method: lessons for developing a risk identification and analysis system. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0101-3.
  51. 51.
    DuMouchel B, Ryan PB, Schuemie MJ, Madigan D. Evaluation of disproportionality safety signaling applied to health care databases. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0106-y.
  52. 52.
    Schuemie MJ, Gini R, Coloma PM, Straatman H, Herings RMC, Pedersen L, et al. Replication of the OMOP experiment in Europe: evaluating methods for risk identification in electronic health record databases. Drug Saf. 2013 (in this supplement issue). doi: 10.1007/s40264-013-0109-8.
  53. 53.
    Harpaz R, DuMouchel W, Shah NH, Madigan D, Ryan P, Friedman C. Novel data-mining methodologies for adverse drug event discovery and analysis. Clin Pharmacol Ther. 2012;91(6):1010–21.Google Scholar
  54. 54.
    Winkelmayer WC, Waikar SS, Mogun H, Solomon DH. Nonselective and cyclooxygenase-2-selective NSAIDs and acute kidney injury. Am J Med. 2008;121(12):1092–8.PubMedCrossRefGoogle Scholar
  55. 55.
    Murray MD, Brater DC, Tierney WM, Hui SL, McDonald CJ. Ibuprofen-associated renal impairment in a large general internal medicine practice. Am J Med Sci. 1990;299(4):222–9.PubMedCrossRefGoogle Scholar
  56. 56.
    Hung CC, Liu WC, Kuo MC, Lee CH, Hwang SJ, Chen HC. Acute renal failure and its risk factors in Stevens–Johnson syndrome and toxic epidermal necrolysis. Am J Nephrol. 2009;29(6):633–8.PubMedCrossRefGoogle Scholar
  57. 57.
    García Rodríguez LA, Duque A, Castellsague J, Perez-Gutthann S, Stricker BH. A cohort study on the risk of acute liver injury among users of ketoconazole and other antifungal drugs. Br J Clin Pharmacol. 1999;48(6):847–52.PubMedCrossRefGoogle Scholar
  58. 58.
    Fischer MA, Winkelmayer WC, Rubin RH, Avorn J. The hepatotoxicity of antifungal medications in bone marrow transplant recipients. Clin Infect Dis. 2005;41(3):301–7.PubMedCrossRefGoogle Scholar
  59. 59.
    Ouyang DW, Shapiro DE, Lu M, Brogly SB, French AL, Leighty RM, et al. Increased risk of hepatotoxicity in HIV-infected pregnant women receiving antiretroviral therapy independent of nevirapine exposure. AIDS. 2009;23(18):2425–30.PubMedCrossRefGoogle Scholar
  60. 60.
    Bruno S, Maisonneuve P, Castellana P, Rotmensz N, Rossi S, Maggioni M, et al. Incidence and risk factors for non-alcoholic steatohepatitis: prospective study of 5408 women enrolled in Italian tamoxifen chemoprevention trial. BMJ. 2005;330(7497):932.PubMedCrossRefGoogle Scholar
  61. 61.
    Solomon DH, Avorn J, Stürmer T, Glynn RJ, Mogun H, Schneeweiss S. Cardiovascular outcomes in new users of coxibs and nonsteroidal antiinflammatory drugs: high-risk subgroups and time course of risk. Arthritis Rheum. 2006;54(5):1378–89.PubMedCrossRefGoogle Scholar
  62. 62.
    Khader YS, Rice J, John L, Abueita O. Oral contraceptives use and the risk of myocardial infarction: a meta-analysis. Contraception. 2003;68(1):11–7.PubMedCrossRefGoogle Scholar
  63. 63.
    Mangoni AA, Woodman RJ, Gaganis P, Gilbert AL, Knights KM. Use of non-steroidal anti-inflammatory drugs and risk of incident myocardial infarction and heart failure, and all-cause mortality in the Australian veteran community. Br J Clin Pharmacol. 2010;69(6):689–700.PubMedCrossRefGoogle Scholar
  64. 64.
    Warner JJ, Weideman RA, Kelly KC, Brilakis ES, Banerjee S, Cunningham F, et al. The risk of acute myocardial infarction with etodolac is not increased compared to naproxen: a historical cohort analysis of a generic COX-2 selective inhibitor. J Cardiovasc Pharmacol Ther. 2008;13(4):252–60.PubMedCrossRefGoogle Scholar
  65. 65.
    Dalton SO, Johansen C, Mellemkjaer L, Norgard B, Sorensen HT, Olsen JH. Use of selective serotonin reuptake inhibitors and risk of upper gastrointestinal tract bleeding: a population-based cohort study. Arch Intern Med. 2003;163(1):59–64.PubMedCrossRefGoogle Scholar
  66. 66.
    de Abajo FJ, García-Rodríguez LA. Risk of upper gastrointestinal tract bleeding associated with selective serotonin reuptake inhibitors and venlafaxine therapy: interaction with nonsteroidal anti-inflammatory drugs and effect of acid-suppressing agents. Arch Gen Psychiatry. 2008;65(7):795–803.PubMedCrossRefGoogle Scholar
  67. 67.
    de Abajo FJ, Rodriguez LA, Montero D. Association between selective serotonin reuptake inhibitors and upper gastrointestinal bleeding: population based case-control study. BMJ. 1999;319(7217):1106–9.PubMedCrossRefGoogle Scholar
  68. 68.
    Helin-Salmivaara A, Huttunen T, Gronroos JM, Klaukka T, Huupponen R. Risk of serious upper gastrointestinal events with concurrent use of NSAIDs and SSRIs: a case-control study in the general population. Eur J Clin Pharmacol. 2007;63(4):403–8.PubMedCrossRefGoogle Scholar
  69. 69.
    Lewis JD, Strom BL, Localio AR, Metz DC, Farrar JT, Weinrieb RM, et al. Moderate and high affinity serotonin reuptake inhibitors increase the risk of upper gastrointestinal toxicity. Pharmacoepidemiol Drug Saf. 2008;17(4):328–35.PubMedCrossRefGoogle Scholar
  70. 70.
    Targownik LE, Bolton JM, Metge CJ, Leung S, Sareen J. Selective serotonin reuptake inhibitors are associated with a modest increase in the risk of upper gastrointestinal bleeding. Am J Gastroenterol. 2009;104(6):1475–82.PubMedCrossRefGoogle Scholar
  71. 71.
    Vidal X, Ibanez L, Vendrell L, Conforti A, Laporte LR, Spanish-Italian Collaborative Group for the Epidemiology of Gastrointestinal B. Risk of upper gastrointestinal bleeding and the degree of serotonin reuptake inhibition by antidepressants: a case-control study. Drug Safety. 2008;31(2):159–68.PubMedCrossRefGoogle Scholar
  72. 72.
    Ashworth NL, Peloso PM, Muhajarine N, Stang M. Risk of hospitalization with peptic ulcer disease or gastrointestinal hemorrhage associated with nabumetone, arthrotec, diclofenac, and naproxen in a population based cohort study. J Rheumatol. 2005;32(11):2212–7.PubMedGoogle Scholar
  73. 73.
    García Rodríguez LA, Hernandez-Diaz S. Relative risk of upper gastrointestinal complications among users of acetaminophen and nonsteroidal anti-inflammatory drugs. Epidemiology. 2001;12(5):570–6.PubMedCrossRefGoogle Scholar
  74. 74.
    de Abajo FJ, García Rodríguez LA. Risk of upper gastrointestinal bleeding and perforation associated with low-dose aspirin as plain and enteric-coated formulations. BMC Clin Pharmacol. 2001;1:1.PubMedCrossRefGoogle Scholar
  75. 75.
    García Rodríguez LA, Barreales Tolosa L. Risk of upper gastrointestinal complications among users of traditional NSAIDs and COXIBs in the general population. Gastroenterology. 2007;132(2):498–506.PubMedCrossRefGoogle Scholar
  76. 76.
    Grimaldi-Bensouda L, Abenhaim L, Michaud L, Mouterde O, Jonville-Bera AP, Giraudeau B, et al. Clinical features and risk factors for upper gastrointestinal bleeding in children: a case-crossover study. Eur J Clin Pharmacol. 2010;66(8):831–7.PubMedCrossRefGoogle Scholar
  77. 77.
    Hippisley-Cox J, Coupland C, Logan R. Risk of adverse gastrointestinal outcomes in patients taking cyclo-oxygenase-2 inhibitors or conventional non-steroidal anti-inflammatory drugs: population based nested case-control analysis. BMJ. 2005;331(7528):1310–6.PubMedCrossRefGoogle Scholar
  78. 78.
    Lanas A, García-Rodríguez LA, Arroyo MT, Gomollon F, Feu F, Gonzalez-Perez A, et al. Risk of upper gastrointestinal ulcer bleeding associated with selective cyclo-oxygenase-2 inhibitors, traditional non-aspirin non-steroidal anti-inflammatory drugs, aspirin and combinations. Gut. 2006;55(12):1731–8.PubMedCrossRefGoogle Scholar
  79. 79.
    Laporte JR, Ibanez L, Vidal X, Vendrell L, Leone R. Upper gastrointestinal bleeding associated with the use of NSAIDs: newer versus older agents. Drug Safety. 2004;27(6):411–20.PubMedCrossRefGoogle Scholar
  80. 80.
    Massó Gonzalez EL, Patrignani P, Tacconelli S, García Rodríguez LA. Variability among nonsteroidal antiinflammatory drugs in risk of upper gastrointestinal bleeding. Arthritis Rheum. 2010;62(6):1592–601.PubMedCrossRefGoogle Scholar
  81. 81.
    Mellemkjaer L, Blot WJ, Sorensen HT, Thomassen L, McLaughlin JK, Nielsen GL, et al. Upper gastrointestinal bleeding among users of NSAIDs: a population-based cohort study in Denmark. Br J Clin Pharmacol. 2002;53(2):173–81.PubMedCrossRefGoogle Scholar
  82. 82.
    Rahme E, Nedjar H. Risks and benefits of COX-2 inhibitors vs non-selective NSAIDs: does their cardiovascular risk exceed their gastrointestinal benefit? A retrospective cohort study. Rheumatology. 2007;46(3):435–8.PubMedCrossRefGoogle Scholar
  83. 83.
    Lanas A, Serrano P, Bajador E, Fuentes J, Sainz R. Risk of upper gastrointestinal bleeding associated with non-aspirin cardiovascular drugs, analgesics and nonsteroidal anti-inflammatory drugs. Eur J Gastroenterol Hepatol. 2003;15(2):173–8.PubMedCrossRefGoogle Scholar
  84. 84.
    Opatrny L, Delaney JA, Suissa S. Gastro-intestinal haemorrhage risks of selective serotonin receptor antagonist therapy: a new look. Br J Clin Pharmacol. 2008;66(1):76–81.PubMedCrossRefGoogle Scholar
  85. 85.
    Levesque LE, Brophy JM, Zhang B. The risk for myocardial infarction with cyclooxygenase-2 inhibitors: a population study of elderly adults. Ann Intern Med. 2005;142(7):481–9.PubMedCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Patrick B. Ryan
    • 1
    • 6
  • Martijn J. Schuemie
    • 2
    • 6
  • Emily Welebob
    • 6
  • Jon Duke
    • 3
    • 4
  • Sarah Valentine
    • 5
  • Abraham G. Hartzema
    • 5
    • 6
  1. 1.Janssen Research and Development LLCTitusvilleUSA
  2. 2.Department of Medical InformaticsErasmus University Medical Center RotterdamRotterdamThe Netherlands
  3. 3.Indiana University School of MedicineINUSA
  4. 4.Regenstrief InstituteINUSA
  5. 5.College of PharmacyUniversity of FloridaGainesvilleUSA
  6. 6.Observational Medical Outcomes Partnership, Foundation for the National Institutes of HealthBethesdaUSA

Personalised recommendations