Pharmaceutical Medicine

, Volume 33, Issue 1, pp 29–43 | Cite as

Evaluation of the US Food and Drug Administration Sentinel Analysis Tools Using a Comparator with a Different Indication: Comparing the Rates of Gastrointestinal Bleeding in Warfarin and Statin Users

  • Ryan M. CarnahanEmail author
  • Joshua J. Gagne
  • Christian Hampp
  • Charles E. Leonard
  • Sengwee Toh
  • Candace C. Fuller
  • Sean Hennessy
  • Laura Hou
  • Noelle M. Cocoros
  • Genna Panucci
  • Tiffany Woodworth
  • Austin Cosgrove
  • Aarthi Iyer
  • Elizabeth A. Chrischilles
Original Research Article



The US Food and Drug Administration’s Sentinel System was established to monitor safety of regulated medical products. Sentinel investigators identified known associations between drugs and adverse events to test reusable analytic tools developed for Sentinel. This test case used a comparator with a different indication.


We tested the ability of Sentinel’s reusable analytic tools to identify the known association between warfarin and gastrointestinal bleeding (GIB). Statins, expected to have no effect on GIB, were the comparator. We further explored the impact of analytic features, including matching ratio and stratifying Cox regression analyses, on matched pairs.


This evaluation included data from 14 Sentinel Data Partners. New users of warfarin and statins, aged 18 years and older, who had not received other anticoagulants or had recent GIB were matched on propensity score using 1:1 and 1:n variable ratio matching, matching statin users with warfarin users to estimate the average treatment effect in warfarin-treated patients. We compared the risk of GIB using Cox proportional hazards regression, following patients for the duration of their observed continuous treatment or until a GIB. For the 1:1 matched cohort, we conducted analyses with and without stratification on matched pair. The variable ratio matched cohort analysis was stratified on the matched set.


We identified 141,398 new users of warfarin and 2,275,694 new users of statins. In analyses stratified on matched pair/set, the hazard ratios (HR) for GIB in warfarin users compared with statin users were 2.78 (95% confidence interval [CI] 2.36–3.28) in the 1:1 matched cohort and 3.10 (95% CI 2.76–3.49) in the variable ratio matched cohort. The HR was lower in the analysis of the 1:1 matched cohort not stratified by matched pair (2.22, 95% CI 1.97–2.49), and highest early in treatment. Follow-up for warfarin users tended to be shorter than for statin users.


This study identified the expected GIB risk with warfarin compared with statins using an analytic tool developed for Sentinel. Our findings suggest that comparators with different indications may be useful in surveillance in select circumstances. Finally, in the presence of differential censoring, stratification by matched pair may reduce the potential for bias in Cox regression analyses.



The authors would like to thank Patrick Archdeacon, MD, for his contributions in planning the study, and Zilu Zhang, MS, for assistance with analyses. The authors would also like to gratefully acknowledge the contributions of the following organizations that provided data used in the analysis: Aetna, Blue Bell, PA; Blue Cross Blue Shield of Massachusetts, Boston, MA; Harvard Pilgrim Health Care Institute, Boston, MA; HealthCore, Inc., Translational Research for Affordability and Quality, Alexandria, VA; HealthPartners Institute (formerly Health Partners Research Foundation), Minneapolis, MN; Humana, Inc., Comprehensive Health Insights, Miramar, FL; Kaiser Permanente Colorado Institute for Health Research, Denver, CO; Kaiser Permanente Center for Health Research Hawai’i, Honolulu, HI; Kaiser Permanente Mid-Atlantic States, Mid-Atlantic Permanente Research Institute, Rockville, MD; Kaiser Permanente Northern California, Division of Research, Oakland, CA; Kaiser Permanente Northwest Center for Health Research, Portland, OR; Kaiser Permanente Washington Health Research Institute (formerly Group Health Research Institute), Seattle, WA; Marshfield Clinic Research Institute, Marshfield, WI; Meyers Primary Care Institute, Worcester, MA; OptumInsight Life Sciences Inc., Boston, MA; Vanderbilt University Medical Center, Department of Health Policy, Nashville, TN, which is indebted to the Tennessee Division of TennCare of the Department of Finance and Administration, which provided data.

Compliance with Ethical Standards


The Sentinel System is sponsored by the US FDA and funded by the FDA through the Department of Health and Human Services (HHS) Contract Number HHSF223201400030I. This study was supported under Mini-Sentinel Contract Numbers HHSF223200910006I/HHSF22301008T.

Conflicts of Interest

Ryan Carnahan, Joshua Gagne, Charles Leonard, Sengwee Toh, Candace Fuller, Sean Hennessy, Laura Hou, Noelle Cocoros, Genna Panucci, Tiffany Woodworth, Austin Cosgrove, Aarthi Iyer, and Elizabeth Chrischilles report no conflicts of interest. Christian Hampp works for the FDA, which funded the study.

Ethics Approval

Sentinel has been deemed a public health activity under the auspices of the FDA and not under the purview of Institutional Review Boards [42, 43].

Supplementary material

40290_2018_265_MOESM1_ESM.pdf (864 kb)
Supplementary material 1 (PDF 864 kb)


  1. 1.
    Platt R, Wilson M, Chan KA, Benner JS, Marchibroda J, McClellan M. The new Sentinel Network: improving the evidence of medical-product safety. N Engl J Med. 2009;361(7):645–7.CrossRefGoogle Scholar
  2. 2.
    Platt R, Brown JS, Robb M, et al. The FDA Sentinel Initiative: an Evolving National Resource. N Engl J Med. 2018;379(22):2091–3.CrossRefGoogle Scholar
  3. 3.
    Sentinel Distributed Database and Common Data Model. Available at: Accessed 19 Nov 2018.
  4. 4.
    Sentinel surveillance tools, routine querying system. Available at: Accessed 12 Mar 2018.
  5. 5.
    Gagne JJ, Han X, Hennessy S, et al. Successful comparison of US food and drug administration sentinel analysis tools to traditional approaches in quantifying a known drug-adverse event association. Clin Pharmacol Ther. 2016;100(5):558–64.CrossRefGoogle Scholar
  6. 6.
    Zhou M, Wang SV, Leonard CE, et al. Sentinel modular program for propensity-score matched cohort analyses: application to glyburide, glipizide, and serious hypoglycemia. Epidemiology. 2017;28(6):838–46.CrossRefGoogle Scholar
  7. 7.
    Carnahan RM, Kuntz JL, Wang SV, et al. Evaluation of the US Food and Drug Administration sentinel analysis tools in confirming previously observed drug-outcome associations: the case of clindamycin and Clostridium difficile infection. Pharmacoepidemiol Drug Saf. 2018;27(7):731–9.CrossRefGoogle Scholar
  8. 8.
    Iowa Health Fact Book: Alzheimer’s Disease Mortality. Available at:’s%20Disease%20Mortality/0. Accessed 24 Jan 2018.
  9. 9.
    Schulman S, Beyth RJ, Kearon C, Levine MN. Hemorrhagic complications of anticoagulant and thrombolytic treatment: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest. 2008;133(6 Suppl):257s–98s.CrossRefGoogle Scholar
  10. 10.
    Johnsen SP, Sorensen HT, Mellemkjoer L, et al. Hospitalisation for upper gastrointestinal bleeding associated with use of oral anticoagulants. Thromb Haemost. 2001;86(2):563–8.Google Scholar
  11. 11.
    Assiri A, Al-Majzoub O, Kanaan AO, Donovan JL, Silva M. Mixed treatment comparison meta-analysis of aspirin, warfarin, and new anticoagulants for stroke prevention in patients with nonvalvular atrial fibrillation. Clin Ther. 2013;35(7):967–984.e962.CrossRefGoogle Scholar
  12. 12.
    Setoguchi S, Glynn RJ, Avorn J, Mogun H, Schneeweiss S. Statins and the risk of lung, breast, and colorectal cancer in the elderly. Circulation. 2007;115(1):27–33.CrossRefGoogle Scholar
  13. 13.
    Curtis JR, Delzell E, Chen L, et al. The relationship between bisphosphonate adherence and fracture: is it the behavior or the medication? Results from the placebo arm of the fracture intervention trial. J Bone Miner Res. 2011;26(4):683–8.CrossRefGoogle Scholar
  14. 14.
    Shrank WH, Patrick AR, Brookhart MA. Healthy user and related biases in observational studies of preventive interventions: a primer for physicians. J Gen Intern Med. 2011;26(5):546–50.CrossRefGoogle Scholar
  15. 15.
    Lund JL, Richardson DB, Sturmer T. The active comparator, new user study design in pharmacoepidemiology: historical foundations and contemporary application. Curr Epidemiol Rep. 2015;2(4):221–8.CrossRefGoogle Scholar
  16. 16.
    Gulmez SE, Lassen AT, Aalykke C, et al. Do statins protect against upper gastrointestinal bleeding? Br J Clin Pharmacol. 2009;67(4):460–5.CrossRefGoogle Scholar
  17. 17.
    Hackam DG, Woodward M, Newby LK, et al. Statins and intracerebral hemorrhage: collaborative systematic review and meta-analysis. Circulation. 2011;124(20):2233–42.CrossRefGoogle Scholar
  18. 18.
    Badillo R, Schmidt R, Mortensen EM, Frei CR, Mansi I. Statin therapy and gastrointestinal hemorrhage: a retrospective cohort study with propensity score-matching. Pharmacoepidemiol Drug Saf. 2015;24(8):849–57.CrossRefGoogle Scholar
  19. 19.
    van Rein N, Cannegieter SC, le Cessie S, et al. Statins and risk of bleeding: an analysis to evaluate possible bias due to prevalent users and healthy user aspects. Am J Epidemiol. 2016;183(10):930–6.CrossRefGoogle Scholar
  20. 20.
    Huitfeldt A, Hernan MA, Kalager M, Robins JM. Comparative effectiveness research using observational data: active comparators to emulate target trials with inactive comparators. EGEMS (Wash DC). 2016;4(1):1234.Google Scholar
  21. 21.
    Leonard CE, Brensinger CM, Bilker WB, et al. Gastrointestinal bleeding and intracranial hemorrhage in concomitant users of warfarin and antihyperlipidemics. Int J Cardiol. 2017;228:761–70.CrossRefGoogle Scholar
  22. 22.
    Brazauskas R, Logan BR. Observational studies: matching or regression? Biol Blood Marrow Transpl. 2016;22(3):557–63.CrossRefGoogle Scholar
  23. 23.
    Cummings P, McKnight B, Greenland S. Matched cohort methods for injury research. Epidemiol Rev. 2003;25:43–50.CrossRefGoogle Scholar
  24. 24.
    Sutradhar R, Baxter NN, Austin PC. Terminating observation within matched pairs of subjects in a matched cohort analysis: a Monte Carlo simulation study. Stat Med. 2016;35(2):294–304.CrossRefGoogle Scholar
  25. 25.
    Schelleman H, Bilker WB, Brensinger CM, Wan F, Yang YX, Hennessy S. Fibrate/Statin initiation in warfarin users and gastrointestinal bleeding risk. Am J Med. 2010;123(2):151–7.CrossRefGoogle Scholar
  26. 26.
    Schelleman H, Brensinger CM, Bilker WB, Hennessy S. Antidepressant-warfarin interaction and associated gastrointestinal bleeding risk in a case-control study. PLoS One. 2011;6(6):e21447.CrossRefGoogle Scholar
  27. 27.
    Cunningham A, Stein CM, Chung CP, Daugherty JR, Smalley WE, Ray WA. An automated database case definition for serious bleeding related to oral anticoagulant use. Pharmacoepidemiol Drug Saf. 2011;20(6):560–6.CrossRefGoogle Scholar
  28. 28.
    Nakasian SS, Rassen JA, Franklin JM. Effects of expanding the look-back period to all available data in the assessment of covariates. Pharmacoepidemiol Drug Saf. 2017;26(8):890–9.CrossRefGoogle Scholar
  29. 29.
    Kerlin MP, Tokar JL. Acute gastrointestinal bleeding. Ann Intern Med. 2013;159(11):793–4.CrossRefGoogle Scholar
  30. 30.
    Tielleman T, Bujanda D, Cryer B. Epidemiology and Risk Factors for Upper Gastrointestinal Bleeding. Gastrointest Endosc Clin N Am. 2015;25(3):415–28.CrossRefGoogle Scholar
  31. 31.
    Patrick AR, Schneeweiss S, Brookhart MA, et al. The implications of propensity score variable selection strategies in pharmacoepidemiology: an empirical illustration. Pharmacoepidemiol Drug Saf. 2011;20(6):551–9.CrossRefGoogle Scholar
  32. 32.
    Myers JA, Rassen JA, Gagne JJ, et al. Effects of adjusting for instrumental variables on bias and precision of effect estimates. Am J Epidemiol. 2011;174(11):1213–22.CrossRefGoogle Scholar
  33. 33.
    Normand ST, Landrum MB, Guadagnoli E, et al. Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores. J Clin Epidemiol. 2001;54(4):387–98.CrossRefGoogle Scholar
  34. 34.
    Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar Behav Res. 2011;46(3):399–424.CrossRefGoogle Scholar
  35. 35.
    Rassen JA, Glynn RJ, Rothman KJ, Setoguchi S, Schneeweiss S. Applying propensity scores estimated in a full cohort to adjust for confounding in subgroup analyses. Pharmacoepidemiol Drug Saf. 2012;21(7):697–709.CrossRefGoogle Scholar
  36. 36.
    Rassen JA, Shelat AA, Myers J, Glynn RJ, Rothman KJ, Schneeweiss S. One-to-many propensity score matching in cohort studies. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 2):69–80.CrossRefGoogle Scholar
  37. 37.
    Binswanger I, Blatchford P, Smiley-Mcdonald H, Ellis C. NDI-DCRP data linking task final report. 2014.Google Scholar
  38. 38.
    Austin PC. The performance of different propensity score methods for estimating marginal hazard ratios. Stat Med. 2013;32(16):2837–49.CrossRefGoogle Scholar
  39. 39.
    McMahan DA, Smith DM, Carey MA, Zhou XH. Risk of major hemorrhage for outpatients treated with warfarin. J Gen Intern Med. 1998;13(5):311–6.CrossRefGoogle Scholar
  40. 40.
    Copland M, Walker ID, Tait RC. Oral anticoagulation and hemorrhagic complications in an elderly population with atrial fibrillation. Arch Intern Med. 2001;161(17):2125–8.CrossRefGoogle Scholar
  41. 41.
    Zuccaro G. Epidemiology of lower gastrointestinal bleeding. Best Pract Res Clin Gastroenterol. 2008;22(2):225–32.CrossRefGoogle Scholar
  42. 42.
    McGraw D, Rosati K, Evans B. A policy framework for public health uses of electronic health data. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):18–22.CrossRefGoogle Scholar
  43. 43.
    Forrow S, Campion DM, Herrinton LJ, et al. The organizational structure and governing principles of the Food and Drug Administration’s Mini-Sentinel pilot program. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):12–7.CrossRefGoogle Scholar
  44. 44.
    Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64(7):749–59.CrossRefGoogle Scholar
  45. 45.
    Schneeweiss S, Seeger JD, Maclure M, Wang PS, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol. 2001;154(9):854–64.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Ryan M. Carnahan
    • 1
    Email author
  • Joshua J. Gagne
    • 2
  • Christian Hampp
    • 3
  • Charles E. Leonard
    • 4
  • Sengwee Toh
    • 5
  • Candace C. Fuller
    • 5
  • Sean Hennessy
    • 4
  • Laura Hou
    • 5
  • Noelle M. Cocoros
    • 5
  • Genna Panucci
    • 5
  • Tiffany Woodworth
    • 5
  • Austin Cosgrove
    • 5
  • Aarthi Iyer
    • 5
  • Elizabeth A. Chrischilles
    • 1
  1. 1.Department of Epidemiology, College of Public HealthUniversity of IowaIowa CityUSA
  2. 2.Division of Pharmacoepidemiology and Pharmacoeconomics, Department of MedicineBrigham and Women’s Hospital and Harvard Medical SchoolBostonUSA
  3. 3.Center for Drug Evaluation and Research, US Food and Drug AdministrationSilver SpringUSA
  4. 4.Center for Pharmacoepidemiology Research and Training, Center for Clinical Epidemiology and Biostatistics, and Department of Biostatistics, Epidemiology, and InformaticsUniversity of Pennsylvania Perelman School of MedicinePhiladelphiaUSA
  5. 5.Harvard Pilgrim Health Care Institute and Harvard Medical SchoolBostonUSA

Personalised recommendations