Drug Safety

, Volume 36, Supplement 1, pp 5–14

Desideratum for Evidence Based Epidemiology

  • J. Marc Overhage
  • Patrick B. Ryan
  • Martijn J. Schuemie
  • Paul E. Stang
COMMENTARY

DOI: 10.1007/s40264-013-0102-2

Cite this article as:
Overhage, J.M., Ryan, P.B., Schuemie, M.J. et al. Drug Saf (2013) 36(Suppl 1): 5. doi:10.1007/s40264-013-0102-2

Abstract

Background

There is great variation in choices of method and specific analytical details in epidemiological studies, resulting in widely varying results even when studying the same drug and outcome in the same database. Not only does this variation undermine the credibility of the research but it limits our ability to improve the methods.

Methods

In order to evaluate the performance of methods and analysis choices we used standard references and a literature review to identify 164 positive controls (drug–outcome pairs believed to represent true adverse drug reactions), and 234 negative controls (drug–outcome pairs for which we have confidence there is no direct causal relationship). We tested 3,748 unique analyses (methods in combination with specific analysis choices) that represent the full range of approaches to adjusting for confounding in five large observational datasets on these controls. We also evaluated the impact of increasingly specific outcome definitions, and performed a replication study in six additional datasets. We characterized the performance of each method using the area under the receiver operator curve (AUC), bias, and coverage probability. In addition, we developed simulated datasets that closely matched the characteristics of the observational datasets into which we inserted data consistent with known drug–outcome relationships in order to measure the accuracy of estimates generated by the analyses.

Discussion

We expect the results of this systematic, empirical evaluation of the performance of these analyses across a moderate range of outcomes and databases to provide important insights into the methods used in epidemiological studies and to increase the consistency with which methods are applied, thereby increasing the confidence in results and our ability to systematically improve our approaches.

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • J. Marc Overhage
    • 1
    • 4
    • 5
  • Patrick B. Ryan
    • 2
    • 5
  • Martijn J. Schuemie
    • 3
    • 5
  • Paul E. Stang
    • 2
    • 5
  1. 1.Siemens Health ServicesMalvernUSA
  2. 2.Janssen Research and Development LLCTitusvilleUSA
  3. 3.Department of Medical InformaticsErasmus University Medical CenterRotterdamThe Netherlands
  4. 4.Siemens Medical Solutions USA, Inc.MalvernUSA
  5. 5.Observational Medical Outcomes Partnership, Foundation for the National Institutes of HealthBethesdaUSA

Personalised recommendations