Drug Safety

, Volume 38, Issue 1, pp 113–114 | Cite as

Comment on: “Zoo or Savannah? Choice of Training Ground for Evidence-Based Pharmacovigilance”

  • Rave Harpaz
  • William DuMouchel
  • Nigam H. Shah
Letter to the Editor

To the Editor:

We read the article by Norén et al. [1] with great interest and commend their effort in bringing forward a critical issue in the evaluation of signal detection methodologies, namely the choice of a benchmark (a reference standard) and an associated evaluation strategy.

Norén et al. argue that signal detection is fundamentally a prognostic activity. Therefore, evaluation strategies should aim to emulate a prospective analysis of signal detection in lieu of a commonly applied yet unsatisfactory approach of retrospective analysis based on well established associations such as those comprising the Observational Medical Outcome Partnership (OMOP) [2] and EU-ADR benchmarks [3]. Norén et al. demonstrate that the two evaluation strategies may lead to different conclusions. They partially attribute this discrepancy to biasing effects (e.g., the influence of publicity on spontaneous reporting and on patient management), which are a consequence of examining well established...


Signal Detection Evaluation Strategy Prospective Analysis Drug Ingredient Observational Medical Outcome Partnership 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Rave Harpaz and Nigam H. Shah acknowledge support by NIH Grant U54-HG004028 for the National Center for Biomedical Ontology and by NIGMS Grant GM101430-01A1.

Competing financial interests

Rave Harpaz, William DuMouchel and Nigam H. Shah declare no competing financial interests. Rave Harpaz and William DuMouchel are employed by Oracle Health Sciences. Nigam H. Shah is a Science Advisor to ApixioInc (, and Kyron Inc (


  1. 1.
    Noren GN, Caster O, Juhlin K, Lindquist M. Zoo or savannah? Choice of training ground for evidence-based pharmacovigilance. Drug Saf. 2014;37(9):655–9.PubMedCrossRefGoogle Scholar
  2. 2.
    Ryan PB, Schuemie MJ, Welebob E, Duke J, Valentine S, Hartzema AG. Defining a reference set to support methodological research in drug safety. Drug Saf. 2013;36(Suppl. 1):S33–47.PubMedCrossRefGoogle Scholar
  3. 3.
    Coloma PM, Avillach P, Salvo F, et al. A reference standard for evaluation of methods for drug safety signal detection using electronic healthcare record databases. Drug Saf. 2013;36(1):13–23.PubMedCrossRefGoogle Scholar
  4. 4.
    Harpaz R, Odgers D, Gaskin G, et al. A time-indexed reference standard of adverse drug reactions. Nat Sci Data 1. 2014. Art ID 140043. doi: 10.1038/sdata.2014.43.
  5. 5.
    Harpaz R, Dumouchel W, Lependu P, Bauer-Mehren A, Ryan P, Shah NH. Performance of pharmacovigilance signal-detection algorithms for the FDA adverse event reporting system. Clin Pharmacol Ther. 2013;93(6):539–46.PubMedCrossRefGoogle Scholar
  6. 6.
    Boyce RD, Ryan PB, Noren GN, et al. Bridging islands of information to establish an integrated knowledge base of drugs and health outcomes of interest. Drug Saf. 2014;37(8):557–67.PubMedCentralPubMedCrossRefGoogle Scholar
  7. 7.
    LePendu P, Iyer SV, Bauer-Mehren A, et al. Pharmacovigilance using clinical notes. Clin Pharmacol Ther. 2013;93(6):547–55.PubMedCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Rave Harpaz
    • 1
  • William DuMouchel
    • 1
  • Nigam H. Shah
    • 2
  1. 1.Oracle Health SciencesBedfordUSA
  2. 2.Stanford UniversityStanfordUSA

Personalised recommendations