Comment on: “Zoo or Savannah? Choice of Training Ground for Evidence-Based Pharmacovigilance”
To the Editor:
We read the article by Norén et al.  with great interest and commend their effort in bringing forward a critical issue in the evaluation of signal detection methodologies, namely the choice of a benchmark (a reference standard) and an associated evaluation strategy.
Norén et al. argue that signal detection is fundamentally a prognostic activity. Therefore, evaluation strategies should aim to emulate a prospective analysis of signal detection in lieu of a commonly applied yet unsatisfactory approach of retrospective analysis based on well established associations such as those comprising the Observational Medical Outcome Partnership (OMOP)  and EU-ADR benchmarks . Norén et al. demonstrate that the two evaluation strategies may lead to different conclusions. They partially attribute this discrepancy to biasing effects (e.g., the influence of publicity on spontaneous reporting and on patient management), which are a consequence of examining well established...
KeywordsSignal Detection Evaluation Strategy Prospective Analysis Drug Ingredient Observational Medical Outcome Partnership
Rave Harpaz and Nigam H. Shah acknowledge support by NIH Grant U54-HG004028 for the National Center for Biomedical Ontology and by NIGMS Grant GM101430-01A1.
Competing financial interests
Rave Harpaz, William DuMouchel and Nigam H. Shah declare no competing financial interests. Rave Harpaz and William DuMouchel are employed by Oracle Health Sciences. Nigam H. Shah is a Science Advisor to ApixioInc (http://www.apixio.com), and Kyron Inc (http://www.kyron.com).
- 4.Harpaz R, Odgers D, Gaskin G, et al. A time-indexed reference standard of adverse drug reactions. Nat Sci Data 1. 2014. Art ID 140043. doi: 10.1038/sdata.2014.43.