Drug Safety

, Volume 30, Issue 8, pp 645–655 | Cite as

Gold Standards in Pharmacovigilance

The Use of Definitive Anecdotal Reports of Adverse Drug Reactions as Pure Gold and High-Grade Ore
  • Manfred HaubenEmail author
  • Jeffrey K. Aronson
Current Opinion


Anecdotal reports of adverse drug reactions are generally regarded as being of poor evidential quality. This is especially relevant for postmarketing drug safety surveillance, which relies heavily on spontaneous anecdotal reports. The numerous limitations of spontaneous reports cannot be overemphasised, but there is another side to the story: these datasets also contain anecdotal reports that can be considered to describe definitive adverse reactions, without the need for further formal verification. We have previously defined four categories of such adverse reactions: (i) extracellular or intracellular tissue deposition of the drug or a metabolite; (ii) a specific anatomical location or pattern of injury; (iii) physiological dysfunction or direct tissue damage demonstrable by physicochemical testing; and (iv) infection, as a result of the administration of an infective agent as the therapeutic substance or because of demonstrable contamination. In this article, we discuss the implications of these definitive (‘between-the-eyes’) adverse effects for pharmacovigilance.


Adverse Drug Reaction Pure Gold Spontaneous Reporting System Proportional Reporting Ratio Bayesian Confidence Propagation Neural Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We are grateful for the helpful comments of David Madigan, Robin Ferner, Paul Glasziou, Yoon Loke and Jan Vandenbroucke, who reviewed draft versions of the manuscript, and Andrew Bate, François Girardin and Sheila Weiss Smith, who refereed our original paper in the British Medical Journal.

Dr Hauben is an employee of Pfizer Inc. and owns stock and stock options in Pfizer and other pharmaceutical companies. The authors have no conflicts of interest that are directly relevant to the content of this manuscript. No funding was provided for the preparation of this review.


  1. 1.
    Aronson JK, Hauben M. Anecdotes that provide definitive evidence. BMJ 2006; 333: 1267–9PubMedCrossRefGoogle Scholar
  2. 2.
    Bisckford CL, Spencer AP. Biliary sludge and hyperbilirubinema associated with ceftriaxone in an adult: case report and review of the literature. Pharmacotherapy 2005; 25(10): 1389–95CrossRefGoogle Scholar
  3. 3.
    Tokumine F, Sunagawa T, Shiohira Y, et al. Drug-associated cholelithiasis: a case of sulindac stone formation and the incorporation of sulindac metabolites into the gallstones. Am J Gastroenterol 1999; 94(8): 2285–8PubMedCrossRefGoogle Scholar
  4. 4.
    Eda A, Yanaka I, Tamada K, et al. Sulindac-associated choledocholithiasis. Am J Gastroenterol 2001; 96(7): 2283–5PubMedCrossRefGoogle Scholar
  5. 5.
    Daudon M, Jungers P. Drug-induced renal calculi: epidemiology, prevention, and management. Drugs 2004; 64(3): 245–75PubMedCrossRefGoogle Scholar
  6. 6.
    Hauben M, Reich L, Gerrits C. Comparative performance or proportional reporting rations (PRR) and multi-item gamma-Poisson shrinker (MGPS) for the identification of crystalluria and urinary tract calculi caused by drugs. Pharmacoepidemiol Drug Saf 2005; S014: 7Google Scholar
  7. 7.
    Messmer E, Font RL, Sheldon G, et al. Pigmented conjunctival cysts following tetracycline/minocycline therapy. Ophthalmology 1983; 90: 1462–8PubMedGoogle Scholar
  8. 8.
    Morrison VL, Kikkawa DO, Herndier BG, et al. Tetracycline induced green conjunctival deposits. Br J Ophthalmol 2005; 89: 1372–3PubMedCrossRefGoogle Scholar
  9. 9.
    Bandla HPR, Davis SH, Hopkins NE. Lipoid pneumonia: a silent complication of mineral oil aspiration. Pediatrics 1999; 103(2): E19PubMedCrossRefGoogle Scholar
  10. 10.
    Taylor JR, Streetman DS, Castle SS. Medication bezoars: a literature review and a report of a case. Ann Pharmacother 1998; 32(9): 940–6PubMedCrossRefGoogle Scholar
  11. 11.
    Champman AH, el Hasani S. Colon ischaemia secondary to barolith obstruction. Br J Radiol 1998; 71(849): 983–4PubMedGoogle Scholar
  12. 12.
    Eiferman RA, Snyder JP, Nordquist RE. Ciprofloxacin microprecipitates and macroprecipitates in the human corneal epithelium. J Cataract Refract Surg 2001; 27: 1701–2PubMedCrossRefGoogle Scholar
  13. 13.
    Lopez JD, del Castillo JMB, Lopez CD, et al. Confocal microscopy in ocular chrysiasis. Cornea 2003; 22(6): 573–5PubMedCrossRefGoogle Scholar
  14. 14.
    Nadin F, Haddad W, Adib J. The differential diagnosis of crystals in the retina. Int Ophthalmol 2001; 24(3): 113–21CrossRefGoogle Scholar
  15. 15.
    Goralczyk R, Barker FM, Buser S, et al. Dose dependency of canthaxanthin crystals in monkey retina and spatial distribution of its metabolites. Invest Ophthalmol Vis Sci 2000; 41(6): 1513–22PubMedGoogle Scholar
  16. 16.
    Culora GA, Ramsay AD, Theaker JM. Aluminium and injection site reactions. J Clin Pathol 1996; 49(10): 844–7PubMedCrossRefGoogle Scholar
  17. 17.
    Sukpanichnant S, Hargrove NS, Kachintorn U, et al. Clofazimine-induced crystal-storing histiocytosis producing chronic abdominal pain in a leprosy patient. Am J Surg Pathol 2000; 24(1): 129–35PubMedCrossRefGoogle Scholar
  18. 18.
    Zanetta G, Maurice-Estepa L, Mousson C, et al. Foscarnet-induced crystalline glomerulonephritis with nephritic syndrome and acute renal failure after kidney transplantation. Transplantation 1999; 67(10): 1376–8PubMedCrossRefGoogle Scholar
  19. 19.
    Rollins SD, Craig JP. Gold-associated lymphadenopathy in a patient with rheumatoid arthritis. Histologic and scanning electron microscopic features. Arch Pathol Lab Med 1991; 115(2): 175–7 654PubMedGoogle Scholar
  20. 20.
    Hendricks A. Yellow lunulae with fluorescence after tetracycline therapy. Arch Dermatol 1980; 116: 438–40PubMedCrossRefGoogle Scholar
  21. 21.
    Dixit VB, Chaudhary SD, Jain VK. Clofazimine induced nail changes. Indian J Lepr 1989; 61: 476–8PubMedGoogle Scholar
  22. 22.
    Adams PC, Holt DW, Storey GCA, et al. Amiodarone and its desethyl metabolite: tissue distribution and morphologic changes during long-term therapy. Circulation 1985; 72(5): 1064–75PubMedCrossRefGoogle Scholar
  23. 23.
    O’Neill JL, Remington TI. Drug-induced esophageal injuries and dysphagia. Ann Pharmacother 2003; 37(11): 1675–84PubMedCrossRefGoogle Scholar
  24. 24.
    Adami NP, de Gutierrez MG, da Fonseca SM, et al. Risk management of extravasation of cytostatic drugs at the Adult Chemotherapy Outpatient Clinic of a university hospital. J Clin Nurs 2005; 14(7): 876–82PubMedCrossRefGoogle Scholar
  25. 25.
    van der Leede H, Jorens PG, Parizel P, et al. Inadvertent intrathecal use of ionic contrast agent leading to nervous system toxicity. Eur Radiol 2002; 12 Suppl. 3: S86–93PubMedGoogle Scholar
  26. 26.
    Alcaraz A, Rey C, Conch A, et al. Intrathecal vincristine and fatal myeloencephalopathy despite cerebrospinal fluid perfusion. J Toxicol Clin Toxicol 2002; 40(5): 557–61PubMedCrossRefGoogle Scholar
  27. 27.
    Manson AJ, Hanagasi H, Turner K, et al. Intravenous apomorphine therapy in Parkinson’s disease: clinical and pharmacokinetic observations. Brain 2001; 124 (Pt 2): 331–40PubMedCrossRefGoogle Scholar
  28. 28.
    Sapir S, Bimstein E. Cholinsalicylate gel induced oral lesion: report of case. J Clin Pediatr Dent 2000; 24(2): 103–6PubMedGoogle Scholar
  29. 29.
    Brazier WJ, Dhariwal DK, Patton DW, et al. Ecstasy related periodontitis and mucosal ulceration: a case report. Br Dent J 2003; 194(4): 197–9PubMedCrossRefGoogle Scholar
  30. 30.
    Seyer BA, Grist W, Muller S. Aggressive destructive midfacial lesion from cocaine abuse. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2002; 94(4): 465–70PubMedCrossRefGoogle Scholar
  31. 31.
    Yamasaki K, Yamasaki A, Tosaki M, et al. Tissue distribution of Thorotrast and role of internal irradiation in carcinogenesis. Oncol Rep 2004; 12(4): 733–8PubMedGoogle Scholar
  32. 32.
    Ben-Zeev B, Watemberg N, Augarten A, et al. Oligohydrosis and hyperthermia: a pilot study of a novel topiramate adverse event. J Child Neurol 2003; 18(4): 254–7PubMedCrossRefGoogle Scholar
  33. 33.
    Shimizu T, Yamashita Y, Satoi M, et al. Heat stroke-like episode in a child caused by zonisamide. Brain Dev 1997; 19: 366–8PubMedCrossRefGoogle Scholar
  34. 34.
    Okumura A, Hayakawa F, Kuno K, et al. Oligohidrosis caused by zonisamide. No To Hattatsu 1996; 28(1): 44–7PubMedGoogle Scholar
  35. 35.
    Lee A, Joo H, Chey W, et al. Photopatch testing in seven cases of photosensitive drug eruptions. Ann Pharmacother 2002; 35: 1584–7CrossRefGoogle Scholar
  36. 36.
    Martin-Lazaro J, Bujan JG, Arrondo AP, et al. Is photopatch testing useful in the investigation of photosensitivity due to flutamide? Contact Dermatitis 2004; 50(5): 325–6PubMedCrossRefGoogle Scholar
  37. 37.
    Jeanmougin M, Manciet JR, De Prost Y, et al. Fenofibrate photoallergy. Ann Dermatol Venereol 1993; 120(8): 549–54PubMedGoogle Scholar
  38. 38.
    Schiffman SS, Zervakis J, Westall HL, et al. Effects of anti-microbial and anti-inflammatory medications on the sense of taste. Physiol Behav 2000; 69: 413–24PubMedCrossRefGoogle Scholar
  39. 39.
    Teare JP, Spedding C, Whitehead MW, et al. Omeprazole and dry mouth. Scand J Gastroenterol 1995; 30(3): 216–8PubMedCrossRefGoogle Scholar
  40. 40.
    Trevenzoli M, Cattelan AM, Marino F, et al. Sepsis and granulomatous hepatitis after bacillus Calmette-Guerin intravesical installation. J Infect 2004; 48(4): 363–7PubMedCrossRefGoogle Scholar
  41. 41.
    Land MH, Rouster-Stevens K, Woods CR, et al. Lactobacillus sepsis associated with probiotic therapy. Pediatrics 2005; 115(1): 178–81PubMedGoogle Scholar
  42. 42.
    Kashiwagi Y, Kawashima H, Takekuma K, et al. Detection of mumps virus genome directly from clinical samples and a simple method for genetic differentiation of the Hoshino vaccine strain from wild strains of mumps virus. J Med Virol 1997; 52: 195–9PubMedCrossRefGoogle Scholar
  43. 43.
    Hauben M, Reich L. Endotoxin-like reactions with intravenous gentamicin: results from pharmacovigilance tools under investigation. Infect Control Hosp Epidemiol 2005; 26(4): 391–4PubMedCrossRefGoogle Scholar
  44. 44.
    Bennett SN, McNeil MM, Bland LA, et al. Postoperative infections traced to contamination of an intravenous anesthetic, propofol. N Engl J Med 1995; 333(3): 147–54PubMedCrossRefGoogle Scholar
  45. 45.
    Koh Y, Shu CL. A new algorithm to identify the causality of ADRs. Drug Saf 2005; 28(12): 1159–61PubMedCrossRefGoogle Scholar
  46. 46.
    Edwards IR, Aronson JK. Adverse drug reactions: definitions, diagnosis, and management. Lancet 2000; 356(9237): 1255–9PubMedCrossRefGoogle Scholar
  47. 47.
    Valenstein PN. Evaluating diagnostic tests with imperfect standards. Am J Clin Pathol 1990; 93: 252–8PubMedGoogle Scholar
  48. 48.
    Obuchowski NA. Special topics III: bias. Radiology 2003; 229: 617–21PubMedCrossRefGoogle Scholar
  49. 49.
    Streiner DL. Diagnosing tests: using and misusing diagnostic and screening tests. J Pers Assess 2003; 81(3): 209–19PubMedCrossRefGoogle Scholar
  50. 50.
    Coggon D, Martyn C, Palmer KT, et al. Assessing case definitions in the absence of a diagnostic gold standard. Int J Epidemiol 2005; 34: 949–52PubMedCrossRefGoogle Scholar
  51. 51.
    Wacholder S, Armstrong B, Hartge P. Validation studies using an alloyed gold standard. Am J Epidemiol 1993; 137: 1251–8PubMedGoogle Scholar
  52. 52.
    Bossuyt PM, Reitsma JB, Bruns DE, et al. Standards for reporting of diagnostic accuracy. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin Chem 2003; 49(1): 7–18PubMedCrossRefGoogle Scholar
  53. 53.
    Pfeiffer RM, Castle PE. With or without a gold standard. Epidemiology 2005; 16(5): 595–7PubMedCrossRefGoogle Scholar
  54. 54.
    Greenland S. Multiple-bias modeling for analysis of observational data. J R Stat Soc 2005; 168: 267–308CrossRefGoogle Scholar
  55. 55.
    Phillips CV. Quantifying and reporting uncertainty from systematic errors. Epidemiology 2003; 14: 459–66PubMedGoogle Scholar
  56. 56.
    Aronson JK, Ferner RE. Clarification of terminology in drug safety. Drug Saf 2005; 28(10): 851–70PubMedCrossRefGoogle Scholar
  57. 57.
    Hauben M, Reich L, Gabbay F. Extension of points on clarifying terminology in drug safety. Drug Saf 2006; 29(3): 273–5PubMedCrossRefGoogle Scholar
  58. 58.
    Buehler JW, Hopkins RS, Overhage JM, et al. Framework for evaluating public health surveillance systems for early detection of outbreaks. MMWR Recomm Rep 2004; 53 (RR-5): 1–11PubMedGoogle Scholar
  59. 59.
    Aronson JK. Unity from diversity: the evidential use of anecdotal reports of adverse drug reactions and interactions. J Eval Clin Pract 2005; 11(2): 195–208PubMedCrossRefGoogle Scholar
  60. 60.
    Hauben M, Madigan D, Gerrits C, et al. The role of data mining in pharmacovigilance. Exp Opinion Drug Saf 2005; 4(5): 929–48CrossRefGoogle Scholar
  61. 61.
    Hauben M, Patadia V, Gerrits C, et al. Data mining in pharmacovigilance: the need for a balanced perspective. Drug Saf 2005; 28(10): 835–42PubMedCrossRefGoogle Scholar
  62. 62.
    Banks D, Woo EJ, Burren DR, et al. Comparing data mining methods in the VAERS database. Pharmacoepidemiol Drug Saf 2005; 14(9): 601–9PubMedCrossRefGoogle Scholar
  63. 63.
    Lilienfeld DE. A challenge to data miners. Pharmacoepidemiol Drug Saf 2005; 13(12): 881–4CrossRefGoogle Scholar
  64. 64.
    Hauben M, Bate A. Data mining in drug safety. In: Aronson JK, editor. Side effects of drugs annual29. Amsterdam: Elsevier Science Ltd, 2007: xxxiii–xlviCrossRefGoogle Scholar
  65. 65.
    Hauben M, Reich L. Response to Levine et al. Br J Clin Pharmacol 2006; 61(1): 115–7 655CrossRefGoogle Scholar
  66. 66.
    Fricker RD, Rolka H. Protecting against biological terrorism: statistical issues in electronic biosurveillance. Chance 2007; 19(4): 4–13Google Scholar
  67. 67.
    Tramèr MR, Moore RA, Reynolds DJ, et al. Quantitative estimation of rare adverse events which follow a biological progression: a new model applied to chronic NSAID use. Pain 2000; 85(1–2): 169–82PubMedCrossRefGoogle Scholar
  68. 68.
    Wald NJ, Morris JK. Teleoanalysis: combining data from different types of study. BMJ 2003; 327(7415): 616–8PubMedCrossRefGoogle Scholar
  69. 69.
    Hauben M. Trimethoprim-induced hyperkalemia: lessons in data mining. Br J Clin Pharmacol 2004; 58: 338–9PubMedCrossRefGoogle Scholar
  70. 70.
    Hauben M, Reich L. Drug-induced pancreatitis: lessons in data mining. Br J Clin Pharmacol 2004; 58(5): 560–2PubMedCrossRefGoogle Scholar
  71. 71.
    Bate A, Lindquist M, Orre R, et al. Data-mining analyses of pharmacovigilance signals in relation to relevant comparison drugs. Eur J Clin Pharmacol 2002; 58(7): 483–9PubMedCrossRefGoogle Scholar
  72. 72.
    Szarfman A, Machado SG, O’Neill RT. Use of screening algorithms and computer systems to efficiently signal higher-than-normal expected combinations of drugs and events in the FDA spontaneous reports database. Drug Saf 2002; 25(6): 381–92PubMedCrossRefGoogle Scholar
  73. 73.
    Korantzopoulos P, Pappa E, Karanikis P, et al. Acute low-back pain during intravenous administration of amiodarone: a report of two cases. Int J Cardiol 2005; 98(2): 355–7PubMedCrossRefGoogle Scholar
  74. 74.
    Smyth J. Determining optimal therapy-randomized trials in individual patients. N Engl J Med 1986; 315: 767–8PubMedCrossRefGoogle Scholar
  75. 75.
    Hofler M. Analytical perspective. The Bradford Hill considerations on causality: a counterfactual perspective. Emerg Themes Epidemiol 2005; 2: 11PubMedCrossRefGoogle Scholar
  76. 76.
    Wirth GJ, Teuscher J, Graf JD, et al. Efavirenz-induced urolithiasis. Urol Res 2006; 34: 288–9PubMedCrossRefGoogle Scholar
  77. 77.
    McGee AM, Davison PM. Skin necrosis following injection of non-steroidal anti-inflammatory drug. Br J Anaesthesia 2002; 88: 139–40CrossRefGoogle Scholar
  78. 78.
    Cherasse A, Kahn M, Mistrih R, et al. Nicolau’s syndrome after local glucocorticoid injection. Joint Bone Spine 2003; 70(5): 390–2PubMedCrossRefGoogle Scholar
  79. 79.
    Kohler LD, Schwedler S, Worret WI. Embolia cutis medicamentosa. Int J Dermatol 1997; 36(3): 197PubMedCrossRefGoogle Scholar
  80. 80.
    Albrecht KH, Littman K, Richter HJ, et al. Animal experiment studies of embolia cutis medicamentosa. Langenbecks Arch Chir 1984; 362(1): 17–23PubMedCrossRefGoogle Scholar
  81. 81.
    Shojaei AR, Haas DA. Local anesthetic cartridges and latex allergy: a literature review. J Can Dent Assoc 2002; 68(10): 622–6PubMedGoogle Scholar
  82. 82.
    Towse A, O’Brien M, Twarog FJ, et al. Local reaction secondary to insulin injection: a potential role for latex antigens in insulin vials and syringes. Diabetes Care 1995; 18(8): 1195–7PubMedCrossRefGoogle Scholar
  83. 83.
    Hauben M. Data mining in pharmacovigilance: integrating statistical and computational approaches. Drug Safety Research Unit 3rd Biennial Conference on Signal Detection and Evaluation; 2005 Sep 28–29; LondonGoogle Scholar
  84. 84.
    Stricker B. Pharmacovigilance: a case of phantom ships and Russian roulette. Ned Tijdschr Geneeskd 2002; 146(27): 1258–61PubMedGoogle Scholar
  85. 85.
    Mazzotta P, Magee LA. A risk-benefit assessment of pharmacological and nonpharmacological treatments for nausea and vomiting of pregnancy. Drugs 2000; 59(4): 781–800PubMedCrossRefGoogle Scholar
  86. 86.
    Hauben M, Reich L, van Puijenbroek, et al. Data mining in pharmacovigilance: lessons from phantom ships. Eur J Clin Pharmacol 2006; 62: 967–70PubMedCrossRefGoogle Scholar
  87. 87.
    Moore D. Drug-induced cutaneous photosensitivity: incidence, mechanism, prevention and management. Drug Saf 2002; 25(5): 345–72PubMedCrossRefGoogle Scholar
  88. 88.
    Moons KGM, Biesheuvel CJ, Grobbee DE. Test research versus diagnostic research. Clin Chem 2004; 50(3): 473–6PubMedCrossRefGoogle Scholar
  89. 89.
    Vandenbroucke JP. In defense of case reports and case series. Ann Intern Med 2001; 134(4): 330–4PubMedGoogle Scholar

Copyright information

© Adis Data Information BV 2007

Authors and Affiliations

  1. 1.Risk Management Strategy, Pfizer Inc.New YorkUSA
  2. 2.Department of MedicineNew York University School of MedicineNew YorkUSA
  3. 3.Department of Community and Preventive MedicineNew York Medical CollegeValhallaUSA
  4. 4.Department of PharmacologyNew York Medical CollegeValhallaUSA
  5. 5.Department of Clinical PharmacologyRadcliffe InfirmaryOxfordEngland

Personalised recommendations