, Volume 33, Issue 2, pp 295–312 | Cite as

Hunting Side Effects and Explaining Them: Should We Reverse Evidence Hierarchies Upside Down?

  • Barbara OsimaniEmail author


Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special attention to the privileged role assigned to randomized controlled trials (RCTs) in evidence based medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended effects, in that the unknowns at stake are heterogonous in the two contexts. However, although “lower level” evidence is increasingly acknowledged to be a valid source of information contributory to assessing the risk profile of medications on theoretical or empirical grounds, current practices have difficulty in assigning a precise epistemic status to this kind of evidence because they are more or less implicitly parasitic on the (statistical) methods developed to test drug efficacy. My thesis is that (1) “lower level” evidence is justified on distinct grounds and at different conditions depending on the different epistemologies which one wishes to endorse, in that each impose different constraints on the methods we adopt to collect and evaluate evidence; (2) such constraints ought to be understood to be different in the case of evidence for risk versus benefit assessment for a series of reasons which I will illustrate on the basis of the recent debate on the causal association between acetaminophen (a.k.a. paracetamol) and asthma.


Evidence hierarchies Randomized controlled trials Clinical methodology Causal inference Scientific inference Induction Abduction Hypothetico-deductive method Precautionary principle Acetaminophen Paracetamol 



Thanks go to two anonymous referees and to: Nancy Cartwright, Jeremy Howick, Phyllis Illari, Bert Leuridan, David Papineau, Julian Reiss, Federica Russo, David Teira, and Jan Vandenbroucke. All provided precious comments on earlier drafts of the paper. I am also indebted to the audiences to which the paper has been presented at different stages of the research project: Evidence and Causality Conference (Canterbury); Conference of the Italian Society for Analytical Philosophy (Alghero); IFOM-IEO Campus (Milan); Concepts of Health Seminar at King’s College London. Funding for the paper has been provided by an intramural project conducted with Prof. Fiorenzo Mignini at the University of Camerino: “Epistemic Asymmetries in Benefit vs. Risk Assessment of Pharmaceuticals”.


  1. Agency for Healthcare Research and Quality (2007) Methods reference guide for effectiveness and comparative effectiveness reviews, Version 1.0 [Draft posted October 2007). Available at:
  2. Allmers H, Skudlik C, John SM (2009) Acetaminophen use: a risk for asthma? Curr Allergy Asthma Rep 9(2):164–167CrossRefGoogle Scholar
  3. Amberbir A, Medhin G, Alem A, Britton J, Davey G, Venn A (2011) The role of acetaminophen and geohelminth infection on the incidence of wheeze and eczema: a longitudinal birth-cohort study. Am J Respir Crit Care Med 183(2):165–170CrossRefGoogle Scholar
  4. Anderson HR (1989) Increase in hospital admissions for childhood asthma: trends in referral, severity and readmissions from 1970 to 1985 in a health region of the United Kingdom. Thorax 44:614–619CrossRefGoogle Scholar
  5. Aronson JK, Hauben M (2006) Anecdotes that provide definitive evidence. BMJ 333(16):1267–1269CrossRefGoogle Scholar
  6. Ayer AJ (1956) The problem of knowledge. MacMillan, LondonGoogle Scholar
  7. Ayer AJ (1957) The conception of probability as a logical relation. In: Korner S (ed) Observation and interpretation in the philosophy of physics. Dover Publications, New YorkGoogle Scholar
  8. Barr RG, Wentowski CC, Curhan GC et al (2004) Prospective study of acetaminophen use and newly diagnosed asthma among women. Am J Respir Crit Care Med 169(7):836–841CrossRefGoogle Scholar
  9. Basu D (1980) Randomization analysis of experimental data: the fisher randomization test. J Am Stat Assoc 75(371):593–595Google Scholar
  10. Beasley RW, Clayton TO, Crane J, ISAAC Phase Three Study Group et al (2008) Association between acetaminophen use in infancy and childhood, and risk of asthma, rhinoconjunctivitis, and eczema in children aged 6–7 years: analysis from phase three of the ISAAC programme. Lancet 372(9643):1039–1048CrossRefGoogle Scholar
  11. Beasley RW, Clayton TO, Crane J, ISAAC Phase Three Study Group et al (2011a) Acetaminophen use and risk of asthma, rhinoconjunctivitis, and eczema in adolescents: International Study of Asthma and Allergies in Childhood Phase Three. Am J Respir Crit Care Med 183(2):171–178CrossRefGoogle Scholar
  12. Beasley RW, Crane J, Lai C, Stewart A, Clayton T (2011b) Acetaminophen and asthma: spurious association? Am J Respir Critical Care Med; Jun 1;183(11): author reply 1570–1Google Scholar
  13. Benson K, Hartz AJ (2000) A comparison of observational studies and randomised, controlled trials. N Engl J Med 342(25):1878–1886CrossRefGoogle Scholar
  14. Broadbent A (2011) Inferring causation in epidemiology: mechanisms, black-boxes, and constraints. In: Illari PM, Russo F, Williamson J (eds) Causality in the Sciences. Oxford University Press, OxfordGoogle Scholar
  15. Burr ML, Butland BK, King S, Vaughan Williams E (1989) Changes in asthma prevalence: two surveys 15 years apart. Arch Dis Child 64:1452–1456CrossRefGoogle Scholar
  16. Carnap R (1947) On the application of inductive logic. Philos Phenomenol Res 8(1):133–148CrossRefGoogle Scholar
  17. Cartwright N (2007) Are RCTs the gold standard? Biosocieties 2:11–20CrossRefGoogle Scholar
  18. Cartwright N (2010) Presidential address. Am Philos Assoc Pac Div, Vancouver, April 10Google Scholar
  19. Chafee FH, Settipane GA (1967) Asthma caused by FD&C approved dyes. J Allergy 40:347–351CrossRefGoogle Scholar
  20. Chalmers I (2005) Statistical theory was not the reason that randomization was used in the British Medical Research Council’s clinical trial of streptomycin for pulmonary tubercolosis. In: Jorland G, Opinel A, Weisz G (eds) Body counts: medical quantification in historical and sociological perspectives. McGill-Queens University Press, Montreal, pp 309–334Google Scholar
  21. Chang KC, Leung CC, Tam CM, Kong FY (2011) Acetaminophen and asthma: spurious association? Am J Respir Crit Care Med 183(11):1570CrossRefGoogle Scholar
  22. Clarke B, Gillies D, Illari P, Russo F, Williamson J (2012) The evidence that evidence-based medicine omits. Prev Med. doi: 10.1016/j.ypmed.2012.10.020 Google Scholar
  23. Clarke B, Gillies D, Illari P, Russo F, Williamson J (forthcoming) Mechanisms and the evidence hierarchy. TOPOI, special issue “Evidence and Causality in the Sciences”Google Scholar
  24. Concato J, Shah MPH, Horwitz RI (2000) Randomized, controlled trials, observational studies and the hierarchy of research designs. N Engl J Med 342(25):1887–1892CrossRefGoogle Scholar
  25. de Paramo BJ, Gancedo SQ, Cuevas M et al (2000) Acetaminophen (acetaminophen) hypersensitivity. Ann Allergy Asthma Immunol 85:508–511CrossRefGoogle Scholar
  26. Eder W, Ege JM, von Mutius E (2006) The asthma epidemic. NEJM 355(21):2226–2235CrossRefGoogle Scholar
  27. Eneli I, Sadri K, Camargo C, Barr RG (2005) Acetaminophen and the risk of asthma: the epidemiologic and pathophysiologic evidence. CEST 127(2):604–612Google Scholar
  28. Evidence Based Medicine Working Group (1992) Evidence-based medicine: a new approach to the teaching the practice of medicine. JAMA 268(17):2420–2425CrossRefGoogle Scholar
  29. Farquhar H, Stewart A, Mitchell E, Crane J, Eyers S, Weatherall M, Beasley R (2010) The role of paracetamol in the pathogenesis of asthma. Clin Exp Allergy 40(1):32–41Google Scholar
  30. Freedman B (1987) Equipoise and the ethics of clinical research. N Engl J Med 317:141–145CrossRefGoogle Scholar
  31. Galindo PA, Borja J, Mur P et al (1998) Anaphylaxis to acetaminophen. Allergol Immunopathol 26:199–200Google Scholar
  32. Gifford F (1995) Community equipoise and the ethics of randomized clinical trials. Bioethics 9:127–148CrossRefGoogle Scholar
  33. Gifford F (2007a) So-called ‘equipose’ and the argument form design. J Med Philos 32:135–150CrossRefGoogle Scholar
  34. Gifford F (2007b) Taking equipoise seriously: The failure of clinical or community equipoise to resolve ethical dilemmas in randomized clinical trials. In: Kincaid H, McKitrick J (eds) Establishing medical reality essays in the metaphysiscs and epistemology of biomedical science. Springer, New YorkGoogle Scholar
  35. Glasziou P, Chalmers I, Rawlins M, McCullock P (2007) When are randomized trials necessary? Picking signal from noise. Br Med J 334:349–351CrossRefGoogle Scholar
  36. Golder S, Loke YK, Bland M (2011) Meta-analyses of adverse effects data derived from randomized controlled trials as compared to observational studies: methodological overview. PLoS Med 8(5):1–13CrossRefGoogle Scholar
  37. Graham NM, Burrell CJ, Douglas RM, Debelle P, Davies L (1990) Adverse effects of aspirin, acetaminophen, and ibuprofen on immune finction, viral shedding, and clinical status in rhinovirus-infected volunteers. J Infect Dis 162(6):1277–1282CrossRefGoogle Scholar
  38. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ (2011) GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 64(4):383–394Google Scholar
  39. Guyatt GH, Oxman AD, Vist GE et al (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Br Med J 336:924–926CrossRefGoogle Scholar
  40. Henderson AJ, Shaheen SO (2013) Acetaminophen and asthma. Paediatr Respir Rev 14(1):9–16CrossRefGoogle Scholar
  41. Herxheimer A (2012) Pharmacovigilance on the turn? Adverse reactions methods in 2012. Br J Gen Pract 62(601):401–402Google Scholar
  42. Holgate ST (2011) The acetaminophen enigma in asthma. Am J Respir Crit Care Med 183:147–148CrossRefGoogle Scholar
  43. Howick J (2011) The Philosophy of Evidence-Based Medicine. Wiley-Blackwell, NYCrossRefGoogle Scholar
  44. Howick J, Glasziou P, Aronson JK (2009) The evolution of evidence hierarchies: what can Hill’s ‘guidelines for causation’ contribute? J R Soc Med 102:186–194CrossRefGoogle Scholar
  45. Howick J, Chalmers I, Glasziou P, Greenhalgh T, Heneghan C, Liberati A, Moschetti I, Phillips B, and Thornton H (2011) The 2011 Oxford CEBM evidence levels of evidence (introductory document). Oxford Centre for Evidence-Based Medicine.
  46. Howick J, Glasziou P, Aronson JK (2013) Can understanding mechanisms solve the problem of extrapolating from study to target populations (the problem of ‘external validity’)? J R Soc Med 106:81–86CrossRefGoogle Scholar
  47. Howson C, Urbach P (2006) Scientific reasoning: the Bayesian approach. Open Court, IllinoisGoogle Scholar
  48. Jick H (1977) The discovery of drug-induced illness. N Engl J Med 296:481–485CrossRefGoogle Scholar
  49. Jick H, Vessey M (1978) Case-control studies in evaluation of drug-induced illness. Am J Epidemiol 107:1–7Google Scholar
  50. Joffe M (2011) Causality and evidence discovery in epidemiology. In: Dieks D, Gonzales WJ, Hartmann S, Uebel T, Wever M (eds) Explanation, prediction, and confirmation. Springer, Berlin, pp 153–166CrossRefGoogle Scholar
  51. Johnson CC, Ownby DR (2011) Have the efforts to prevent aspirin-related Reye’s syndrome fuelled an increase in asthma? Clin Exp Allergy 42(3):296–298CrossRefGoogle Scholar
  52. Karimi M, Mirzaei M, Ahmadieh MH (2006) Acetaminophen use and the symptoms of asthma, allergic rhinitis and eczema in children. Iran J Allergy Asthma Immunol 5(2):63–67Google Scholar
  53. Keynes JM (1921) A treatise on probability. MacMillan, LondonGoogle Scholar
  54. Kyburg HE, Teng CM (2001) Uncertain inference. Cambridge University Press, CambridgeGoogle Scholar
  55. La Caze A (2009) Evidence based medicine must be. J Med Philos 34(5):509–527CrossRefGoogle Scholar
  56. La Caze A (2013) Why randomized interventional studies. J Med Philos 38(4):352–368Google Scholar
  57. Lesko SM, Louik C, Vezina RM, Mitchell AA (2002) Asthma morbidity after the short-term use of ibuprofen in children. Pediatrics 109(2):e20. Available at Google Scholar
  58. Lindley DV (1982) The role of randomization in inference. PSA: Proceedings of the Biennial meeting of the philosophy of science association, vol II: symposia and invited papers, pp 431–446Google Scholar
  59. Lowe A, Carlin JB, Bennett CM, Hosking CS, Allen KJ, Robertson CF, Axelrad C, Abramson MJ, Hill DJ, Dharmage SC (2010) Paracetamol use in early life and asthma: prospective birth cohort study. BMJ 341:c4616CrossRefGoogle Scholar
  60. Mannino DM, Homa DM, Pertowski CA, Ashizawa A, Nixon LL, Johnson CA, Ball LB, Jack E, Kang DS (1998) Surveillance for Asthma—United States, 1960–1995. MMWR CDC Surveill Summ 47(SS-1):1–28Google Scholar
  61. Mannino DM, Homa DM, Akinbami LJ, Moorman JE, Gwynn C, Redd SC (2002) Surveillance for Asthma—United States, 1980–1999. MMWR CDC Surveill Summ 51(SS01):1–13Google Scholar
  62. Martinez-Gimeno A, García-Marcos L (2013) The association between acetaminophen and asthma: should its pediatric use be banned? Expert Rev Respir Med. 7(2):113–122CrossRefGoogle Scholar
  63. McBride JT (2011) The association of acetaminophen and asthma prevalence and severity. Pediatrics 128(6):1181–1185. Available at: Google Scholar
  64. McKeever TM, Lewis SA, Smit HA, Burney P, Britton JR, Cassano PA (2005) The association of acetaminophen, aspirin, and ibuprofen with respiratory disease and lung function. Am J Respir Crit Care Med 171(9):966–1071CrossRefGoogle Scholar
  65. Meheus J (2011) A formal logic for the abduction of singular hypotheses. In: Diecks D, Gonzales WJ, Hartmann S, Uebel T, Weber M (eds) Explanation, Prediction, and Confirmation. SpringerGoogle Scholar
  66. Miettinen OS (1983) The need for randomization in the study of intended effects. Stat Med 2:267–271CrossRefGoogle Scholar
  67. Nassini R, MAterazzi S, Andre E, Sartiani L, Aldini G, Trevisani M et al (2010) Acetaminophen, via its reactive metabolite N-acetyl-p-benzo-quinonemine and transient receptor potential ankyrin-1 stimulation, causes neurigenic inflammation in the airways and other tissues in rodents. FASEB J 24:4904–4916CrossRefGoogle Scholar
  68. Newson RB, Shaheen SO, Chinn S, Burney PG (2000) Acetaminophen sales and atopic diseases in children and adults: an ecological analysis. Eur Respir J 16(5):817–823CrossRefGoogle Scholar
  69. Ninan TK, Russell G (1992) Respiratory symptoms and atopy in Aberdeen schoolchildren: evidence from two surveys 25 years apart. Br Med J 304:873–875CrossRefGoogle Scholar
  70. Osimani B (2007) Probabilistic information and decision-making in the health context: Package leaflets as basis for informed consent. PhD Thesis, University of LuganoGoogle Scholar
  71. Osimani B (2013a) The precautionary principle in the pharmaceutical domain: a philosophical enquiry into probabilistic reasoning and risk aversion. Health, Risk & Society (special issue: Health Care through the Lens of Risk) 15(2):123–143Google Scholar
  72. Osimani B (2013b) Until RCT-proven? On the asymmetry of evidence requirements for risk assessment. J Eval Clin Pract 19:454–462Google Scholar
  73. Osimani B, Russo F, Williamson J (2011) Scientific evidence and the law: an objective Bayesian formalisation of the precautionary principle in pharmaceutical regulation. J Philos Sci Law 11Google Scholar
  74. Papanikolaou PN, Christidi GD, Ioannidis JPA (2006) Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. Can Med Assoc J 174(5):635–641CrossRefGoogle Scholar
  75. Papineau D (1994) The virtues of randomization. Br J Philos Sci 45(2):437–450CrossRefGoogle Scholar
  76. Platts-Mills TA, Erwin E, Heymann P, Woodfolk J (2005) Is the hygiene hypothesis still a viable explanation for the increased prevalence of asthma? Allergy 60(suppl 79):25–31Google Scholar
  77. Psaty B, Vandenbroucke JP (2008) Opportunities for enhancing the FDA guidance on pharmacovigilance. JAMA 300(8):952–953CrossRefGoogle Scholar
  78. Russo F, Williamson J (2007) Interpreting causality in the health sciences. Int Stud Philos Sci 21(2):157–170CrossRefGoogle Scholar
  79. Russo F, Williamson J (2011) Epistemic causality and evidence-based medicine. Hist Philos Life Sci 33(4):563–582Google Scholar
  80. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS (1996) Evidence based medicine: what it is and what it isn’t. Br Med J 312:71–72CrossRefGoogle Scholar
  81. Seaton A, Godden DJ, Brown K (1994) Increase in Asthma: a more toxic environment or a more susceptible population? Thorax 49:171–174CrossRefGoogle Scholar
  82. Shaheen SO, Sterne JAC, Songhurst CE, Burney PGJ (2000) Frequent acetaminophen use and asthma in adults. Thorax 55:266–270CrossRefGoogle Scholar
  83. Shaheen S, Potts J, Gnatiuc L et al (2008) Selenium and Asthma Research Integration Project; GA2LEN. The relation between acetaminophen use and asthma: a GA2LEN European case-control study. Eur Respir J 32(5):1231–1236CrossRefGoogle Scholar
  84. Stegenga J (2011) Is meta-analysis the platinum standard of evidence? Stud Hist Philos Biol Biomed Sci 42:497–507CrossRefGoogle Scholar
  85. Straus SE, McAlister FA (2000) Evidence-based medicine: a commentary on common criticisms. Can Med Assoc J 163(7):837–841Google Scholar
  86. Stricker B, Psaty B (2004) Detection, verification, and quantification of adverse drug reactions. Br Med J 329:44–47CrossRefGoogle Scholar
  87. Subbarao P, MAndhane PJ, Sears MR (2009) Asthma: epidemiology, etiology, risk factors. CMAJ 181(9):181–190CrossRefGoogle Scholar
  88. Suppes P (1982) Arguments for randomizing. PSA 2:464–475Google Scholar
  89. Teira D (2011) Frequentist versus Bayesian Clinical Trials. In: Gifford F (eds) Handbook of the philosophy of science, vol 16: Philosophy of Medicine pp 255–298Google Scholar
  90. Teira D, Reiss J (2013) Causality, impartiality and evidence-based policy. In: Chao H-K, Chen S-T, Millstein R (eds) Towards the Methodological Turn in the Philosophy of Science: Mechanism and Causality in Biology and Economics, Springer, New York, pp 207–224Google Scholar
  91. Urbach P (1994) Reply to David Papineau. Br J Philos Sci 45(2):712–715CrossRefGoogle Scholar
  92. Vandenbroucke JP (2004) When are observational studies as credible as randomized trials? Lancet 363:1728–1731CrossRefGoogle Scholar
  93. Vandenbroucke JP (2006) What is the best evidence for determining harms of medical treatment? Commentary. Can Med Assoc J 174(5):645–646CrossRefGoogle Scholar
  94. Vandenbroucke JP (2007) Letter to the editor. JAMA 297(19):2077–2078CrossRefGoogle Scholar
  95. Vandenbroucke JP (2008) Observational research, randomised trials, and two views of medical science. Plos Med 5(3):339–343 (quote in the text are from the longer version: 1–28)CrossRefGoogle Scholar
  96. Vandenbroucke JP, Psaty BM (2008) Benefits and risks of drug treatments. How to combine the best evidence on benefits with the best data about adverse effects. JAMA 300(20):2417–2419CrossRefGoogle Scholar
  97. Varner AE, Busse WW, Lemanske RF Jr (1998) Hypothesis: decreased use of pediatric aspirin has contributed to the increasing prevalence of childhood asthma. Ann Allergy Asthma Immunol 81(4):347–351CrossRefGoogle Scholar
  98. Wickens K, Beasley R, Town I, Epton M, Pattemore P, Ingham T, Crane J; New Zealand Asthma and Allergy Cohort Study Group. (2011) The effects of early and late acetaminophen exposure on asthma and atopy: a birth cohort. Clin Exp Allergy. 2011 Mar; 41(3):399–406. doi: 10.1111/j.1365-2222.2010.03610.x. Epub 2010 Sep 29Google Scholar
  99. Worral J (2007a) Evidence in medicine and evidence-based medicine. Philos Compass 2(6):981–1022CrossRefGoogle Scholar
  100. Worral J (2007b) Why there’s no cause to randomize? Br J Philos Sci 58:451–488CrossRefGoogle Scholar
  101. Worral J (2010) Do we need some large, simple randomized trials in medicine? In: Suárez, Mauricio and Dorato, Mauro and Rédei, Miklós, (eds.) EPSA philosophical issues in the sciences: launch of the European Philosophy of Science Association. Springer, LondonGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.School of PharmacologyUniversity of CamerinoCamerinoItaly

Personalised recommendations