Mechanisms and the Evidence Hierarchy

Abstract

Evidence-based medicine (EBM) makes use of explicit procedures for grading evidence for causal claims. Normally, these procedures categorise evidence of correlation produced by statistical trials as better evidence for a causal claim than evidence of mechanisms produced by other methods. We argue, in contrast, that evidence of mechanisms needs to be viewed as complementary to, rather than inferior to, evidence of correlation. In this paper we first set out the case for treating evidence of mechanisms alongside evidence of correlation in explicit protocols for evaluating evidence. Next we provide case studies which exemplify the ways in which evidence of mechanisms complements evidence of correlation in practice. Finally, we put forward some general considerations as to how the two sorts of evidence can be more closely integrated by EBM.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    We recognise that hierarchies are capable of playing other roles in facilitating medical decision making too, such as providing safeguards against litigation, or simplifying prescription practices (Timmermans and Berg 2003). However, our focus here is firmly on their epistemological role.

  2. 2.

    ‘Mechanism-based reasoning’ seems to refer roughly to what we call evidence of mechanisms. Howick writes ‘Mechanistic reasoning is an inferential chain (or web) linking the intervention (such as HRT) with a patient-relevant outcome, via relevant mechanisms.’ (Howick 2011, p. 929).

  3. 3.

    The International Agency for Research on Cancer (IARC) is one of the few agencies that now tries to systematically consider evidence of mechanisms (IARC 2006, \((\S B.4)\) ). One way it does this is by formulating a two dimensional hierarchy that considers evidence obtained in experimental studies on animals along one dimension and evidence obtained on humans along the other. However, on each dimension the emphasis is still on evidence obtained from statistical trials.

  4. 4.

    There is an established tradition in sociology and social science on social mechanisms, see for instance the classic text Hedström and Swedberg (1988) and the recent contribution Demeulenaere (2011).

  5. 5.

    The remainder of this section will deal with the reference class problem as applied to trial design. However, as the epidemiological terminology differs significantly between trial design and interpretation of trial data, it’s worth making a few brief remarks here regarding the problem as it affects interpretation, particularly the practice of stratification. Stratification involves dividing up trial results to examine outcomes in partitions of the trial population thought to be interestingly different from the general population. For instance, a trial of an antihypertensive agent might stratify the trial population into age groups at the data analysis stage, to see if the drug response differs. This kind of practice is limited by various kinds of information bias, including the sparse data problem: the smaller the strata, the greater the variability of apportionment ratios (Rothman et al. 2008), and the lower the precision of any resulting causal claims. In short, the problem is identical in either trial design or interpretation.

  6. 6.

    And see the literature on sample size calculation for examples of this (Rothman et al. 2008, pp. 149ff).

  7. 7.

    Howick (2011) provides other examples of such compelling stories and how they led to the development of EBM.

  8. 8.

    MRC (1948, 1949, 1950) contain the reports on these trials published in the British Medical Journal. There is an overview of the trials in Daniels and Bradford Hill (1952), while Bradford Hill (1990) gives some interesting reminiscences.

  9. 9.

    In a sense, we follow up the suggestion of Solomon (2011) that EBM largely ignores basic science, particularly mechanisms, and we offer an account of how to integrate grading evidence of mechanisms and grading evidence of correlation.

References

  1. Bassler D, Briel M, Montori VM, Lane M, Glasziou P, Zhou Q, Heels-Ansdell D, Walter SD, Guyatt GH, Flynn DN et al (2010) Stopping randomized trials early for benefit and estimation of treatment effects. J Am Med Assoc 303(12):1180–1187

    Article  Google Scholar 

  2. Bechtel W (2008) Mental mechanisms: philosophical perspectives on cognitive neuroscience. Routledge, Oxford

    Google Scholar 

  3. Bell G (2008) Selection: the mechanism of evolution, 2nd edn. OUP

  4. Bem DJ (2011) Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect. J Pers Soc Psychol 100:407–425

    Article  Google Scholar 

  5. Bernard C (1856) An introduction to the study of experimental medicine. Macmillan, New York (1927 edition)

  6. Bradford Hill A (1965) The environment and disease: association or causation? Proc R Soc Med 58:295–300

    Google Scholar 

  7. Bradford Hill A (1990) Memories of the British streptomycin trial. Controlled Clin Trials 2:77–79

    Article  Google Scholar 

  8. Broadbent A (2011) Inferring causation in epidemiology: mechanisms, black boxes, and contrasts. In: Illari P, Russo F, Williamson J (eds) Causality in the sciences. Oxford University Press, Oxford, pp 45–69

    Google Scholar 

  9. Brock TD (1988) Robert Koch: a life in medicine and bacteriology, 2 edn. ASM Press

  10. Campaner R (2011) Understanding mechanisms in the health sciences. Theor Med Bioethics 32:5–17

    Article  Google Scholar 

  11. Campaner R, Galavotti MC (2012) Evidence and the assessment of causal relations in the health sciences. Int Stud Philos Sci 26(1):27–45

    Article  Google Scholar 

  12. Carter KC (2012) The decline of therapeutic bloodletting and the collapse of traditional medicine. Transaction Publishers, New Brunswick

    Google Scholar 

  13. Cartwright N (2007) Causal powers: what are they? why do we need them? what can be done with them and what cannot? LSE discussion paper. http://www.lse.ac.uk/CPNSS/projects/CoreResearchProjects/ContingencyDissentInScience/DP/CausalPowersMonographCartwrightPrint%20Numbers%20Corrected.pdf

  14. Cartwright N (2010) What are randomised controlled trials good for? Philos Stud 147:59–70

    Article  Google Scholar 

  15. Cartwright N, Hardie J (2012) Evidence based policy: a practical guide to doing it better. OUP, USA

  16. Cartwright N, Munro E (2010) The limitations of randomized controlled trials in predicting effectiveness. J Eval Clin Pract 16:260–266

    Article  Google Scholar 

  17. Clarke B (2011a) Causality in medicine with particular reference to the viral causation of cancers. PhD thesis. Department of Science and Technology Studies, University College London, London

  18. Clarke B (2011b) Causation and melanoma classification. Theor Med Bioethics 32:19–32

    Article  Google Scholar 

  19. Clarke B, Gillies D, Illari P, Russo F, Williamson J (2013) The evidence that evidence-based medicine omits. Prev Med. doi:10.1016/j.ypmed.2012.10.020

  20. Claveau F (2012) The Russo–Williamson theses in the social sciences: causal inference drawing on two types of evidence. Stud Hist Philos Biol Biomed Sci 43(4):806–813

    Google Scholar 

  21. Cook T, Campbell D (1979) Quasi-experimentation: design and analysis issues for field settings. Rand MacNally, Chicago

    Google Scholar 

  22. Craver C (2007) Explaining the brain. Clarendon Press, Oxford

    Google Scholar 

  23. Cucherat M, Haugh MC, Gooch M, Boissel J-P (2000) Evidence of clinical efficacy of homeopathy: a meta-analysis of clinical trials. Eur J Clin Pharmacol 56:27–33

    Article  Google Scholar 

  24. Daniels M, Bradford Hill A (1952) Chemotherapy of pulmonary tuberculosis in young adults. An analysis of the combined results of three medical research council trials. Br Med J 1162–1168

  25. Darby G, Williamson J (2011) Imaging technology and the philosophy of causality. Philos Technol 24(2):115–136

    Article  Google Scholar 

  26. Darden L (2006) Reasoning in biological discoveries. Cambridge University Press, Cambridge

    Google Scholar 

  27. Debré P (1994) Louis Pasteur. The John Hopkins University Press. English translation by Elborg Forster, 1998

  28. Demeulenaere P (ed) (2011) Analytical sociology and social mechanisms. Cambridge University Press, Cambridge

  29. Dragulinescu S (2012) On ‘stabilising’ medical mechanisms, truth-makers and epistemic causality: a critique to Williamson and Russo’s approach. Synthese 187(2):785–800

    Google Scholar 

  30. Fayad ZA, Mani V, Woodward M, Kallend D, Abt M, Burgess T, Fuster V, Ballantyne CM, Stein EA, Tardif J-C, Rudd JHF, Farkouh ME, Tawakol A (2011) Safety and efficacy of dalcetrapib on atherosclerotic disease using novel non-invasive multimodality imaging (dal-PLAQUE): a randomised clinical trial. Lancet 378(9802):1547–1559

    Article  Google Scholar 

  31. Florey ME (1961) The clinical application of antibiotics. Streptomycin and other antibiotics active Against tuberculosis. vol 2, Oxford University Press, London

    Google Scholar 

  32. Fox W, Ellard G, Mitchison D (1999) Studies on the treatment of tuberculosis undertaken by the british medical research council tuberculosis units, 1946–1986, with relevant subsequent publications. Int J Tuberc Lung Dis 3(10s2):S231–S279

    Google Scholar 

  33. Gillies DA (2011) The Russo–Williamson thesis and the question of whether smoking causes heart disease. In: Illari PM, Russo F, Williamson J (eds) Causality in the sciences, Oxford University Press, Oxford, pp 110–125

    Google Scholar 

  34. Hedström P, Swedberg R (1988) Social mechanism: an analytical approach to social theory. Cambridge University Press, Cambridge

    Google Scholar 

  35. Hitchcock C (2001) A tale of two effects. Philos Rev 110(3):361–396

    Article  Google Scholar 

  36. Howick J (2011) Exposing the vanities—and a qualified defence—of mechanistic evidence in clinical decision-making. Philos Sci 78(5):926–940. Proceedings of the Biennial PSA 2010

    Google Scholar 

  37. IARC (2006) IARC monographs on the evaluation of carcinogenic risks to humans: preamble. Int Agency Res Cancer, http://monographs.iarc.fr/ENG/Preamble/CurrentPreamble.pdf

  38. Illari PM (2011) Mechanistic evidence: disambiguating the Russo–Williamson thesis. Int Stud Philos Sci 25:139–157

    Article  Google Scholar 

  39. Illari PM, Williamson J (2012) What is a mechanism? thinking about mechanisms across the sciences. Eur J Philos Sci 2:119–135

    Article  Google Scholar 

  40. Kshirsagar AV, Carpenter M, Bang H, Wyatt SB, Colindres RE (2006) Blood pressure usually considered normal is associated with an elevated risk of cardiovascular disease. Am J Med 119(2):133–141

    Article  Google Scholar 

  41. La Caze A (2008) Evidence-based medicine can’t be …. Soc Epistemol 22(4):353–370

    Article  Google Scholar 

  42. La Caze A (2009) Evidence-based medicine must be …. J Med Philos 34:509–527

    Article  Google Scholar 

  43. La Caze A (2011) The role of basic science in evidence-based medicine. Biol Philos 26(1):81–98

    Article  Google Scholar 

  44. La Caze A, Djulbegovic B, Senn S (2012) What does randomisation achieve?. Evid Med 17:1–2

    Article  Google Scholar 

  45. Leibovici L (2001) Effects of remote, retroactive intercessory prayer on outcomes in patients with bloodstream infection: randomised controlled trial. Br Med J 323:1450–1451

    Article  Google Scholar 

  46. Lucas R (1976) Econometric policy evaluation. In: Brunner K, Meltzer A (eds) The Phillips curve and labor markets, volume 1 of Carnegie–Rochester conference series on public policy. North-Holland, Amsterdam, pp 161–168

  47. Machamer P, Darden L, Craver C (2000) Thinking about mechanisms. Philos Sci 67:1–25

    Article  Google Scholar 

  48. McLaren L, Ghali LM, Lorenzetti D, Rockl M (2006) Out of context? Translating evidence from the North Karelia project over place and time. Health Educ Res 22(3):414–424

    Article  Google Scholar 

  49. MRC (1948) Streptomycin treatment of pulmonary tuberculosis. Br Med J 769–782

  50. MRC (1949) Treatment of pulmonary tuberculosis with para-aminosalicylic acid and streptomycin: preliminary report. Br Med J 1521

  51. MRC (1950) Treatment of pulmonary tuberculosis with streptomycin and para-amino-salicylic acid. Br Med J 1073–1085

  52. Munro SA, Lewin SA, Smith HJ, Engel ME, Fretheim A, Volmink J (2007) Patient adherence to tuberculosis treatment: a systematic review of qualitative research. PLoS Med 4(7):e238

    Article  Google Scholar 

  53. NICE (2006) The guidelines manual. National Institute for Health and Clinical Excellence, London. Available from: http://www.nice.org.uk

  54. NICE (2009) The guidelines manual. National Institute for Health and Clinical Excellence, London. Available from: http://www.nice.org.uk

  55. NICE (2011a) CG117: tuberculosis. National Institute for Health and Clinical Excellence, London. Available from: http://www.nice.org.uk

  56. NICE (2011b) CG127: hypertension: full guideline. National Institute for Health and Clinical Excellence, London. Available from: http://www.nice.org.uk

  57. NICE (2011c) CG127: hypertension: quick reference guide. National Institute for Health and Clinical Excellence, London. Available from: http://www.nice.org.uk

  58. Northcott R (2012) How necessary are randomized controlled trials? In: Munson R (ed) Intervention and reflection: basic issues in medical ethics, 9th edn. Thomson Wadsworth, pp 187–191

  59. OCEBM Levels of Evidence Working Group (2011) The Oxford 2011 levels of evidence. Oxford Centre for Evidence-Based Medicine, http://www.cebm.net/index.aspx?o=5653

  60. Papineau D (1994) The virtues of randomization. Br J Philos Sci 45:437–450

    Article  Google Scholar 

  61. Puska P, Vartiaine E, Laatikainen T, Jousilahti P, Paavola M (eds) (2009) The North Karelia Project: from North Karelia to national action. National Institute for Health and Welfare

  62. Reichenbach H (1949) The theory of probability: an inquiry into the logical and mathematical foundations of the calculus of probability. University of California Press, Berkeley, CA

    Google Scholar 

  63. Rothman K, Greenland S, Lash T (2008) Modern epidemiology. Lippincott Williams & Wilkins, Philadelphia, PA

    Google Scholar 

  64. Russo F, Williamson J (2007) Interpreting causality in the health sciences. Int Stud Philos Sci 21(2):157–170

    Article  Google Scholar 

  65. Russo F, Williamson J (2011a) Epistemic causality and evidence-based medicine. History Philos Life Sci 33(4):563–582

    Google Scholar 

  66. Russo F, Williamson J (2011b) Generic versus single-case causality: the case of autopsy. Eur J Philos Sci 1(1):47–69

    Article  Google Scholar 

  67. Russo F, Williamson J (2012) EnviroGenomarkers: the interplay between mechanisms and difference making in establishing causal claims. Med Stud 3(4):249–262

    Article  Google Scholar 

  68. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS (1996) Evidence based medicine: what it is and what it isn’t. Br Med J 312(7023):71–72

    Article  Google Scholar 

  69. Salmon WC (1977) Objectively homogeneous reference classes. Synthese 36(4):399–414

    Article  Google Scholar 

  70. Salmon WC, Jeffrey RC, Greeno JG (1971) Statistical explanation and statistical relevance. University of Pittsburgh Press, Pittsburgh, PA

    Google Scholar 

  71. Schwartz GG, Olsson AG, Abt M, Ballantyne CM, Barter PJ, Brumm J, Chaitman BR, Holme IM, Kallend D, Leiter LA, Leitersdorf E, McMurray JJ, Mundl H, Nicholls SJ, Shah PK, Tardif J-C, Wright RS (2012) Effects of dalcetrapib in patients with a recent acute coronary syndrome. N Engl J Med 367(22):2089–2099. PMID: 23126252

    Google Scholar 

  72. Sober E (1988) The principle of the common cause. In: Fetzer JH (ed) Probability and causality: essays in honour of Wesley C. Salmon. Reidel, Dordrecht, pp 211–228

  73. Solomon M (2011) Just a paradigm: evidence-based medicine in epistemological context. Eur J Philos Sci 1:451–466

    Article  Google Scholar 

  74. Steel D (2008) Across the boundaries: extrapolation in biology and social science. Oxford University Press, Oxford

    Google Scholar 

  75. Stegenga J (2011) Is meta-analysis the platinum standard of evidence. Stud History Philos Biol Biomed Sci 42:497–507

    Article  Google Scholar 

  76. Stein EA, Roth EM, Rhyne JM, Burgess T, Kallend D, Robinson JG (2010) Safety and tolerability of dalcetrapib (RO4607381/JTT-705): results from a 48-week trial. Eur Heart J 31(4):480–488

    Article  Google Scholar 

  77. Thompson RP (2011) Causality, theories and medicine. In: Illari PM, Russo F, Williamson J (eds) Causality in the sciences. Oxford University Press, Oxford, pp 25–44

    Google Scholar 

  78. Timmermans S, Berg M (2003) The gold standard: the challenge of evidence-based medicine and standardization in health care. Temple University Press, Philadelphia

    Google Scholar 

  79. Victora CG, Habicht J-P, Bryce J (2004) Evidence-based public health: moving beyond randomized trials. Am J Public Health 94:400–405

    Article  Google Scholar 

  80. Wagner EH (1982) The North Karelia Project: what it tells us about the prevention of cardiovascular disease. Am J Public Health 72(1):51–53

    Article  Google Scholar 

  81. Weber E (2009) How probabilistic causation can account for the use of mechanistic evidence. Int Stud Philos Sci 23(3):277–295

    Article  Google Scholar 

  82. Williamson J (2005) Bayesian nets and causality: philosophical and computational foundations. Oxford University Press, Oxford

    Google Scholar 

  83. Williamson J (2013) How can causal explanations explain? Erkenntnis. doi:10.1007/s10670-013-9512-x

  84. Worrall J (2002) What evidence in evidence-based medicine. Philos Sci 69:S316–S330

    Article  Google Scholar 

  85. Worrall J (2007) Why there’s no cause to randomize. Br J Philos Sci 58:451–488

    Article  Google Scholar 

  86. Worrall J (2010) Evidence: philosophy of science meets medicine. J Eval Clin Pract 16:356–362

    Article  Google Scholar 

Download references

Acknowledgments

We thank the UK Arts and Humanities Research Council for supporting this research. F. Russo also acknowledges financial support from the FWO-Flanders (2012–2013) as Pegasus Marie Curie Fellow. We are extremely grateful to the very many people who came to various events we organised during 2012, and participated in the discussions that allowed us to develop these ideas. We owe particular thanks to Ian McKay, Barbara Osimani, Jacob Stegenga and David Teira for extensive comments leading to significant improvements to the paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Phyllis Illari.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Clarke, B., Gillies, D., Illari, P. et al. Mechanisms and the Evidence Hierarchy. Topoi 33, 339–360 (2014). https://doi.org/10.1007/s11245-013-9220-9

Download citation

Keywords

  • Mechanism
  • Difference-making
  • Evidence
  • Evidence of mechanism
  • Evidence in medicine
  • Evidence-based medicine