Skip to main content
SpringerLink
Log in

Randomized Trials and Observational Studies: The Current Philosophical Controversy

  • Living reference work entry
  • First Online:
Handbook of the Philosophy of Medicine

Abstract

The supposed superiority of randomized over non-randomized studies is used to justify claims about therapeutic effectiveness of medical interventions and also inclusion criteria for many systematic reviews of therapeutic interventions. However, the view that randomized trials provide better evidence has been challenged by philosophers of science. In addition, empirical evidence for average differences between randomized trials and observational studies (which we would expect if one method were superior) has proven difficult to find. This chapter reviews the controversy surrounding the relative merits of randomized trials and observational studies. It is concluded that while (well-conducted) observational can often provide the same level of evidential support as randomized trials, merits of (well-conducted) randomized trials warrant claims about their superiority, especially where results from the two methods are contradictory.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Adler UC, Krüger S, Teut M et al (2013) Homeopathy for depression: a randomized, partially double-blind, placebo-controlled, four-armed study (DEP-HOM). PLoS One 8:e74537

    Article  Google Scholar 

  • Altman DG (2002) Poor-quality medical research: what can journals do? JAMA J Am Med Assoc 287:2765–2767

    Article  Google Scholar 

  • Anglemyer A, Horvath HT, Bero L (2014) Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev 4:MR000034

    Google Scholar 

  • Barbui C, Cipriani A (2007) Publication bias in systematic reviews. Arch Gen Psychiatry 64:868

    Article  Google Scholar 

  • Berkson J, Magath T, Hurn M (1939) The error of estimate of the blood cell count as made with the hemocytometer. Am J Physiol 128:309–323

    Google Scholar 

  • Borgerson K (2009) Valuing evidence: bias and the evidence hierarchy of evidence-based medicine. Perspect Biol Med 52:218–233

    Article  Google Scholar 

  • Boswell K, Cook C, Burch S, Eaddy M, Cantrell R (2012) Associating medication adherence with improved outcomes: a systematic literature review. Am J Pharm Benefits 4:e97–e108

    Google Scholar 

  • Broadbent A (2013) Jeremy Howick: the philosophy of evidence-based medicine. Philos Sci 80:165–168

    Article  Google Scholar 

  • Bylund DB, Reed AL (2007) Childhood and adolescent depression: why do children and adults respond differently to antidepressant drugs? Neurochem Int 51:246–253

    Article  Google Scholar 

  • Carlson MD, Morrison RS (2009) Study design, precision, and validity in observational studies. J Palliat Med 12:77–82

    Article  Google Scholar 

  • Cartwright N (2007) Are RCTs the gold standard? Biosocieties 2:11–20

    Article  Google Scholar 

  • Cocco G (2009) Erectile dysfunction after therapy with metoprolol: the Hawthorne effect. Cardiology 112:174–177

    Article  Google Scholar 

  • Coronary Drug Project (1980) Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med 303:1038–1041

    Article  Google Scholar 

  • DeCensi A, Puntoni M, Goodwin P et al (2010) Metformin and cancer risk in diabetic patients: a systematic review and meta-analysis. Cancer Prev Res 3:1451–1461

    Article  Google Scholar 

  • Deupree JD, Reed AL, Bylund DB (2007) Differential effects of the tricyclic antidepressant desipramine on the density of adrenergic receptors in juvenile and adult rats. J Pharmacol Exp Ther 321:770–776

    Article  Google Scholar 

  • Di Blasi Z, Harkness E, Ernst E, Georgiou A, Kleijnen J (2001) Influence of context effects on health outcomes: a systematic review. Lancet 357:757–762

    Article  Google Scholar 

  • Eisenach JC, Lindner MD (2004) Did experimenter bias conceal the efficacy of spinal opioids in previous studies with the spinal nerve ligation model of neuropathic pain? Anesthesiology 100:765–767

    Article  Google Scholar 

  • Every-Palmer S, Howick J (2014) How evidence-based medicine is failing due to biased trials and selective publication. J Eval Clin Pract 20:908–914

    Article  Google Scholar 

  • Glasziou P, Chalmers I, Rawlings M, McCulloch P (2007) When are randomised trials unnecessary? Picking signal from noise. Br Med J 334:349

    Article  Google Scholar 

  • Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico (GISSI) (1986) Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet 22:397–402

    Google Scholar 

  • Guyatt GH, Oxman AD, Vist GE et al (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Br Med J 336:924–926

    Article  Google Scholar 

  • Higgins JJ, Green S (2008) The Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. Wiley Blackwell, Chichester

    Google Scholar 

  • Howick J (2011) The philosophy of evidence-based medicine. Wiley Blackwell/BMJ Books, Chichester

    Book  Google Scholar 

  • Howick J, Mebius A (2014). In search of justification for the unpredictability paradox. Trials 15:480

    Article  Google Scholar 

  • Jefferson T, Demichell V, Di Pietrantonj C, Jones M, Rivetti D (2006). Neuraminidase inhibitors for preventing and treating influenza in healthy adults. Cochrane Database Syst Rev 19(3)

    Google Scholar 

  • Jadad A (1998) Randomized controlled trials. BMJ Books, London

    Google Scholar 

  • Jefferson T, Jones MA, Doshi P et al (2014) Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev 4:CD008965

    Google Scholar 

  • Jones B, Howick J, Hopewell J, Liew SM (2014) Response to ‘Position statement on ethics, equipoise and research on charged particle therapy’. J Med Ethics 40:576–577

    Article  Google Scholar 

  • Keen HI, Pile K, Hill CL (2005) The prevalence of underpowered randomized clinical trials in rheumatology. J Rheumatol 32:2083–2088

    Google Scholar 

  • Knekt P, Reunanen A, Jarvinen R et al (1994) Antioxidant vitamin intake and coronary mortality in a longitudinal population study. Am J Epidemiol 139:1180–1189

    Google Scholar 

  • La Caze A, Djulbegovic B, Senn S (2012) What does randomization achieve? Evid Based Med 17:1–3

    Google Scholar 

  • Mant D (1999) Can randomised trials inform clinical decisions about individual patients? Lancet 35:743–746

    Article  Google Scholar 

  • McCarney R, Warner J, Iliffe S et al (2007) The Hawthorne effect: a randomised, controlled trial. BMC Med Res Methodol 7:30

    Article  Google Scholar 

  • McCloskey RV, Straube RC, Sanders C, Smith SM, Smith CR (1994) Treatment of septic shock with human monoclonal antibody HA-1A. A randomized, double-blind, placebo-controlled trial. Ann Intern Med 121:1–5

    Article  Google Scholar 

  • McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D (2002) Randomised trials in surgery: problems and possible solutions. Br Med J 324:1448–1451

    Article  Google Scholar 

  • Mebius A (2014) Corroborating evidence-based medicine. J Eval Clin Pract 20:915–920

    Article  Google Scholar 

  • Muthuri SG, Venkatesan S, Myles PR et al (2014) Effectiveness of neuraminidase inhibitors in reducing mortality in patients admitted to hospital with influenza A H1N1pdm09 virus infection: a meta-analysis of individual participant data. Lancet Respir Med 2:395–404

    Article  Google Scholar 

  • Oberai P, Balachandran I, Janardhanan N et al (2013) Homoeopathic management in depressive episodes: a prospective, unicentric, non-comparative, open-label observational study. Indian J Res Homoeopath 7:116–125

    Article  Google Scholar 

  • OCEBM Levels of Evidence Working Group (2011) The Oxford 2011 levels of evidence. Oxford Centre for Evidence-Based Medicine. Available at: http://www.cebm.net/index.aspx?o=5653. Accessed 17 June 2011

  • Odgaard-Jensen J, Vist GE, Timmer A et al (2011) Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev 4:MR000012

    Google Scholar 

  • Penston J (2003) Fact and fiction in medical research: the large-scale randomised trial. The London Press, London

    Google Scholar 

  • Petitti DB, Perlman JA, Sidney S (1987) Noncontraceptive estrogens and mortality: long-term follow-up of women in the Walnut Creek Study. Obstet Gynecol 70:289–293

    Google Scholar 

  • Rosenthal R, Lawson R (1964) A longitudinal study of the effects of experimenter bias on the operant learning of laboratory rats. J Psychiatr Res 69:61–72

    Article  Google Scholar 

  • Rossouw JE, Anderson GL, Prentice RL et al (2002) Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women’s Health Initiative randomized controlled trial. JAMA J Am Med Assoc 288:321–333

    Article  Google Scholar 

  • Schulz KF, Grimes DA (2002) Blinding in randomised trials: hiding who got what. Lancet 359:696–700

    Article  Google Scholar 

  • Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA J Am Med Assoc 273:408–412

    Article  Google Scholar 

  • Sesso HD, Buring JE, Christen WG et al (2008) Vitamins E and C in the prevention of cardiovascular disease in men: the physicians’ health study II randomized controlled trial. JAMA J Am Med Assoc 300:2123–2133

    Article  Google Scholar 

  • Sierevelt IN, van Oldenrijk J, Poolman RW (2007) Is statistical significance clinically important? A guide to judge the clinical relevance of study findings. J Long Term Eff Med Implants 17:173–179

    Article  Google Scholar 

  • Smith GD, Ebrahim S (2002) Data dredging, bias, or confounding. Br Med J 325:1437–1438

    Article  Google Scholar 

  • Stampfer MJ, Colditz GA (1991) Estrogen replacement therapy and coronary heart disease: a quantitative assessment of the epidemiologic evidence. Prev Med 20:47–63

    Article  Google Scholar 

  • Stevens RJ, Ali R, Bankhead CR et al (2012) Cancer outcomes and all-cause mortality in adults allocated to metformin: systematic review and collaborative meta-analysis of randomised clinical trials. Diabetologia 55:2593–2603

    Article  Google Scholar 

  • Straus SE, Glasziou P, Richardson WS, Haynes RB (2011) Evidence-based medicine: how to practice and teach EBM, 4th edn. Churchill Livingston, Edinburgh

    Google Scholar 

  • Stroup DF, Berlin JA, Morton SC et al (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA J Am Med 283:2008–2012

    Article  Google Scholar 

  • Vist GE, Bryant D, Somerville L, Birminghem T, Oxman AD (2008) Outcomes of patients who participate in randomized controlled trials compared to similar patients receiving similar interventions who do not participate. Cochrane Database Syst Rev 3:MR000009

    Google Scholar 

  • Wartolowska K, Judge A, Collins G et al (2014) Use of placebo controls in the evaluation of surgery: systematic review. Br Med J 348:g3253

    Article  Google Scholar 

  • Wood L, Egger M, Gluud LL et al (2008) Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. Br Med J 336:601–605

    Article  Google Scholar 

  • Worrall J (2002) What evidence in evidence-based medicine? Philos Sci 69:S316–S330

    Article  Google Scholar 

  • Worrall J (2007) Evidence in medicine. Philos Compass 2:981–1022

    Article  Google Scholar 

  • Zetin M, Hoepner CT (2007) Relevance of exclusion criteria in antidepressant clinical trials: a replication study. J Clin Psychopharmacol 27:295–301

    Article  Google Scholar 

  • Ziegler EJ, Fisher CJ Jr, Sprung CL et al (1991) Treatment of gram-negative bacteremia and septic shock with HA-1A human monoclonal antibody against endotoxin. A randomized, double-blind, placebo-controlled trial. N Engl J Med 324:429–436

    Article  Google Scholar 

  • Zimmerman M, Posternak MA, Chelminski I (2002) Symptom severity and exclusion from antidepressant efficacy trials. J Clin Psychopharmacol 22:610–614

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kellogg College, University of Oxford, Banbury Road, OX2 6PN, Oxford, UK

    Jeremy Howick

  2. Royal Institute of Technology (KTH), Teknikringen 78 B, 100 44, Stockholm, Sweden

    Alexander Mebius

  3. Nuffield Department of Primary Care Health Sciences, University of Oxford, New Radcliffe House, OX2 6GG, Oxford, UK

    Alexander Mebius

Authors
  1. Jeremy Howick
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Mebius
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeremy Howick .

Editor information

Editors and Affiliations

  1. Philosophisches Seminar, Universität Hamburg, Hamburg, Germany

    Thomas Schramme

  2. Philosophy, History and Law, Swansea University, Swansea, United Kingdom

    Steven Edwards

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media Dordrecht

About this entry

Cite this entry

Howick, J., Mebius, A. (2015). Randomized Trials and Observational Studies: The Current Philosophical Controversy. In: Schramme, T., Edwards, S. (eds) Handbook of the Philosophy of Medicine. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-8706-2_45-1

Download citation

  • DOI: https://doi.org/10.1007/978-94-017-8706-2_45-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Online ISBN: 978-94-017-8706-2

  • eBook Packages: Springer Reference Religion and PhilosophyReference Module Humanities and Social SciencesReference Module Humanities

Publish with us

Policies and ethics

Search

Navigation