Summary
The tenets of evidence-based medicine include an emphasis on hierarchies of research design (i.e., study architecture). Often, a single randomized, controlled trial is considered to provide “truth,” whereas results from any observational study are viewed with suspicion. This paper describes information that contradicts and discourages such a rigid approach to evaluating the quality of research design. Unless a more balanced strategy evolves, new claims of methodological authority may be just as problematic as the traditional claims of medical authority that have been criticized by proponents of evidence-based medicine.
Article PDF
Similar content being viewed by others
References
Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine.JAMA 268: 2420–2425, 1992.
Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users’ guide to the medical literature. IX. A method for grading health care recommendations.JAMA [Erratum 275:1232, 1996] 274: 1800–1804, 1995.
U. S. Preventive Services Task Force. Guide to clinical preventive services, Ed 2, p 862. Baltimore: U. S. Preventive Services Task Force, 1996.
Byar DP, Simon RM, Friedewald WT, Schlesselman JJ, DeMets DL, Ellengerg JH et al. Randomized clinical trials. Perspectives on some recent ideas.N Engl J Med 295: 74–80, 1976.
Feinstein AR. Current problems and future challenges in randomized clinical trials.Circulation 70: 767–774, 1984.
Abel U, Koch A. The role of randomization in clinical studies: myths and beliefs.J Clin Epidemiol 52: 487–497, 1999.
Sacks H, Chalmers TC, Smith HJ. Randomized versus historical controls for clinical trials.Am J Med 72: 233–240, 1982.
Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine: how to practice and teach EBM, p 106. London: Churchill Livingstone, 2000.
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs.N Engl J Med 342: 1887–1892, 2000.
MacMahon S, Peto R, Cutler J, Collins R, Sorlie P, Neaton J et al. Blood pressure, stroke, and coronary heart disease. Part 1, prolonged differences in blood pressure: prospective observational studies corrected for the regression dilution bias.Lancet 335: 765–774, 1990.
Colditz GA, Brewer TF, Berkey CS, Wilson ME, Burdick E, Feinberg HV et al. Efficacy of bacillus Calmette-Guerin vaccine in the prevention of tuberculosis. Meta-analysis of the published literature.JAMA 271: 698–702, 1994.
Kerlikowske K, Grady D, Rubin SM, Sandrock C, Ernster VL. Efficacy of screening mammography. A meta-analysis.JAMA 273: 149–154, 1995.
Cummings P, Psaty BM. The association between cholesterol and death from injury.Ann Intern Med 120: 848–855, 1994.
Jacobs D, Blackburn H, Higgins M, Reed D, Iso H, McMillan G et al. Report of the conference on low blood cholesterol: mortality associations.Circulation 86: 1046–1060, 1992.
Collins R, Peto R, MacMahon S, Hebert P, Fiebach NH, Eberlein KA et al. Blood pressure, stroke, and coronary heart disease. Part 2, Short-term reductions in blood pressure: overview of randomised drug trials in their epidemiological context.Lancet 335: 827–838, 1990.
Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials.N Engl J Med 342: 1878–1886, 2000.
Demissie K, Mills OF, Rhoads GG. Empirical comparison of the results of randomized, controlled trials and case-control studies in evaluating the effectiveness of screening mammography.J Clin Epidemiol 51: 81–91, 1998.
McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Methods in health services research: interpreting the evidence: choosing between randomised and non-randomised studies.BMJ 319: 312–315, 1999.
Horwitz RI, Viscoli CM, Clemens JD, Sadock RT. Developing improved observational methods for evaluating therapeutic effectiveness.Am J Med 89: 630–638, 1990.
Beta-blocker heart attack trail research group. A randomized trial of propanolol in patients with acute myocardial infarction.JAMA 247: 1707–1714, 1982.
Lipsey MW, Wilson DB. The efficacy of psychological, educational, and behavioral treatment: confirmation from meta-analysis.Am Psychol 48: 1181–1209, 1993.
Horwitz RI. Complexity and contradiction in clinical trial research.Am J Med 82: 498–510, 1987.
Bates D. Practical problems in the organisation of clinical trials in multiple sclerosis.Neuroepidemiology 6: 6–16, 1987.
McKhann GM. The trials of clinical trials.Arch Neurol 46: 611–614, 1989.
Loeb C, Gandolfo C. Methodological problems of clinical trials in multi-infarct dementia.Neuroepidemiology 9: 223–227, 1990.
Riggs JE, Hobbs GR. Clinical trials, outcomes, and statistics: how better can be worse.Neurology 51: 1234–1235, 1998.
Chadwick D, Privitera M. Placebo-controlled studies in neurology: where do they stop?Neurology 52: 682–685, 1999.
Glauser TA. Integrating clinical trial data into clinical practice.Neurology 58: S6-S12, 2002.
Horn KD. Evolving strategies in the treatment of sepsis and systemic inflammatory response syndrome (SIRS).QJM 91: 265–277, 1998.
Angus DC, Birmingham MC, Balk RA, Scannon PJ, Collins D, Kruse JA et al. E5 murine monoclonal antiendotoxin antibody in gram-negative sepsis.JAMA 283: 1723–1730, 2000.
LeLorier L, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large, randomized, controlled trials.N Engl J Med 337: 536–542, 1997.
Yusuf S, Dagenais G, Pogue J, Bosch J, Sleight P. Vitamin E supplementation and cardiovascular events in high-risk patients. The Heart Outcomes Prevention Evaluation Study Investigators.N Engl J Med 342: 154–160, 2000.
Pocock SJ, Elbourne DR. Randomized trials or observational tribulations?N Engl J Med 342: 1907–1909, 2000.
Stampfer MJ, Hennekens CH, Manson JE, Colditz GA, Rosner B, Willett WC. Vitamin E consumption and the risk of coronary disease in women.N Engl J Med 328: 1444–1449, 1993.
Rimm EB, Stampfer MJ, Ascherio A, Giovannucci E, Colditz GA, Willett WC. Vitamin E consumption and the risk of coronary disease in men.N Engl J Med 328: 1450–1456, 1993.
Hodis HN, Mack WJ, LaBree L, Cashin-Hemphill L, Sevanian A, Johnson R et al. Serial coronary angiographic evidence that anti-oxidant vitamin intake reduces progression of coronary artery atherosclerosis.JAMA 273: 1849–1854, 1995.
The Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group. The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers.N Engl J Med 330: 1029–1035, 1994.
GISSI-Prevenzione Investigators (Gruppo Italiano per lo Studio della Sopravvivenza nell’Infarto Miocardico). Dietary supplementation with n-3 polyunsaturated fatty acids and vitamin E after myocardial infarction: results of the GISSI-Prevenzione trial.Lancet 354: 447–455, 1999.
Blot WJ, Li JY, Taylor PR, Guo W, Dawsey S, Wang GQ et al. Nutrition intervention trials in Linxian, China: supplementation with specific vitamin/mineral combinations, cancer incidence, and disease-specific mortality in the general population.J Natl Cancer Inst 85: 1483–1492, 1993.
Stephens NG, Parsons A, Schofield PM, Kelly F, Cheeseman K, Michinson MJ. Randomised controlled trial of vitamin E in patients with coronary disease: Cambridge Heart Antioxidant Study.Lancet 347: 781–786, 1996.
Grodstein F, Manson JE, Colditz GA, Willett WC, Speizer FE, Stampfer MJ. A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease.Ann Intern Med 133: 933–941, 2000.
Writing Group for the Women’s Health Initiative Investigators. Risk and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial.JAMA 288: 321–333, 2002.
Hulley S, Grady D, Bush T, Furberg C, Herrington D, Riggs B et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women.JAMA 280: 605–613, 1998.
Grodstein F, Clarkson TB, Manson JE. Understanding the divergent data on postmenopausal therapy.N Engl J Med 348: 645–650, 2003.
Garbe E, Suissa S. Hormone replacement therapy and acute coronary outcomes: methodological issues between randomized and observational studies.Hum Reprod 19: 8–13, 2004.
Angell M, Kassirer JP. Alternative medicine—the risks of untested and unregulated remedies.N Engl J Med 339: 839–841, 1998.
Goodman SN, The mammography dilemma: a crisis for evidence-based medicine?Ann Intern Med 137: 363–364, 2002.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Concato, J. Observationalversus experimental studies: What’s the evidence for a hierarchy?. Neurotherapeutics 1, 341–347 (2004). https://doi.org/10.1602/neurorx.1.3.341
Issue Date:
DOI: https://doi.org/10.1602/neurorx.1.3.341