Abstract
Evidence-based medicine (EBM) is becoming the guiding principle for clinical treatment decisions. But evidence remains a loosely defined term. Multiple criteria for evidence criteria have been proposed. Most influential evidence criteria give priority to meta-analyses because they promise an objective procedure to combine the outcomes of all informative, putatively conflicting studies on the same issue in an overall score. However, we claim that meta-analyses are of limited informative value for the following six reasons: (1) meta-analyses are often “overpowered” with clinically irrelevant results that might emerge as highly significant; (2) there is serious concern of publication biases with “negative” studies not being published; (3) meta-analyses consider the variation in the results of the empirical studies included to be random noise, however, the variability of results across studies can be informative; (4) the result of a meta-analysis depends on the strategy used to identify the included empirical studies; (5) the quality of conclusions from meta-analyses depends on the statistical tests used to combine the results of the separate studies; (6) the qualitative conclusions drawn from the meta-analytical combination of individual studies may depend on specific design aspects of the individual studies. Thus, meta-analyses are primarily a method to generate hypotheses through an a posteriori analysis of treatment effects.
Similar content being viewed by others
References
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gotzsche PC, Lang T (2001) The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134:663–694
Anderson IM (1998) SSRIS versus tricyclic antidepressants in depressed inpatients: a meta-analysis of efficacy and tolerability. Depress Anxiety 7(Suppl 1):11–17
Anderson IM (2000) Selective serotonin reuptake inhibitors versus tricyclic antidepressants: a meta-analysis of efficacy and tolerability. J Affect Disord 58:19–36
Bandelow B, Zohar J, Hollander E, Kasper S, Möller HJ et al (2008) World Federation of Societies of Biological Psychiatry (WFSBP) guidelines for the pharmacological treatment of anxiety, obsessive–compulsive and post-traumatic stress disorders—first revision. World J Biol Psychiatry 9:248–312
Bandelow B, Zohar J, Kasper S, Möller HJ (2008) How to grade categories of evidence. World J Biol Psychiatry 9:242–247
Bauer M, Whybrow P, Angst J, Versiani M, Möller H-J (2004) Biologische Behandlung unipolarer depressiver Störungen. Behandlungsleitlinien der World Federation of Societies of Biological Psychiatry (WFSBP), Stuttgart, Wissenschaftliche Verlagsgesellschaft
Broich K (2009) Committee for Medicinal Products for Human Use (CHMP) assessment on efficacy of antidepressants. Eur Neuropsychopharmacol 19:305–308
Chan AW (2008) Bias, spin, and misreporting: time for full access to trial protocols and results. PLoS Med 5:e230
Cipriani A, Furukawa TA, Salanti G, Geddes JR, Higgins JPT, Churchill R, Watanabe N, Nakagawa A, Omori IM, McGuire H, Tansella M, Barbui C (2009) Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet 273:746–758
Feinstein AR (1995) Meta-analysis: statistical alchemy for the 21st century. J Clin Epidemiol 48:71–79
Fergusson D, Doucette S, Glass KC, Shapiro S, Healy D, Hebert P, Hutton B (2005) Association between suicide attempts and selective serotonin reuptake inhibitors: systematic review of randomised controlled trials. BMJ 330:396. doi:10.1136/bmj.330.7488.396
Geddes J, Freemantle N, Harrison P, Bebbington P (2000) Atypical antipsychotics in the treatment of schizophrenia: systematic overview and meta-regression analysis. BMJ 321:1371–1376
Geddes J, Harrison P, Freemantle N (2003) New generation versus conventional antipsychotics. Lancet 362:404–405
Gibbons RD, Hur K, Bhaumik DK, Mann JJ (2005) The relationship between antidepressant medication use and rate of suicide. Arch Gen Psychiatry 62:165–172
Gunnell D, Saperia J, Ashby D (2005) Selective serotonin reuptake inhibitors (SSRIs) and suicide in adults: meta-analysis of drug company data from placebo controlled, randomised controlled trials submitted to the MHRA’s safety review. BMJ 330:385. doi:10.1136/bmj.330.7488.385
Hayashino Y, Noguchi Y, Fukui T (2005) Systematic evaluation and comparison of statistical tests for publication bias. J Epidemiol 15:235–243
Ioannidis JP, Trikalinos TA (2007) The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. CMAJ 176:1091–1096
Jüni P, Altman DG, Egger M (2001) Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 323:42–46
Khan A, Kolts RL, Thase ME, Krishnan KR, Brown W (2004) Research design features and patient characteristics associated with the outcome of antidepressant clinical trials. Am J Psychiatry 161:2045–2049
Khan A, Schwartz K (2005) Study designs and outcomes in antidepressant clinical trials. Essent Psychopharmacol 6:221–226
Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT (2008) Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med 5:e45
Leucht S, Barnes TR, Kissling W, Engel RR, Correll C, Kane JM (2003) Relapse prevention in schizophrenia with new-generation antipsychotics: a systematic review and exploratory meta-analysis of randomized, controlled trials. Am J Psychiatry 160:1209–1222
Leucht S, Wahlbeck K, Hamann J, Kissling W (2003) New generation antipsychotics versus low-potency conventional antipsychotics: a systematic review and meta-analysis. Lancet 361:1581–1589
Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B (2003) Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 326:1171–1173
Möller H-J (2008) Isn’t the efficacy of antidepressants clinically relevant? A critical comment on the results of the metaanalysis by Kirsch et al. 2008. Eur Arch Psychiatry Clin Neurosci 258:451–455
Möller H-J, Maier W (2009) Evidence-based medicine in psychopharmacotherapy: possibilities, problems and limitations. Eur Arch Psychiatry Clin Neurosci [Epub ahead of print]
Moncrieff J, Kirsch I (2005) Efficacy of antidepressants in adults. BMJ 331:155–157
National Institute for Health and Clinical Excellence Depression Management of depression in primary and secondary care—NICE guidance (2009) Available at http://www.nice.org.uk/nicemedia/pdf/CG23fullguideline.pdf.Anonymous
Rising K, Bacchetti P, Bero L (2008) Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med 5:e217
Stahl SM (2002) Antipsychotic polypharmacy: evidence based or eminence based? Acta Psychiatr Scand 106:321–322
Sterne JA, Gavaghan D, Egger M (2000) Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol 53:1119–1129
Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358:252–260
Walsh BT, Seidman SN, Sysko R, Gould M (2002) Placebo response in studies of major depression: variable, substantial, and growing. JAMA 287:1840–1847
Williams DD, Garner J (2002) The case against “the evidence”: a different perspective on evidence-based medicine. Br J Psychiatry 180:8–12
Winblad B, Jones RW, Wirth Y, Stoffler A, Mobius HJ (2007) Memantine in moderate to severe Alzheimer’s disease: a meta-analysis of randomised clinical trials. Dement Geriatr Cogn Disord 24:20–27
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Maier, W., Möller, HJ. Meta-analyses: a method to maximise the evidence from clinical studies?. Eur Arch Psychiatry Clin Neurosci 260, 17–23 (2010). https://doi.org/10.1007/s00406-009-0068-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00406-009-0068-3