Meta-analyses: a method to maximise the evidence from clinical studies?

Special Issue


Evidence-based medicine (EBM) is becoming the guiding principle for clinical treatment decisions. But evidence remains a loosely defined term. Multiple criteria for evidence criteria have been proposed. Most influential evidence criteria give priority to meta-analyses because they promise an objective procedure to combine the outcomes of all informative, putatively conflicting studies on the same issue in an overall score. However, we claim that meta-analyses are of limited informative value for the following six reasons: (1) meta-analyses are often “overpowered” with clinically irrelevant results that might emerge as highly significant; (2) there is serious concern of publication biases with “negative” studies not being published; (3) meta-analyses consider the variation in the results of the empirical studies included to be random noise, however, the variability of results across studies can be informative; (4) the result of a meta-analysis depends on the strategy used to identify the included empirical studies; (5) the quality of conclusions from meta-analyses depends on the statistical tests used to combine the results of the separate studies; (6) the qualitative conclusions drawn from the meta-analytical combination of individual studies may depend on specific design aspects of the individual studies. Thus, meta-analyses are primarily a method to generate hypotheses through an a posteriori analysis of treatment effects.


Meta-analyses Evidence-based medicine Clinical studies 


  1. 1.
    Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gotzsche PC, Lang T (2001) The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134:663–694PubMedGoogle Scholar
  2. 2.
    Anderson IM (1998) SSRIS versus tricyclic antidepressants in depressed inpatients: a meta-analysis of efficacy and tolerability. Depress Anxiety 7(Suppl 1):11–17CrossRefPubMedGoogle Scholar
  3. 3.
    Anderson IM (2000) Selective serotonin reuptake inhibitors versus tricyclic antidepressants: a meta-analysis of efficacy and tolerability. J Affect Disord 58:19–36CrossRefPubMedGoogle Scholar
  4. 4.
    Bandelow B, Zohar J, Hollander E, Kasper S, Möller HJ et al (2008) World Federation of Societies of Biological Psychiatry (WFSBP) guidelines for the pharmacological treatment of anxiety, obsessive–compulsive and post-traumatic stress disorders—first revision. World J Biol Psychiatry 9:248–312CrossRefPubMedGoogle Scholar
  5. 5.
    Bandelow B, Zohar J, Kasper S, Möller HJ (2008) How to grade categories of evidence. World J Biol Psychiatry 9:242–247CrossRefPubMedGoogle Scholar
  6. 6.
    Bauer M, Whybrow P, Angst J, Versiani M, Möller H-J (2004) Biologische Behandlung unipolarer depressiver Störungen. Behandlungsleitlinien der World Federation of Societies of Biological Psychiatry (WFSBP), Stuttgart, Wissenschaftliche VerlagsgesellschaftGoogle Scholar
  7. 7.
    Broich K (2009) Committee for Medicinal Products for Human Use (CHMP) assessment on efficacy of antidepressants. Eur Neuropsychopharmacol 19:305–308CrossRefPubMedGoogle Scholar
  8. 8.
    Chan AW (2008) Bias, spin, and misreporting: time for full access to trial protocols and results. PLoS Med 5:e230CrossRefPubMedGoogle Scholar
  9. 9.
    Cipriani A, Furukawa TA, Salanti G, Geddes JR, Higgins JPT, Churchill R, Watanabe N, Nakagawa A, Omori IM, McGuire H, Tansella M, Barbui C (2009) Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet 273:746–758Google Scholar
  10. 10.
    Feinstein AR (1995) Meta-analysis: statistical alchemy for the 21st century. J Clin Epidemiol 48:71–79 CrossRefPubMedGoogle Scholar
  11. 11.
    Fergusson D, Doucette S, Glass KC, Shapiro S, Healy D, Hebert P, Hutton B (2005) Association between suicide attempts and selective serotonin reuptake inhibitors: systematic review of randomised controlled trials. BMJ 330:396. doi:10.1136/bmj.330.7488.396 CrossRefPubMedGoogle Scholar
  12. 12.
    Geddes J, Freemantle N, Harrison P, Bebbington P (2000) Atypical antipsychotics in the treatment of schizophrenia: systematic overview and meta-regression analysis. BMJ 321:1371–1376CrossRefPubMedGoogle Scholar
  13. 13.
    Geddes J, Harrison P, Freemantle N (2003) New generation versus conventional antipsychotics. Lancet 362:404–405CrossRefPubMedGoogle Scholar
  14. 14.
    Gibbons RD, Hur K, Bhaumik DK, Mann JJ (2005) The relationship between antidepressant medication use and rate of suicide. Arch Gen Psychiatry 62:165–172CrossRefPubMedGoogle Scholar
  15. 15.
    Gunnell D, Saperia J, Ashby D (2005) Selective serotonin reuptake inhibitors (SSRIs) and suicide in adults: meta-analysis of drug company data from placebo controlled, randomised controlled trials submitted to the MHRA’s safety review. BMJ 330:385. doi:10.1136/bmj.330.7488.385 CrossRefPubMedGoogle Scholar
  16. 16.
    Hayashino Y, Noguchi Y, Fukui T (2005) Systematic evaluation and comparison of statistical tests for publication bias. J Epidemiol 15:235–243CrossRefPubMedGoogle Scholar
  17. 17.
    Ioannidis JP, Trikalinos TA (2007) The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. CMAJ 176:1091–1096PubMedGoogle Scholar
  18. 18.
    Jüni P, Altman DG, Egger M (2001) Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 323:42–46CrossRefPubMedGoogle Scholar
  19. 19.
    Khan A, Kolts RL, Thase ME, Krishnan KR, Brown W (2004) Research design features and patient characteristics associated with the outcome of antidepressant clinical trials. Am J Psychiatry 161:2045–2049CrossRefPubMedGoogle Scholar
  20. 20.
    Khan A, Schwartz K (2005) Study designs and outcomes in antidepressant clinical trials. Essent Psychopharmacol 6:221–226PubMedGoogle Scholar
  21. 21.
    Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT (2008) Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med 5:e45CrossRefPubMedGoogle Scholar
  22. 22.
    Leucht S, Barnes TR, Kissling W, Engel RR, Correll C, Kane JM (2003) Relapse prevention in schizophrenia with new-generation antipsychotics: a systematic review and exploratory meta-analysis of randomized, controlled trials. Am J Psychiatry 160:1209–1222CrossRefPubMedGoogle Scholar
  23. 23.
    Leucht S, Wahlbeck K, Hamann J, Kissling W (2003) New generation antipsychotics versus low-potency conventional antipsychotics: a systematic review and meta-analysis. Lancet 361:1581–1589CrossRefPubMedGoogle Scholar
  24. 24.
    Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B (2003) Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 326:1171–1173CrossRefPubMedGoogle Scholar
  25. 25.
    Möller H-J (2008) Isn’t the efficacy of antidepressants clinically relevant? A critical comment on the results of the metaanalysis by Kirsch et al. 2008. Eur Arch Psychiatry Clin Neurosci 258:451–455CrossRefPubMedGoogle Scholar
  26. 26.
    Möller H-J, Maier W (2009) Evidence-based medicine in psychopharmacotherapy: possibilities, problems and limitations. Eur Arch Psychiatry Clin Neurosci [Epub ahead of print]Google Scholar
  27. 27.
    Moncrieff J, Kirsch I (2005) Efficacy of antidepressants in adults. BMJ 331:155–157CrossRefPubMedGoogle Scholar
  28. 28.
    National Institute for Health and Clinical Excellence Depression Management of depression in primary and secondary care—NICE guidance (2009) Available at
  29. 29.
    Rising K, Bacchetti P, Bero L (2008) Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med 5:e217CrossRefPubMedGoogle Scholar
  30. 30.
    Stahl SM (2002) Antipsychotic polypharmacy: evidence based or eminence based? Acta Psychiatr Scand 106:321–322CrossRefPubMedGoogle Scholar
  31. 31.
    Sterne JA, Gavaghan D, Egger M (2000) Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol 53:1119–1129CrossRefPubMedGoogle Scholar
  32. 32.
    Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358:252–260CrossRefPubMedGoogle Scholar
  33. 33.
    Walsh BT, Seidman SN, Sysko R, Gould M (2002) Placebo response in studies of major depression: variable, substantial, and growing. JAMA 287:1840–1847CrossRefPubMedGoogle Scholar
  34. 34.
    Williams DD, Garner J (2002) The case against “the evidence”: a different perspective on evidence-based medicine. Br J Psychiatry 180:8–12CrossRefPubMedGoogle Scholar
  35. 35.
    Winblad B, Jones RW, Wirth Y, Stoffler A, Mobius HJ (2007) Memantine in moderate to severe Alzheimer’s disease: a meta-analysis of randomised clinical trials. Dement Geriatr Cogn Disord 24:20–27CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.Department of Psychiatry and PsychotherapyRheinische Friedrich-Wilhelms-UniversityBonnGermany
  2. 2.Department of Psychiatry and PsychotherapyLudwig-Maximilians-UniversityMunichGermany

Personalised recommendations