European Journal of Clinical Pharmacology

, Volume 71, Issue 6, pp 645–647 | Cite as

Channeling the flood of meta-analyses

  • Uwe FuhrEmail author
  • Martin Hellmich

A meta-analysis is a potentially powerful tool to summarize results from separate, however, closely related research projects. Meta-analyses are conducted at negligible costs and efforts as compared to those required for the acquisition of new raw data, e.g., by clinical trials. As part of systematic assessment of drug effects, meta-analyses form the basis for evidence-based clinical decision making and, moreover, are also used by policy makers, e.g., to determine reimbursement. These benefits have increasingly been recognized in recent years.

The Cochrane Collaboration founded in 1993 pioneered in and still promotes the preparation and maintenance of high-quality systematic reviews including meta-analysis. It also supports the development of software to conduct meta-analyses. Their Review Manager (RevMan, current version: 5.3.5) is an excellent, comprehensive, simple to use program which is distributed free of charge to academic researchers (see,, accessed March 8, 2015, and linked sites of the Cochrane Collaboration). A structured approach addressing most pitfalls is supported by concerted initiatives such as PRISMA for meta-analysis of clinical trials [2] and MOOSE for meta-analysis of observational studies [4]. With these resources, doing a meta-analysis has become a technically simple, straightforward procedure. It may therefore falsely appear that no specific in-depth expertise is required to conduct a meta-analysis, neither medical nor statistical nor pharmacological. Because of the need for information for decision-making and the excellent reception of meta-analyses, becoming a “meta-analyst” may seem as an intriguing idea to easily acquire many impact points and reputation and make a splendid career, with the excellent cost to benefit ratio of meta-analysis extending to the “meta-analyst” as an individual scientist—or does it?

Some researchers and some science policy makers certainly would agree, although in 2014 published meta-analyses accounted for not more than about 1.3 % of all publications in PubMed (, accessed March 8, 2015). For several years, editors of many journals including the European Journal of Clinical Pharmacology are virtually flooded with meta-analyses. This phenomenon is not limited to the area of clinical pharmacology [1]. The annual exponential increase of published meta-analyses calculated from 1990 to 2014 is pretty constant (Fig. 1). It reaches 1.17-fold (8 %) and considerably exceeds the 1.04-fold (2 %) increase seen for all publications (given as geometric mean and coefficient of variation). From our perception, the number of submitted manuscripts reporting a meta-analysis is growing even faster. The technical/statistical quality of submitted meta-analyses is typically high and shows proper use of the tools available. Does clinical pharmacology or, more generally, biomedical science benefit from this phenomenon?
Fig. 1

PubMed publication counts of meta-analyses from 1980–2014

From a scientific point of view, the appropriate starting point for a meta-analysis is a specific research question which has been addressed by a variety of individual research projects, however, which has not been answered with sufficient precision by any of these, say, because individual projects were not large enough. Existing information however typically should be sufficient to generate a specific hypothesis, also taking clinical relevance into account where applicable. For instance, to demonstrate that treatment A is as effective in lowering blood pressure as treatment B, “as effective as” needs to be well defined beforehand, e.g., as “with 95 % confidence, difference in systolic blood pressure reduction is not larger than 5 mmHg.” If a meta-analysis is part of a data-mining process in absence of a guiding hypothesis, this should be clearly described and justified.

Which studies can be considered as addressing a similar research question? Would it be sensible to summarize the effects of any “antihypertensive therapy” in a meta-analysis? Or the effects of any beta-blocker? Of those of the beta-blocker metoprolol at any dose? The answer would be different for different response criteria, such as blood pressure-lowering effect or emergence of hypoglycemia in diabetics. Provided the selection criteria of individual studies to be included in a meta-analysis are covered, the detailed rationale for this selection should also be integral part of an eventually published meta-analysis.

How many studies are required to make conduction of a meta-analysis sensible? If two studies are almost identical with regard to study design and covariates, a number of two might be sufficient. Like for studies in individual patients, increasing variability between studies results in a larger number required to test the hypothesis. It remains to be addressed whether an average effect of a specific drug treatment calculated from a large number of heterogeneous studies conducted on different populations is of any clinical use, because at the end of the day, the average result cannot be applied to a patient belonging to one of these populations. Thus, once all studies fulfilling the selection criteria have been identified, the next step to be addressed and to be described in a manuscript is whether observed heterogeneity is appropriate for the conduction of the intended meta-analysis.

An appropriate meta-analysis is original research because it uses specific methods and provides novel information, and the process of its generation is as vigorous as experimental research. Accordingly, the process should be described prior to its start in a respective protocol as recently recommended in detail [3]. Such protocols should be registered “in order to promote and maintain transparency in the systematic review process, to assist in minimizing the risk of bias(es), and help to reduce unnecessary duplication of reviews” [3]. Proper reporting of meta-analysis of clinical trials is covered by the broadly accepted recommendations [2, 4]. For meta-analysis on pharmacological interventions, additional key information of individual research projects includes exposure towards the drug(s) to be examined, i.e., substance, dose, route of administration, and duration of treatment. Potential covariates such as demographic characteristics, disease state, co-medication, etc. of individual research projects included in the analysis should be described in sufficient detail.

Many of the manuscripts on meta-analyses submitted to the European Journal of Clinical Pharmacology appear to reveal a different procedure and seem to be driven primarily not by scientific but rather by publication motivations. To this end, the starting point is to identify any area where a new meta-analysis could be accepted for publication. Identifying respective gaps of knowledge is very challenging and highly competitive task, especially without detailed knowledge in the respective medical area, because most evidence which can reasonably be summarized was already meta-analyzed. The result often is without benefit for the scientific community. Undesired but frequent attempts include but are not limited to the following:
  • Lack of proper description of the research question and of implications of the results.

  • Separate manuscripts for each adverse event or for each efficacy endpoint of a given drug therapy; occasionally, such manuscripts are prepared simultaneously and submitted in parallel to different journals in order to conceal this approach.

  • Meta-analyses summarizing remotely related studies.

  • Meta-analyses summarizing an inappropriately small number of studies.

  • Repetition of recently published meta-analyses with only a minor amount of information added.

The European Journal of Clinical Pharmacology continues to welcome the submission of manuscripts reporting high-quality meta-analyses irrespective of geographic origin. We recently adopted our “instructions of authors” to clarify our expectations. As outlined above, beyond being technically/statistically appropriate, scientific and/or clinical relevance is the second major indispensable requirement.


  1. 1.
    Ioannidis JP, Chang CQ, Lam TK, Schully SD, Khoury MJ (2013) The geometric increase in meta-analyses from China in the genomic era. ssPLoS One 8(6):e65602CrossRefGoogle Scholar
  2. 2.
    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7):e1000100CrossRefPubMedCentralPubMedGoogle Scholar
  3. 3.
    Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, PRISMA-P Group (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 349:g7647CrossRefPubMedGoogle Scholar
  4. 4.
    Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group. JAMA 283(15):2008–2012CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of Pharmacology, Clinical Pharmacology UnitHospital of the University of CologneKölnGermany
  2. 2.Institute of Medical Statistics, Informatics and EpidemiologyUniversity of CologneKölnGermany

Personalised recommendations