Channeling the flood of meta-analyses
- 705 Downloads
A meta-analysis is a potentially powerful tool to summarize results from separate, however, closely related research projects. Meta-analyses are conducted at negligible costs and efforts as compared to those required for the acquisition of new raw data, e.g., by clinical trials. As part of systematic assessment of drug effects, meta-analyses form the basis for evidence-based clinical decision making and, moreover, are also used by policy makers, e.g., to determine reimbursement. These benefits have increasingly been recognized in recent years.
The Cochrane Collaboration founded in 1993 pioneered in and still promotes the preparation and maintenance of high-quality systematic reviews including meta-analysis. It also supports the development of software to conduct meta-analyses. Their Review Manager (RevMan, current version: 5.3.5) is an excellent, comprehensive, simple to use program which is distributed free of charge to academic researchers (see http://www.cochrane.org/, http://tech.cochrane.org/revman, accessed March 8, 2015, and linked sites of the Cochrane Collaboration). A structured approach addressing most pitfalls is supported by concerted initiatives such as PRISMA for meta-analysis of clinical trials  and MOOSE for meta-analysis of observational studies . With these resources, doing a meta-analysis has become a technically simple, straightforward procedure. It may therefore falsely appear that no specific in-depth expertise is required to conduct a meta-analysis, neither medical nor statistical nor pharmacological. Because of the need for information for decision-making and the excellent reception of meta-analyses, becoming a “meta-analyst” may seem as an intriguing idea to easily acquire many impact points and reputation and make a splendid career, with the excellent cost to benefit ratio of meta-analysis extending to the “meta-analyst” as an individual scientist—or does it?
From a scientific point of view, the appropriate starting point for a meta-analysis is a specific research question which has been addressed by a variety of individual research projects, however, which has not been answered with sufficient precision by any of these, say, because individual projects were not large enough. Existing information however typically should be sufficient to generate a specific hypothesis, also taking clinical relevance into account where applicable. For instance, to demonstrate that treatment A is as effective in lowering blood pressure as treatment B, “as effective as” needs to be well defined beforehand, e.g., as “with 95 % confidence, difference in systolic blood pressure reduction is not larger than 5 mmHg.” If a meta-analysis is part of a data-mining process in absence of a guiding hypothesis, this should be clearly described and justified.
Which studies can be considered as addressing a similar research question? Would it be sensible to summarize the effects of any “antihypertensive therapy” in a meta-analysis? Or the effects of any beta-blocker? Of those of the beta-blocker metoprolol at any dose? The answer would be different for different response criteria, such as blood pressure-lowering effect or emergence of hypoglycemia in diabetics. Provided the selection criteria of individual studies to be included in a meta-analysis are covered, the detailed rationale for this selection should also be integral part of an eventually published meta-analysis.
How many studies are required to make conduction of a meta-analysis sensible? If two studies are almost identical with regard to study design and covariates, a number of two might be sufficient. Like for studies in individual patients, increasing variability between studies results in a larger number required to test the hypothesis. It remains to be addressed whether an average effect of a specific drug treatment calculated from a large number of heterogeneous studies conducted on different populations is of any clinical use, because at the end of the day, the average result cannot be applied to a patient belonging to one of these populations. Thus, once all studies fulfilling the selection criteria have been identified, the next step to be addressed and to be described in a manuscript is whether observed heterogeneity is appropriate for the conduction of the intended meta-analysis.
An appropriate meta-analysis is original research because it uses specific methods and provides novel information, and the process of its generation is as vigorous as experimental research. Accordingly, the process should be described prior to its start in a respective protocol as recently recommended in detail . Such protocols should be registered “in order to promote and maintain transparency in the systematic review process, to assist in minimizing the risk of bias(es), and help to reduce unnecessary duplication of reviews” . Proper reporting of meta-analysis of clinical trials is covered by the broadly accepted recommendations [2, 4]. For meta-analysis on pharmacological interventions, additional key information of individual research projects includes exposure towards the drug(s) to be examined, i.e., substance, dose, route of administration, and duration of treatment. Potential covariates such as demographic characteristics, disease state, co-medication, etc. of individual research projects included in the analysis should be described in sufficient detail.
Lack of proper description of the research question and of implications of the results.
Separate manuscripts for each adverse event or for each efficacy endpoint of a given drug therapy; occasionally, such manuscripts are prepared simultaneously and submitted in parallel to different journals in order to conceal this approach.
Meta-analyses summarizing remotely related studies.
Meta-analyses summarizing an inappropriately small number of studies.
Repetition of recently published meta-analyses with only a minor amount of information added.
The European Journal of Clinical Pharmacology continues to welcome the submission of manuscripts reporting high-quality meta-analyses irrespective of geographic origin. We recently adopted our “instructions of authors” to clarify our expectations. As outlined above, beyond being technically/statistically appropriate, scientific and/or clinical relevance is the second major indispensable requirement.
- 2.Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7):e1000100CrossRefPubMedCentralPubMedGoogle Scholar
- 4.Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group. JAMA 283(15):2008–2012CrossRefPubMedGoogle Scholar