Meta-analysis I

Abstract

Meta-analysis is now used in a wide range of disciplines, in particular epidemiology and evidence-based medicine, where the results of some meta-analyses have led to major changes in clinical practice and health care policies. Meta-analysis is applicable to collections of research that produce quantitative results, examine the same constructs and relationships, and have findings that can be configured in a comparable statistical form called an effect size (e.g. correlation coefficients, odds ratios, proportions, etc.), that is, are comparable given the question at hand. These results from several studies that address a set of related research hypotheses are then quantitatively combined using statistical methods. This chapter provides an in-depth discussion of the various statistical methods currently available, with a focus on bias adjustment in meta-analysis.

Appendices

Appendix 1: Need for an Overdispersion Correction

In a study with overdispersed data, the mean or expectation structure (θ) is adequate but the variance structure [σ ²(θ)] is inadequate. Individuals in the study can have the outcome with some degree of dependence on study-specific parameters unrelated to the intervention. If such data are analysed as if the outcomes were independent, then sampling variances tend to be too small, giving a false sense of precision. One approach is to think of the true variance structure as following the form [ϕ(θ)σ ²(θ)]; however, it is complex to fit such a form. As a simpler approach, we suppose ϕ(θ) = c, so that the true variance structure [cσ ²(θ)] is some constant multiplier of the theoretical variance structure. A common method of estimating c suggested used by Lindsey (1999) or Tjur (1998) is to use the observed chi-squared goodness of fit statistic for the pooled studies divided by its degrees of freedom:

$$ c={\chi^2}/\mathrm{df} $$

If there is no overdispersion or lack of fit, c = 1 (because the expected value of the chi-squared statistic is equal to its degrees of freedom) and if there is, then c > 1. In a meta-analysis, this goodness of fit chi-squared divided by its df is equal to H ² as defined by Higgins and Thompson (2002).

The problem of using the overdispersion parameter as a constant multiplier of the variances of each study in the meta-analysis presupposes that, for a constant increase in this parameter, there is a constant increase in variance. This means that the impact of the parameter is not capped and a point is eventually reached where there is overinflation of the variances for a given level of overdispersion resulting in overcorrection and confidence intervals that are too wide. In order to reduce the impact of large values of H ², we can transform H ² to its reciprocal and use this to proportionally inflate the variances. Higgins and Thompson (2002) also defined an I ² parameter, which is an index of dispersion that is restricted between 0 (no dispersion) and 1. If we reverse the I ² scale (by subtracting it from 1) so that no dispersion (only sampling error) is now 1 as opposed to 0, then (1 − I ²) is indeed the reciprocal of H ². We thus used (1 − I ²) as an exponent to proportionally inflate study variances < 1. For variance > 1, we used 2 minus this overdispersion parameter (which reduces to [I ² + 1]) as the inflation factor. Additional rescaling was done by scaling (1 − I ²) to various roots and using the simulation described above to see the impact on coverage of the confidence interval. The fourth root was found to result in an acceptable simulated coverage of the confidence interval around 95 %. We thus used [(1 − I ²)^1/4] as the final overdispersion correction factor. This is also equivalent to (1/H ²)^1/4. This correction was then used to inflate the variances of individual studies resulting in a more conservative meta-analysis pooled variance. Even if the accuracy of this approximation is questionable, common sense suggests that it is better to perform this correction, implicitly making the (more or less incorrect) assumption that the distribution of c is approximated well enough by a χ ² distribution with k − 1 degrees of freedom than not to perform any correction at all, implicitly making the (certainly incorrect) assumption that there is no overdispersion in the data (Tjur 1998). This adjustment in the QE model corrects for overdispersion within studies that affect the precision of the pooled estimate, not for heterogeneity between studies that affect the estimate itself.

Appendix 2: Quality Scores and Population Impact Scores

For a QE type of meta-analysis, a reproducible and effective scheme of quality assessment is required. However, any quality score can be used with the method and thus we are not constrained to any one method. There are many different quality assessment instruments and most have parameters that allow us to assess the likelihood for bias. Although the importance of such quality assessment of experimental studies is well established, quality assessment of other study designs in systematic reviews is far less well developed. The feasibility of creating one quality checklist to apply to various study designs has been explored by Downs and Black (1998), and research has gone into developing instruments to measure the methodological quality of observational studies in meta-analyses (see Chap. 13). Nevertheless, there is as yet no consensus on how to synthesize information about quality from a range of study designs within a systematic review, although many quality assessment schemes exist. Concato (2004) suggests that a more balanced view of observational and experimental evidence is necessary. The way Q _i is computed from the score for each study and the additional use of population weights (for burden of disease or type C studies) is depicted in Table 14.1. The population weights are applied as a method of standardization of the group pooled estimates where there is a single estimate per group. The population weighted analysis does not use inverse variance weighting and if a rate is being pooled would give an equivalent result to direct standardization used in epidemiology. Rates have a problematic variance but can be based on a normal approximation to the Poisson distribution:

$$ \mathrm{Va}{\mathrm{{r}}_\mathrm{{r}\mathrm{ate}}}={\it O}\times {{\left( {\frac{{\it K}}{{\it P}}} \right)}^2} $$

where O are the observed events, P is the person-time of observation and K is a constant multiplier. In the computation, zero rates can be imputed to have variances based on a single observed event as a continuity correction.

Table 14.1 Hypothetical calculation of Q _i for use in QE meta-analyses^a