Planning a Meta-analysis in a Systematic Review

Abstract

This chapter provides guidance on planning a meta-analysis. The topics covered include choosing moderators for effect size models, considerations for choosing between fixed and random effects models, issues in conducting moderator models in meta-analysis such as confounding of predictors, and computing meta-regression. Examples are provided using data from a meta-analysis by Sirin (2005). The chapter’s appendix also provides SPSS and SAS program code for the analyses in the examples.

Appendix

3.1.1 Computing the Variance Component Using SAS

The example in Sect. 3.4.1 involves the computation of the variance component using restricted maximum likelihood. Below is a SAS program that computes the estimates for the analyses in Tables 3.1–3.3. The first program enters the data used in this example. The variables are the study name, the correlation, Fisher’s z corresponding to the correlation, the variance of Fisher’s z, a code that takes the value 1 for studies using state tests for achievement, and 2 for standardized tests, and the study weight. The second program provides the code to compute the variance component using restricted maximum likelihood. The first line calls Proc Mixed, with cl proving the confidence limits, and method indicating the used of restricted maximum likelihood. The model statement indicates that Fisher’s z is the outcome, and the model is a simple random effects model with no predictors. The options S and CL provide the fixed-effects parameter estimates, and the confidence interval, respectively. The class statement indicates that study is a class variable, which will be designated as the random effect later in the code. The random statement designates the random effect, and the option solution prints the estimate of the random effect, or in this case, the random effects variance. The repeated statement specifies the covariance structure of the error term. For meta-analysis, we use the option group to allow between-group (in this case, study) heterogeneity. The parms statement provides the starting values for the covariance parameters. The first value is the starting value for the overall variance component for this model. The next 11 elements are the within-study estimates of the variance of the effect size. The option eqcons fixes the variances for the 11 studies in the analysis since these are considered known in a meta-analysis model.

data sirinch6ex;
input study $ corr zval varz achmeas wt;
cards;
alspurb	.7190	.9055717	.0278	1	36.00
alsprur	.0720	.0721248	.0097	1	103.00
calban1	.6800	.8291140	.0008	1	1298.00
dixflo	.4670	.5062267	.0122	1	82.00
grelan	.6500	.7752987	.0086	1	116.00
gulbur	.1240	.1246415	.0006	2	1570.00
johnlin	.1750	.1768200	.0006	2	1683.00
kling	.5400	.6041556	.0030	2	329.00
recste	.0600	.0600722	.0077	2	130.00
schu	.4300	.4598967	.0077	2	130.00
shawal	.1660	.1675505	.0030	2	332.00
;
run;

proc mixed cl method=reml data=sirinch6ex;

class study;

model zval =/S CL;

random int/SUBJECT=study S;

repeated/GROUP=study;

Parms (0.01 to 2.00 by 0.01)

(.0278) (.0097) (.0008) (.0122)

(.0086) (.0006) (.0006) (.0030)

(.0077) (.0077) (.0030)

/EQCONS=2 to 12;

run;

3.1.2 Computing the Variance Component Using R

Below is a simple iterative program to obtain the restricted maximum likelihood estimates from R. The estimates are described by Raudenbush (2009). The first two lines set up the starting values for the random effects mean and variance, namely a vector of 100 elements, all having the value of zero. The 100 iterations take place within the brackets, “{}”. First, the value for the new variance for the vector of effect sizes is computed as adding our current value of the random effects variance, sigrmle[i] to the within-study variation, sirin$ztrans. Given this value of the vector of the within-study variances, vstar, we recomputed a new value of the random effects mean, beta0. Given this value of the random effects mean, beta0, we recalculate the value of the variance component sigrmle. We cycle back and forth for 100 iterations in this program. Below the program are the results from R using this example.

beta0<−rep(0, c(100))

sigrmle<−rep(0,c(100))

for (i in 1:100) {

vstar<−sirin$var+sigrmle[i]

beta0[i]<−sum(sirin$ztrans/vstar)/sum(1/vstar)

sigrmle[i+1]<−(sum((((sirin$ztrans-beta0[i])**2)-sirin$var)/(vstar**2))/sum(1/(vstar**2)))+(1/sum(1/vstar))

}

> beta0

[1] 0.3551265 0.4162903 0.4169039 0.4169685 0.4169757 0.4169765 0.4169766

[8] 0.4169766 0.4169766 0.4169766 0.4169766 0.4169766 0.4169766 0.4169766 ………..

> sigrmle

[1] 0.00000000 0.08408039 0.09172159 0.09258992 0.09268741 0.09269834

[7] 0.09269957 0.09269971 0.09269972 0.09269972 0.09269972 0.09269972…….

We can see that by the 8th iteration, both the variance component and the random effects mean do not change. The estimates from R correspond to those obtained using SAS.

3.1.3 Computing the Fixed Effects Meta-regression Using SPSS

The analysis in Table 3.6 can be computed from the Analyze menu in SPSS, under Regression, and then under Linear Regression. The Dependent variable will be the Fisher z-transformations of the correlation coefficients. In this example, the Independent variables are the percent minority in the sample, the dummy code indicating whether free lunch was used as the measure of socio-economic status, the dummy code indicating whether parent education level was used as the measure of socio-economic status, and grade level. The WLS Weight for the analysis is the inverse of the Fisher’s z-transformation variance, or 1/(n – 3).

3.1.4 Computing the Fixed Effects Meta-regression Using SAS

To compute the fixed effects meta-regression in SAS, we use Proc Reg as illustrated in the code given below. The model statement begins with the outcome, the Fisher z-transformation, an equals sign, and the list of predictors. An option to the model statement is given by the slash /and followed by I which indicates that we want the inverse of the crossproducts matrix. The inverse of the crossproducts matrix will provide the correct variances for the regression coefficients. The weight statement indicates that the variable wt should be used in the weighted least squares regression.

proc reg data=Chap. 3;

model ztrans=grade percmin freelunch educ/I;

weight wt;

print;

run;

The output provided by SAS is first the inverse of the crossproducts matrix:

X′X Inverse, Parameter Estimates, and SSE

	Intercept	Grade	% minority	Free lunch	Parent’s education
Intercept	0.000218	0.0000594	−4.746E-7	−0.0000666	−0.0000709
Grade	−0.0000594	0.0000198	3.834E-8	0.0000126	0.0000143
% minority	−4.745E-7	3.834E-8	9.403E-9	7.456E-8	5.824E-8
Free lunch	−0.0000666	0.0000126	7.456E-8	0.000154	0.0000362
Parent’s education	−0.0000709	0.0000143	5.824E-8	0.0000362	0.000117

The diagonal elements of this matrix are the correct variances for the weighted regression coefficients. For example, the correct standard error for Parent’s education level is √0.000117 = 0.0108, which corresponds to that computed using the adjusted SPSS values in Table 3.6. Next, SAS provides the ANOVA table for the regression and the R-squared values.

Analysis of Variance

Source	DF	Sum of squares	Mean square	F value	Pr > F
Model	4	1081.426	270.356	8.98	<.0001
Error	35	1053.852	30.110
Corrected total	39	2135.278

Root MSE	5.487	R-Square	0.5065
Dependent mean	0.272	Adj R-Sq	0.4501
Coeff var	2014.675

As in the example with SPSS, we will use the square root of the mean square error given in the output above to correct the standard errors and the significance tests of the regression coefficients given below.

Variable	DF	Parameter estimate	Standard error	t-value	Pr >
Intercept	1	0.0408	0.0811	0.50	0.6176
Grade	1	0.0729	0.0244	2.98	0.0052
% minority	1	−0.000999	0.00053	−1.88	0.0686
Free lunch	1	0.353	0.0681	5.19	<.0001
Parent’s education	1	0.104	0.0594	1.75	0.0885