Power of Statistical Tests for Subgroup Analysis in Meta-Analysis

Abstract

Meta-analysis is used extensively in health and medicine to examine the average effect of a treatment and the variation among studies estimating this effect. However, the estimate of the average treatment effect is not the only concern in a systematic review. Researchers also need to understand how the treatment effect may vary. The potential challenges for subgroup analyses in meta-analysis parallel those in randomized trials where many have urged caution in the conduct and interpretation of subgroup analyses. This chapter will discuss the power of statistical tests for subgroup analysis in order to help in both the planning and interpretation of subgroup tests in a meta-analysis. The chapter will begin with an overview of subgroup analyses in a meta-analysis, reviewing recent research on the interpretation of these results. The chapter will then discuss the a priori power of subgroup analyses under the fixed and the random effects model and provide examples of power computations. The chapter will also present a general overview of power in meta-regression with directions for future research on power analysis for examining subgroup effects.

Appendices

R Function pchisq to Compute Power

In R, the function pchisq gives the distribution function for the chi-square distribution, the area either above or below a particular value. To compute the power for the examples of the chapter, use the following code:

$$ 1\hbox{--} \mathrm{pchisq}\left(x, df, ncp=y,\mathrm{lower}.\mathrm{tail}=\mathrm{TRUE},\log .\mathrm{p}=\mathrm{FALSE}\right) $$

The value x is the critical value for the central chi-square for a test at the α level of significance with df degrees of freedom (the number of groups in the test – 1). The value y in ncp = y is the value of the non-centrality parameter for the alternative hypothesis being tested. The argument lower.tail = TRUE gives the area in the non-central chi-square distribution that is less than the value of x, or the lower tail of the distribution. By default, the function returns the result in log units; setting the argument log.p = FALSE provides the value needed for computing power.

To obtain the critical value of the central chi-square distribution at the α level of significance, use the following code:

$$ \mathrm{qchisq}\left(p, df, ncp= 0,\mathrm{lower}.\mathrm{tail}=\mathrm{TRUE},\log .\mathrm{p}=\mathrm{FALSE}\right) $$

The value p is 1 − α, and df are the degrees of freedom for the test (the number of groups in the test – 1). The expression ncp = 0 indicates that we are interested in the central chi-square distribution.

R Code for Fig. 17.1, Power for Subgroup Differences with the Standardized Mean Difference

# Power analysis for fixed effects subgroup analysis of two groups - SMD

library(ggplot2)

# Create a dataframe with N = 15 values for power analysis

# subdiff is a range of subgroup mean differences

# m is the common number of studies in both groups

# ssize is the within-study sample size

# vi is the common within-study variance for the effect size

N < −15

powerparms1 < − data.frame(subdiff = numeric(N), ssize = (N), vi = numeric (N))

# add values for the power parameters

# seq is a function that creates a sequence of numbers for the subgroup differences

# rep is a function that repeats a value for the common within-study sample size

powerparms1$subdiff <− c(seq(from = 0.05, to = 0.75, by = 0.05))

powerparms1$ssize <− c(rep(20, times = 15))

# use effsize and ssize to create vi for standardized mean difference

# I use 1/2 of the subgroup difference of interest for the overall mean effect size

powerparms1$vi < − ((powerparms1$ssize)/((0.5*powerparms1$ssize) ^ 2)) +

(((0.5*powerparms1$subdiff) ^ 2)/ (2*powerparms1$ssize))

# copy the dataframe 3 times to create parameters for 3 different

# scenarios for number of studies within groups m = 5, 10, 15

powerp1 < − rbind(powerparms1, powerparms1, powerparms1)

# create values for equal m within groups of 5, 10, 15

powerp1$m < − c(rep(5, times = 15), rep(10, times = 15), rep(15, times = 15))

powerp1$mlab <− c(rep(“3:m = 5”, times = 15), rep(“2:m = 10”, times = 15),

rep(“1:m = 15”, times = 15))

# get the sums of the weights for values of the vi and m

powerp1$w1sum < − powerp1$m*(1/powerp1$vi)

powerp1$w2sum < − powerp1$m*(1/powerp1$vi)

# get lambda the non-centrality parameter

powerp1$lambda <− (powerp1$w1sum*((0.5*powerp1$subdiff) ^ 2))+

(powerp1$w2sum*((0.5*powerp1$subdiff) ^ 2))

#get power

powerp1$power < − 1 - pchisq(3.84, 1, ncp = powerp1$lambda, lower.tail = T,

log.p = F)

# plot the power curves

ggplot(powerp1, aes(x = powerp1$subdiff, y = powerp1$power, group = powerp1$mlab)) +

geom_line() +

geom_point(aes(shape = powerp1$mlab)) +

ggtitle(“Power for subgroup differences with SMD”) +

labs(y = “Power”, x = “Subgroup difference”) +

labs(shape = “Subgroup size”)

R Code for Fig. 17.2, Power for Subgroup Differences with Log-Odds Ratio and Varying Degrees of Heterogeneity

# Power analysis for random effects subgroup analysis of two groups - LOR

library(ggplot2)

# Create a dataframe with values for power analysis

# subdiff is a range of subgroup mean differences

# m is the common number of studies in both groups

# ssize is the within-study sample size

# vi is the common within-study variance for the effect size

N < -15

powerparms2 < − data.frame(subdiff = numeric(N), m = numeric(N), ssize = numeric(N), vi = numeric(N))

# add values for the power parameters

# seq is a function that creates a sequence of numbers for the subgroup differences

# rep is a function that repeats a value

powerparms2$subdiff <− c(seq(from = 0.05, to = 2.85, by = 0.20))

powerparms2$ssize <− c(rep(20, times = 15))

powerparms2$m < − c(rep(10, times = 15))

# use an LOR = 1 for overall effect size to compute vi

# ssize/4 for each cell to obtain common estimate of vi

cellcounts <− powerparms2$ssize/4

powerparms2$vi < − 4*(1/cellcounts)

# copy the dataframe 3 times to create parameters for 3 different

# scenarios for heterogeneity: low, medium, high

powerp2 < − rbind(powerparms2, powerparms2, powerparms2)

# create factor for each value of heterogeneity

powerp2$levelh <− c(rep(“1:low”, times = 15), rep(“2:medium”, times = 15),

rep(“3:high”, times = 15))

# create values for tau for the three different levels of heterogeneity

powerp2$tausq[1:15] < − powerp2$vi[1:15]/3

powerp2$tausq[16:30] < − powerp2$vi[16:30]

powerp2$tausq[31:45] < − powerp2$vi[31:45]*3

# get the sums of the weights for values of the vi and m

powerp2$w1sum < − powerp2$m*(1/(powerp2$vi + powerp2$tau))

powerp2$w2sum < − powerp2$m*(1/(powerp2$vi + powerp2$tau))

# get lambda the non-centrality parameter

powerp2$lambda <− (powerp2$w1sum*((0.5*powerp2$subdiff) ^ 2))+

(powerp2$w2sum*((0.5*powerp2$subdiff) ^ 2))

# get power

powerp2$power < − 1 - pchisq(3.84, 1, ncp = powerp2$lambda, lower.tail = T,

log.p = F)

# plot the power curves

ggplot(powerp2, aes(x = powerp2$subdiff, y = powerp2$power, group = powerp2$levelh)) +

geom_line() +

geom_point(aes(shape = powerp2$levelh)) +

ggtitle(“Power for subgroup differences with LOR”) +

labs(y = “Power”, x = “Subgroup difference”) +

labs(shape = “Heterogeneity”)

R Code for Fig. 17.3, Power for the Log-Odds Ratio with Varying Numbers of Studies Within Groups

# Power analysis for random effects subgroup analysis of two groups - LOR

# Moderate heterogeneity and unbalanced groups

library(ggplot2)

# Create a dataframe with values for power analysis

# subdiff is a range of subgroup mean differences

# m is the common number of studies in both groups

# ssize is the within-study sample size

# vi is the common within-study variance for the effect size

N < -15

powerparms3 < − data.frame(subdiff = numeric(N), ssize = numeric(N), vi = numeric(N))

# add values for the power parameters

# seq is a function that creates a sequence of numbers

# rep is a function that repeats a value

powerparms3$subdiff <− c(seq(from = 0.05, to = 2.85, by = 0.20))

powerparms3$ssize <− c(rep(20, times = 15))

# use an LOR = 1 for overall effect size to compute vi

# ssize/4 for each cell to obtain common estimate of vi

cellcounts <− powerparms3$ssize/4

powerparms3$vi < − 4*(1/cellcounts)

# copy the dataframe 3 times to create parameters for 3 different

# scenarios for balance of subgroups - balanced, 1/2 and 1/5

powerp3 < − rbind(powerparms3, powerparms3, powerparms3)

# create factor for balance scenario

powerp3$balance <− c(rep(“1:balanced”, times = 15), rep(“2:m1 = 2 m2”, times = 15),

rep(“3:m1 = 5 m2”, times = 15))

# create values for m1 and m2

powerp3$m2[1:15] < − 9

powerp3$m1[1:15] < − powerp3$m2[1:15]

powerp3$m2[16:30] < − 6

powerp3$m1[16:30] < − 2* powerp3$m2[16:30]

powerp3$m2[31:45] < − 3

powerp3$m1[31:45] < − 5* powerp3$m2[31:45]

# moderate level of tau: vi = tau

powerp3$tausq <− powerp3$vi

# get the sums of the weights for values of the vi and m

powerp3$w1sum < − powerp3$m1*(1/(powerp3$vi + powerp3$tau))

powerp3$w2sum < − powerp3$m2*(1/(powerp3$vi + powerp3$tau))

# get the value of the overall mean effect size given subdiff

# and the values of m1 and m2

powerp3$meaneff <− (powerp3$m2*powerp3$subdiff)/(powerp3$m1 + powerp3$m2)

# get lambda assuming group 1 has mean 0 and group 2 has mean = subdiff

powerp3$lambda <− (powerp3$w1sum*((0 - powerp3$meaneff) ^ 2))+

(powerp3$w2sum*((powerp3$subdiff - powerp3$meaneff) ^ 2))

powerp3$power < − 1 - pchisq(3.84, 1, ncp = powerp3$lambda, lower.tail = T,

log.p = F)

# plot the power curves

ggplot(powerp3, aes(x = powerp3$subdiff, y = powerp3$power, group = powerp3$balance)) +

geom_line() +

geom_point(aes(shape = powerp3$balance)) +

ggtitle(“Power for LOR: Unbalanced groups”) +

labs(y = “Power”, x = “Subgroup difference”) +

labs(shape = “Group Balance”)