Statistical methodologies to pool across multiple intervention studies

Abstract

Combining and analyzing data from heterogeneous randomized controlled trials of complex multiple-component intervention studies, or discussing them in a systematic review, is not straightforward. The present article describes certain issues to be considered when combining data across studies, based on discussions in an NIH-sponsored workshop on pooling issues across studies in consortia (see Belle et al. in Psychol Aging, 18(3):396–405, 2003). Several statistical methodologies are described and their advantages and limitations are explored. Whether weighting the different studies data differently, or via employing random effects, one must recognize that different pooling methodologies may yield different results. Pooling can be used for comprehensive exploratory analyses of data from RCTs and should not be viewed as replacing the standard analysis plan for each study. Pooling may help to identify intervention components that may be more effective especially for subsets of participants with certain behavioral characteristics. Pooling, when supported by statistical tests, can allow exploratory investigation of potential hypotheses and for the design of future interventions.

APPENDIX

Modeling overall intervention effects

• Model 1a: Modeling individual-level outcomes from multiple single-component intervention trials using fixed effects

  Let Y _ij denote the outcome for the jth person in the ith study, and I _ij is a dummy variable denoting the study arm assignment for the same individual (=1 for active arm, =0 for control arm), then

  $$ {Y}_{ij}={\beta}_0+{\beta}_1{I}_{ij}+{\beta}_2{X}_{1i}+{\beta}_3\left[{I}_{ij}*{X}_{1i}\right]+{\beta}_4{X}_{2ij}+{e}_{ij}, $$

  (1a)

  could be a potential model examining the effect β ₁ of the intervention after accounting for the interaction of the intervention with a study-level covariate X _1i and with a subject-level covariate X _2ij . The error terms e _ij are assumed to follow an N(0,σ²) distribution. If X _1i is a categorical-level variable, this is essentially stratification (or blocking) analysis. If X _i1 is a series of dummy variables identifying the various studies, one is then treating each study as a stratum in a single “stratified large study.”

• Model 1b: Modeling study-level outcomes from multiple single-component intervention trials using fixed effects

  Since we do not have the individual level information, let E _i denote the observed effect in the ith study, which could be a difference in means between the intervention and control arms for continuous variables, or the log odds ratio of the probability of an event for binary outcomes, then

  $$ {E}_i = \mu +{\beta}_1{X}_{1i}+{\beta}_2{X}_{2i}+{\beta}_3{X}_{3i}+{e}_i, $$

  (1b)

  could be a potential model examining the overall effect μ of the intervention after accounting for the effects of three study-level covariates (X _1i , X _2i , X _3i ). The error terms e _i are assumed to have an N(0,σ²) distribution for the variation in each study’s estimate of the common effect μ.

• Model 1c: Modeling study-level outcomes from multiple single-component intervention trials using random effects

  Since we do not have the individual level information, let e _i denote the observed effect in the ith study, which could be a difference in means between the intervention and control arms for continuous variables, or the log odds ratio of the probability of an event for binary outcomes, then

  $$ {E}_i = \mu +{\zeta}_i+{\beta}_1{X}_{1i}+{\beta}_2{X}_{2i}+{\beta}_3{X}_{3i}+{e}_i, $$

  (1c)

  could be a potential model examining the overall effect μ of the intervention after accounting for the fixed effects of three study-level covariates (X _1i , X _2i , X _3i ) and the random effects ζ_i, assumed to be N(0,τ²) and independent of the errors e _i . The random effects help decompose the total variance in study effects into a component due to across study variation (τ²) and a within-study variation (σ²).

• Model 2: Modeling study-level outcomes from multicomponent intervention trials using random effects meta-regression

  For illustration purposes, we use the COPTR consortium, where the primary outcome is body mass index change [ΔBMI]. For simplicity of illustration, assume that each of the four studies in the consortium has multicomponent interventions addressing obesity but that three main modalities are common to all—C₁ = education modality, C₂ = physical activity modality, and C₃ = dietary modality. Note that all studies do not necessarily have to offer all modalities as part of their multicomponent intervention and that what they offer may differ within a modality. The Cs above denote indicator variables for whether it is offered or not as part of the intervention in a given study. For example, C_1i = 1 if an education component is offered in the ith study, =0 if not. If in addition to the intervention components we have two study-level covariates—say, W₁ = proportion of males (assume as a mediator) and W₂ = mean age of individuals (assume as a moderator of education component), for example, then, the random-effects meta-regression model with only study-level covariates would be

  $$ \left[\varDelta BM{I}_i\right]=\mu +{\zeta}_i+{\beta}_1{C}_{1i}+{\beta}_2{C}_{2i}+{\beta}_3{C}_{3i} + {\beta}_4{W}_{1i}+{\beta}_5{W}_{2i}+{\beta}_6\left[{C}_{1i}*{W}_{2i}\right]\Big)+{e}_i, $$

  (2)

  where we are interested in the overall effect μ but also in the fixed coefficients βs. The ζs are the study random effects that help study the variance components.

• Model 3: Modeling individual-level outcomes from multicomponent intervention trials using random effects multilevel meta-regression

  In a consortium, one expects to be able to have individual-level information and can thus model the individual change or effect within a person. Using the COPTR example of Model 2, but for individual-level outcomes, we now have that the jth subject in the ith study may have been randomized to receive or not the kth component C_k. Thus, we can model the within-person effect with individual level covariates sex (S) and age (A):

  $$ \Delta BM{I}_{ij}=\mu + {\zeta}_i+{\beta}_1{C}_{1ij}+{\beta}_2{C}_{2ij}+{\beta}_3{C}_{3ij}+{\beta}_4{S}_{ij}+{\beta}_5{A}_{ij}+{\beta}_6\left[{C}_{1ij}*{A}_{ij}\right]\Big)+{e}_i, $$

  (3)

  where we are interested in the overall effect μ but also in the fixed coefficients βs. We should point out that any subject-level covariate would ideally be measured using the same instrument or method (i.e., have a set of common metrics) across studies.