Assessing generalizability and variability of single-case design effect sizes using two-stage multilevel modeling including moderators

Abstract

This study introduces an innovative meta-analytic approach, two-stage multilevel meta-analysis that considers the hierarchical structure of single-case experimental design (SCED) data. This approach is unique as it is suitable to include moderators at the intervention level, participant level, and study level, and is therefore especially recommended for the meta-analyst interested in moving beyond estimating the overall intervention effectiveness. Using this approach, the between-participant variability and between-study variability in intervention effectiveness can be evaluated in addition to obtaining a generalized effect size estimate across studies. This is a timely contribution to the SCED field, as the source(s) of variability in effect size can be identified, and moderators at the corresponding level(s) (participant level and/or study level) can be added to explain the variability. The two-stage multilevel meta-analytic approach, with the inclusion of moderators, can provide evidence-based recommendations about the effectiveness of an intervention taking into account intervention, participant, and study characteristics. First, a conceptual introduction to two-stage multilevel meta-analysis is given to provide a good understanding of its full potentials and modeling options. Second, the usage of this approach will be demonstrated by applying it to a published meta-analytic data set. The goal of this study is to disseminate the two-stage multilevel meta-analysis approach in the hope that SCED meta-analyst will consider this methodology in future meta-analyses.