Skip to main content

Empirically Based Mean Effect Size Distributions for Universal Prevention Programs Targeting School-Aged Youth: A Review of Meta-Analyses

Abstract

This review of reviews presents an empirically based set of mean effect size distributions for judging the relative impact of the effects of universal mental health promotion and prevention programs for school-age youth (ages 5 through 18) across a range of program targets and types of outcomes. Mean effect size distributions were established by examining the findings from 74 meta-analyses of universal prevention and promotion programs that included more than 1100 controlled outcome studies involving over 490,000 school-age youth. The distributions of mean effect sizes from these meta-analyses indicated considerable variability across program targets and outcomes that differed substantially from Cohen’s (1988, Statistical power analysis for the behavioral sciences (2nd ed.)) widely used set of conventions for assessing if effects are small, medium, or large. These updated mean effect size distributions will provide researchers, practitioners, and funders with more appropriate evidence-based standards for judging the relative effects of universal prevention programs for youth. Limitations in current data and directions for future work are also discussed.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    Namely, a meta-analysis might have synthesized findings from universal prevention programs reporting substance use outcomes, regardless of the primary target of the intervention. In this instance, results from a delinquency prevention trial reporting substance use outcomes might have been included in the meta-analysis. In our coding, this meta-analysis would have been categorized as a “substance use program,” unless the meta-analysis authors reported additional subgroup results by actual program target.

  2. 2.

    We conducted sensitivity analyses (shown in the online Appendix) to assess how the inclusion of dependent mean effect sizes within-meta-analyses influenced the results. Those sensitivity analyses included one mean effect size estimate from each meta-analysis, where we selected the mean effect size based on the largest number of primary studies. Results from those sensitivity analyses are substantively similar to those reported in Table 2.

  3. 3.

    As shown in the follow-up mean effect size distributions in Appendix B, there was consistent evidence that follow-up mean effects were smaller than immediate posttest mean effects. These findings should be interpreted cautiously, however, given that only 25 of the 74 included meta-analyses reported follow-up effect sizes.

References

  1. Bloom, H. S., Hill, C. J., Black, A. R., & Lipsey, M. W. (2008). Performance trajectories and performance gaps as achievement effect-size benchmarks for educational interventions. Journal of Research on Educational Effectiveness, 1, 289–328. https://doi.org/10.1080/19345740802400072.

    Article  Google Scholar 

  2. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex: Wiley. https://doi.org/10.1002/9780470743386.

    Book  Google Scholar 

  3. Bradshaw, C. P., Waasdorp, T. E., & Leaf, P. J. (2012). Effects of school-wide positive behavioral interventions and supports on child behavior problems. Pediatrics, 130, e1136–e1145. https://doi.org/10.1542/peds.2012-0243.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  5. Cooper, H., & Koenka, A. C. (2012). The overview of reviews: Unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. American Psychologist, 67, 446–462. https://doi.org/10.1037/a0027119.

    Article  PubMed  Google Scholar 

  6. Cox, D. R., & Snell, E. J. (1989). Analysis of binary data (2nd ed.). New York: Chapman & Hall/CRC.

    Google Scholar 

  7. Durlak, J. A. (1995). School-based prevention programs for children and adolescents. Thousand Oaks: Sage. https://doi.org/10.4135/9781483327396.

    Book  Google Scholar 

  8. Durlak, J. A. (1997). Effective prevention programs for children and adolescents. New York: Plenum. https://doi.org/10.1007/978-1-4899-0065-4.

    Book  Google Scholar 

  9. Durlak, J. A. (2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34, 917–928. https://doi.org/10.1093/jpepsy/jsp004.

    Article  PubMed  Google Scholar 

  10. Furr-Holden, C. D., Ialongo, N. S., Anthony, J. C., Petras, H., & Kellam, S. G. (2004). Developmentally inspired drug prevention: Middle school outcomes in a school-based randomized prevention trial. Drug and Alcohol Dependence, 73, 149–158. https://doi.org/10.1016/j.drugalcdep.2003.10.002.

    Article  PubMed  Google Scholar 

  11. Grucza, R. A., Plunk, A. D., Hipp, P. R., Cavazos-Rehg, P., Krauss, M. J., Brownson, R. C., & Bierut, L. J. (2013). Long-term effects of laws governing youth access to tobacco. American journal of Public Health, 103(8), 1493–1499.

    Article  Google Scholar 

  12. Harris, D. N. (2009). Toward policy-relevant benchmarks for interpreting effect sizes: Combining effects with costs. Educational Evaluation and Policy Analysis, 31, 3–29. https://doi.org/10.3102/0162373708327524.

    Article  Google Scholar 

  13. Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2007). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2, 172–177. https://doi.org/10.1111/j.1750-8606.2008.00061.x.

    Article  Google Scholar 

  14. Kumpfer, K. L., & Alvarado, R. (2003). Family-strengthening approaches for the prevention of youth problem behaviors. American Psychologist, 58, 457. https://doi.org/10.1037/0003-066X.58.6-7.457.

    Article  PubMed  Google Scholar 

  15. Lippke, S., Nigg, C. R., & Maddock, J. E. (2012). Health-promoting and health-risk behaviors: Theory-driven analyses of multiple health behavior change in three international samples. International Journal of Behavioral Medicine, 19, 1–13. https://doi.org/10.1007/s12529-010-9135-4.

    Article  PubMed  Google Scholar 

  16. Lipsey, M. W., & Cullen, F. T. (2007). The effectiveness of correctional rehabilitation: A review of systematic reviews. Annual Review of Law and Social Science, 3, 297–320. https://doi.org/10.1146/annurev.lawsocsci.3.081806.112833.

    Article  Google Scholar 

  17. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6, e1000097. https://doi.org/10.1371/journal.pmed.1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Peters, L. W., Kok, G., Ten Dam, G. T., Buijs, G. J., & Paulussen, T. G. (2009). Effective elements of school health promotion across behavioral domains: A systematic review of reviews. BMC Public Health, 9, 182. https://doi.org/10.1186/1471-2458-9-182.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64, 878–912. https://doi.org/10.1111/lang.12079.

    Article  Google Scholar 

  20. Sandler, I., Wolchik, S. A., Cruden, G., Mahrer, N. E., Ahn, S., Brincks, A., & Brown, C. H. (2014). Overview of meta-analyses of the prevention of mental health, substance use and conduct problems. Annual Review of Clinical Psychology, 10, 243. https://doi.org/10.1146/annurev-clinpsy-050212-185524.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Substance Abuse & Mental Health Data Archive (SAMHDA). (2016). National Survey on Drug Use and Health 2015 (NSDUH-2016-DS0001). Retrieved from https://www.datafiles.samhsa.gov/study-dataset/national-survey-drug-use-and-health-2015-nsduh-2015-ds0001-nid16894.

  22. Wagenaar, A. C., Salois, M. J., & Komro, K. A. (2009). Effects of beverage alcohol price and tax levels on drinking: A meta-analysis of 1003 estimates from 112 studies. Addiction, 104, 179–190. https://doi.org/10.1111/j.1360-0443.2008.02438.x.

    Article  Google Scholar 

  23. Wiefferink, C. H., Peters, L., Hoekstra, F., Ten Dam, G., Buijs, G. J., & Paulussen, T. G. (2006). Clustering of health-related behaviors and their determinants: Possible consequences for school health interventions. Prevention Science, 7, 127–149. https://doi.org/10.1007/s11121-005-0021-2.

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Emily E. Tanner-Smith.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflicts of interest. JAD acknowledges that he is an author of several meta-analyses included in this review, but he was not involved in extracting any data from those meta-analyses for the purposes of this review.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Informed Consent

For this type of study, formal consent is not required.

Electronic Supplementary Material

ESM 1

(DOCX 275 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tanner-Smith, E.E., Durlak, J.A. & Marx, R.A. Empirically Based Mean Effect Size Distributions for Universal Prevention Programs Targeting School-Aged Youth: A Review of Meta-Analyses. Prev Sci 19, 1091–1101 (2018). https://doi.org/10.1007/s11121-018-0942-1

Download citation

Keywords

  • Effect sizes
  • Meta-analysis
  • Universal prevention
  • Youth