Long-term Impacts and Benefit–Cost Analysis of the Communities That Care Prevention System at Age 23, 12 Years After Baseline

Abstract

This study estimated sustained impacts and long-term benefits and costs of the Communities That Care (CTC) prevention system, implemented and evaluated in a longitudinal cluster-randomized trial involving 24 communities in seven states. Analyses utilized reports from a longitudinal panel of 4407 participants, followed since the study’s baseline in grade 5, with most recent follow-up 12 years later at age 23. Impacts on lifetime abstinence from primary outcomes of substance use and antisocial behavior were estimated using generalized linear mixed Poisson regression analysis, adjusted for individual and community-level covariates. Possible cascading effects on 4-year college completion, major depressive disorder, and generalized anxiety disorder through age 23 were evaluated as secondary outcomes. CTC had a statistically significant global effect on primary outcomes and also on combined primary and secondary outcomes. Among primary outcomes, point estimates suggested absolute improvements in lifetime abstinence of 3.5 to 6.1% in the intervention arm and relative improvements of 13 to 55%; 95% confidence intervals revealed some uncertainty in estimates. Among secondary outcomes, 4-year college completion was 1.9% greater among young adults from intervention communities, a 20% relative improvement. Mental health outcomes were approximately the same across trial arms. Although CTC had small sustained effects through age 23, benefit–cost analyses indicated CTC was reliably cost beneficial, with a net present value of $7152 (95% credible interval: $1253 to $15,268) per participant from primary impacts and $17,919 ($306 to $39,186) when secondary impacts were also included. It remained cost beneficial even when impacts were adjusted downward due to the involvement of CTC’s developer in the trial. Findings suggest that broader dissemination of CTC could improve public health and individual lives in the long term and generate positive net benefits to society.

This is a preview of subscription content, access via your institution.

References

  1. Arthur, M. W., Hawkins, J. D., Pollard, J. A., Catalano, R. F., & Baglioni, A., Jr. (2002). Measuring risk and protective factors for use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Survey. Evaluation Review, 26(6), 575–601.

    PubMed  Google Scholar 

  2. Bersani, B. E., & Doherty, E. E. (2018). Desistance from offending in the twenty-first century. Annual Review of Criminology, 1, 311–334.

    Article  Google Scholar 

  3. Briney, J. S., Brown, E. C., Hawkins, J. D., & Arthur, M. W. (2012). Predictive validity of established cut points for risk and protective factor scales from the Communities That Care Youth Survey. The Journal of Primary Prevention, 33(5–6), 249–258.

    Article  Google Scholar 

  4. Brown, E. C., Graham, J. W., Hawkins, J. D., Arthur, M. W., Baldwin, M. M., Oesterle, S., & Abbott, R. D. (2009). Design and analysis of the Community Youth Development Study longitudinal cohort sample. Evaluation Review, 33(4), 311–334.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Brown, E. C., Hawkins, J. D., Rhew, I. C., Shapiro, V. B., Abbott, R. D., Oesterle, S., & Catalano, R. F. (2014). Prevention system mediation of Communities That Care effects on youth outcomes. Prevention Science, 15(5), 623–632.

    Article  Google Scholar 

  6. Chilenski, S. M., Frank, J., Summers, N., & Lew, D. (2019). Public health benefits 16 years after a statewide policy change: Communities That Care in Pennsylvania. Prevention Science, 20(6), 947–958.

    Article  Google Scholar 

  7. Crowley, D. M., Dodge, K. A., Barnett, W. S., Corso, P., Duffy, S., Graham, P., & Jones, D. E. (2018). Standards of evidence for conducting and reporting economic evaluations in prevention science. Prevention Science, 19(3), 366–390.

    Article  Google Scholar 

  8. Enthoven, A. (2019). How systems analysis, cost-effectiveness analysis, or benefit-cost analysis first became influential in federal government program decision-making. Journal of Benefit-Cost Analysis, 10(2), 146–155.

    Article  Google Scholar 

  9. Fagan, A. A., Arthur, M. W., Hanson, K., Briney, J. S., & Hawkins, J. D. (2011). Effects of Communities That Care on the adoption and implementation fidelity of evidence-based prevention programs in communities: Results from a randomized controlled trial. Prevention Science, 12(3), 223–234.

    Article  Google Scholar 

  10. Fagan, A. A., Bumbarger, B. K., Barth, R. P., Bradshaw, C. P., Cooper, B. R., Supplee, L. H., & Walker, D. K. (2019). Scaling up evidence-based interventions in US public systems to prevent behavioral health problems: Challenges and opportunities. Prevention Science, 20(8), 1147–1168.

    Article  Google Scholar 

  11. Fagan, A. A., Hanson, K., Briney, J. S., & Hawkins, J. D. (2012). Sustaining the utilization and high quality implementation of tested and effective prevention programs using the Communities That Care prevention system. American Journal of Community Psychology, 49(3–4), 365–377.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2009). Translational research in action: Implementation of the Communities That Care prevention system in 12 communities. Journal of Community Psychology, 37(7), 809–829.

    Article  Google Scholar 

  13. Fagan, A. A., Hawkins, J. D., Catalano, R. F., & Farrington, D. P. (2019). Communities That Care: Building community engagement and capacity to prevent youth behavior problems. Oxford University Press.

    Google Scholar 

  14. Feinberg, M. E., Jones, D. E., Greenberg, M. T., Osgood, D. W., & Bontempo, D. (2010). Effects of the Communities That Care model in Pennsylvania on change in adolescent risk and problem behaviors. Prevention Science, 11(2), 163–171.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Feng, Z., & Thompson, B. (2002). Some design issues in a community intervention trial. Controlled Clinical Trials, 23(4), 431–449.

    Article  Google Scholar 

  16. Glaser, R. R., Van Horn, M. L., Arthur, M. W., Hawkins, J. D., & Catalano, R. F. (2005). Measurement properties of the Communities That Care Youth Survey across demographic groups. Journal of Quantitative Criminology, 21(1), 73–102.

    Article  Google Scholar 

  17. Gloppen, K. M., Brown, E. C., Wagenaar, B. H., Hawkins, J. D., Rhew, I. C., & Oesterle, S. (2016). Sustaining adoption of science-based prevention through Communities That Care. Journal of Community Psychology, 44(1), 78–98.

    Article  PubMed  Google Scholar 

  18. Gottfredson, D. C., Cook, T. D., Gardner, F. E., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science, 16(7), 893–926.

    Article  Google Scholar 

  19. Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbott, R. D., & Catalano, R. F. (2008). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health, 43(1), 15–22.

    Article  Google Scholar 

  20. Hawkins, J. D., Catalano, R. F., Arthur, M. W., Egan, E., Brown, E. C., Abbott, R. D., & Murray, D. M. (2008). Testing Communities That Care: The rationale, design and behavioral baseline equivalence of the Community Youth Development Study. Prevention Science, 9(3), 178–190.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Hawkins, J. D., Oesterle, S., Brown, E. C., Abbott, R. D., & Catalano, R. F. (2014). Youth problem behaviors 8 years after implementing the Communities That Care prevention system: A community-randomized trial. JAMA Pediatrics, 168(2), 122–129.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbott, R. D., Fagan, A. A., & Catalano, R. F. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities That Care. Archives of Pediatrics and Adolescent Medicine, 163(9), 789–798.

  23. Hawkins, J. D., Oesterle, S., Brown, E. C., Monahan, K. C., Abbott, R. D., Arthur, M. W., & Catalano, R. F. (2012). Sustained decreases in risk exposure and youth problem behaviors after installation of the Communities That Care prevention system in a randomized trial. Archives of Pediatrics and Adolescent Medicine, 166(2), 141–148.

    Article  PubMed  Google Scholar 

  24. Institute of Medicine and National Research Council. (2015). Investing in the health and well-being of young adults. Washington, DC: The National Academies Press.

  25. Jones, T. M., Hill, K. G., Epstein, M., Lee, J. O., Hawkins, J. D., & Catalano, R. F. (2016). Understanding the interplay of individual and social-developmental factors in the progression of substance use and mental health from childhood to adulthood. Development and Psychopathology, 28(3), 721-741.

    Article  Google Scholar 

  26. Kosterman, R., Hawkins, J. D., Hill, K. G., Bailey, J. A., Catalano, R. F., & Abbott, R. D. (2019). Effects of social development intervention in childhood on adult life at ages 30 to 39. Prevention Science, 20(7), 986–995.

    Article  Google Scholar 

  27. Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613.

    CAS  Article  Google Scholar 

  28. Kuklinski, M. R., Briney, J. S., Hawkins, J. D., & Catalano, R. F. (2012). Cost-benefit analysis of Communities That Care outcomes at eighth grade. Prevention Science, 13(2), 150–161.

    Article  Google Scholar 

  29. Kuklinski, M. R., Fagan, A. A., Hawkins, J. D., Briney, J. S., & Catalano, R. F. (2015). Benefit–cost analysis of a randomized evaluation of Communities That Care: Monetizing intervention effects on the initiation of delinquency and substance use through grade 12. Journal of Experimental Criminology, 11(2), 165–192.

    Article  Google Scholar 

  30. Levin, H. M., & McEwan, P. J. (2001). Cost-effectiveness analysis: Methods and applications (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  31. Lynne-Landsman, S. D., Bradshaw, C. P., & Ialongo, N. S. (2010). Testing a developmental cascade model of adolescent substance use trajectories and young adult adjustment. Development and Psychopathology, 22(4), 933–948.

    Article  Google Scholar 

  32. National Academies of Sciences, Engineering, and Medicine. (2016). Advancing the power of economic evidence to inform investments in children, youth, and families. Washington, DC: National Academies Press.

    Google Scholar 

  33. Neumann, P. J., Kim, D. D., Trikalinos, T. A., Sculpher, M. J., Salomon, J. A., Prosser, L. A., & Krahn, M. (2018). Future directions for cost-effectiveness analyses in health and medicine. Medical Decision Making, 38(7), 767–777.

    Article  Google Scholar 

  34. Oesterle, S., Hawkins, J. D., Fagan, A. A., Abbott, R. D., & Catalano, R. F. (2010). Testing the universality of the effects of the Communities That Care prevention system for preventing adolescent drug use and delinquency. Prevention Science, 11(4), 411–424.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Oesterle, S., Hawkins, J. D., Kuklinski, M. R., Fagan, A. A., Fleming, C., Rhew, I. C., & Catalano, R. F. (2015). Effects of Communities That Care on males’ and females’ drug use and delinquency 9 years after baseline in a community-randomized trial. American Journal of Community Psychology, 56(3), 217–228.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Oesterle, S., Kuklinski, M. R., Hawkins, J. D., Skinner, M. L., Guttmannova, K., & Rhew, I. C. (2018). Long-term effects of the Communities That Care trial on substance use, antisocial behavior, and violence through age 21 years. American Journal of Public Health, 108(5), 659–665.

    Article  Google Scholar 

  37. Quinby, R. K., Hanson, K., Brooke-Weiss, B., Arthur, M. W., Hawkins, J. D., & Fagan, A. A. (2008). Installing the Communities That Care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology, 36(3), 313–332.

    Article  Google Scholar 

  38. Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., Congdon, R., & Du Toit, M. (2011). Hierarchical Linear and Nonlinear Modeling (HLM7). Lincolnwood, IL: Scientific Software International.

    Google Scholar 

  39. Reynolds, A. J., Ou, S.-R., & Temple, J. A. (2018). A multicomponent, preschool to third grade preventive intervention and educational attainment at 35 years of age. JAMA Pediatrics, 172(3), 247–256.

    Article  Google Scholar 

  40. Rhew, I. C., Brown, E. C., Hawkins, J. D., & Briney, J. S. (2013). Sustained effects of Communities That Care on prevention service system transformation. American Journal of Public Health, 103(3), 529–535.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York, NY: John Wiley and Sons.

    Book  Google Scholar 

  42. Sánchez-Meca, J., Marín-Martínez, F., & Chacón-Moscoso, S. (2003). Effect-size indices for dichotomized outcomes in meta-analysis. Psychological Methods8(4), 448.

  43. Schweinhart, L. J. (2019). Lessons on sustaining early gains from the life-course study of Perry Preschool. In A. J. Reynolds & J. A. Temple (Eds.), Sustaining early childhood learning gains: Program, school, and family influences (pp. 254–267). New York, NY: Cambridge University Press.

    Chapter  Google Scholar 

  44. Social Development Research Group. (2005–2016). Community Youth Development Study, Youth Development Survey [grades 5 - age 23]. Seattle, WA: Social Development Research Group, School of Social Work, University of Washington.

  45. Spitzer, R. L., Kroenke, K., Williams, J. B., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097.

    Article  Google Scholar 

  46. Sweeten, G., Piquero, A. R., & Steinberg, L. (2013). Age and the explanation of crime, revisited. Journal of Youth and Adolescence, 42(6), 921–938.

    Article  Google Scholar 

  47. Toumbourou, J. W., Rowland, B., Williams, J., Smith, R., & Patton, G. C. (2019). Community intervention to prevent adolescent health behavior problems: Evaluation of Communities That Care in Australia. Health Psychology, 38(6), 536-544.

    Article  Google Scholar 

  48. Washington State Institute for Public Policy. (2019). Benefit-cost technical documentation: Washington State Institute for Public Policy Benefit-Cost Model. Olympia, WA: Author.

    Google Scholar 

  49. White, D., & VanLandingham, G. (2015). Benefit-cost analysis in the states: Status, impact, and challenges. Journal of Benefit-Cost Analysis, 6(2), 369–399.

    Article  Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge Stephanie Lee and her staff at the Washington State Institute for Public Policy for allowing us to use their BCA model and software tool in this study.

Funding

This research was supported by National Institute on Drug Abuse (NIDA) grants R01 DA015183 and R01 DA015183-15A1, with co-funding from the National Institute of Child Health and Human Development, the National Cancer Institute, the National Institute of Mental Health, the Center for Substance Abuse Prevention, and the National Institute on Alcohol Abuse and Alcoholism.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Margaret R. Kuklinski.

Ethics declarations

Disclaimer

No endorsement by WSIPP of the views expressed in this paper should be inferred. The paper’s content is solely the authors’ responsibility and does not necessarily represent official views of study funders, who played no role in the study design; data collection, analysis, and interpretation; writing of the report; or publication decisions.

Ethical Approval

All research protocols involving human participants were approved by the University of Washington Institutional Review Board and were in accordance with the ethical standards of the 1964 Helsinki declaration and its later amendments.

Informed Consent

Informed consent was obtained from all study participants.

Conflict of Interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 77 KB)

Supplementary file2 (XLSX 31 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kuklinski, M.R., Oesterle, S., Briney, J.S. et al. Long-term Impacts and Benefit–Cost Analysis of the Communities That Care Prevention System at Age 23, 12 Years After Baseline. Prev Sci 22, 452–463 (2021). https://doi.org/10.1007/s11121-021-01218-7

Download citation

Keywords

  • Communities That Care
  • Long-term impacts
  • Benefit–cost analysis