Abstract
Data collected during an evaluation of a multi-site trial of an enhanced after-school program were used to relate quality of program implementation to student experiences after school. The enhanced after-school program incorporated a drug use and violence prevention component that was shown to be effective in previous research. Building on Durlak and Dupre’s (Am J Community Psychol 41:327–350, 2008) dimensions of implementation, we assessed the level of dosage, quality of management and climate, participant responsiveness, and staffing quality achieved at the five program sites. We evaluated how these characteristics co-varied with self-reported positive experiences after-school. The study illustrates how multiple dimensions of program implementation can be measured, and shows that some but not all dimensions of implementation are related to the quality of student after-school experiences. Measures of quality of management and climate, participant responsiveness, and staffing stability were most clearly associated with youth experiences. The importance of measuring multiple dimensions of program implementation in intervention research is discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Armstrong, T., & Armstrong, G. (2004). The organizational, community and programmatic characteristics that predict the effective implementation of after-school programs. Journal of School Violence, 3, 93–109.
Beckett, M., Hawken, A., & Jacknowitz, A. (2001). Accountability for after-school care: Devising standards and measuring adherence to them. Santa Monica, CA: Rand Corporation.
Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.
Durlak, J. A., & Weissberg, R. P. (2007). The impact of after-school programs that promote personal and social skills. Chicago, IL: Collaborative for Academic, Social, and Emotional Learning. Retrieved February 12, 2007, from www.casel.org.
Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.
Dynarski, M., James-Burdumy, S., Moore, M., Rosenberg, L., Deke, J., & Mansfield, W. (2003). When schools stay open late: The national evaluation of the 21st century community learning centers program: New findings: US Department of Education, National Center for Education Evaluation and Regional Assistance. Washington, DC: US Government Printing Office.
Fashola, O. (1998). Review of extended day and after school programs and their effectiveness. Retrieved January, 2008, from www.csos.jhu.edu/crespar/techReports/Report24.pdf.
Finance Project. (2007). Estimated federal investment in out-of-school time. Retrieved April, 2008, from www.financeproject.org/publications/estimatedfederalOST_TFPflyer.pdf.
Gerstenblith, S. A., Soulé, D. A., Gottfredson, D. C., Lu, S., Kellstrom, M. A., Womer, S. C., et al. (2005). ASPs, antisocial behavior, and positive youth development: An exploration of the relationship between program implementation and changes in youth development. In J. L. Mahoney, R. W. Larson, & J. S. Eccles (Eds.), Organized activities as contexts of development: Extracurricular activities, after-school and community programs (pp. 457–478). Mahwah, NH: Lawrence Erlbaum & Associates.
Gottfredson, D. C., Cross, A. B., & Soulé, D. A. (2007). Distinguishing characteristics of effective and ineffective after-school programs to prevention delinquency and victimization. Criminology and Public Policy, 6, 601–631.
Gottfredson, D. C., Cross, A. B., Wilson, D. M., Rorie, M., & Connell, N. (in press). Effects of participation in after-school programs for middle school students: A randomized trial. Journal of Research on Educational Effectiveness.
Gottfredson, D. C., Gerstenblith, S. A., Soulé, D. A., Womer, S. C., & Lu, S. (2004). Do ASPs reduce delinquency? Prevention Science, 5, 253–266.
Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39, 3–35.
Gottfredson, D. C., Gottfredson, G. D., & Weisman, S. A. (2001). The timing of delinquent behavior and its implications for ASPs. Criminology & Public Policy, 1(1), 61–80.
Granger, R. C. (2008). After-school programs and academics: Implications for policy, practice, and research. Social Policy Report, 22, 3–19.
Granger, R. C., Durlak, J. A., Yohalem, N., & Reisner, E. (2007). Improving after-school program quality. The William T. Grant foundation working paper. Retrieved May, 2008, from www.wtgrantfoundation.org/usr_doc/Improving_After-School_Program_Quality.pdf.
Grossman, J., Campbell, M., & Raley, B. (2007). Quality time after School: What instructors can do to enhance learning. Philadelphia: Public/Private Ventures.
Hansen, W. B., & Dusenbury, L. (2004). All Stars Plus: A competence and motivation enhancement approach to prevention. Health Education, 104, 371–381.
Hansen, W. B., & Larson, R. (2005). The youth experiences survey 2.0: Instrument revisions and validity testing. Urbana-Champaign, IL: University of Illinois.
Harrington, N. G., Giles, S. M., Hoyle, R. H., Feeney, G. J., & Youngbluth, S. C. (2001). Evaluation of the All Stars™ character education and problem behavior prevention program: Effects on mediator and outcome variables for middle school students. Health Education Research, 28, 533–546.
Kane, T. J. (2004). The impact of after school programs: Interpreting the results of four recent evaluations. The William T. Grant foundation working paper. Retrieved May, 2008, from http://www.wtgrantfoundation.org/usr_doc/After-school_paper.pdf.
Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. (2006). Out-of-school time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76, 275–313.
Mahoney, J. L. (2000). School extracurricular activity participation as a moderator in the development of antisocial patterns. Child Development, 71, 502–516.
McNeal, R. B., Hansen, W. B., Harrington, N. G., & Giles, S. M. (2004). How All Stars™ works: An examination of program effects on mediating variables. Health Education and Behavior, 31, 165–178.
Pierce, K. M., Hamm, J. V., & Vandell, D. L. (1999). Experiences in after-school programs and children’s adjustment in first grade classrooms. Child Development, 70, 756–767.
Rosenthal, R., & Vandell, D. L. (1996). Quality of care at school-aged child-care programs: Regulatable features, observed experiences, child perspectives, and parent perspectives. Child Development, 67, 2434–2445.
Sickmund, M., Snyder, H. N., & Poe-Yamagata, E. (1997). Juvenile offenders and victims: 1997 update on violence. Washington, DC: Office of Juvenile Justice and Delinquency Prevention.
Simpkins-Chaput, S., Little, P. M. D., & Weiss, H. B. (2004). Understanding and measuring attendance in out-of-school time programs. Issues and opportunities in out-of-school time evaluation brief no. 7. Cambridge, MA: Harvard Family Research Project.
Weisman, S. A., Womer, S. C., Kellstrom, M. A., Bryner, S. L., Kahler, A., Slocum, L., et al. (2002). Maryland after school community grant program, report on the 2001–2002 school year evaluation of the phase 3 after school programs. Technical report.
Weiss, H. B., Little, P. M., & Bouffard, S. M. (2005). More than just being there: Balancing the participation equation. New Directions for Youth Development, 2005, 15–31.
Whitebook, M., Howes, C., & Phillips, D. (1998). Worthy work, unlivable wages: The national child care staffing study, 1988–1997. Washington, DC: National Center for the Early Childhood Work Force.
Acknowledgments
This work was supported through grant number R305F050069 from the U.S. Department of Education Institute of Educational Sciences to the University of Maryland. We acknowledge the support of the Baltimore County Local Management Board, the Baltimore County Public Schools, and the Baltimore County Department of Recreation and Parks for implementing and managing the after school programs. We especially acknowledge the assistance of Elise Andrews of the Baltimore County Local Management Board and Beahta Davis of the Baltimore County Department of Recreation and Parks. We would like to thank Gordon Bonham for collecting data from the school system and Stephanie DiPietro, Mathew Gugino, Lynda Okeke, Matthew Brigham, Freshta Rahimi, and Jaynie Trageser for research assistance. Finally, we thank three anonymous reviewers for helpful comments on an earlier draft, and Joe Durlak for major editorial assistance in improving the paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cross, A.B., Gottfredson, D.C., Wilson, D.M. et al. Implementation Quality and Positive Experiences in After-School Programs. Am J Community Psychol 45, 370–380 (2010). https://doi.org/10.1007/s10464-010-9295-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10464-010-9295-z