Promoting Afterschool Quality and Positive Youth Development: Cluster Randomized Trial of the Pax Good Behavior Game

Abstract

This randomized trial tested a strategy originally developed for school settings, the Pax Good Behavior Game (PAX GBG), in the new context of afterschool programs. We examined this approach in afterschool since 70% of all juvenile crime occurs between the hours of 3–6 pm, making afterschool an important setting for prevention and promotion. Dual-career and working families need monitoring and supervision for their children in quality settings that are safe and appropriately structured. While substantial work has identified important features of afterschool programs, increasing attention is being given to how to foster quality. PAX GBG, with its focus on shared norms, cooperative teams, contingent activity rewards, and liberal praise, could potentially enhance not only appropriate structure and supportive relationships, but also youth self-regulation, co-regulation, and socio-emotional development. This study examined the PAX GBG among 76 afterschool programs, serving 811 youth ages 5–12, who were diverse in race-ethnicity, socio-economic status, and geographic locale. Demographically matched pairs of afterschool programs were randomized to PAX GBG or treatment-as-usual. Independent observers conducted ratings of implementation fidelity and program quality across time; along with surveys of children’s problem and prosocial behavior. Interaction effects were found using hierarchical linear models such that experimental programs evidencing higher implementation fidelity demonstrated better program quality than controls, (i.e., less harshness, increased appropriate structure, support, and engagement), as well as reduced child-reported hyperactivity and intent-to-treat effects on prosocial behavior. This study demonstrates that best practices fostered by PAX GBG and implemented with fidelity in afterschool result in higher quality contexts for positive youth development.

This is a preview of subscription content, access via your institution.

References

  1. Afterschool Alliance (2014). America After 3pm; Afterschool Programs in Demand. Washington, D.C.

  2. Arnett, J. (1989). Caregivers in day-care centers: Does training matter? Journal of Applied Developmental Psychology, 10, 541–552.

    Article  Google Scholar 

  3. Baldwin, C. K., & Wilder, Q. (2014). Inside quality: Examination of quality improvement processes in afterschool youth programs. Child & Youth Services, 35, 152–168. doi:10.1080/0145935X.2014.924346.

    Article  Google Scholar 

  4. Barker, R. G. (1968). Ecological psychology: Concepts and methods for studying the environment of human behavior. Palo Alto: Stanford University Press.

    Google Scholar 

  5. Barrish, H., Saunders, M., & Wolf, M. M. (1969). Good behavior game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavioral Analysis, 2, 119–124.

    CAS  Article  Google Scholar 

  6. Becker, K. D., Bradshaw, C. P., Domitrovich, C., & Ialongo, N. S. (2013). Coaching teachers to improve implementation of the good behavior game. Administration and Policy in Mental Health, 40, 482–493.

    Article  PubMed  Google Scholar 

  7. Belgrave, F. Z., Reed, M. C., Plybon, L. E., Butler, D. S., Allison, K. W., & Davis, T. (2004). An evaluation of Sisters of Nia: A cultural program for African American girls. Journal of Black Psychology, 30, 329–343.

    Article  Google Scholar 

  8. Browne, D. (2015). Growing together, learning together: What cities have discovered about building afterschool systems. New York, NY: Wallace Foundation.

  9. Catalano, R. F., Berglund, M. L., Ryan, J. A., Lonczak, H. S., & Hawkins, J. D. (2002). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Prevention & Treatment, 5, 15a.

    Google Scholar 

  10. Collins, L. M., Murphy, S. A., & Strecher, V. (2007). The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): New methods for more potent eHealth interventions. American Journal of Preventive Medicine, 32, S112–S118.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Cross, A. B., Gottfredson, D. C., Wilson, D. M., Rorie, M., & Connell, N. (2010). Implementation quality and positive experiences in after-school programs. American Journal of Community Psychology, 45, 370–380.

    Article  PubMed  Google Scholar 

  12. Domitrovich, C. E., Gest, S. D., Gill, S., Bierman, K. L., Welsh, J. A., & Jones, D. (2009). Fostering highquality teaching with an enriched curriculum and professional development support: The head start REDI program. American Educational Research Journal, 567–597. doi:10.3102/0002831208328089.

  13. Domitrovich, C. E., Bradshaw, C. P., Greenberg, M. T., Embry, D., Poduska, J. M., & Ialongo, N. S. (2010). Integrated models of school-based prevention: Logic and theory. Psychology in the Schools, 47, 71–88.

    PubMed  PubMed Central  Google Scholar 

  14. Durlak, J. A., & DuPre, E. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.

    Article  PubMed  Google Scholar 

  15. Durlak, J. A., Weissberg, R. P., & Pachan, M. (2010). A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Community Psychology, 5, 294–309. doi:10.1007/s10464-010.

    Article  Google Scholar 

  16. Eccles, J., & Gootman, J. A. (2002). Community programs to promote youth development. Committee on community-level programs for youth. Board on children, youth and families, division of behavioral and social sciences and education, national research council and institute of medicine. Washington, DC: National Academies of Science.

  17. Embry, D. D., Richardson, C., Schaffer, K., et al. (2010). PAX good behavior game (3rd ed.). Tucson: PAXIS Institute.

    Google Scholar 

  18. Fairweather, G. W. (1972). Social change: The challenge to survival. Morristown: General Learning Press.

    Google Scholar 

  19. Fixsen, D. L., Naoom, S. F., Blase, K. A., & Friedman, R. M. (2005). Implementation research: A synthesis of the literature. http://www.popline.org/node/266329.

  20. Fleiss, J. L. (1981). The measurement of interrater agreement. In J. L. Fleiss (Ed.), Statistical methods for rates and proportions (pp. 212–236). New York: John Wiley.

    Google Scholar 

  21. Frazier, S. L., Cappella, E., & Atkins, M. S. (2007). Linking mental health and after school systems for children in urban poverty: Preventing problems, promoting possibilities. Administration and Policy in Mental Health and Mental Health Services Research, 34, 389–399.

    Article  PubMed  Google Scholar 

  22. Frazier, S. L., Mehta, T. G., Atkins, M. S., Hur, K., & Rusch, D. (2013). Not just a walk in the park: Efficacy to effectiveness for after school programs in communities of concentrated urban poverty. Administration and Policy in Mental Health and Mental Health Services Research, 40, 406–418.

    Article  PubMed  Google Scholar 

  23. Fredricks, J. A., Bohnert, A. M., & Burdette, K. (2014). Moving beyond attendance: Lessons learned from assessing engagement in afterschool contexts. New Directions for Youth Development, 144, 45–58.

    Article  Google Scholar 

  24. Goodman, R., Meltzer, H., & Bailey, V. (2003). The strengths and difficulties questionnaire: A pilot study on the validity of the self-report version. International Review of Psychiatry, 15, 173–177.

    CAS  Article  PubMed  Google Scholar 

  25. Gottfredson, D. C., Gerstenblith, S. A., Soulé, D. A., Womer, S. C., & Lu, S. (2004). Do after school programs reduce delinquency? Prevention Science, 5, 253–266.

    Article  PubMed  Google Scholar 

  26. Gottfredson, D., Cross, A. B., Wilson, D., Rorie, M., & Connell, N. (2010). Effects of participation in after-school programs for middle school students: A randomized trial. Journal of Research on Educational Effectiveness, 3, 282–313. doi:10.1080/19345741003686659.

    Article  Google Scholar 

  27. Granger, R. C. (2010). Understanding and improving the effectiveness of after-school practice. American Journal of Community Psychology, 45, 441–446.

    Article  PubMed  Google Scholar 

  28. Heath, S. B., & McLaughlin, M. W. (1994). The best of both worlds: Connecting schools and community youth organizations for all-day, all-year learning. Educational Administration Quarterly, 30, 278–300.

    Article  Google Scholar 

  29. Hirschi, T. (1969). Causes of delinquency. Berkeley: University of California Press.

    Google Scholar 

  30. Hynes, K., & Sanders, F. (2011). Diverging experiences during out-of-school time: The race gap in exposure to after-school programs. The Journal of Negro Education, 80, 464–476.

    Google Scholar 

  31. Hynes, K., Smith, E., & Perkins, D. (2009). Piloting a classroom-based intervention in after-school Programmes: A case study in science migration. Journal of Children's Services, 4, 4–20.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27, 599–641.

    CAS  Article  PubMed  Google Scholar 

  33. Ialongo, N., Poduska, J., Werthamer, L., & Kellam, S. (2001). The distal impact of two first grade preventive interventions on conduct problems and disorder and mental health service need and utilization in early adolescence. Journal of Emotional and Behavioral Disorders, 9, 146–160.

    Article  Google Scholar 

  34. James-Burdumy, S., Dynarski, M., Moore, M., Deke, J., Mansfield, W., & Pistorino, C. (2005). When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program: Final Report. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, 2005. Available at http://www.ed.gov/ies/ncee.

  35. Kellam, S. G., Brown, C. H., Poduska, J. M., Ialongoc, N. S., Wang, W., Toyinbo, P., Petras, H., Ford, C., Windham, A., & Wilcox, C. H. (2008). Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence, 95, SS5–S28. doi:10.1016/j.drugalcdep.2008.01.004.

    Article  Google Scholar 

  36. Kuperminc, G. P., Smith, E. P., & Henrich, C. C. (2013). Introduction to the special issue on “Social and motivational processes in after-school settings: Bridging gaps between theory, research, and practice”. The Journal of Early Adolescence, 33, 5–16.

    Article  Google Scholar 

  37. Lannie, A. L., & McCurdy, B. L. (2007). Preventing disruptive behavior in the urban classroom: Effects of the good behavior game on student and teacher behavior. Education & Treatment of Children, 30, 85–98.

    Article  Google Scholar 

  38. Larson, R. (2000). Toward a psychology of positive youth development. American Psychologist, 55, 170–183.

    CAS  Article  PubMed  Google Scholar 

  39. Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. (2006). Out-of-school-time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76, 275–313.

    Article  Google Scholar 

  40. Lerner, R. M., Lerner, J. V., Almerigi, J. B., Theokas, C., Phelps, E., Gestsdottir, S., et al. (2005). Positive youth development, participation in community youth development programs, and community contributions of fifth-grade adolescents findings from the first wave of the 4-H study of positive youth development. The Journal of Early Adolescence, 25, 17–71.

    Article  Google Scholar 

  41. Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education and Behavior, 31, 242–257.

    Article  Google Scholar 

  42. Little, P., Wimer, C., & Weiss, H. B. (2008). After school programs in the 21st century: Their potential and what it takes to achieve it. Issues and opportunities in out-of-school time evaluation, 10, 1–12.

    Google Scholar 

  43. Mahoney, J. L., & Zigler, E. F. (2006). Translating science to policy under the No Child Left Behind Act of 2001: Lessons from the national evaluation of the 21st-Century Community Learning Centers. Journal of Applied Developmental Psychology, 27, 282–294.

    Article  Google Scholar 

  44. Mahoney, J. L., Stattin, H., & Lord, H. (2004). Unstructured youth recreation centre participation and antisocial behavior development: Selection influences and the moderating role of antisocial peers. International Journal of Behavioral Development, 28, 553–560.

    Article  Google Scholar 

  45. Mahoney, J. L., Lord, H., & Carryl, E. (2005). An ecological analysis of after-school program participation and the development of academic performance and motivational attributes for disadvantaged children. Child Development, 76, 811–825.

    Article  PubMed  Google Scholar 

  46. Mellor, D. (2004). Furthering the use of the strengths and difficulties questionnaire: Reliability with younger child respondents. Psychological Assessment, 16, 396–401.

    Article  PubMed  Google Scholar 

  47. Miller, B. M. (2005). Pathways to success for youth: What counts in after-school. Wellesley: National Institute on Out-of-School-Time http://www.uwmb.org/news/05-mars-study.html.

    Google Scholar 

  48. Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247–266.

    Article  Google Scholar 

  49. Nunnally, J., & Berstein, I. (1994). Psychometric theory. New York: McGraw-Hill.

    Google Scholar 

  50. Odgers, C. L., Moffett, T. E., Tach, M. L., Sampson, R. J., Taylor, A., & Matthews, C. L. (2009). The protective effects of neighborhood collective efficacy on British children growing up in deprivation: A developmental analysis. Developmental Psychology, 45, 942–957.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Oh, Y., Osgood, D. W., & Smith, E. P. (2015). Measuring afterschool program quality using setting-level observational approaches. The Journal of Early Adolescence, 35, 681–713.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Osgood, D. W., & Anderson, A. L. (2004). Unstructured socializing and rates of delinquency. Criminology, 3, 519.

    Article  Google Scholar 

  53. Pas, E. T., Waasdorp, T. E., & Bradshaw, C. P. (2015). Examining contextual influences on classroom-based implementation of positive behavior support strategies: Findings from a randomized controlled effectiveness trial. Prevention Science, 16(8), 1096–1106.

  54. Pianta, R. C., & Hamre, B. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38, 109–119.

    Article  Google Scholar 

  55. Pierce, K. M., Hamm, J. V., & Vandell, D. L. (1999). Experiences in after-school programs and children’s adjustment in first-grade classrooms. Child Development, 70, 756–767.

    CAS  Article  PubMed  Google Scholar 

  56. Pierce, K. M., Bolt, D. M., & Vandell, D. L. (2010). Specific features of after-school program quality: Associations with children’s functioning in middle childhood. American Journal of Community Psychology, 45, 381–393. doi:10.1007/s10464-010-9304-2.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Pittman, K. J. (1991). Promoting youth development: Strengthening the role of youth serving and community organizations. Washington, D.C.: Center for Youth Development and Family Research.

    Google Scholar 

  58. Rasbash, J., Steele, F., Browne, W. J., & Goldstein, H. (2012). A User's Guide to MLwiN Version 2.26. Bristol: University of Bristol.

  59. Raudenbush, S. W., Martinez, A., Bloom, H., Zhu, P., & Lin, F. (2011). Studying the reliability of group-level measures with implications for statistical power: A six-step paradigm. Retrieved from http://wtgrantfoundation.org/FocusAreas#youthsocial-settings.

  60. Reid, J. B., Patterson, G. R., & Snyder, J. E. (2002). Antisocial behavior in children and adolescents: A developmental analysis and model for intervention. Washington, D.C.: American Psychological Association.

    Google Scholar 

  61. Riggs, N. R., Bohnert, A. M., Guzman, M. D., & Davidson, D. (2010). Examining the potential of community-based after-school programs for Latino youth. American Journal of Community Psychology, 45, 417–429.

    Article  PubMed  Google Scholar 

  62. Russo, M. F., Stokes, G. S., Lahey, B. B., Christ, M. A. G., McBurnett, K., Loeber, R., et al. (1993). A sensation seeking scale for children: Further refinement and psychometric development. Journal of Psychopathology and Behavioral Assessment, 15, 69–86.

    Article  Google Scholar 

  63. Sampson, R. J., Raudenbush, S. W., & Earls, F. (1997). Neighborhoods and violent crime: A multilevel study of collective efficacy. Science, 277, 918–924.

    CAS  Article  PubMed  Google Scholar 

  64. Sheldon, J., Arbreton, A., Hopkins, L., & Grossman, J. B. (2010). Investing in success: Key strategies for building quality in after-school programs. American Journal of Community Psychology, 45, 394–404.

    Article  PubMed  Google Scholar 

  65. Shinn, M., & Rapkin, B. D. (2000). Cross-level research without cross-ups in community psychology. In Handbook of community psychology (pp. 669–695). USA: Springer.

    Google Scholar 

  66. Smith, C., & Hohmann, C. (2005). Full findings from the Youth PQA validation study. Ypsilanti: High/Scope Educational Research Foundation.

    Google Scholar 

  67. Smith, E. P., Boutte, G. S., Zigler, E., & Finn-Stevenson, M. (2004). Opportunities for schools to promote resilience in children and youth. In K. I. Maton, C. J. Schellenbach, B. J. Leadbetter, & A. L. Solarz (Eds.), Investing in children, youth, families, and communities: Strengths-based research and policy. Washington, D. C: American Psychological Association.

    Google Scholar 

  68. Smith, C., Akiva, T., Sugar, S. A., Devaney, T., Lo, Y.-J., Frank, K., Peck, S., & Cortina, K. (2012). Continuous quality improvement in afterschool settings: Impact findings from the Youth Program Quality Intervention study. Ypsilanti, MI: David P. Weikart Center for Youth Program Quality. Washington, DC: Forum for Youth Investment.

    Google Scholar 

  69. Smith, E. P., Osgood, D. W., Caldwell, L. C., Hynes, K., & Perkins, D. F. (2013). Measuring collective efficacy among children in community-based afterschool: Pathways toward prevention and positive youth development. American Journal of Community Psychology, 52, 27-40. doi:10.1007/s10464-013-9574-6.

  70. Smith, E. P., Wise, E., Rosen, H., Rosen, A., Childs, S., & McManus, M. (2014). Top-down, bottom-up, and around the jungle gym. A social processes and networks approaches to building learning communities in afterschool. American Journal of Community Psychology, 53, 491–502. doi:10.1007/s10464-014-9656-0.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Snyder, H. N., & Sickmund, M. (2006). Juvenile offenders and victims: 2006 National Report (pp. 1–261). Washington, D.C.: U.S. Department of Justice.

    Google Scholar 

  72. Stuhlman, M. W., Hamre, B. K., Downer, J. T., & Pianta, R. C. (2010). A practitioner’s guide to conducting classroom observations: What the research tells us about choosing and using observational systems. Charlottesville: University of Virginia Center for Advanced Study of Teaching and Learning.

  73. Tebes, J. K., Feinn, R., Vanderploeg, J. J., Chinman, M. J., Shepard, J., Brabham, T., et al. (2007). Impact of a positive youth development program in urban after-school settings on the prevention of adolescent substance use. Journal of Adolescent Health, 41, 239–247.

    Article  PubMed  Google Scholar 

  74. Tseng, V., & Seidman, E. (2007). A systems framework for understanding social settings. American Journal of Community Psychology., 39, 217–228. doi:10.1007/s10464-007-9101-8.

    Article  PubMed  Google Scholar 

  75. Vandell, D. L., Reisner, E. R., Brown, B. B., Pierce, K. M., Dadisman, K., & Pechman, E. M. (2004). The study of promising after-school programs: Descriptive report of the promising programs. Retrieved from http://www.wcer.wisc.edu/childcare/statements.html.

  76. Wanless, S. B., & Domitrovich, C. E. (2015). Readiness to implement school-based social-emotional learning interventions: Using research on factors related to implementation to maximize quality. Prevention Science, 16(8), 1037–1043.

  77. Weiss, H. B., Little, P., & Bouffard, S. M. (2005). More than just being there: Balancing the participation equation. New Directions for Youth Development, 2005, 15–31.

    Article  Google Scholar 

  78. Yohalem, N., & Wilson-Ahlstrom, A. (2010). Inside the black box: Assessing and improving quality in youth programs. American Journal of Community Psychology, 45, 350–357. doi:10.1007/s10464-010-9311.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

We acknowledge funding support from William T. Grant Foundation [Grant # 8529]; the Wallace Foundation [Grant #20080489]; and the National Institute for Drug Abuse [Grant # R01 DA025187]. We acknowledge former W. T. Grant Executives, Robert Granger, Edward Seidman and Vivian Tseng, whose feedback on this study was invaluable. We are also grateful for the many staff, parents, and children whose participation made this study possible.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Emilie Phillips Smith.

Ethics declarations

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. A process for obtaining informed consent from all individual participants was included in the study. Research involving Human Participants was approved and monitored by The Pennsylvania State University Institutional Review Board (IRB # 23990).

Conflict of Interest

The authors declare that they have no conflict of interest.

Electronic supplementary material

ESM 1

(DOC 50 kb)

ESM 2

(DOCX 19 kb)

ESM 3

(DOC 218 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Smith, E.P., Osgood, D.W., Oh, Y. et al. Promoting Afterschool Quality and Positive Youth Development: Cluster Randomized Trial of the Pax Good Behavior Game. Prev Sci 19, 159–173 (2018). https://doi.org/10.1007/s11121-017-0820-2

Download citation

Keywords

  • Afterschool quality
  • Child socio-emotional outcomes
  • Co-regulation
  • Implementation fidelity
  • PAX GBG
  • Positive youth development
  • PAX GBG
  • Randomized trial
  • Self-regulation
  • Setting-level effects