Prevention Science

, Volume 19, Issue 2, pp 159–173 | Cite as

Promoting Afterschool Quality and Positive Youth Development: Cluster Randomized Trial of the Pax Good Behavior Game

  • Emilie Phillips SmithEmail author
  • D. Wayne Osgood
  • Yoonkyung Oh
  • Linda C. Caldwell


This randomized trial tested a strategy originally developed for school settings, the Pax Good Behavior Game (PAX GBG), in the new context of afterschool programs. We examined this approach in afterschool since 70% of all juvenile crime occurs between the hours of 3–6 pm, making afterschool an important setting for prevention and promotion. Dual-career and working families need monitoring and supervision for their children in quality settings that are safe and appropriately structured. While substantial work has identified important features of afterschool programs, increasing attention is being given to how to foster quality. PAX GBG, with its focus on shared norms, cooperative teams, contingent activity rewards, and liberal praise, could potentially enhance not only appropriate structure and supportive relationships, but also youth self-regulation, co-regulation, and socio-emotional development. This study examined the PAX GBG among 76 afterschool programs, serving 811 youth ages 5–12, who were diverse in race-ethnicity, socio-economic status, and geographic locale. Demographically matched pairs of afterschool programs were randomized to PAX GBG or treatment-as-usual. Independent observers conducted ratings of implementation fidelity and program quality across time; along with surveys of children’s problem and prosocial behavior. Interaction effects were found using hierarchical linear models such that experimental programs evidencing higher implementation fidelity demonstrated better program quality than controls, (i.e., less harshness, increased appropriate structure, support, and engagement), as well as reduced child-reported hyperactivity and intent-to-treat effects on prosocial behavior. This study demonstrates that best practices fostered by PAX GBG and implemented with fidelity in afterschool result in higher quality contexts for positive youth development.


Afterschool quality Child socio-emotional outcomes Co-regulation Implementation fidelity PAX GBG Positive youth development PAX GBG Randomized trial Self-regulation Setting-level effects 



We acknowledge funding support from William T. Grant Foundation [Grant # 8529]; the Wallace Foundation [Grant #20080489]; and the National Institute for Drug Abuse [Grant # R01 DA025187]. We acknowledge former W. T. Grant Executives, Robert Granger, Edward Seidman and Vivian Tseng, whose feedback on this study was invaluable. We are also grateful for the many staff, parents, and children whose participation made this study possible.

Compliance with Ethical Standards

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. A process for obtaining informed consent from all individual participants was included in the study. Research involving Human Participants was approved and monitored by The Pennsylvania State University Institutional Review Board (IRB # 23990).

Conflict of Interest

The authors declare that they have no conflict of interest.

Supplementary material

11121_2017_820_MOESM1_ESM.doc (50 kb)
ESM 1 (DOC 50 kb)
11121_2017_820_MOESM2_ESM.docx (19 kb)
ESM 2 (DOCX 19 kb)
11121_2017_820_MOESM3_ESM.doc (218 kb)
ESM 3 (DOC 218 kb)


  1. Afterschool Alliance (2014). America After 3pm; Afterschool Programs in Demand. Washington, D.C.Google Scholar
  2. Arnett, J. (1989). Caregivers in day-care centers: Does training matter? Journal of Applied Developmental Psychology, 10, 541–552.CrossRefGoogle Scholar
  3. Baldwin, C. K., & Wilder, Q. (2014). Inside quality: Examination of quality improvement processes in afterschool youth programs. Child & Youth Services, 35, 152–168. doi: 10.1080/0145935X.2014.924346.CrossRefGoogle Scholar
  4. Barker, R. G. (1968). Ecological psychology: Concepts and methods for studying the environment of human behavior. Palo Alto: Stanford University Press.Google Scholar
  5. Barrish, H., Saunders, M., & Wolf, M. M. (1969). Good behavior game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavioral Analysis, 2, 119–124.CrossRefGoogle Scholar
  6. Becker, K. D., Bradshaw, C. P., Domitrovich, C., & Ialongo, N. S. (2013). Coaching teachers to improve implementation of the good behavior game. Administration and Policy in Mental Health, 40, 482–493.CrossRefPubMedGoogle Scholar
  7. Belgrave, F. Z., Reed, M. C., Plybon, L. E., Butler, D. S., Allison, K. W., & Davis, T. (2004). An evaluation of Sisters of Nia: A cultural program for African American girls. Journal of Black Psychology, 30, 329–343.CrossRefGoogle Scholar
  8. Browne, D. (2015). Growing together, learning together: What cities have discovered about building afterschool systems. New York, NY: Wallace Foundation.Google Scholar
  9. Catalano, R. F., Berglund, M. L., Ryan, J. A., Lonczak, H. S., & Hawkins, J. D. (2002). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Prevention & Treatment, 5, 15a.Google Scholar
  10. Collins, L. M., Murphy, S. A., & Strecher, V. (2007). The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): New methods for more potent eHealth interventions. American Journal of Preventive Medicine, 32, S112–S118.CrossRefPubMedPubMedCentralGoogle Scholar
  11. Cross, A. B., Gottfredson, D. C., Wilson, D. M., Rorie, M., & Connell, N. (2010). Implementation quality and positive experiences in after-school programs. American Journal of Community Psychology, 45, 370–380.CrossRefPubMedGoogle Scholar
  12. Domitrovich, C. E., Gest, S. D., Gill, S., Bierman, K. L., Welsh, J. A., & Jones, D. (2009). Fostering highquality teaching with an enriched curriculum and professional development support: The head start REDI program. American Educational Research Journal, 567–597. doi: 10.3102/0002831208328089.
  13. Domitrovich, C. E., Bradshaw, C. P., Greenberg, M. T., Embry, D., Poduska, J. M., & Ialongo, N. S. (2010). Integrated models of school-based prevention: Logic and theory. Psychology in the Schools, 47, 71–88.PubMedPubMedCentralGoogle Scholar
  14. Durlak, J. A., & DuPre, E. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.CrossRefPubMedGoogle Scholar
  15. Durlak, J. A., Weissberg, R. P., & Pachan, M. (2010). A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Community Psychology, 5, 294–309. doi: 10.1007/s10464-010.CrossRefGoogle Scholar
  16. Eccles, J., & Gootman, J. A. (2002). Community programs to promote youth development. Committee on community-level programs for youth. Board on children, youth and families, division of behavioral and social sciences and education, national research council and institute of medicine. Washington, DC: National Academies of Science.Google Scholar
  17. Embry, D. D., Richardson, C., Schaffer, K., et al. (2010). PAX good behavior game (3rd ed.). Tucson: PAXIS Institute.Google Scholar
  18. Fairweather, G. W. (1972). Social change: The challenge to survival. Morristown: General Learning Press.Google Scholar
  19. Fixsen, D. L., Naoom, S. F., Blase, K. A., & Friedman, R. M. (2005). Implementation research: A synthesis of the literature.
  20. Fleiss, J. L. (1981). The measurement of interrater agreement. In J. L. Fleiss (Ed.), Statistical methods for rates and proportions (pp. 212–236). New York: John Wiley.Google Scholar
  21. Frazier, S. L., Cappella, E., & Atkins, M. S. (2007). Linking mental health and after school systems for children in urban poverty: Preventing problems, promoting possibilities. Administration and Policy in Mental Health and Mental Health Services Research, 34, 389–399.CrossRefPubMedGoogle Scholar
  22. Frazier, S. L., Mehta, T. G., Atkins, M. S., Hur, K., & Rusch, D. (2013). Not just a walk in the park: Efficacy to effectiveness for after school programs in communities of concentrated urban poverty. Administration and Policy in Mental Health and Mental Health Services Research, 40, 406–418.CrossRefPubMedGoogle Scholar
  23. Fredricks, J. A., Bohnert, A. M., & Burdette, K. (2014). Moving beyond attendance: Lessons learned from assessing engagement in afterschool contexts. New Directions for Youth Development, 144, 45–58.CrossRefGoogle Scholar
  24. Goodman, R., Meltzer, H., & Bailey, V. (2003). The strengths and difficulties questionnaire: A pilot study on the validity of the self-report version. International Review of Psychiatry, 15, 173–177.CrossRefPubMedGoogle Scholar
  25. Gottfredson, D. C., Gerstenblith, S. A., Soulé, D. A., Womer, S. C., & Lu, S. (2004). Do after school programs reduce delinquency? Prevention Science, 5, 253–266.CrossRefPubMedGoogle Scholar
  26. Gottfredson, D., Cross, A. B., Wilson, D., Rorie, M., & Connell, N. (2010). Effects of participation in after-school programs for middle school students: A randomized trial. Journal of Research on Educational Effectiveness, 3, 282–313. doi: 10.1080/19345741003686659.CrossRefGoogle Scholar
  27. Granger, R. C. (2010). Understanding and improving the effectiveness of after-school practice. American Journal of Community Psychology, 45, 441–446.CrossRefPubMedGoogle Scholar
  28. Heath, S. B., & McLaughlin, M. W. (1994). The best of both worlds: Connecting schools and community youth organizations for all-day, all-year learning. Educational Administration Quarterly, 30, 278–300.CrossRefGoogle Scholar
  29. Hirschi, T. (1969). Causes of delinquency. Berkeley: University of California Press.Google Scholar
  30. Hynes, K., & Sanders, F. (2011). Diverging experiences during out-of-school time: The race gap in exposure to after-school programs. The Journal of Negro Education, 80, 464–476.Google Scholar
  31. Hynes, K., Smith, E., & Perkins, D. (2009). Piloting a classroom-based intervention in after-school Programmes: A case study in science migration. Journal of Children's Services, 4, 4–20.CrossRefPubMedPubMedCentralGoogle Scholar
  32. Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27, 599–641.CrossRefPubMedGoogle Scholar
  33. Ialongo, N., Poduska, J., Werthamer, L., & Kellam, S. (2001). The distal impact of two first grade preventive interventions on conduct problems and disorder and mental health service need and utilization in early adolescence. Journal of Emotional and Behavioral Disorders, 9, 146–160.CrossRefGoogle Scholar
  34. James-Burdumy, S., Dynarski, M., Moore, M., Deke, J., Mansfield, W., & Pistorino, C. (2005). When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program: Final Report. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, 2005. Available at
  35. Kellam, S. G., Brown, C. H., Poduska, J. M., Ialongoc, N. S., Wang, W., Toyinbo, P., Petras, H., Ford, C., Windham, A., & Wilcox, C. H. (2008). Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence, 95, SS5–S28. doi: 10.1016/j.drugalcdep.2008.01.004.CrossRefGoogle Scholar
  36. Kuperminc, G. P., Smith, E. P., & Henrich, C. C. (2013). Introduction to the special issue on “Social and motivational processes in after-school settings: Bridging gaps between theory, research, and practice”. The Journal of Early Adolescence, 33, 5–16.CrossRefGoogle Scholar
  37. Lannie, A. L., & McCurdy, B. L. (2007). Preventing disruptive behavior in the urban classroom: Effects of the good behavior game on student and teacher behavior. Education & Treatment of Children, 30, 85–98.CrossRefGoogle Scholar
  38. Larson, R. (2000). Toward a psychology of positive youth development. American Psychologist, 55, 170–183.CrossRefPubMedGoogle Scholar
  39. Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. (2006). Out-of-school-time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76, 275–313.CrossRefGoogle Scholar
  40. Lerner, R. M., Lerner, J. V., Almerigi, J. B., Theokas, C., Phelps, E., Gestsdottir, S., et al. (2005). Positive youth development, participation in community youth development programs, and community contributions of fifth-grade adolescents findings from the first wave of the 4-H study of positive youth development. The Journal of Early Adolescence, 25, 17–71.CrossRefGoogle Scholar
  41. Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education and Behavior, 31, 242–257.CrossRefGoogle Scholar
  42. Little, P., Wimer, C., & Weiss, H. B. (2008). After school programs in the 21st century: Their potential and what it takes to achieve it. Issues and opportunities in out-of-school time evaluation, 10, 1–12.Google Scholar
  43. Mahoney, J. L., & Zigler, E. F. (2006). Translating science to policy under the No Child Left Behind Act of 2001: Lessons from the national evaluation of the 21st-Century Community Learning Centers. Journal of Applied Developmental Psychology, 27, 282–294.CrossRefGoogle Scholar
  44. Mahoney, J. L., Stattin, H., & Lord, H. (2004). Unstructured youth recreation centre participation and antisocial behavior development: Selection influences and the moderating role of antisocial peers. International Journal of Behavioral Development, 28, 553–560.CrossRefGoogle Scholar
  45. Mahoney, J. L., Lord, H., & Carryl, E. (2005). An ecological analysis of after-school program participation and the development of academic performance and motivational attributes for disadvantaged children. Child Development, 76, 811–825.CrossRefPubMedGoogle Scholar
  46. Mellor, D. (2004). Furthering the use of the strengths and difficulties questionnaire: Reliability with younger child respondents. Psychological Assessment, 16, 396–401.CrossRefPubMedGoogle Scholar
  47. Miller, B. M. (2005). Pathways to success for youth: What counts in after-school. Wellesley: National Institute on Out-of-School-Time Scholar
  48. Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247–266.CrossRefGoogle Scholar
  49. Nunnally, J., & Berstein, I. (1994). Psychometric theory. New York: McGraw-Hill.Google Scholar
  50. Odgers, C. L., Moffett, T. E., Tach, M. L., Sampson, R. J., Taylor, A., & Matthews, C. L. (2009). The protective effects of neighborhood collective efficacy on British children growing up in deprivation: A developmental analysis. Developmental Psychology, 45, 942–957.CrossRefPubMedPubMedCentralGoogle Scholar
  51. Oh, Y., Osgood, D. W., & Smith, E. P. (2015). Measuring afterschool program quality using setting-level observational approaches. The Journal of Early Adolescence, 35, 681–713.CrossRefPubMedPubMedCentralGoogle Scholar
  52. Osgood, D. W., & Anderson, A. L. (2004). Unstructured socializing and rates of delinquency. Criminology, 3, 519.CrossRefGoogle Scholar
  53. Pas, E. T., Waasdorp, T. E., & Bradshaw, C. P. (2015). Examining contextual influences on classroom-based implementation of positive behavior support strategies: Findings from a randomized controlled effectiveness trial. Prevention Science, 16(8), 1096–1106.Google Scholar
  54. Pianta, R. C., & Hamre, B. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38, 109–119.CrossRefGoogle Scholar
  55. Pierce, K. M., Hamm, J. V., & Vandell, D. L. (1999). Experiences in after-school programs and children’s adjustment in first-grade classrooms. Child Development, 70, 756–767.CrossRefPubMedGoogle Scholar
  56. Pierce, K. M., Bolt, D. M., & Vandell, D. L. (2010). Specific features of after-school program quality: Associations with children’s functioning in middle childhood. American Journal of Community Psychology, 45, 381–393. doi: 10.1007/s10464-010-9304-2.CrossRefPubMedPubMedCentralGoogle Scholar
  57. Pittman, K. J. (1991). Promoting youth development: Strengthening the role of youth serving and community organizations. Washington, D.C.: Center for Youth Development and Family Research.Google Scholar
  58. Rasbash, J., Steele, F., Browne, W. J., & Goldstein, H. (2012). A User's Guide to MLwiN Version 2.26. Bristol: University of Bristol.Google Scholar
  59. Raudenbush, S. W., Martinez, A., Bloom, H., Zhu, P., & Lin, F. (2011). Studying the reliability of group-level measures with implications for statistical power: A six-step paradigm. Retrieved from
  60. Reid, J. B., Patterson, G. R., & Snyder, J. E. (2002). Antisocial behavior in children and adolescents: A developmental analysis and model for intervention. Washington, D.C.: American Psychological Association.Google Scholar
  61. Riggs, N. R., Bohnert, A. M., Guzman, M. D., & Davidson, D. (2010). Examining the potential of community-based after-school programs for Latino youth. American Journal of Community Psychology, 45, 417–429.CrossRefPubMedGoogle Scholar
  62. Russo, M. F., Stokes, G. S., Lahey, B. B., Christ, M. A. G., McBurnett, K., Loeber, R., et al. (1993). A sensation seeking scale for children: Further refinement and psychometric development. Journal of Psychopathology and Behavioral Assessment, 15, 69–86.CrossRefGoogle Scholar
  63. Sampson, R. J., Raudenbush, S. W., & Earls, F. (1997). Neighborhoods and violent crime: A multilevel study of collective efficacy. Science, 277, 918–924.CrossRefPubMedGoogle Scholar
  64. Sheldon, J., Arbreton, A., Hopkins, L., & Grossman, J. B. (2010). Investing in success: Key strategies for building quality in after-school programs. American Journal of Community Psychology, 45, 394–404.CrossRefPubMedGoogle Scholar
  65. Shinn, M., & Rapkin, B. D. (2000). Cross-level research without cross-ups in community psychology. In Handbook of community psychology (pp. 669–695). USA: Springer.CrossRefGoogle Scholar
  66. Smith, C., & Hohmann, C. (2005). Full findings from the Youth PQA validation study. Ypsilanti: High/Scope Educational Research Foundation.Google Scholar
  67. Smith, E. P., Boutte, G. S., Zigler, E., & Finn-Stevenson, M. (2004). Opportunities for schools to promote resilience in children and youth. In K. I. Maton, C. J. Schellenbach, B. J. Leadbetter, & A. L. Solarz (Eds.), Investing in children, youth, families, and communities: Strengths-based research and policy. Washington, D. C: American Psychological Association.Google Scholar
  68. Smith, C., Akiva, T., Sugar, S. A., Devaney, T., Lo, Y.-J., Frank, K., Peck, S., & Cortina, K. (2012). Continuous quality improvement in afterschool settings: Impact findings from the Youth Program Quality Intervention study. Ypsilanti, MI: David P. Weikart Center for Youth Program Quality. Washington, DC: Forum for Youth Investment.Google Scholar
  69. Smith, E. P., Osgood, D. W., Caldwell, L. C., Hynes, K., & Perkins, D. F. (2013). Measuring collective efficacy among children in community-based afterschool: Pathways toward prevention and positive youth development. American Journal of Community Psychology, 52, 27-40. doi: 10.1007/s10464-013-9574-6.
  70. Smith, E. P., Wise, E., Rosen, H., Rosen, A., Childs, S., & McManus, M. (2014). Top-down, bottom-up, and around the jungle gym. A social processes and networks approaches to building learning communities in afterschool. American Journal of Community Psychology, 53, 491–502. doi: 10.1007/s10464-014-9656-0.CrossRefPubMedPubMedCentralGoogle Scholar
  71. Snyder, H. N., & Sickmund, M. (2006). Juvenile offenders and victims: 2006 National Report (pp. 1–261). Washington, D.C.: U.S. Department of Justice.Google Scholar
  72. Stuhlman, M. W., Hamre, B. K., Downer, J. T., & Pianta, R. C. (2010). A practitioner’s guide to conducting classroom observations: What the research tells us about choosing and using observational systems. Charlottesville: University of Virginia Center for Advanced Study of Teaching and Learning.Google Scholar
  73. Tebes, J. K., Feinn, R., Vanderploeg, J. J., Chinman, M. J., Shepard, J., Brabham, T., et al. (2007). Impact of a positive youth development program in urban after-school settings on the prevention of adolescent substance use. Journal of Adolescent Health, 41, 239–247.CrossRefPubMedGoogle Scholar
  74. Tseng, V., & Seidman, E. (2007). A systems framework for understanding social settings. American Journal of Community Psychology., 39, 217–228. doi: 10.1007/s10464-007-9101-8.CrossRefPubMedGoogle Scholar
  75. Vandell, D. L., Reisner, E. R., Brown, B. B., Pierce, K. M., Dadisman, K., & Pechman, E. M. (2004). The study of promising after-school programs: Descriptive report of the promising programs. Retrieved from
  76. Wanless, S. B., & Domitrovich, C. E. (2015). Readiness to implement school-based social-emotional learning interventions: Using research on factors related to implementation to maximize quality. Prevention Science, 16(8), 1037–1043.Google Scholar
  77. Weiss, H. B., Little, P., & Bouffard, S. M. (2005). More than just being there: Balancing the participation equation. New Directions for Youth Development, 2005, 15–31.CrossRefGoogle Scholar
  78. Yohalem, N., & Wilson-Ahlstrom, A. (2010). Inside the black box: Assessing and improving quality in youth programs. American Journal of Community Psychology, 45, 350–357. doi: 10.1007/s10464-010-9311.CrossRefPubMedGoogle Scholar

Copyright information

© Society for Prevention Research 2017

Authors and Affiliations

  • Emilie Phillips Smith
    • 1
    Email author
  • D. Wayne Osgood
    • 2
  • Yoonkyung Oh
    • 2
  • Linda C. Caldwell
    • 2
  1. 1.Department of Human Development and Family ScienceThe University of GeorgiaAthensUSA
  2. 2.The Pennsylvania State UniversityState CollegeUSA

Personalised recommendations