Check-in/Check-out Implementation in Schools: a Meta-Analysis of Group Design Studies

Abstract

This meta-analysis study synthesized the intervention effects of Check-in/Check-out (CICO), the most common Tier 2 behavior intervention used in schools. Systematic review procedures were employed to examine six CICO studies that were evaluated using a group research design and that involved 146 students. Studies were coded to examine their characteristics as well as quality indicators concerning their design features. The results indicated that in group design studies, the overall effect size of CICO across student outcomes (problem behavior, appropriate behavior, social skills, and academic performance) was medium (g = 0.42). No differences in effects were found for outcome domains, Tier 1 fidelity, and implementation fidelity, but grade level was associated with statistically significant differences in effect sizes (Q = 3.95, p < .05). Elementary school students (g = 0.70) showed greater improvements than middle school students (g = 0.27). Further research is needed to examine the intervention features impacting the effects of CICO.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

Studies prefixed with an asterisk (*) were included in the meta-analysis

  1. Algozzine, R. F., Barrett, S., Eber, L., George, H., Horner, R. H., Lewis, T. J., Putnam, B., Swain-Bradway, J., McIntosh, K., & Sugai, G. (2019). SWPBIS Tiered Fidelity Inventory. Eugene, OR: OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. Available from http://www.pbis.org.

  2. Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101. https://doi.org/10.2307/2533446.

    Article  PubMed  Google Scholar 

  3. Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2011). Introduction to meta-analysis. Chichester, UK: Wiley.

    Google Scholar 

  4. Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2013). Comprehensive Meta-analysis version 3. Englewood, NJ: Biostat.

    Google Scholar 

  5. Bowman-Perrott, L., Burke, M. D., Zaini, S., Zhang, N., & Vannest, K. (2016). Promoting positive behavior using the good behavior game: A meta-analysis of single-case research. Journal of Positive Behavior Interventions, 18, 180–190. https://doi.org/10.1177/1098300715592355.

    Article  Google Scholar 

  6. Boyd, R. J., & Anderson, C. M. (2013). Breaks are better: A tier II social behavior intervention. Journal of Behavioral Education, 22, 348–365. https://doi.org/10.1007/s10864-013-9184-2.

    Article  Google Scholar 

  7. Bradshaw, C. P., Waasdorp, T. E., & Leaf, P. J. (2012). Effects of school-wide positive behavioral interventions and supports on child behavior problems. Pediatrics, 130, e1136–e1145. https://doi.org/10.1542/peds.2012-0243.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Bruhn, A. L., & Hirsch, S. E. (2017). From good intentions to great implementation. Report on Emotional & Behavioral Disorders in Youth, 17, 64–70.

    Google Scholar 

  9. Bruhn, A. L., Lane, K. L., & Hirsch, S. E. (2014). A review of tier 2 interventions conducted within multitiered models of behavioral prevention. Journal of Emotional and Behavioral Disorders, 22, 171–189. https://doi.org/10.1177/1063426613476092.

    Article  Google Scholar 

  10. Bruhn, A. L., McDaniel, S. C., Rila, A., & Estrapala, S. (2018). A step-by-step guide to tier 2 behavioral progress monitoring. Beyond Behavior, 27, 15–27. https://doi.org/10.1177/1074295618756984.

    Article  Google Scholar 

  11. Campbell, A., & Anderson, C. M. (2011). Check-in/check-out: A systematic evaluation and component analysis. Journal of Applied Behavior Analysis, 44, 315–326. https://doi.org/10.1901/jaba.2011.44-315.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Campbell, A., Rodriguez, B. J., Anderson, C., & Barnes, A. (2013). Effects of a tier 2 intervention on classroom disruptive behavior and academic engagement. Journal of Curriculum and Instruction, 7, 32–54.

    Article  Google Scholar 

  13. *Cheney, D. A., Stage, S. A., Hawken, L. S., Lynass, L., Mielenz, C., & Waugh, M. (2009). A 2-year outcome study of the check, connect, and expect intervention for students at risk for severe behavior problems. Journal of Emotional and Behavioral Disorders, 17, 226–243. https://doi.org/10.1177/1063426609339186.

  14. Childs, K. E., Kincaid, D., & George, H. P. (2010). A model for statewide evaluation of a universal positive behavior support initiative. Journal of Positive Behavior Interventions, 12, 198–210. https://doi.org/10.1177/1098300709340699.

    Article  Google Scholar 

  15. Cohen, J. (1988). Statistical power analysis for the behavioural sciences. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  16. Cohen, R., Kincaid, D., & Childs, K. E. (2007). Measuring schoolwide positive behavior support implementation: Development and validation of the benchmarks of quality. Journal of Positive Behavior Interventions, 9, 203–213. https://doi.org/10.1177/10983007070090040301.

    Article  Google Scholar 

  17. Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis. New York, NY: Russell Sage Foundation.

    Google Scholar 

  18. Crone, D. A., Hawken, L. S., & Horner, R. H. (2010). Responding to problem behavior in schools: The behavior education program. New York, NY: Guilford Press.

    Google Scholar 

  19. Drevon, D. D., Hixson, M. D., Wyse, R. D., & Rigney, A. M. (2019). A meta-analytic review of the evidence for check-in check-out. Psychology in the Schools, 56, 393–412.

    Article  Google Scholar 

  20. DuPaul, G. J., Eckert, T. L., & Vilardo, B. (2012). The effects of school-based interventions for attention deficit hyperactivity disorder: A meta-analysis 1996-2010. School Psychology Review, 41, 387.

    Google Scholar 

  21. Eldevik, S., Hastings, R. P., Jahr, E., & Hughes, J. C. (2012). Outcomes of behavioral intervention for children with autism in mainstream pre-school settings. Journal of Autism and Developmental Disorders, 42, 210–220. https://doi.org/10.1007/s10803-011-1234-9.

    Article  PubMed  Google Scholar 

  22. Ennis, R. P., Jolivette, K., Swoszowski, N. C., & Johnson, M. L. (2012). Secondary prevention efforts at a residential facility for students with emotional and behavioral disorders: Function-based check-in, check-out. Residential Treatment for Children & Youth, 29, 79–102. https://doi.org/10.1080/0886571X.2012.669250.

    Article  Google Scholar 

  23. Ennis, C. R., Kwang-Sun, C. B., & George, H. P. (2016). An evaluation of group contingency interventions: The role of teacher preference. Journal of Positive Behavior Interventions, 18, 17–28. https://doi.org/10.1177/1098300715577663.

    Article  Google Scholar 

  24. Evans, C., & Weiss, S. L. (2014). Teachers working together: How to communicate, collaborate, and facilitate positive behavior in inclusive classrooms. Journal of the International Association of Special Education, 15, 142–146.

    Google Scholar 

  25. *Filter, K. J., McKenna, M. K., Benedict, E. A., Horner, R. H., Todd, A., & Watson, J. (2007). Check in/check out: A post-hoc evaluation of an efficient, secondary-level targeted intervention for reducing problem behaviors in schools. Education and Treatment of Children, 30, 69–84. https://doi.org/10.1353/etc.2007.0000.

  26. Flannery, K. B., Fenning, P., Kato, M. M., & McIntosh, K. (2014). Effects of school-wide positive behavioral interventions and supports and fidelity of implementation on problem behavior in high schools. School Psychology Quarterly, 29, 111–124. https://doi.org/10.1037/spq0000039.

    Article  PubMed  Google Scholar 

  27. Gresham, F. M., & Elliott, S. N. (1990). Social skills rating system. Circle Pines, MN: American Guidance Service.

    Google Scholar 

  28. Gresham, F. M., Van, M. B., & Cook, C. R. (2006). Social skills training for teaching replacement behaviors: Remediating acquisition deficits in at-risk students. Behavioral Disorders, 31, 363–377. https://doi.org/10.1177/019874290603100402.

    Article  Google Scholar 

  29. *Hawken, L. S. (2006). School psychologists as leaders in the implementation of a targeted intervention: The Behavior Education Program. School Psychology Quarterly, 21, 91–111. https://doi.org/10.1521/scpq.2006.21.1.91.

  30. Hawken, L. S., & Horner, R. H. (2003). Evaluation of a targeted intervention within a schoolwide system of behavior support. Journal of Behavioral Education, 12, 225–240.

    Article  Google Scholar 

  31. Hawken, L. S., Sandra MacLeod, K., & Rawlings, L. (2007). Effects of the behavior education program (BEP) on office discipline referrals of elementary school students. Journal of Positive Behavior Interventions, 9, 94–101. https://doi.org/10.1177/10983007070090020601.

    Article  Google Scholar 

  32. *Hawken, L. S., O'Neill, R. E., & MacLeod, K. S. (2011). An investigation of the impact of function of problem behavior on effectiveness of the behavior education program (BEP). Education and Treatment of Children, 34, 551–574. https://doi.org/10.1353/etc.2011.0031.

  33. Hawken, L. S., Bundock, K., Kladis, K., O'Keeffe, B., & Barrett, C. A. (2014). Systematic review of the check-in, check-out intervention for students at risk for emotional and behavioral disorders. Education and Treatment of Children, 37, 635–658. https://doi.org/10.1353/etc.2014.0030.

    Article  Google Scholar 

  34. Hawken, L. S., Bundock, K., Barrett, C. A., Eber, L., Breen, K., & Phillips, D. (2015). Large-scale implementation of check-in, check-out: A descriptive study. Canadian Journal of School Psychology, 30, 304–319. https://doi.org/10.1177/0829573515601005.

    Article  Google Scholar 

  35. Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539–1558. https://doi.org/10.1002/sim.1186.

    Article  Google Scholar 

  36. Horner, R. H., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A. W., & Esparanza, J. (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–144.

    Article  Google Scholar 

  37. Hoyle, C. G., Marshall, K. J., & Yell, M. L. (2011). Positive behavior supports: Tier 2 interventions in middle schools. Preventing School Failure, 55, 164–170. https://doi.org/10.1080/1045988X.2010.503975.

    Article  Google Scholar 

  38. Huffcutt, A. I., & Arthur, W. (1995). Development of a new outlier statistic for meta-analytic data. Journal of Applied Psychology, 80, 327–334. https://doi.org/10.1037/0021-9010.80.2.327.

    Article  Google Scholar 

  39. Huizenga, H. M., Visser, I., & Dolan, C. V. (2011). Testing overall and moderator effects in random effects meta-regression. British Journal of Mathematical and Statistical Psychology, 64, 1–19. https://doi.org/10.1348/000711010X522687.

    Article  PubMed  Google Scholar 

  40. Kilgus, S. P., Fallon, L. M., & Feinberg, A. B. (2016). Function-based modification of check-in/check-out to influence escape-maintained behavior. Journal of Applied School Psychology, 32, 24–45. https://doi.org/10.1080/15377903.2015.1084965.

    Article  Google Scholar 

  41. Klingbeil, D. A., Dart, E. H., & Schramm, A. L. (2019). A systematic review of function-modified check-in/check-out. Journal of Positive Behavior Interventions, 21, 77–92. https://doi.org/10.1177/1098300718778032.

    Article  Google Scholar 

  42. Maggin, D. M., Zurheide, J., Pickett, K. C., & Baillie, S. J. (2015). A systematic evidence review of the check-in/check-out program for reducing student challenging behaviors. Journal of Positive Behavior Interventions, 17, 197–208. https://doi.org/10.1177/1098300715573630.

    Article  Google Scholar 

  43. Maggin, D. M., Pustejovsky, J. E., & Johnson, A. H. (2017). A meta-analysis of school-based group contingency interventions for students with challenging behavior: An update. Remedial and Special Education, 38, 353–370. https://doi.org/10.1177/0741932517716900.

    Article  Google Scholar 

  44. *March, R. E., & Horner, R. H. (2002). Feasibility and contributions of functional behavioral assessment in schools. Journal of Emotional and Behavioral Disorders, 10, 158–170.

  45. Martens, K., & Andreen, K. (2013). School counselors’ involvement with a school-wide positive behavior support intervention: Addressing student behavior issues in a proactive and positive manner. Professional School Counseling, 16, 313–322. https://doi.org/10.1177/2156759X1201600504.

    Article  Google Scholar 

  46. McDaniel, S. C., Houchins, D. E., & Robinson, C. (2016). The effects of check, connect, and expect on behavioral and academic growth. Journal of Emotional and Behavioral Disorders, 24, 42–53. https://doi.org/10.1177/1063426615573262.

    Article  Google Scholar 

  47. *McIntosh, K., Campbell, A. L., Carter, D. R., & Rossetto Dickey, C. (2009). Differential effects of a tier two behavior intervention based on function of problem behavior. Journal of Positive Behavior Interventions, 11, 82–93. https://doi.org/10.1177/1098300708319127.

  48. Miller, L. M., Dufrene, B. A., Olmi, D. J., Tingstrom, D., & Filce, H. (2015a). Self-monitoring as a viable fading option in check-in/check-out. Journal of School Psychology, 53, 121–135. https://doi.org/10.1016/j.jsp.2014.12.004.

    Article  PubMed  Google Scholar 

  49. Miller, L. M., Dufrene, B. A., Sterling, H. E., Olmi, D. J., & Bachmayer, E. (2015b). The effects of check-in/check-out on problem behavior and academic engagement in elementary school students. Journal of Positive Behavior Interventions, 17, 28–38. https://doi.org/10.1177/1098300713517141.

    Article  Google Scholar 

  50. Mitchell, B. S., Stormont, M., & Gage, N. A. (2011). Tier two interventions implemented within the context of a tiered prevention framework. Behavioral Disorders, 36, 241–261. https://doi.org/10.1177/019874291103600404\.

    Article  Google Scholar 

  51. Mitchell, B. S., Adamson, R., & McKenna, J. W. (2017). Curbing our enthusiasm: An analysis of the check-in/check-out literature using the council for exceptional children’s evidence-based practice standards. Behavior Modification, 41, 343–367. https://doi.org/10.1177/0145445516675273.

    Article  PubMed  Google Scholar 

  52. Mong, M. D., Johnson, K. N., & Mong, K. W. (2011). Effects of check-in/check-out on behavioral indices and mathematics generalization. Behavioral Disorders, 36, 225–240.

    Article  Google Scholar 

  53. Reynolds, C., & Kamphaus, R. (2004). Behavior assessment system for children-second edition (BASC-2). Circle Pines, MN: American Guidance Services.

    Google Scholar 

  54. Rodriguez, B. J., Campbell, A., Fairbanks Falcon, S., & Borgmeier, C. (2015). Examination of critical features and lessons learned for implementation of a tier 2 intervention system for social behavior. Journal of Educational and Psychological Consultation, 25, 224–251. https://doi.org/10.1080/10474412.2014.929953.

    Article  Google Scholar 

  55. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638–641. https://doi.org/10.1037/0033-2909.86.3.638.

    Article  Google Scholar 

  56. Ruiz, M. I., Smith, T. N., Naquin, G. M., Morgan-D’atrio, C., & Dellinger, A. B. (2014). Assessing the implementation fidelity of check-in check-out behavioral interventions in elementary and middle schools. Preventing School Failure: Alternative Education for Children and Youth, 58, 42–49. https://doi.org/10.1080/1045988X.2012.755667.

    Article  Google Scholar 

  57. Scott, T. M., Alter, P. J., Rosenberg, M., & Borgmeier, C. (2010). Decision-making in secondary and tertiary interventions of school-wide systems of positive behavior support. Education and Treatment of Children, 33, 513–535. https://doi.org/10.1353/etc.2010.0003.

    Article  Google Scholar 

  58. *Simonsen, B., Myers, D., & Briere III, D. E. (2011). Comparing a behavioral check-in/check-out (CICO) intervention to standard practice in an urban middle school setting using an experimental group design. Journal of Positive Behavior Interventions, 13, 31–48. https://doi.org/10.1177/1098300709359026.

  59. Sugai, G., Horner, R. H., & Todd, A. W. (2000). PBIS SelfAssessment survey 2.0. Eugene, OR: Educational and Community Supports Available from http://www.pbisapps.org.

    Google Scholar 

  60. Sugai, G., Lewis-Palmer, T. L., Todd, A. W., & Horner, R. H. (2001). School-wide evaluation tool (SET). Eugene, OR: Educational and community supports. Available from http://www.pbis.org.

  61. Sugai, G., Horner, R., Lewis-Palmer, T., & Rossetto Dickey, C. (2014a). PBIS team implementation checklist (TIC 3.1). Eugene, OR: Educational and Community Supports Available from http://www.pbisapps.org.

    Google Scholar 

  62. Sugai, G., Simonsen, B., Bradshaw, C., Horner, R., & Lewis, T. J. (2014b). Delivering high quality school-wide positive behavior support. In J. McLeskey, N. L. Waldron, F. Spooner, & B. Algozzine (Eds.), Handbook of effective inclusive schools: Research and practice (pp. 306–321). New York, NY: Routledge.

    Google Scholar 

  63. Swoszowski, N. C., McDaniel, S. C., Jolivette, K., & Melius, P. (2013). The effects of tier II check-in/check-out including adaptation for non-responders on the off-task behavior of elementary students in a residential setting. Education and Treatment of Children, 36, 63–79. https://doi.org/10.1353/etc.2013.0024.

    Article  Google Scholar 

  64. Swoszowski, N. C., Evanovich, L. L., Ennis, R. P., & Jolivette, K. (2017). Evaluating implementation of check in/check out in alternative educational settings: Stakeholder perspectives. Residential Treatment for Children & Youth, 34, 107–121. https://doi.org/10.1080/0886571X.2017.1283283.

    Article  Google Scholar 

  65. Ttofi, M. M., & Farrington, D. P. (2012). Bullying prevention programs: The importance of peer intervention, disciplinary methods and age variations. Journal of Experimental Criminology, 8, 443–462. https://doi.org/10.1007/s11292-012-9161-0.

    Article  Google Scholar 

  66. Virués-Ortega, J. (2010). Applied behavior analytic intervention for autism in early childhood: Meta-analysis, meta-regression and dose–response meta-analysis of multiple outcomes. Clinical Psychology Review, 30, 387–399. https://doi.org/10.1016/j.cpr.2010.01.008.

    Article  PubMed  Google Scholar 

  67. Walker, H. M., Kavanagh, K., Stiller, B., Golly, A., Severson, H. H., & Feil, E. G. (1998). First step to success: An early intervention approach for preventing school antisocial behavior. Journal of Emotional and Behavioral Disorders, 6, 66–80. https://doi.org/10.1177/106342669800600201.

    Article  Google Scholar 

  68. Walker, V. L., Chung, Y. C., & Bonnet, L. K. (2018). Function-based intervention in inclusive school settings: A meta-analysis. Journal of Positive Behavior Interventions, 20, 203–216. https://doi.org/10.1177/1098300717718350.

    Article  Google Scholar 

  69. Weber, M. A., Rich, S. E., Gann, C. J., Duhon, G. J., & Kellen, S. S. (2019). Can less be more for students at-risk for emotional and behavioral disorders: Evaluating components of check-in/check-out. Education and Treatment of Children, 42, 469–488. https://doi.org/10.1353/etc.2019.0022.

    Article  Google Scholar 

  70. What Works Clearinghouse. (2020). Standards handbook (version 4.01). Washington, DC: Institute of Education Sciences.

  71. White, W. A. (1988). A meta-analysis of the effects of direct instruction in special education. Education and Treatment of Children, 11, 364–374.

    Google Scholar 

  72. Wolfe, K., Pyle, D., Charlton, C. T., Sabey, C. V., Lund, E. M., & Ross, S. W. (2016). A systematic review of the empirical support for check-in check-out. Journal of Positive Behavior Interventions, 18, 74–88. https://doi.org/10.1177/1098300715595957.

    Article  Google Scholar 

Download references

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Eun-Young Park.

Ethics declarations

Conflict of Interest

The authors declare that they have no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Park, E., Blair, K.C. Check-in/Check-out Implementation in Schools: a Meta-Analysis of Group Design Studies. Educ. Treat. Child. (2020). https://doi.org/10.1007/s43494-020-00030-2

Download citation

Keywords

  • Check-in/check-out
  • Behavior education program
  • Tier 2
  • Group design
  • Meta-analysis