Advertisement

Implementing Effective Educational Practices at Scales of Social Importance

  • Robert H. Horner
  • George Sugai
  • Dean L. Fixsen
Article

Abstract

Implementing evidence-based practices is becoming both a goal and standard across medicine, psychology, and education. Initial successes, however, are now leading to questions about how successful demonstrations may be expanded to scales of social importance. In this paper, we review lessons learned about scaling up evidence-based practices gleaned from our experience implementing school-wide positive behavioral interventions and supports (PBIS) across more than 23,000 schools in the USA. We draw heavily from the work of Flay et al. (Prev Sci 6:151–175, 2005. doi: 10.1007/s11121-005-5553-y) related to defining evidence-based practices, the significant contributions from the emerging “implementation science” movement (Fixsen et al. in Implementation research: a synthesis of the literature, University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231), Tampa 2005), and guidance we have received from teachers, family members, students, and administrators who have adopted PBIS.

Keywords

Evidence-based practices Scaling up Implementation science 

Notes

Acknowledgements

Development of this paper was supported by the Office of Special Education Programs U.S. Department of Education (H326S980003 and H326K120005). The authors extend appreciation to Dr. Karen Blase for editorial feedback on initial drafts of this manuscript.

Compliance with Ethical Standards

Conflict of interest

The three authors each serve as principal investigators on federal technical assistance grants targeting large-scale implementation of evidence-based practices. The first two authors direct OSEP Grant H326K120005, and the third author directs OSEP Grant H326S980003.

Research Involving Human Subjects

For the results reported in this paper, no formal consent of human participants was required.

References

  1. Alberto, P. A., & Troutman, A. C. (2007). Applied behavior analysis for teachers (8th ed.). Upper Saddle River, NJ: Merrill.Google Scholar
  2. APA Publications and Communications Board Working Group on Journal Article Reporting Standards. (2008). Reporting standards for research in Psychology: Why do we need them? What might they be? American Psychologist, 63, 839–851.CrossRefPubMedCentralGoogle Scholar
  3. Biglan, A. (1995). Changing cultural practices: A contextualist framework for intervention research. Reno, NV: Context Press.Google Scholar
  4. Biglan, A., Glasgow, R. E., & Singer, G. (1990). The need for a science of larger social units: A contextual approach. Behavior Therapy, 21, 195–215. doi: 10.1016/S0005-7894(05)80277-1.CrossRefGoogle Scholar
  5. Biglan, A., Cody, C., Aldridge, W. A., II, Dabroski, A., & Kjellstrand, J. (2011). The promise neighborhoods initiative: Improving developmental outcomes through comprehensive interventions. The Journal of the Institute for Comprehensive Community Development, 2, 25–34.Google Scholar
  6. Blase, K., & Fixsen, D. (2013). Core intervention components: Identifying and operationalizing what makes programs work. Washington, DC: ASPE Research Brief, Office of the Assistant Secretary for Planning and Evaluation, Office of Human Services Policy, U.S. Department of Health and Human Services.Google Scholar
  7. Blase, K. A., Fixsen, D. L., Sims, B. J., & Ward, C. S. (2015). Implementation science Changing hearts, minds, behavior, and systems to improve educational outcomes. Berkeley, CA: Paper presented at the Wing Institute’s Ninth Annual Summit on Evidence-based Education. Retrieved from http://nirn.fpg.unc.edu/resources/implementation-science%09changing-hearts-minds-behavior-and-systems-to-improve.
  8. Bradshaw, C., Koth, C., Bevans, K., Ialongo, N., & Leaf, P. (2008a). The impact of school-wide positive behavioral interventions and supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23, 462–473. doi: 10.1037/a0012883.CrossRefGoogle Scholar
  9. Bradshaw, C. P., Reinke, W. M., Brown, L. D., Bevans, K. B., & Leaf, P. J. (2008b). Implementation of school-wide positive behavioral interventions and supports (PBIS) in elementary schools: Observations from a randomized trial. Education and Treatment of Children, 31, 1–26. doi: 10.1353/etc.0.0025.CrossRefGoogle Scholar
  10. Bradshaw, C. P., Koth, C. W., Thornton, L. A., & Leaf, P. J. (2009). Altering school climate through school-wide positive behavioral interventions and supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10, 100–115. doi: 10.1007/s11121-008-0114.9.CrossRefPubMedGoogle Scholar
  11. Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of school-wide positive behavioral interventions and supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133–148. doi: 10.1177/1098300709334798.CrossRefGoogle Scholar
  12. Carr, E. G., Horner, R. H., Turnbull, A. P., Marquis, J. G., McLaughlin, D. M., McAtee, M. L., et al. (1999). Positive behavior support for people with developmental disabilities: A research synthesis. Washington, DC: American Association on Mental Retardation Monograph Series.Google Scholar
  13. Childs, K., Kincaid, D., George, H., & Gage, N. (2016). The relationship between school-wide implementation of positive behavior interventions and supports and student discipline outcomes. Journal of Positive Behavior Interventions, 18, 89–99. doi: 10.1177/1098300715590398.CrossRefGoogle Scholar
  14. Crone, D. A., & Horner, R. H. (2003). Building positive behavior support systems in schools: Functional behavioral assessment. New York, NY: Guilford.Google Scholar
  15. Crone, D., Hawken, L., & Horner, R. H. (2010). Responding to problem behavior in schools: The behavior education program (2nd edn). The Guilford Practical Intervention in the Schools Series. New York, NY: Guilford Press.Google Scholar
  16. Didden, R., Duker, P., & Korzilius, H. (1997). Meta-analytic study on treatment effectiveness for problem behavior with individuals who have mental retardation. American Journal on Mental Retardation, 101, 387–399.PubMedGoogle Scholar
  17. Embry, D. D. (2004). Community-based prevention using simple, low-cost, evidence-based kernels and behavior vaccines. Journal of Community Psychology, 32, 575–591. doi: 10.1002/jcop.20020.CrossRefGoogle Scholar
  18. Embry, D. D., & Biglan, A. (2008). Evidence-based kernels: Fundamental units of behavioral influence. Clinical Child and Family Psychology Review, 11, 75. doi: 10.1007/s10567-008-0036-x.CrossRefPubMedPubMedCentralGoogle Scholar
  19. Fallon, L. M., O’Keeffe, B. V., & Sugai, G. (2012). Consideration of culture and context in school-wide positive behavior support: A review of current literature. Journal of Positive Behavior Interventions, 14, 209–219. doi: 10.1177/1098300712442242.CrossRefGoogle Scholar
  20. Fisher, E., Shortell, S., & Savitz, S. (2016). Implementation science: A potential catalyst for delivery system reform. Journal of the American Medical Association, 315, 339–340. doi: 10.1001/jama.2015.17949.CrossRefPubMedGoogle Scholar
  21. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature (Vol. 11, pp. 247–266). Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).Google Scholar
  22. Fixsen, D. L., Blase, K. A., Horner, R. H., & Sugai, G. (2010). An overview of the state implementation and scaling up of evidence-based practices center (SISEP): Scaling up brief, 4. Chapel Hill, NC: FPG Child Development Institute, University of North Carolina Chapel Hill.Google Scholar
  23. Fixsen, D. L., Blase, K. A., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230. doi: 10.1177/001440291307900206.CrossRefGoogle Scholar
  24. Flannery, K. B., Frank, J. L., Doren, B., Kato, M. M., & Fenning, P. (2013). Implementing schoolwide-positive behavior support in high school settings: Analysis of eight high schools. The High School Journal, 96, 267–282. doi: 10.1353/hsj.2013.0015.CrossRefGoogle Scholar
  25. Flay, B., Biglan, A., Boruch, R., Castro, F., Gottfredson, D., Kellam, S., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151–175. doi: 10.1007/s11121-005-5553-y.CrossRefPubMedGoogle Scholar
  26. Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18, 37–50.Google Scholar
  27. Gresham, F. M. (2004). Current status and future directions of school-based behavioral interventions. School Psychology Review, 33, 326–343.Google Scholar
  28. Hagermoser Sanetti, L. M., & Kratochwill, T. R. (Eds.). (2014). Treatment integrity: A foundation for evidence-based practice in applied psychology. Washington, DC: American Psychological Association.Google Scholar
  29. Horner, R. H., Todd, A. W., Lewis-Palmer, T., Irvin, L. K., Sugai, G., & Boland, J. B. (2004). The school-wide evaluation tool (SET): A research instrument for assessing school wide positive behavior support. Journal of Positive Behavior Interventions, 6, 3–12. doi: 10.1177/10983007040060010201.CrossRefGoogle Scholar
  30. Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., et al. (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–145. doi: 10.1177/1098300709332067.CrossRefGoogle Scholar
  31. Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42, 1–14.Google Scholar
  32. Horner, R. H., Kincaid, D., Sugai, G., Lewis, T., Eber, L., Barrett, S., et al. (2014). Scaling up school-wide positive behavioral interventions and supports: The experiences of seven states with documented success. Journal of Positive behavioral Interventions, 16, 197–208. doi: 10.1177/1098300713503685.CrossRefGoogle Scholar
  33. Ingram, K., Lewis-Palmer, T., & Sugai, G. (2005). Function-based intervention planning: Comparing the effectiveness of FBA indicated and contra-indicated intervention plans. Journal of Positive Behavior Interventions, 7, 224–236.CrossRefGoogle Scholar
  34. Irvin, L. K., Tobin, T. J., Sprague, J. R., Sugai, G., & Vincent, C. G. (2004). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions, 6, 131–147. doi: 10.1177/10983007040060030201.CrossRefGoogle Scholar
  35. Jones, C., Caravaca, L., Cizek, S., Horner, R. H., & Vincent, C. G. (2006). Culturally responsive schoolwide positive behavior support: A case study in one school with a high proportion of Native American students. Multiple Voices for Ethnically Diverse Exceptional Learners, 9, 108–119.Google Scholar
  36. Kelm, J. L., McIntosh, K., & Cooley, S. (2014). Effects of implementing school-wide positive behavioural interventions and supports on problem behaviour and academic achievement in a Canadian elementary school. Canadian Journal of School Psychology, 29, 195–212. doi: 10.1177/0829573514540266.CrossRefGoogle Scholar
  37. Klingner, J. K., Artiles, A. J., Kozleski, A., Harry, B., Zion, S., Tate, W., et al. (2005). Addressing the disproportionate representation of culturally and linguistically diverse students in special education through culturally responsive educational systems. Education Policy Analysis Archives, 13, 1–42.CrossRefGoogle Scholar
  38. Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single case designs technical documentation. Retrieved from What Works Clearinghouse website: http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.
  39. Latham, G. I. (1992). Interacting with at-risk children: The positive approach. Principal, 72, 26–30.Google Scholar
  40. Lau, A. S. (2006). Making the case for selective and directed cultural adaptations of evidence based treatments: Examples from parent training. Clinical Psychology: Science and Practice, 13, 295–310. doi: 10.1111/j.1468-2850.2006.00042.x.Google Scholar
  41. Lewis, T. J., & Sugai, G. (1999). Effective behavior support: A systems approach to proactive school-wide management. Focus on Exceptional Children, 31, 1–24.Google Scholar
  42. Maggin, D. M., Zurheide, J., Pickett, K. C., & Baillie, S. J. (2015). A systematic evidence review of the Check-In/Check-Out program for reducing student challenging behaviors. Journal of Positive Behavior Interventions, 17, 197–208. doi: 10.1177/1098300715573630.CrossRefGoogle Scholar
  43. Mayer, G. R. (1995). Preventing anti-social behavior in the schools. Journal of Applied Behavior Analysis, 28, 267–278.CrossRefGoogle Scholar
  44. McIntosh, K., & Turri, M. G. (2014). Positive behavior support: Sustainability and continuous regeneration. In C. R. Reynolds, K. J. Vannest, & E. Fletcher-Janzen (Eds.), Encyclopedia of special education: A reference for the education of children, adolescents, and adults with disabilities and other exceptional individuals (4th ed., pp. 2061–2064). Hoboken, NJ: Wiley. doi: 10.1002/9781118660584.ese1903.Google Scholar
  45. McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., & Mathews, S. (2013). Factors related to sustained implementation of school-wide positive behavior interventions and supports. Exceptional Children, 79, 293–311.Google Scholar
  46. McIntosh, K., Kim, J., Mercer, S. H., Strickland-Cohen, M. K., & Horner, R. H. (2015). Variables associated with enhanced sustainability of school-wide positive behavior interventions and supports. Assessment for Effective Intervention, 40, 184–191. doi: 10.1177/1534508414556503.CrossRefGoogle Scholar
  47. McIntosh, K., Massar, M. M., Algozzine, R. F., George, H. P., Horner, R. H., Lewis, T. J., et al. (2016). Technical adequacy of the SWPBIS tiered fidelity inventory. Journal of Positive Behavior Interventions. doi: 10.1177/1098300716637193.Google Scholar
  48. Metz, A., & Easterling, D. (2016). Using implementation science to translate foundation strategy. The Foundation Review, 8, 12. doi: 10.9707/1944-5660.1302.Google Scholar
  49. Meyer, L. H., & Evans, I. M. (1989). Non aversive intervention for behavior problems. Baltimore, MD: Paul H. Brookes.Google Scholar
  50. Moreno, G., Wong-Lo, M., & Bullock, L. M. (2014). Assisting students from diverse backgrounds with challenging behaviors: Incorporating a culturally attuned functional behavioral assessment in prereferral services. Preventing school Failure: Alternative Education for Children and Youth, 58, 58–68. doi: 10.1080/1045988X.2012.763156.CrossRefGoogle Scholar
  51. Newton, J. S., Horner, R. H., Algozzine, R. R., Todd, A. W., & Algozzine, K. M. (2009). Using a problem-solving model to enhance data-based decision making in schools. In W. Sailor, G. Dunlap, G. Sugai, & R. Horner (Eds.), Handbook of positive behavior support (pp. 551–580), a volume in the Issues in Clinical Child Psychology series. New York, NY: Springer. doi: http://dx.doi.org/10.1007/978-0-387-09632-2_23.
  52. O’Neill, R. E., Albin, R. W., Storey, K., Horner, R. H., & Sprague, J. R. (Eds.). (2015). Functional assessment and program development for problem behavior: A practical handbook, instructor’s edition (3rd ed.). Stamford, CT: Cengage Learning.Google Scholar
  53. Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–149. doi: 10.1177/001440290507100201.CrossRefGoogle Scholar
  54. Seligman, M. E. P., & Csikszentmihalyi, M. (2000). Positive psychology: An introduction. American Psychologist, 55, 5–14. doi: 10.1037/0003-066X.55.1.5.CrossRefPubMedGoogle Scholar
  55. Simonsen, B., & Sugai, G. (2013). PBIS in alternative educational settings: Positive support for youth with high-risk behavior. Education and Treatment of Children, 36, 3–14.CrossRefGoogle Scholar
  56. Skinner, B. F. (1953). Science and human behavior. New York, NY: Simon and Schuster.Google Scholar
  57. Sprick, R., Sprick, M., & Garrison, M. (1992). Foundations: Developing positive school-wide discipline policies. Longmont, CO: Sopris West.Google Scholar
  58. Sugai, G., & Horner, R. H. (1999). Discipline and behavioral support: Preferred processes and practices. Effective School Practices, 17, 10–22.Google Scholar
  59. Sugai, G., & Horner, R. H. (2009). Defining and describing schoolwide positive behavior support. In W. Sailor, G. Dunlap, G. Sugai, & R. Horner (Eds.), Handbook of positive behavior support (pp. 307–326). New York, NY: Springer.CrossRefGoogle Scholar
  60. Sugai, G., O’Keeffe, B. V., & Fallon, L. M. (2012). A contextual consideration of culture and school-wide positive behavior support. Journal of Positive Behavior Interventions, 14, 197–208. doi: 10.1177/1098300711426334.CrossRefGoogle Scholar
  61. Sugai, G., Horner, R. H. & Lewis, T. (2015). Behavioral interventions and supports implementation blueprint. Retrieved from www.pbis.org.
  62. Tobin, T. J., Vincent, C. G., Horner, R. H., Rossetto Dickey, C., & May, S. A. (2012). Fidelity measures to improve implementation of behavioural support. International Journal of Positive Behavioural Support, 2, 12–19.Google Scholar
  63. Todd, A. W., Horner, R. H., Newton, J. S., Algozzine, R. F., Algozzine, K. M., & Frank, J. L. (2011). The effects of using team-initiated problem solving (TIPS) on problem solving practices of school-wide behavior support teams. Journal of Applied School Psychology, 27, 42–59.CrossRefGoogle Scholar
  64. Waasdorp, T., Bradshaw, C. P., & Leaf, P. J. (2012). The impact of school-wide positive behavioral interventions and support (SWPBIS) on bullying and peer rejection: A randomized control effectiveness trial. Archive of Pediatric Adolescent Medicine, 166, 149–156.CrossRefGoogle Scholar
  65. Walker, H., Kavanagh, K., Stiller, B., Golly, A., Severson, H., & Feil, E. (1998). First step to success: An early intervention approach for preventing school antisocial behavior. Journal of Emotional and Behavioral Disorders, 6, 66–80. doi: 10.1177/106342669800600201.CrossRefGoogle Scholar
  66. Walker, H. M., Seeley, J. R., Small, J., Severson, H. H., Graham, B. A., Feil, E. G., et al. (2009). A randomized controlled trial of the first step to success early intervention: Demonstration of program efficacy outcomes in a diverse, urban school district. Journal of Emotional and Behavioral Disorders, 17, 197–212. doi: 10.1177/1063426609341645.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.University of OregonEugeneUSA
  2. 2.University of ConnecticutStorrsUSA
  3. 3.University of North CarolinaChapel HillUSA

Personalised recommendations