Abstract
Schools, like other service sectors, are confronted with an implementation gap, with the slow adoption and uneven implementation of evidence-based practices (EBP) as part of routine service delivery, undermining efforts to promote better youth behavioral health outcomes. Implementation researchers have undertaken systematic efforts to publish taxonomies of implementation strategies (i.e., methods or techniques that are used to facilitate the uptake, use, and sustainment of EBP), such as the Expert Recommendations for Implementing Change (ERIC) Project. The 73-strategy ERIC compilation was developed in the context of healthcare and largely informed by research and practice experts who operate in that service sector. Thus, the comprehensibility, contextual appropriateness, and utility of the existing compilation to other service sectors, such as the educational setting, remain unknown. The purpose of this study was to initiate the School Implementation Strategies, Translating ERIC Resources (SISTER) Project to iteratively adapt the ERIC compilation to the educational sector. The results of a seven-step adaptation process resulted in 75 school-adapted strategies. Surface-level changes were made to the majority of the original ERIC strategies (52 out of 73), while five of the strategies required deeper modifications for adaptation to the school context. Six strategies were deleted and seven new strategies were added based on existing school-based research. The implications of this study’s findings for prevention scientists engaged in implementation research (e.g., creating a common nomenclature for implementation strategies) and limitations are discussed.
Similar content being viewed by others
References
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in child welfare. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.
Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9, 45.
Abrahamson, E. (2004). Change without pain: How managers can overcome initiative overload, organizational chaos, and employee burnout. Boston: Harvard Business School Press.
Barrera, M., & González-Castro, F. (2006). A heuristic framework for the cultural adaptation of interventions. Clinical Psychology: Science and Practice, 13, 311–316.
Barrish, H. H., Saunders, M., & Wolfe, M. D. (1969). Good behavior game. Effects of individual contingencies for group consequences and disruptive behavior in a classroom. Journal of Applied Behavioral Analysis, 2, 119–124.
Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30.
Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity for outcome research: Issues for the cultural adaptation and development of psychosocial treatments with Hispanics. Journal of Abnormal Child Psychology, 23, 67–82.
Bernal, G., Jimenez-Chafey, M. I., & Domenech-Rodríguez, M. M. (2009). Cultural adaptation of treatments: A resource for considering culture in evidence-based practice. Professional Psychology: Research and Practice, 40, 361–368.
Bookstein, A. (1990). Informetric distributions, part I: Unified overview. Journal of the American Society for Information Science, 41, 368–375.
Boyd, M. R., Powell, B. J., Endicott, D., & Lewis, C. C. (2017). A method for tracking implementation strategies: An exemplar implementing measurement-based care in community behavioral health clinics. Behavior Therapy. https://doi.org/10.1016/j.beth.2017.11.012.
Brackett, M. A., & Rivers, S. E. (2013). Transforming students’ lives with social and emotional learning. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 368–388). New York: Taylor & Francis.
Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133–148.
Bruns, E. J., Duong, M. T., Lyon, A. R., Pullmann, M. D., Cook, C. R., Cheney, D., & McCauley, E. (2016). Fostering SMART partnerships to develop an effective continuum of behavioral health services and supports in schools. American Journal of Orthopsychiatry, 86, 156–170.
Bunger, A. C., Powell, B. J., Robertson, H. A., MacDowell, H., Birken, S. A., & Shea, C. (2017). Tracking implementation strategies: A description of a practical approach and early findings. Health Research Policy and Systems, 15, 1–12.
Burnes, B. (2004). Emergent change and planned change: Competitors or allies? The case of XYZ construction. International Journal of Operations & Production Management, 24, 886–902.
Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79, 135–144.
Cook, C. R., Lyon, A. R., Kubergovic, D., Wright, D. B., & Zhang, Y. (2015). A supportive beliefs intervention to facilitate the implementation of evidence-based practices within a multi-tiered system of supports. School Mental Health, 7, 49–60.
Cook, C. R., Miller, F., Fiat, A., Renshaw, T. L., Frye, M., & Joseph, G. (2017a). Promoting secondary teachers’ well-being and intentions to implement evidence-based practices: Randomized evaluation of the ACHIEVER Resilience Curriculum. Psychology in the Schools, 54, 13–28.
Cook, C. R., Pauling, S., McCaslin, S., Larson, M., Thayer, A. J., & Fiat, A. (2017b). Brief reminders as a low threshold strategy to facilitate teacher delivery of evidence-based classroom practices. Manuscript in preparation.
Dalkey, N. C., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9, 458–467.
Damian, J., Gallo, J., Leaf, P., & Mendelson, T. (2017). Organizational and provider level factors in implementation of trauma-informed care after a city-wide training: An explanatory mixed methods assessment. BMC Health Services Research, 17, 750.
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Science, 4, 50.
Dart, E., Cook, C. R., Gresham, F. M., & Cheneir, J. (2012). Test driving to increase treatment integrity and student outcomes. School Psychology Review, 41, 467–481.
DiGennaro, F. D., Martens, B. K., & McIntyre, L. L. (2005). Increasing treatment integrity through negative reinforcement: Effects on teacher and student behavior. School Psychology Review, 34, 220–231.
Dingfelder, H. E., & Mandell, D. S. (2011). Bridging the research-to-practice gap in autism intervention: An application of diffusion of innovation theory. Journal of Autism and Developmental Disorders, 41, 597–609.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.
Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157.
Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2015). Going above and beyond for implementation: The development and validity testing of the Implementation Citizenship Behavior Scale (ICBS). Implementation Science, 10, 65.
Farmer, E. M., Burns, B. J., Phillips, S. D., Angold, A., & Costello, E. J. (2003). Pathways into and through mental health services for children and adolescents. Psychiatric Services, 54, 60–66.
Fishbein, D. H., Ridenour, T. A., Stahl, M., & Sussman, S. (2016). The full translational spectrum of prevention science: Facilitating the transfer of knowledge to practices and policies that prevent behavioral health problems. Translational Behavioral Medicine, 6, 5–16.
Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–232.
Flottorp, S. A., Oxman, A. D., Krause, J., et al. (2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science, 8, 1–11. https://doi.org/10.1186/1748-5908-8-35 [PMC free article] [PubMed] [CrossRef].
Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. E., Reddy, L. A., Rosenfield, S. A., et al. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77–100.
Franks, R., & Bory, T. (2015). Who supports the successful implementation and sustainability of evidence-based practices? Defining and understanding the roles of intermediary and purveyor organisations. New Directions for Child and Adolescent Development, 149, 41–56.
Gottfredson, D., & Gottfredson, G. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime & Delinquency, 39, 3–35.
Grol, R. (2001). Successes and failures in the implementation of evidence-based guidelines for clinical practice. Medical Care, 39, 46–54.
Herlihy, B., & Corey, G. (2006). Boundary issues in counseling: Multiple roles and responsibilities. Alexandria: American Counseling Association.
Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466.
Horner, R. H., Sugai, G., & Fixsen, D. L. (2017). Implementing effective educational practices at scales of social importance. Clinical Child & Family Psychology Review, 20, 25–35.
Johnson, L. D., Wehby, J. H., Symons, F. J., Moore, T. C., Maggin, D. M., & Sutherland, K. S. (2014). An analysis of preference relative to teacher implementation of intervention. The Journal of Special Education, 48, 214–224.
Kane, M., & Trochim, W. M. K. (2007). Concept mapping for planning and evaluation. Thousand Oaks: Sage Publications.
Kazdin, A. E. (2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 1–27.
Khamisa, N., Peltzer, K., & Oldenburg, B. (2013). Burnout in relation to specific contributing factors and health outcomes among nurses: A systematic review. International Journal of Environmental Research and Public Health, 10, 2214–2240.
Kohler, F., Crillery, K., Shearer, D., & Good, G. (1997). Effects of peer coaching on teacher and student outcomes. Journal of Educational Research, 90, 240–250.
Krause, J., Van Lieshout, J., Klomp, R., Huntink, E., Aakhus, E., Flottorp, S., et al. (2014). Identifying determinants of care for tailoring implementation in chronic diseases: An evaluation of different methods. Implementation Science, 9, 102.
Leeman, J., Birken, S. A., Powell, B. J., Rohweder, C., & Shea, C. M. (2017). Beyond “implementation strategies”: Classifying the full range of strategies used in implementation science and practice. Implementation Science, 12, 125–134.
Leong, F. T. (1996). Toward an integrative model of cross-cultural counseling and psychotherapy. Applied and Preventive Psychology, 5, 189–209.
Lewis, C. C. (2017). Systematic review of D & I Mechanisms in health. Keynote address at 4th biennial Society for Implementation Research Collaboration Conference.
Lewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., et al. (2018). From classification to causality: advancing understanding of mechanisms of change in implementation science. Frontiers of Public Health, 6, 1–6.
Lochman, J. E., & Wells, K. C. (2002). Contextual social–cognitive mediators and child outcome: A test of the theoretical model in the Coping Power program. Development and Psychopathology, 14, 945–967.
Locke, J., Beidas, R. S., Marcus, S., Stahmer, A., Aarons, G. A., Lyon, A. R., et al. (2016). A mixed methods study of individual and organizational factors that affect implementation of interventions for children with autism in public schools. Implementation Science, 11, 135.
Lyon, A. R., & Bruns, E. J. (2019). From evidence to impact: Joining our best school mental health practices with our best implementation strategies. School Mental Health, 11, 106–114.
Lyon, A. R., Lewis, C. C., Melvin, A., Boyd, M., Nicodimos, S., Liu, F. F., & Jungbluth, N. (2016). Health Information Technologies—Academic and Commercial Evaluation (HIT-ACE) methodology: Description and application to clinical feedback systems. Implementation Science, 11, 128.
Lyon, A. R., Pullmann, M. D., Walker, S. C., & D’Angelo, G. (2017). Community-sourced intervention programs: Review of submissions in response to a statewide call for “promising practices.”. Administration and Policy in Mental Health and Mental Health Services Research, 44, 16–28.
Lyon, A., Cook, C. R., Brown, E., Locke, J., Ehrhart, M., Davis, C., & Aarons, G. (2018). Assessing the organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13, 5.
Lyon, A. R., Comtois, K. A., Kerns, S. E. U., Landes, S. J., & Lewis, C. C. (2019). Closing the science-practice gap in implementation before it widens. In A. Shlonsky, R. Mildon, & B. Albers (Eds.), The science of implementation. Springer (in press).
McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. American Psychologist, 65, 73–84.
McKleroy, V. S., Galbraith, J. S., Cummings, B., Jones, P., Harshbarger, C., et al. (2006). Adapting evidence-based behavioral interventions for new settings and target populations. AIDS Education & Prevention, 18, 59–73.
Morina, N., Koerssen, R., & Pollet, T. V. (2016). Interventions for children and adolescents with posttraumatic stress disorder: A meta-analysis of comparative outcome studies. Clinical Psychology Review, 47, 41–54.
Nadeem, E., & Ringle, V. (2016). De-adoption of an evidence-based trauma intervention in schools: A retrospective report from an urban school district. School Mental Health, 8, 132–143.
Novins, D. K., Green, A. E., Legha, R. K., et al. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child & Adolescent Psychiatry, 52, 1009–1025.
Owens, J., Lyon, A. R., Brandt, N. E., Maisa Warner, M., Nadeem, E., Spiel, C., & Wagner, M. (2014). Implementation science in school mental health: Key constructs and a proposed research agenda. School Mental Health, 6, 99–111.
Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., et al. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69, 123–157.
Powell, B. J., Proctor, E. K., & Glass, J. E. (2014). A systematic review of strategies for implementing empirically supported mental health interventions. Research Social Work & Practice., 24, 192–212.
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., et al. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 1–14.
Powell, B. J., Beidas, R., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., & Mandell, D. S. (2017). Methods to improve the selection and tailoring of implementation strategies. The Journal of Behavioral Health Services & Research, 44, 177–194.
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, C. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36, 24–34.
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8, 1–11.
Resnicow, K., Baranowski, T., Ahluwalia, J., & Braithwaite, R. (1990). Cultural sensitivity in public health: Defined and demystified. Ethnicity and Disease, 9, 10–21.
Ringwalt, C. L., Vincus, A., Ennett, S., Johnson, R., & Rohrbach, L. A. (2004). Reasons for teachers’ adaptation of substance use prevention curricula in schools with non-white student populations. Prevention Science, 5, 61–67.
Ringwalt, C. R., Vincus, A., Ennett, S. T., Hanley, S., Bowling, J. M., & Rohrbach, L. A. (2009). The prevalence of evidence-based substance use prevention curricula in U.S. middle schools in 2005. Prevention Science, 10, 33–40.
Russ, S. J., Sevdalis, N., Moorthy, K., et al. (2014). A qualitative evaluation of the barriers and facilitators toward implementation of the WHO surgical safety checklist across hospitals in England: Lessons from the “surgical checklist implementation project”. Annual Surgery, 261, 81–91.
Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S. A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73, 106–115.
St. Peter Pipkin, C., Vollmer, T. R., & Sloman, K. N. (2010). Effects of treatment integrity failures during differential reinforcement of alternative behavior: A translational model. Journal of Applied Behavior Analysis, 43, 47–70.
Stahmer, A. C., Reed, S., & Lee, E. (2015). Training teachers to use evidence-based practices for autism: Examining procedural implementation fidelity. Psychology in the School, 52, 181–195.
Stetler, C. B., Legro, M. W., Rycroft-Malone, J., Bowman, C., Curran, G., & Guihan, M. (2006). Role of “external facilitation” in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science, 1, 23.
Urbach, J., Moore, B. A., Klinger, J. K., Galman, S., Haager, D., Brownell, M. T., & Dingle, M. (2015). “That’s my job”: Comparing the beliefs of more and less accomplished special educators related to their roles and responsibilities. Teacher Education and Special Education, 38, 323–336.
Vescio, V., Ross, D., & Adams, A. (2008). A review of research on the impact of professional learning communities on teaching practice and student learning. Teaching and Teacher Education, 24, 80e91.
Walker, V. L., Chung, Y.C., & Bonnet, L. K. (2017). Function-based intervention in inclusive school settings: A meta-analysis. Journal of Positive Behavior Interventions.
Waltz, T. J., Powell, B. J., Matthieu, M. M., Damschroder, L. J., Chinman, M. J., Smith, J. L., et al. (2015). Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: Results from the Expert Recommendations for Implementing Change (ERIC) study. Implementation Science, 10, 1–8.
Weiner, B. J., Lewis, C. C., Stanick, C. F., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12, 1–12.
Weisz, J., & Kazdin, A. (2017). Evidence-based psychotherapies for children and adolescents. Guilford.
Wensing, M., & Grol, R. (2005). Methods to identify implementation problems. In R. Grol, M. Wensing, & M. Eccles (Eds.), Improving patient care: The implementation of change in clinical practice (pp. 109–120). Edinburgh: Elsevier.
Wensing, M., Oxman, A., Baker, R., et al. (2011). Tailored implementation for chronic diseases (TICD): A project protocol. Implementation Science, 6, 1–8.
Williams, N. J. (2016). Multilevel mechanisms of implementation strategies in mental health: Integrating theory, research, and practice. Administration and Policy in Mental Health and Mental Health Services Research, 43, 783–798.
Funding
This publication was supported in part by funding from the Institute of Education Sciences (R305A160114 PIs - Lyon and Cook; R305A170292 PIs – Cook and Lyon). BJP was supported by K01MH113806, R25MH080916, and UL1TR001111. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or Institute of Education Sciences.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Ethical Approval
Exempt status was obtained from the university IRB prior to conducting the study.
Informed Consent
No informed consent was necessary as part of this study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Cook, C.R., Lyon, A.R., Locke, J. et al. Adapting a Compilation of Implementation Strategies to Advance School-Based Implementation Research and Practice. Prev Sci 20, 914–935 (2019). https://doi.org/10.1007/s11121-019-01017-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-019-01017-1