Abstract
Key features of the school environment can have a significant impact on teachers’ effective use of evidence-based practices (EBP), yet implementation-specific organizational constructs have rarely been studied in the education sector. This study examined three aspects of the organizational implementation context (implementation leadership, climate, and citizenship behavior), which have been conceptualized and validated in other service settings. Focus groups with central office administrators, principals, and teachers were conducted to understand the applicability and conceptual boundaries of these organizational constructs in schools. Focus group transcripts were coded, and the results indicated both similarities and differences in their conceptualizations of implementation leadership, climate, and citizenship behavior in school. The data indicated that: (1) implementation leadership was largely present in schools with the addition of Distributed Leadership; (2) two implementation climate constructs were most clearly present (i.e., Focus on EBP and Educational Support for EBP) and two additional constructs (i.e., Existing Support to Deliver EBP and Prioritization of EBP) emerged as part of this construct; and (3) implementation citizenship behavior (Helping Others and Keeping Informed) was consistently acknowledged across schools and two new components emerged (i.e., Information Sharing and Observation/Feedback). Recommendations to researchers and community stakeholders are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014a). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9, 45.
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014b). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274.
Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017). The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatric Services, 68, 115–122.
Aarons, G. A., Green, A. E., Palinkas, L. A., Self-Brown, S., Whitaker, D. J., Lutzker, J. R., et al. (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7, 1–9.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.
Aitken, L. M., Hackwood, B., Crouch, S., Clayton, S., West, N., Carney, D., et al. (2011). Creating an environment to implement and sustain evidence-based practice: A developmental process. Australian Critical Care, 24, 244–254.
Angelle, P. S. (2010). An organizational perspective of distributed leadership: A portrait of a middle school. Rmle Online, 33, 1–16.
APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. The American Psychologist, 61, 271.
Atkins, M., Frazier, S., Leathers, S., Graczyk, P., Talbott, E., Jakobsons, L., et al. (2008). Teacher key opinion leaders and mental health consultation in low-income urban schools. Journal of Consulting and Clinical Psychology, 76, 905–908.
Basch, C. E., Sliepcevich, E. M., Gold, R. S., Duncan, D. F., & Kolbe, L. J. (1985). Avoiding type III errors in health education program evaluations: A case study. Health Education Quarterly, 12, 315–331.
Becker-Haimes, E. M., Okamura, K. H., Wolk, C. B., Rubin, R., Evans, A. C., & Beidas, R. S. (2017). Predictors of clinician use of exposure therapy in community mental health settings. Journal of Anxiety Disorders, 49, 88–94.
Beidas, R. S., Aarons, G., Barg, F., Evans, A., Hadley, T., Hoagwood, K., et al. (2013). Policy to implementation: Evidence-based practice in community mental health—Study protocol. Implementation Science, 8, 1–9.
Beidas, R. S., Edmunds, J., Ditty, M., Watkins, J., Walsh, L., Marcus, S., et al. (2014). Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Administration and Policy In Mental Health, 41, 788–799.
Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood, K. E., Schoenwald, S., Evans, A. C., et al. (2016a). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169, 374–382.
Beidas, R. S., Stewart, R. E., Adams, D. R., Fernandez, T., Lustbader, S., Powell, B. J., et al. (2016b). A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research, 43, 893–908.
Birken, S. A., DiMartino, L. D., Kirk, M. A., Lee, S. Y. D., McClelland, M., & Albert, N. M. (2015). Elaborating on theory with middle managers’ experience implementing healthcare innovations in practice. Implementation Science, 11, 2.
Bonham, C., Willging, C., Sommerfeld, D., & Aarons, G. A. (2014). Organizational factors influencing implementation of evidence-based practices for integrated treatment in behavioral health agencies. Psychiatry Journal, 2014, 1–9.
Bradley, E. H., Curry, L. A., & Devers, K. J. (2007). Qualitative data analysis for health services research: Developing taxonomy, themes, and theory. Health Services Research, 42, 1758–1772.
Bruns, E. J., Duong, M. T., Lyon, A. R., Pullmann, M. D., Cook, C. R., Cheney, D., et al. (2016). Fostering SMART partnerships to develop an effective continuum of behavioral health services and supports in schools. American Journal of Orthopsychiatry, 86, 156.
Cook, C. R., Lyon, A. R., Locke, J., Waltz, T., & Powell, B. (submitted). Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prevention Science.
Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional children, 79, 135–144.
Costello, E. J., He, J., Sampson, N. A., Kessler, R. C., & Merikangas, K. R. (2014). Services for adolescents with psychiatric disorders: 12-month data from the National Comorbidity Survey-Adolescent. Psychiatric Services, 65, 359–366.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.
Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157.
Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2015). Going above and beyond for implementation: The development and validity testing of the Implementation Citizenship Behavior Scale (ICBS). Implementation Science, 10, 65.
Ehrhart, M. G., Aarons, G. A., Torres, E. M., Finn, N. K., & Roesch, S. C. (2016). Implementation in child welfare: Leadership, climate, attitudes, and citizenship behavior. In Poster presented at the 31st annual conference of the Society for Industrial and Organizational Psychology, Anaheim, CA.
Ehrhart, M. G., Torres, E. M., Green, A. E., Trott, E. M., Willging, C. E., Moullin, J. C., et al. (2018). Leading for the long haul: A mixed-method evaluation of the Sustainment Leadership Scale (SLS). Implementation Science, 13, 17.
Farmer, E. M. Z., Burns, B. J., Phillips, S. D., Angold, A., & Costello, E. J. (2003). Pathways into and through mental health services for children and adolescents. Prevention Science, 54, 60–66.
Feuerstein, J., Olswang, L. B., Greenslade, K., Pinder, G. L., Dowden, P., & Madden, J. (2017). Moving triadic gaze intervention into practice: Measuring clinician attitude and implementation fidelity. Journal of Speech, Language, and Hearing Research, 60, 1285–1298.
Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230.
Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida.
Forman, S. G., Olin, S. S., Hoagwood, K. E., Crowe, M., & Saka, N. (2009). Evidence-based interventions in schools: Developers’ views of implementation barriers and facilitators. School Mental Health, 1, 26–36.
Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. E., Reddy, L. A., Rosenfeld, S. A., et al. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77–100.
Gottfredson, G. D., & Gottfredson, D. C. (2001). What schools do to prevent problem behavior and promote safe environments. Journal of Educational and Psychological Consultation, 12, 313–344.
Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18, 59–82.
Hoagwood, K., & Johnson, J. (2003). School psychology: A public health framework: I. From evidence-based practices to evidence-based policies. Journal of School Psychology, 41, 3–21.
Jacobs, S. R., Weiner, B. J., & Bunger, A. C. (2014). Context matters: Measuring implementation climate among individuals and groups. Implementation Science, 9, 46.
Langer, D. A., Wood, J. J., Wood, P. A., Garland, A. F., Landsverk, J., & Hough, R. L. (2015). Mental health service use in schools and non-school-based outpatient settings: Comparing predictors of service use. School Mental Health, 7, 161–173.
Langley, A. K., Nadeem, E., Kataoka, S. H., Stein, B. D., & Jaycox, L. H. (2010). Evidence-based mental health programs in schools: Barriers and facilitators of successful implementation. School Mental Health, 2, 105–113.
Lewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018). From confusion to causality: Advancing understanding of mechanisms of change in implementation science.
Locke, J., Beidas, R. S., Marcus, S., Stahmer, A. S., Aarons, G. A., Lyon, A. R., et al. (2016). A mixed methods study of individual and organizational factors that affect implementation of interventions for children with autism in public schools. Implementation Science, 11, 135.
Locke, J., Olsen, A., Wideman, R., Downey, M. M., Kretzmann, M., Kasari, C., et al. (2015). A tangled web: The challenges of implementing an evidence-based social engagement intervention for children with autism in urban public school settings. Behavior Therapy, 46, 54–67.
Lyon, A., Cook, C., Brown, E., Locke, J., Davis, C., Ehrhart, M., et al. (2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13, 5.
Lyon, A. R., Cook, C. R., Locke, J., Powell, B., & Waltz, T. (submitted). Importance and feasibility of an adapted set of strategies for implementing evidence-based behavioral health practices in the education sector. Journal of School Psychology.
Lyon, A. R., Ludwig, K. A., Vander Stoep, A., Gudmundsen, G., & McCauley, E. (2013). Patterns and predictors of mental healthcare utilization in schools and other service sectors among adolescents at risk for depression. School Mental Health, 5, 155–165.
Malloy, M., Acock, A., DuBois, D. L., Vuchinich, S., Silverthorn, N., Ji, P., et al. (2015). Teachers’ perceptions of school organizational climate as predictors of dosage and quality of implementation of a social-emotional and character development program. Prevention Science, 16, 1086–1095.
Martinez, R. G., Lewis, C. C., & Weiner, B. J. (2014). Instrumentation issues in implementation science. Implementation Science, 9, 118.
Merikangas, K. R., He, J., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., et al. (2011). Service utilization for lifetime mental disorders in U.S. adolescents: Results of the National Comorbidity Survey-Adolescent Supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry, 50, 32–45.
Neale, J. (2016). Iterative categorization (IC): A systematic technique for analysing qualitative data. Addiction (Abingdon, England), 111(6), 1096–1106. https://doi.org/10.1111/add.13314.
Odom, S. L., Cox, A. W., Brock, M. E., & National Professional Development Center on ASD. (2013). Implementation science, professional development, and autism spectrum disorders. Exceptional Children, 79, 233–251.
Organ, D. W., Podsakoff, P. M., & MacKenzie, S. B. (2005). Organizational citizenship behavior: Its nature, antecedents, and consequences. London: Sage Publications.
Owens, J. S., Lyon, A. R., Brandt, N. E., Warner, C. M., Nadeem, E., Spiel, C., et al. (2014). Implementation science in school mental health: Key constructs in a developing research agenda. School Mental Health, 6, 99–111.
Powell, B. J., Beidas, R. S., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., et al. (2017). Methods to improve the selection and tailoring of implementation strategies. The Journal of Behavioral Health Services and Research, 44, 177–194.
Powell, B. P., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., et al. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 1–15.
Priestland, A., & Hanig, R. (2005). Developing first-level leaders. Harvard Business Review, 83, 112–120.
Sanetti, L. M. H., Knight, A., Cochrane, W., & Minster, M. (in preparation). Reporting of implementation fidelity of interventions with children in the school psychology literature from 2009 to 2017.
Schneider, B., Ehrhart, M. G., & Macey, W. H. (2013). Organizational climate and culture. Annual Review of Psychology, 64, 361–388.
Wang, V., Maciejewski, M. L., Helfrich, C. D., & Weiner, B. J. (2017). Working smarter not harder: Coupling implementation to de-implementation. Healthcare. https://doi.org/10.1016/j.hjdsi.2017.12.004.
Funding
This study was funded by R305A160114 (third and last author) awarded by the Institute of Education Sciences as well as K01MH100199 (first author) awarded from the National Institute of Mental Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or Institute of Education Sciences.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical Approval
All procedures were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Appendix
Appendix
Rights and permissions
About this article
Cite this article
Locke, J., Lee, K., Cook, C.R. et al. Understanding the Organizational Implementation Context of Schools: A Qualitative Study of School District Administrators, Principals, and Teachers. School Mental Health 11, 379–399 (2019). https://doi.org/10.1007/s12310-018-9292-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12310-018-9292-1