School Mental Health

, Volume 11, Issue 3, pp 379–399 | Cite as

Understanding the Organizational Implementation Context of Schools: A Qualitative Study of School District Administrators, Principals, and Teachers

  • Jill LockeEmail author
  • Kristine Lee
  • Clayton R. Cook
  • Lindsay Frederick
  • Cheryl Vázquez-Colón
  • Mark G. Ehrhart
  • Gregory A. Aarons
  • Chayna Davis
  • Aaron R. Lyon
Original Paper


Key features of the school environment can have a significant impact on teachers’ effective use of evidence-based practices (EBP), yet implementation-specific organizational constructs have rarely been studied in the education sector. This study examined three aspects of the organizational implementation context (implementation leadership, climate, and citizenship behavior), which have been conceptualized and validated in other service settings. Focus groups with central office administrators, principals, and teachers were conducted to understand the applicability and conceptual boundaries of these organizational constructs in schools. Focus group transcripts were coded, and the results indicated both similarities and differences in their conceptualizations of implementation leadership, climate, and citizenship behavior in school. The data indicated that: (1) implementation leadership was largely present in schools with the addition of Distributed Leadership; (2) two implementation climate constructs were most clearly present (i.e., Focus on EBP and Educational Support for EBP) and two additional constructs (i.e., Existing Support to Deliver EBP and Prioritization of EBP) emerged as part of this construct; and (3) implementation citizenship behavior (Helping Others and Keeping Informed) was consistently acknowledged across schools and two new components emerged (i.e., Information Sharing and Observation/Feedback). Recommendations to researchers and community stakeholders are discussed.


Schools Implementation Evidence-based practices Leadership Climate Citizenship behavior 



This study was funded by R305A160114 (third and last author) awarded by the Institute of Education Sciences as well as K01MH100199 (first author) awarded from the National Institute of Mental Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or Institute of Education Sciences.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.


  1. Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014a). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9, 45.CrossRefPubMedGoogle Scholar
  2. Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014b). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274.CrossRefPubMedPubMedCentralGoogle Scholar
  3. Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017). The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatric Services, 68, 115–122.CrossRefPubMedGoogle Scholar
  4. Aarons, G. A., Green, A. E., Palinkas, L. A., Self-Brown, S., Whitaker, D. J., Lutzker, J. R., et al. (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7, 1–9.CrossRefGoogle Scholar
  5. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.CrossRefPubMedGoogle Scholar
  6. Aitken, L. M., Hackwood, B., Crouch, S., Clayton, S., West, N., Carney, D., et al. (2011). Creating an environment to implement and sustain evidence-based practice: A developmental process. Australian Critical Care, 24, 244–254.CrossRefPubMedGoogle Scholar
  7. Angelle, P. S. (2010). An organizational perspective of distributed leadership: A portrait of a middle school. Rmle Online, 33, 1–16.CrossRefGoogle Scholar
  8. APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. The American Psychologist, 61, 271.CrossRefGoogle Scholar
  9. Atkins, M., Frazier, S., Leathers, S., Graczyk, P., Talbott, E., Jakobsons, L., et al. (2008). Teacher key opinion leaders and mental health consultation in low-income urban schools. Journal of Consulting and Clinical Psychology, 76, 905–908.CrossRefPubMedGoogle Scholar
  10. Basch, C. E., Sliepcevich, E. M., Gold, R. S., Duncan, D. F., & Kolbe, L. J. (1985). Avoiding type III errors in health education program evaluations: A case study. Health Education Quarterly, 12, 315–331.CrossRefPubMedGoogle Scholar
  11. Becker-Haimes, E. M., Okamura, K. H., Wolk, C. B., Rubin, R., Evans, A. C., & Beidas, R. S. (2017). Predictors of clinician use of exposure therapy in community mental health settings. Journal of Anxiety Disorders, 49, 88–94.CrossRefPubMedPubMedCentralGoogle Scholar
  12. Beidas, R. S., Aarons, G., Barg, F., Evans, A., Hadley, T., Hoagwood, K., et al. (2013). Policy to implementation: Evidence-based practice in community mental health—Study protocol. Implementation Science, 8, 1–9.CrossRefGoogle Scholar
  13. Beidas, R. S., Edmunds, J., Ditty, M., Watkins, J., Walsh, L., Marcus, S., et al. (2014). Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Administration and Policy In Mental Health, 41, 788–799.CrossRefPubMedPubMedCentralGoogle Scholar
  14. Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood, K. E., Schoenwald, S., Evans, A. C., et al. (2016a). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169, 374–382.CrossRefGoogle Scholar
  15. Beidas, R. S., Stewart, R. E., Adams, D. R., Fernandez, T., Lustbader, S., Powell, B. J., et al. (2016b). A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research, 43, 893–908.CrossRefPubMedGoogle Scholar
  16. Birken, S. A., DiMartino, L. D., Kirk, M. A., Lee, S. Y. D., McClelland, M., & Albert, N. M. (2015). Elaborating on theory with middle managers’ experience implementing healthcare innovations in practice. Implementation Science, 11, 2.CrossRefGoogle Scholar
  17. Bonham, C., Willging, C., Sommerfeld, D., & Aarons, G. A. (2014). Organizational factors influencing implementation of evidence-based practices for integrated treatment in behavioral health agencies. Psychiatry Journal, 2014, 1–9.Google Scholar
  18. Bradley, E. H., Curry, L. A., & Devers, K. J. (2007). Qualitative data analysis for health services research: Developing taxonomy, themes, and theory. Health Services Research, 42, 1758–1772.CrossRefPubMedPubMedCentralGoogle Scholar
  19. Bruns, E. J., Duong, M. T., Lyon, A. R., Pullmann, M. D., Cook, C. R., Cheney, D., et al. (2016). Fostering SMART partnerships to develop an effective continuum of behavioral health services and supports in schools. American Journal of Orthopsychiatry, 86, 156.CrossRefPubMedGoogle Scholar
  20. Cook, C. R., Lyon, A. R., Locke, J., Waltz, T., & Powell, B. (submitted). Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prevention Science.Google Scholar
  21. Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional children, 79, 135–144.Google Scholar
  22. Costello, E. J., He, J., Sampson, N. A., Kessler, R. C., & Merikangas, K. R. (2014). Services for adolescents with psychiatric disorders: 12-month data from the National Comorbidity Survey-Adolescent. Psychiatric Services, 65, 359–366.CrossRefPubMedGoogle Scholar
  23. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.CrossRefPubMedGoogle Scholar
  24. Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157.CrossRefPubMedGoogle Scholar
  25. Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2015). Going above and beyond for implementation: The development and validity testing of the Implementation Citizenship Behavior Scale (ICBS). Implementation Science, 10, 65.CrossRefPubMedGoogle Scholar
  26. Ehrhart, M. G., Aarons, G. A., Torres, E. M., Finn, N. K., & Roesch, S. C. (2016). Implementation in child welfare: Leadership, climate, attitudes, and citizenship behavior. In Poster presented at the 31st annual conference of the Society for Industrial and Organizational Psychology, Anaheim, CA.Google Scholar
  27. Ehrhart, M. G., Torres, E. M., Green, A. E., Trott, E. M., Willging, C. E., Moullin, J. C., et al. (2018). Leading for the long haul: A mixed-method evaluation of the Sustainment Leadership Scale (SLS). Implementation Science, 13, 17.CrossRefPubMedGoogle Scholar
  28. Farmer, E. M. Z., Burns, B. J., Phillips, S. D., Angold, A., & Costello, E. J. (2003). Pathways into and through mental health services for children and adolescents. Prevention Science, 54, 60–66.Google Scholar
  29. Feuerstein, J., Olswang, L. B., Greenslade, K., Pinder, G. L., Dowden, P., & Madden, J. (2017). Moving triadic gaze intervention into practice: Measuring clinician attitude and implementation fidelity. Journal of Speech, Language, and Hearing Research, 60, 1285–1298.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230.Google Scholar
  31. Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida.Google Scholar
  32. Forman, S. G., Olin, S. S., Hoagwood, K. E., Crowe, M., & Saka, N. (2009). Evidence-based interventions in schools: Developers’ views of implementation barriers and facilitators. School Mental Health, 1, 26–36.CrossRefGoogle Scholar
  33. Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. E., Reddy, L. A., Rosenfeld, S. A., et al. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77–100.CrossRefPubMedGoogle Scholar
  34. Gottfredson, G. D., & Gottfredson, D. C. (2001). What schools do to prevent problem behavior and promote safe environments. Journal of Educational and Psychological Consultation, 12, 313–344.CrossRefGoogle Scholar
  35. Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18, 59–82.CrossRefGoogle Scholar
  36. Hoagwood, K., & Johnson, J. (2003). School psychology: A public health framework: I. From evidence-based practices to evidence-based policies. Journal of School Psychology, 41, 3–21.CrossRefGoogle Scholar
  37. Jacobs, S. R., Weiner, B. J., & Bunger, A. C. (2014). Context matters: Measuring implementation climate among individuals and groups. Implementation Science, 9, 46.CrossRefPubMedGoogle Scholar
  38. Langer, D. A., Wood, J. J., Wood, P. A., Garland, A. F., Landsverk, J., & Hough, R. L. (2015). Mental health service use in schools and non-school-based outpatient settings: Comparing predictors of service use. School Mental Health, 7, 161–173.CrossRefPubMedPubMedCentralGoogle Scholar
  39. Langley, A. K., Nadeem, E., Kataoka, S. H., Stein, B. D., & Jaycox, L. H. (2010). Evidence-based mental health programs in schools: Barriers and facilitators of successful implementation. School Mental Health, 2, 105–113.CrossRefPubMedPubMedCentralGoogle Scholar
  40. Lewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018). From confusion to causality: Advancing understanding of mechanisms of change in implementation science.Google Scholar
  41. Locke, J., Beidas, R. S., Marcus, S., Stahmer, A. S., Aarons, G. A., Lyon, A. R., et al. (2016). A mixed methods study of individual and organizational factors that affect implementation of interventions for children with autism in public schools. Implementation Science, 11, 135.CrossRefPubMedGoogle Scholar
  42. Locke, J., Olsen, A., Wideman, R., Downey, M. M., Kretzmann, M., Kasari, C., et al. (2015). A tangled web: The challenges of implementing an evidence-based social engagement intervention for children with autism in urban public school settings. Behavior Therapy, 46, 54–67.CrossRefPubMedGoogle Scholar
  43. Lyon, A., Cook, C., Brown, E., Locke, J., Davis, C., Ehrhart, M., et al. (2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13, 5.CrossRefPubMedGoogle Scholar
  44. Lyon, A. R., Cook, C. R., Locke, J., Powell, B., & Waltz, T. (submitted). Importance and feasibility of an adapted set of strategies for implementing evidence-based behavioral health practices in the education sector. Journal of School Psychology.Google Scholar
  45. Lyon, A. R., Ludwig, K. A., Vander Stoep, A., Gudmundsen, G., & McCauley, E. (2013). Patterns and predictors of mental healthcare utilization in schools and other service sectors among adolescents at risk for depression. School Mental Health, 5, 155–165.CrossRefGoogle Scholar
  46. Malloy, M., Acock, A., DuBois, D. L., Vuchinich, S., Silverthorn, N., Ji, P., et al. (2015). Teachers’ perceptions of school organizational climate as predictors of dosage and quality of implementation of a social-emotional and character development program. Prevention Science, 16, 1086–1095.CrossRefPubMedGoogle Scholar
  47. Martinez, R. G., Lewis, C. C., & Weiner, B. J. (2014). Instrumentation issues in implementation science. Implementation Science, 9, 118.CrossRefPubMedGoogle Scholar
  48. Merikangas, K. R., He, J., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., et al. (2011). Service utilization for lifetime mental disorders in U.S. adolescents: Results of the National Comorbidity Survey-Adolescent Supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry, 50, 32–45.CrossRefPubMedGoogle Scholar
  49. Neale, J. (2016). Iterative categorization (IC): A systematic technique for analysing qualitative data. Addiction (Abingdon, England), 111(6), 1096–1106. Scholar
  50. Odom, S. L., Cox, A. W., Brock, M. E., & National Professional Development Center on ASD. (2013). Implementation science, professional development, and autism spectrum disorders. Exceptional Children, 79, 233–251.Google Scholar
  51. Organ, D. W., Podsakoff, P. M., & MacKenzie, S. B. (2005). Organizational citizenship behavior: Its nature, antecedents, and consequences. London: Sage Publications.Google Scholar
  52. Owens, J. S., Lyon, A. R., Brandt, N. E., Warner, C. M., Nadeem, E., Spiel, C., et al. (2014). Implementation science in school mental health: Key constructs in a developing research agenda. School Mental Health, 6, 99–111.CrossRefPubMedGoogle Scholar
  53. Powell, B. J., Beidas, R. S., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., et al. (2017). Methods to improve the selection and tailoring of implementation strategies. The Journal of Behavioral Health Services and Research, 44, 177–194.CrossRefPubMedGoogle Scholar
  54. Powell, B. P., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., et al. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 1–15.CrossRefGoogle Scholar
  55. Priestland, A., & Hanig, R. (2005). Developing first-level leaders. Harvard Business Review, 83, 112–120.PubMedGoogle Scholar
  56. Sanetti, L. M. H., Knight, A., Cochrane, W., & Minster, M. (in preparation). Reporting of implementation fidelity of interventions with children in the school psychology literature from 2009 to 2017.Google Scholar
  57. Schneider, B., Ehrhart, M. G., & Macey, W. H. (2013). Organizational climate and culture. Annual Review of Psychology, 64, 361–388.CrossRefPubMedGoogle Scholar
  58. Wang, V., Maciejewski, M. L., Helfrich, C. D., & Weiner, B. J. (2017). Working smarter not harder: Coupling implementation to de-implementation. Healthcare. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of WashingtonSeattleUSA
  2. 2.University of MinnesotaMinneapolisUSA
  3. 3.Univeristy of Central FloridaOrlandoUSA
  4. 4.University of CaliforniaSan DiegoUSA

Personalised recommendations