Validation of the Implementation Leadership Scale (ILS) with Supervisors’ Self-Ratings


Although often discussed, there is a lack of empirical research on the role of leadership in the management and delivery of health services. The implementation leadership scale (ILS) assesses the degree to which leaders are knowledgeable, proactive, perseverant, and supportive during evidence-based practice (EBP) implementation. The purpose of this study was to examine the psychometric properties of the ILS for leaders’ self-ratings using a sample of mental health clinic supervisors (N = 119). Supervisors (i.e., leaders) completed surveys including self-ratings of their implementation leadership. Confirmatory factor analysis, reliability, and validity of the ILS were evaluated. The ILS factor structure was supported in the sample of supervisors. Results demonstrated internal consistency reliability and validity. Cronbach alpha’s ranged from 0.92 to 0.96 for the ILS subscales and 0.95 for the ILS overall scale. The factor structure replication and reliability of the ILS in a sample of supervisors demonstrates its applicability with employees across organizational levels.

This is a preview of subscription content, access via your institution.


  1. Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The evidence-based practice attitude scale (EBPAS). Mental Health Services Research, 6(2), 61–74.

    Article  Google Scholar 

  2. Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9(1), 45.

    Article  Google Scholar 

  3. Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015). Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 11.

    Article  Google Scholar 

  4. Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35(1), 255–274.

    Article  Google Scholar 

  5. Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., Sklar, M., & Horowitz, J. (2015). Discrepancies in leader and follower ratings of transformational leadership: Relationships with organizational culture in mental health. Administration and Policy in Mental Health and Mental Health Services Research, 42, 1–12.

    Article  Google Scholar 

  6. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.

    Article  Google Scholar 

  7. Aarons, G. A., & Sommerfeld, D. H. (2012). Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. Journal of the American Academy Child and Adolescent Psychiatry, 51(4), 423–431.

    Article  Google Scholar 

  8. Bass, B. M., & Avolio, B. J. (1990). The implications of transformational and transactional leadership for individual, team, and organizational development. In W. Pasmore & R. W. Woodman (Eds.), Research in organizational change and development (pp. 231–272). Greenwich, CT: JAI Press.

    Google Scholar 

  9. Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood, K. E., Schoenwald, S., Evans, A. C., ... Mandell, D. S., et al. (2015). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169(4), 374–382.

    Article  Google Scholar 

  10. Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9(1), 157.

    Article  Google Scholar 

  11. Hong, Y., Liao, H., Hu, J., & Jiang, K. (2013). Missing link in the service profit chain: A meta-analytic review of the antecedents, consequences, and moderators of service climate. Journal of Applied Psychology, 98(2), 237–267.

    Article  Google Scholar 

  12. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

    Article  Google Scholar 

  13. Martinez, R. G., Lewis, C. C., & Weiner, B. J. (2014). Instrumentation issues in implementation science. Implementation Science, 9(1), 118.

    Article  Google Scholar 

  14. Michaelis, B., Stegmaier, R., & Sonntag, K. (2010). Shedding light on followers’ innovation implementation behavior: The role of transformational leadership, commitment to change, and climate for initiative. Journal of Managerial Psychology, 25(4), 408–429.

    Article  Google Scholar 

  15. Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th edn.). Los Angeles, CA: Muthén & Muthén.

  16. Priestland, A., & Hanig, R. (2005). Developing first-level leaders. Harvard Business Review, 83(6), 112–120.

    PubMed  Google Scholar 

  17. Proctor, E. K., Landsverk, J., Aarons, G. A., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24–34.

    Article  Google Scholar 

  18. Schneider, B., Ehrhart, M. G., Mayer, D. M., Saltz, J. L., & Niles-Jolly, K. (2005). Understanding organization-customer links in service settings. Academy of Management Journal, 48(6), 1017–1032.

    Article  Google Scholar 

Download references


This study was supported by National Institute of Mental Health grants R21MH098124 (PI: Ehrhart), R21MH082731 (PI: Aarons), R01MH072961 (PI: Aarons), K23MH099179 (PI: Beidas) P30MH074678 (PI: Landsverk), R25MH080916 (PI: Proctor). The authors thank the community-based organizations, clinicians, and leaders that made this study possible.

Author information



Corresponding author

Correspondence to Elisa M. Torres.

Ethics declarations

Conflict of interest

The authors report no conflict of interest and certify that this manuscript has not been published or submitted elsewhere.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Torres, E.M., Ehrhart, M.G., Beidas, R.S. et al. Validation of the Implementation Leadership Scale (ILS) with Supervisors’ Self-Ratings. Community Ment Health J 54, 49–53 (2018).

Download citation


  • Implementation
  • Leadership
  • Supervisors
  • Evidence-based practice
  • Measurement