Skip to main content
SpringerLink
Log in

Discrepancies and Agreement in Perceptions of Implementation Leadership: Associations with Dosage of School-Based Evidence-Based Practices for Children with Autism

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Studies demonstrate that discrepancies among leader–follower perceptions of leadership are related to organizational processes that may impact evidence-based practice (EBP) implementation. However, it is unknown whether discrepancies in leadership perceptions also predict EBP use. This study examined the association of principal–staff alignment and discrepancy in perceptions of implementation leadership with the dosage of three Autism-focused school-based EBPs, Discrete Trial Training, Pivotal Response Training (PRT), and Visual Schedules. PRT dosage was higher when principals under-rated their implementation leadership and when it was agreed upon that implementation leadership was low. Findings have implications for leaders support of EBPs implemented in school-based settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9(1), 1–10.

    Article  Google Scholar 

  • Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., Sklar, M., & Horowitz, J. (2015). Discrepancies in leader and follower ratings of transformational leadership: Relationship with organizational culture in mental health. Administration and Policy in Mental Health and Mental Health Services Research, 44(4), 480–491. https://doi.org/10.1007/s10488-015-0672-7.

    Article  Google Scholar 

  • Aarons, G. A., Ehrhart, M. G., Moullin, J. C., Torres, E. M., & Green, A. E. (2017a). Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: A cluster randomized trial study protocol. Implementation Science, 12(1), 1–11. https://doi.org/10.1186/s13012-017-0562-3.

    Article  Google Scholar 

  • Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017b). The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatric Services, 68(2), 115–122. https://doi.org/10.1176/appi.ps.201600062.

    Article  PubMed  Google Scholar 

  • Aarons, G. A., Green, A. E., Trott, E., Willging, C. E., Torres, E. M., Ehrhart, M. G., et al. (2016). The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 991–1008. https://doi.org/10.1007/s10488-016-0751-4.

    Article  PubMed  Google Scholar 

  • Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 28(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7.

    Article  Google Scholar 

  • Aarons, G., & Sommerfeld, D. (2012). Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. Journal of the American Academy of Child and Adolescent Psychiatry, 51(4), 423–431. https://doi.org/10.1016/j.jaac.2012.01.018.

    Article  PubMed  Google Scholar 

  • American Psychological Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

    Book  Google Scholar 

  • Arick, J. R., Young, H. E., Falco, R. A., Loos, L. M., Krug, D. A., Gense, M. H., et al. (2003). Designing an outcome study to monitor the progress of students with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 18(2), 75–87. https://doi.org/10.1177/108835760301800201.

    Article  Google Scholar 

  • Atwater, L., Roush, P., & Fischthal, A. (1995). The influence of upward feedback on self- and follower ratings of leadership. Personnel Psychology, 48(1), 35–59. https://doi.org/10.1111/j.1744-6570.1995.tb01745.x.

    Article  Google Scholar 

  • Atwater, L., Waldman, D., Ostroff, C., Robie, C., Waldman, D., & Johnson, K. M. (2005). Self–other agreement: Comparing its relationship with performance in the U.S. and Europe. International Journal of Selection and Assessment, 13(1), 25–40. https://doi.org/10.1111/j.0965-075X.2005.00297.x.

    Article  Google Scholar 

  • Atwater, L. E., & Yammarino, F. J. (1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45(1), 141–164. https://doi.org/10.1111/j.1744-6570.1992.tb00848.x.

    Article  Google Scholar 

  • Bass, B. M., & Yammarino, F. J. (1991). Congruence of self and others’ leadership ratings of naval officers for understanding successful performance. Applied Psychology, 40(4), 437–454.

    Article  Google Scholar 

  • Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability. In K. J. Klein & S. W. Kozlowski (Eds.), Multilevel theory research, and methods in organizations (pp. 349–381). San Francisco: Jossey-Bass.

    Google Scholar 

  • Bradley, E. H., Curry, L. A., & Devers, K. J. (2007). Qualitative data analysis for health services research: Developing taxonomy, themes, and theory. Health Services Research, 42(4), 1758–1772. https://doi.org/10.1111/j.1475-6773.2006.00684.x.

    Article  PubMed  PubMed Central  Google Scholar 

  • Brookman-Frazee, L., & Stahmer, A. C. (2018). Effectiveness of a multi-level implementation strategy for ASD interventions: Study protocol for two linked cluster randomized trials. Implementation Science, 13(66), 1–14. https://doi.org/10.1186/s13012-018-0757-2.

    Article  Google Scholar 

  • Burns, B. J., Costello, E. J., Angold, A., Tweed, D., Stangl, D., Farmer, E. M. Z., et al. (1995). Children’s mental health service use across service sectors. Health Affairs, 14(3), 147–159.

    Article  CAS  PubMed  Google Scholar 

  • Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79(2), 135–144. https://doi.org/10.1177/001440291307900201.

    Article  Google Scholar 

  • Cook, R. D., & Weisberg, S. (1982). Residuals and influence in regression. New York: Chapman and Hall.

    Google Scholar 

  • Cummings, G. G., Estabrooks, C. A., Midodzi, W. K., Wallin, L., & Hayduk, L. (2007). Influence of organizational characteristics and context on research utilization. Entrepreneurship and Regional Development, 55(4), S24–S39. https://doi.org/10.1080/08985626.2017.1328905.

    Article  Google Scholar 

  • Dettmer, S., Simpson, R. L., Smith Myles, B., & Ganz, J. G. (2000). The use of visual supports to facilitate transition of students with autism. Focus on Autism and Other Developmental Disabilities, 15(3), 163–169.

    Article  Google Scholar 

  • Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350. https://doi.org/10.2307/2666999.

    Article  Google Scholar 

  • Farmer, E. M. Z., Burns, B. J., Phillips, S. D., Angold, A., & Costello, E. J. (2003). Pathways into and through mental health services for children and adolescents. Psychiatric Services, 54(1), 60–66. https://doi.org/10.1176/appi.ps.54.1.60.

    Article  PubMed  Google Scholar 

  • Fleenor, J. W., McCauley, C. D., & Brutus, S. (1996). Self-other rating agreement and leader effectiveness. Leadership Quarterly, 7(4), 487–506. https://doi.org/10.1016/S1048-9843(96)90003-X.

    Article  Google Scholar 

  • Forman, S. G., Olin, S. S., Hoagwood, K. E., Crowe, M., & Saka, N. (2009). Evidence-based interventions in schools: Developers’ views of implementation barriers and facilitators. School Mental Health, 1(1), 26–36. https://doi.org/10.1007/s12310-008-9002-5.

    Article  Google Scholar 

  • Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39(1), 3–35.

    Article  Google Scholar 

  • Henggeler, S. W., Chapman, J. E., Rowland, M. D., Halliday-Boykins, C. A., Randall, J., Shackelford, J., et al. (2008). Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. Journal of Consulting and Clinical Psychology, 76(4), 556–567. https://doi.org/10.1037/0022-006X.76.4.556.

    Article  PubMed  PubMed Central  Google Scholar 

  • James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67(2), 219–229.

    Article  Google Scholar 

  • Koegel, R. L., Schreibman, L., Good, A., Cerniglia, L., Murphy, C., & Koegel, L. K. (1988). How to teach pivotal behaviors to children with autism: A training manual. Santa Barbara: University of California.

    Google Scholar 

  • Langley, A. K., Nadeem, E., Kataoka, S. H., Stein, B. D., & Jaycox, L. H. (2010). Evidence-based mental health programs in schools: Barriers and facilitators of successful implementation. School Mental Health, 2(3), 105–113. https://doi.org/10.1007/s12310-010-9038-1.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lebreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852.

    Article  Google Scholar 

  • Locke, J., Beidas, R. S., Marcus, S., Stahmer, A., Aarons, G. A., Lyon, A. R., et al. (2016). A mixed methods study of individual and organizational factors that affect implementation of interventions for children with autism in public schools. Implementation Science, 11(1), 1–9. https://doi.org/10.1186/s13012-016-0501-8.

    Article  Google Scholar 

  • Locke, J., Olsen, A., Wideman, R., Downey, M. M., Kretzmann, M., Kasari, C., et al. (2015). A tangled web: The challenges of implementing an evidence-based social engagement intervention for children with autism in urban public school settings. Behavior Therapy, 46(1), 54–67. https://doi.org/10.1016/j.beth.2014.05.001.

    Article  PubMed  Google Scholar 

  • Locke, J., Violante, S., Pullmann, M. D., Kerns, S. E. U., Jungbluth, N., & Dorsey, S. (2017). Agreement and discrepancy between supervisor and clinician alliance: Associations with clinicians’ perceptions of psychological climate and emotional exhaustion. Administration and Policy in Mental Health and Mental Health Services Research, 0(0), 1–13. https://doi.org/10.1007/s10488-017-0841-y.

    Article  Google Scholar 

  • Lyon, A. R., Cook, C. R., Brown, E. C., Locke, J., Davis, C., Ehrhart, M., et al. (2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13(1), 1–14. https://doi.org/10.1186/s13012-017-0705-6.

    Article  Google Scholar 

  • Mandell, D. S., Stahmer, A. C., Shin, S., Xie, M., Reisinger, E., & Marcus, S. C. (2013). The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. Autism, 17(3), 281–295. https://doi.org/10.1177/1362361312473666.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mihalic, S. F., Fagan, A. A., & Argamaso, S. (2008). Implementing the LifeSkills training drug prevention program: Factors related to implementation fidelity. Implementation Science, 3(1), 1–16. https://doi.org/10.1186/1748-5908-3-5.

    Article  Google Scholar 

  • National Research Council (U.S.). (2001). Educating children with autism. Washington, DC: National Academy Press.

    Google Scholar 

  • Niemiec, C. P., & Ryan, R. M. (2009). Autonomy, competence, and relatedness in the classroom: Applying self-determination theory to educational practice. Theory and Research in Education, 7(2), 133–144. https://doi.org/10.1177/1477878509104318.

    Article  Google Scholar 

  • Ostroff, C., Atwater, L. E., & Feinberg, B. J. (2004). Understanding self-other agreement: A look at rater and ratee characteristics, context, and outcomes. Personnel Psychology, 57(2), 333–375. https://doi.org/10.1111/j.1744-6570.2004.tb02494.x.

    Article  Google Scholar 

  • Owens, B. P., & Hekman, D. R. (2012). Modeling how to grow: An inductive examination of humble leader behaviors, contingencies, and outcomes. The Academy of Management Journal, 55(4), 787–818.

    Article  Google Scholar 

  • Owens, B. P., & Hekman, D. R. (2016). How does leader humility influence team performance? Exploring the mechanisms of contagion and collective promotion focus. Academy of Management Journal, 59(3), 1088–1111. https://doi.org/10.5465/amj.2013.0660.

    Article  Google Scholar 

  • Ozer, E. J. (2006). Contextual effects in school-based violence prevention programs: A conceptual framework and empirical review. Journal of Primary Prevention, 27(3), 315–340. https://doi.org/10.1007/s10935-006-0036-x.

    Article  PubMed  Google Scholar 

  • Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53. https://doi.org/10.1007/s10488-010-0314-z.

    Article  PubMed  Google Scholar 

  • Pellecchia, M., Connell, J. E., Beidas, R. S., Xie, M., Marcus, S. C., & Mandell, D. S. (2015). Dismantling the active ingredients of an intervention for children with autism. Journal of Autism and Developmental Disorders, 45(9), 2917–2927. https://doi.org/10.1007/s10803-015-2455-0.

    Article  PubMed  PubMed Central  Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7.

    Article  PubMed  Google Scholar 

  • Ryan, R., & Deci, E. (2000). Self-determination theory and the facilitation of intrinsic motivation. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68.

    Article  CAS  Google Scholar 

  • Shanock, L. R., Baran, B. E., Gentry, W. A., Clever Pattison, S., & Heggestad, E. D. (2014). Erratum to: Polynomial regression with response surface analysis: A powerful approach for examining moderation and overcoming limitations of difference scores. Journal of Business and Psychology, 29(1), 161. https://doi.org/10.1007/s10869-013-9317-6.

    Article  Google Scholar 

  • Shanock, L. R., Baran, B. E., Gentry, W. A., Pattison, S. C., & Heggestad, E. D. (2010). Polynomial regression with response surface analysis: A powerful approach for examining moderation and overcoming limitations of difference scores. Journal of Business and Psychology, 25(4), 543–554. https://doi.org/10.1007/s10869-010-9183-4.

    Article  Google Scholar 

  • Smith, T., Groen, A. D., & Wynn, J. W. (2000). Randomized trial of intensive early intervention for children with pervasive developmental disorder. American Journal on Mental Retardation, 105(4), 269–285.

    Article  CAS  PubMed  Google Scholar 

  • Stahmer, A. C., Reith, S., Lee, E., Reisingger, E. M., Mandell, D. S., & Connell, J. E. (2015). Training teachers to use evidence-based practices for autism: Examining procedural implementation fidelity. Psychology in Schools, 52(2), 181–195. https://doi.org/10.1002/pits.

    Article  Google Scholar 

  • Suhrheinrich, J., Stahmer, A. C., Reed, S., Schreibman, L., Reisinger, E., & Mandell, D. (2013). Implementation challenges in translating pivotal response training into community settings. Journal of Autism and Developmental Disorders, 43(12), 2970–2976. https://doi.org/10.1007/s10803-013-1826-7.

    Article  PubMed  Google Scholar 

  • Weiner, B. J., Belden, C. M., Bergmire, D. M., & Johnston, M. (2011). The meaning and measurement of implementation climate. Implementation Science, 6(1), 78. https://doi.org/10.1186/1748-5908-6-78.

    Article  PubMed  Google Scholar 

  • Whitaker, K., Fortier, A., Bruns, E. J., Nicodimos, S., Ludwig, K., Lyon, A. R., et al. (2018). How do school mental health services vary across contexts? Lessons learned from two efforts to implement a research-based strategy. School Mental Health, 10(2), 134–146. https://doi.org/10.1007/s12310-017-9243-2.

    Article  Google Scholar 

  • Wilson, S. J., Lipsey, M. W., & Derzon, J. H. (2003). The effects of school-based intervention programs on aggressive behavior: A meta-analysis. Journal of Consulting and Clinical Psychology, 71(1), 136–149. https://doi.org/10.1037/0022-006X.71.1.136.

    Article  PubMed  Google Scholar 

  • Yammarino, F. J., & Atwater, L. E. (1993). Understanding self-perception accuracy: Implications for human resource management. Human Resource Management, 32(2–3), 231–247. https://doi.org/10.1002/hrm.3930320204.

    Article  Google Scholar 

Download references

Funding

This study was funded by the National Institute of Mental Health (K01 MH100199).

Author information

Authors and Affiliations

  1. University of Washington, Seattle, USA

    Rosemary D. Meza, Shannon Dorsey, Lindsay Frederick & Jill Locke

  2. University of Pennsylvania, Philadelphia, USA

    Rinad S. Beidas & David S. Mandell

  3. University of Central Florida, Orlando, USA

    Mark G. Ehrhart

  4. University of Pittsburgh, Pittsburgh, USA

    Cristine Oh

  5. Department of Psychology, University of Washington, 335 Guthrie Hall, Box 351525, Seattle, WA, 98195, USA

    Rosemary D. Meza

Authors
  1. Rosemary D. Meza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rinad S. Beidas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark G. Ehrhart
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David S. Mandell
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shannon Dorsey
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lindsay Frederick
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Cristine Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jill Locke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rosemary D. Meza.

Ethics declarations

Conflicts of interest

The authors have no conflicts of interest to declare.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meza, R.D., Beidas, R.S., Ehrhart, M.G. et al. Discrepancies and Agreement in Perceptions of Implementation Leadership: Associations with Dosage of School-Based Evidence-Based Practices for Children with Autism. Adm Policy Ment Health 46, 518–529 (2019). https://doi.org/10.1007/s10488-019-00932-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-019-00932-3

Keywords

Search

Navigation