Abstract
In this paper, we propose an implementation science research agenda as it applies to school mental health (SMH). First, we provide an overview of important contextual issues to be considered when addressing research questions pertinent to the implementation of mental health interventions in schools. Next, we critically review three core implementation components: (a) professional development and coaching for school professionals regarding evidence-based practices (EBPs); (b) the integrity of EBPs implemented in schools; and (c) EBP sustainment under typical school conditions. We articulate research questions central to the next generation of research in each of these areas as well as methods to address such questions. Our intent in doing so is to contribute to a developing blueprint to guide community-research partnerships as well as funding agencies in their efforts to advance implementation science in SMH.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
These three topics were selected via a three-step process. First, the authors of the article identified a list of components and processes that have been identified in multiple models of IS research, as well as issues that have emerged in the context of each individual’s program of research. Second, the authors individually rank-ordered the topics, and then, the group discussed the rankings and reached a consensus. Third, to ensure that the topics were important and relevant to those in the practice of SMH and education, the topics were discussed with a variety of community partners (i.e., currently practicing SMH professionals and educators), and their feedback was incorporated.
Because many school professionals may implement a SMH intervention, henceforth we refer to all such professionals as “implementers” or “providers.”
References
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.
Aarons, G. A., & Palinkas, L. A. (2007). Implementation of evidence-based practice in child welfare: Service provider perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 34, 411–419.
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211.
Backer, T. E. (2001). Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state-of-the art review. Rockville, MD: Center for Substance Abuse Prevention.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191.
Barber, J. P., Gallop, R., Crits-Christoph, P., Frank, A., Thase, M. E., Weiss, R. D., et al. (2006). The role of therapist adherence, therapist competence, and alliance in predicting outcome of individual drug counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research, 16, 229–240.
Barkham, M., Hardy, G., & Mellor-Clark, J. (Eds.). (2010). Developing and delivering practice-based evidence: A guide for the psychological therapies. Chichester, UK: John Wiley & Sons.
Becker, K., Haak, J., Domitrovich, C., Keperling, J., & Ialongo, N. (2013). Online training for teachers delivering evidence-based preventive interventions. Manuscript under review.
Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1–30.
Blank, R. K., de las Alas, N., & Smith, C. (2008). Does teacher professional development have effects on teaching and learning? Analysis of evaluation findings from programs for mathematics and science teachers in 14 states. Washington, DC: The Council of Chief State School Officers.
Chu, B. C., & Kendall, P. C. (2004). Positive association of child involvement and treatment outcome within a manual-based cognitive-behavioral treatment for children with anxiety. Journal of Consulting and Clinical Psychology, 72(5), 821–829.
Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79(2), 135–144.
Cunningham, C. E., Vaillancourt, T., Rimas, H., Deal, K., Cunningham, L., Short, K., et al. (2009). Modeling the bullying prevention program preferences of educators: A discrete choice conjoint experiment. Journal of Abnormal Child Psychology, 37, 929–943.
Curran, G., Bauer, M., Mittman, B., Pyne, J., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50, 217–226.
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50.
Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18(1), 23–45.
DuPaul, G. D. (2009). Assessing integrity of intervention implementation: Critical factors and future directions. School Mental Health, 1, 154–157.
Durlak, J. (2013). The importance of quality implementation for research, practice, and policy. ASPE Research Brief. US Department of Health and Human Services.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3–4), 327–350.
Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.
Eccles, M. P., & Mittman, B. S. (2006). Welcome to implementation science. Implementation Science, 1(1), 1–3.
Evans, S. W., Koch, R., Brady, C., Meszaros, P., & Sadler, J. (2013). Community and school mental health professionals’ knowledge and use of evidence-based substance use prevention programs. Administration and Policy in Mental Health and Mental Health Services Research, 40(4), 319–330.
Evans, S. W., Schultz, B. K., & DeMars, C. (2013). Coaching for high school students with ADHD: An examination of academic and social outcomes. Manuscript under review.
Fabiano, G. A., Chafouleas, S. M., Weist, M. D., Sumi, W. C., & Humphrey, N. (2013). Methodology considerations in school mental health research. School Mental Health, under reivew.
Farmer, A., Legare, F., Turcot, L., Grimshaw, J., Harvey, E., McGowan, J., & Wolf, F. M. (2008). Printed educational materials: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, 3. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD004398.pub2/references.
Fixsen, D. L., Blase, K. A., Duda, M., Naoom, S. F., & Van Dyke, M. (2010). Sustainability of evidence-based programs in education. Journal of Evidence-based Practices for Schools, 11(1), 30–46.
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.
Forman, S. G., & Barakat, N. M. (2011). Cognitive-behavioral therapy in the schools: Bringing research to practice through effective implementation. Psychology in the Schools, 48, 283–296.
Forman, S. G., Codding, R. S., Reddy, L. A., Sanetti, L. M. H., Shapiro, E. S., Gonzales, J. E., et al. (2013). Implementation science and school psychology. School Psychology Quarterly, 38(2), 77–100.
Forman, S. G., Olin, S., Hoagwood, K., Crowe, M., & Saka, N. (2009). Evidence-based intervention in schools: Developers’ views of implementation barriers and facilitators. School Mental Health: A Multidisciplinary Research and Practice Journal, 1, 26–36.
Fox, J. K., Herzig-Anderson, K., Colognori, D., Stewart, C. E., & Masia Warner, C. (2014). School-based treatment for anxiety in children and adolescents: New developments in transportability and dissemination. In M. S. Weist, N. A. Lever, C. P. Bradshaw, & J. S. Owens (Eds.), Handbook of school mental health (2nd ed., pp. 355–368). New York: Springer.
Frey, A. J., Cloud, R., Lee, J., Small, J. W., Seeley, J. R., Feil, E. G., et al. (2011). The promise of motivational interviewing in school mental health. School Mental Health, 3, 1–12.
Hallinger, P., & Heck, R. H. (1996). Reassessing the principal’s role in school effectiveness: A review of empirical research, 1980–1995. Educational administration quarterly, 32(1), 5–44.
Han, S. S., & Weiss, B. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33(6), 665–679.
Harn, B., Parisi, D., & Stoolmiller, M. (2013). Balancing fidelity and flexibility and fit: What do we really know about fidelity of implementation in schools? Exceptional Children, 79, 181–193.
Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466.
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., et al.. (2012) Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, 6. Art. no.: CD000259.
Joyce, B., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
Kam, C. M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4(1), 55–63.
Kelly, M. S., Berzin, S. C., Frey, A., Alvarez, M., Shaffer, G., & O’Brie, K. (2010). The state of school social work: Findings from the National School Social Work Survey. School Mental Health, 2, 132–141.
Kutash, K., Cross, B., Madias, A., Duchnowski, A. J., & Green, A. L. (2012). Description of a fidelity implementation system: An example from a community-based children’s mental health program. Journal of Child and Family Studies, 21(6), 1028–1040.
Langley, A. K., Nadeem, E., Kataoka, S. H., Stein, B. D., & Jaycox, L. H. (2010). Evidence-based mental health programs in schools: Barriers and facilitators of successful implementation. School Mental Health, 2(3), 105–113.
Lee, S. J., Altschul, I., & Mowbray, C. T. (2008). Using planned adaptation to implement evidence-based programs with new populations. American Journal of Community Psychology, 41, 290–303.
Leff, S. S., Costigan, T., & Power, T. J. (2004). Using participatory research to develop a playground-based prevention program. Journal of School Psychology, 42(1), 3–21.
Leff, S. S., Waasdorp, T. E., Waanders, C., & Paskewich, B. S. (2014). Better understanding and intervening to prevent relational aggression. In M. S. Weist, N. A. Lever, C. P. Bradshaw, & J. S. Owens (Eds.), Handbook of school mental health (2nd ed., pp. 171–182). New York: Springer.
Lyon, A. R., Charlesworth-Attie, S., Vander Stoep, A., & McCauley, E. (2011a). Modular psychotherapy for youth with internalizing problems: Implementation with therapists in school-based health centers. School Psychology Review, 40(4), 569–581.
Lyon, A. R., Frazier, S. L., Mehta, T., Atkins, M. S., & Weisbach, J. (2011b). Easier said than done: Intervention sustainability in an urban after-school program. Administration and Policy in Mental Health and Mental Health Services Research, 38(6), 504–517.
Lyon, A. R., Ludwig, K., Romano, E., Koltracht, J., Vander Stoep, A., & McCauley, E. (in press). Using modular psychotherapy in school mental health: Provider perspectives on intervention-setting fit. Journal of Clinical Child & Adolescent Psychology.
Lyon, A. R., McCauley, E., & Vander Stoep, A. (2011c). Toward successful implementation of evidence-based practices: Characterizing the intervention context of counselors in school-based health centers. Emotional & Behavioral Disorders in Youth, 11, 19–25.
Lyon, A. R., Stirman, S. W., Kerns, S. E., & Bruns, E. J. (2011d). Developing the mental health workforce: Review and application of training approaches from multiple disciplines. Administration and Policy in Mental Health and Mental Health Services Research, 38(4), 238–253.
Masia Warner, C., Brice, C., Esseling, P. G., Stewart, C. E., Mufson, L., & Herzig, K. (2013). Consultants’ perception of school counselor’s ability to implement an empirically-based intervention for adolescent social anxiety disorder. Administration Policy in Mental Health and Mental Health Services Research, 40(3), 541–554.
Murphy, S. M., Lynch, K. G., Oslin, D., McKay, J. R., & TenHave, T. (2007). Developing adaptive treatment strategies in substance abuse research. Drug and Alcohol Dependence, 88(Suppl 2), S24–S30.
Nastasi, B. K., Varjas, K., Schensul, S. L., Silva, K. T., Schensul, J. J., & Ratnayake, P. (2000). The participatory intervention model: A framework for conceptualizing and promoting intervention acceptability. School Psychology Quarterly, 15(2), 207–232.
Noell, G., Witt, J., Gilbertson, D., Ranier, D., & Freeland, J. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12, 77–88.
Olmstead, T., Carroll, K. M., Canning-Ball, M., & Martino, S. (2011). Cost and cost-effectiveness of three strategies for training clinicians in motivational interviewing. Drug and Alchohol Dependence, 116, 195–202.
Olweus, D., & Limber, S. P. (2010). Bullying in school: Evaluation and dissemination of the Olweus Bullying Prevention Program. American Journal of Orthopsychiatry, 80, 124–134.
Owens, J. S., Andrews, N., Collins, J., Griffeth, J. C., & Mahoney, M. (2011a). Finding common ground: University research guided by community needs for elementary school-aged youth. In L. Harter, J. Hamel-Lambert, & J. Millesen (Eds.), Participatory partnerships for social action and research (pp. 49–71). Dubuque, IA: Kendall Hunt.
Owens, J. S., Dan, M., Alvarez, H., Tener, D., & Oberlin, K. (2007). Using community–university partnerships to advance school mental health programming: Why, what and how? Ohio Mental Health Network for School Success Network News, 4(2), 2–3.
Owens, J. S., Holdaway, A. S., Zoromski, A. K., Evans, S. W., Himawan, L. K., Girio-Herrera, E., et al. (2012). Incremental benefits of a daily report card intervention over time for youth with disruptive behavior. Behavior Therapy, 43, 848–861.
Owens, J. S., Murphy, C. E., Richerson, L., Girio, E. L., & Himawan, L. K. (2008). Science to practice in underserved communities: The effectiveness of school mental health programming. Journal of Clinical Child and Adolescent Psychology, 37, 434–447.
Owens, J. S., Storer, J. L., & Girio, E. L. (2011b). Psychosocial interventions for elementary school-aged children with ADHD. In S. W. Evans & B. Hoza (Eds.), Treating attention deficit hyperactivity disorder: Assessment and intervention in developmental context (pp. 10-2–10-26). Kingston, NJ: Civic Research Institute.
Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53.
Patel, V. (2009). The future of psychiatry in low-and middle-income countries. Psychological Medicine, 39(11), 1759–1762.
Patel, P. G., Stark, K. D., Metz, K. L., & Banneyer, K. N. (2014). School-based interventions for depression. In M. S. Weist, N. A. Lever, C. P. Bradshaw, & J. S. Owens (Eds.), Handbook of school mental health (2nd ed., pp. 369–385). New York: Springer.
Perepletchikova, F. (2009). Treatment integrity in treatment outcome research: Adequacy of procedures, associated factors, implications for research and practice, guidelines and recommendations. Germany: Lambert Academic Publishing.
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24–34.
Proctor, E. K., Powell, B. J., Baumann, A. A., Hamilton, A. M., & Santens, R. L. (2012). Writing implementation research grant proposals: Ten key ingredients. Implementation Science, 7(1), 96.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76.
Reinke, W. M., Herman, K. C., Darney, D., Pitchford, J., Becker, K., Domitrovich, C., et al. (2012). Using the Classroom Check-Up model to support implementation of PATHS to PAX. Advances in School Mental Health Promotion, 5, 220–232.
Rogers, E. M. (2003). Diffusions of innovations (5th ed.). New York, NY: Free Press.
Scheirer, M. A. (2005). Is sustainability possible? A review and commentary on empirical studies of program sustainability. American Journal of Evaluation, 26, 320–347.
Scheirer, M. A., & Dearing, J. W. (2011). An agenda for research on the sustainability of public health programs. American Journal of Public Health, 101, 2059–2067.
Schell, S. F., Luke, D. A., Schooley, M. W., Elliot, M. B., Herbers, S. H., Mueller, N. B., & Bunger, A. C. (2013). Public health program capacity for sustainability: A new framework. Implementation Science, 8(1), 15.
Schoenwald, S. K., Sheidow, A. J., & Letourneau, E. J. (2004). Toward effective quality assurance in evidence-based practice: Links between expert consultation, therapist fidelity, and child outcomes. Journal of Clinical Child & Adolescent Psychology, 33(1), 94–104.
Schouten, L. M. T., Hulscher, M. E. J. L., Everdingen, J. J. E., Huijsman, R., & Grol, R. P. T. M. (2008). Evidence for the impact of quality improvement collaboratives: A systematic review. British Medical Journal, 336(7659), 1491–1494.
Schulte, A. C., Easton, J. E., & Parker, J. (2009). Advances in treatment integrity research: Multidisciplinary perspectives on the conceptualization, measurement, and enhancement of treatment integrity. School Psychology Review, 38, 460–475.
Silverman, W. K., & Hinshaw, S. P. (2008). The second special issue on evidence-based psychosocial treatments for children and adolescents: A 10-year update. Journal of Clinical Child & Adolescent Psychology, 37(1), 1–7.
Southam-Gerow, M. A., & McLeod, B. D. (2013). Advances in applying treatment integrity research for dissemination and implementation science: Introduction to special issue. Clinical Psychology: Science and Practice, 20(1), 1–13.
Stirman, S. W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7(17), 1–19.
Stirman, S. W., Miller, C. J., Toder, K., & Calloway, A. (2013). Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science, 8(1), 65.
Sugai, G., & Horner, R. R. (2006). A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review, 35(2), 245–259.
Swanson, H. L., & Hoskyn, M. (1998). A synthesis of experimental intervention literature for students with learning disabilities: A meta-analysis of treatment outcomes. Review of Educational Research, 68, 277–322.
Thomas, A. E., & Grimes, J. E. (Eds.). (2008). Best practices in school psychology (Vol. V). Bethesda, MD: National Association of School Psychologist.
Wagner, M., Gaylor, E., Fabricant, N., & Shaver, D. (2013). Early childhood language intervention models: Lessons learned about model sustainment and spread. Menlo Park, CA: SRI International.
Williams, J. H., Horvath, V. E., Wei, H. S., Van Dorn, R. A., & Jonson-Reid, M. (2007). Teachers’ perspectives of children’s mental health service needs in urban elementary schools. Children & Schools, 29(2), 95–107.
Yu, J., Wagner, M., & Shaver, D. (2012). Tertiary behavior intervention models in elementary and middle schools: Lessons learned about model sustainability and spread. Menlo Park, CA: SRI International. http://mdcc.sri.com/prod_serv.html.
Acknowledgments
The authors would like to acknowledge Jan Fine, Jacqueline Griffeth, and Priscilla Petrosky, our school and community partners, who reviewed the paper and provided valuable feedback that was grounded in the realities of their daily work in schools. Julie S. Owens was supported by grants from the Department of Education, Institute of Educational Sciences (R305A110059; R324C080006; and R324A110232). This publication was made possible in part by funding from the National Institute of Mental Health (NIMH; K08 MH095939), awarded to Aaron Lyon. Dr. Lyon is an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI). Nicole Evangelista Brandt was partially supported by Project U45 MC 00174 from the Office of Adolescent Health, Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Service Administration, Department of Health and Human Services. This work was partially supported by an NIMH Grant R01MH081881, awarded to Dr. Masia Warner. This work was partially supported by two NIMH Grants, K01 MH083694 (PI Nadeem) and P30 MH090322 (PI: Hoagwood).
Author information
Authors and Affiliations
Corresponding author
Additional information
The last five authors are listed alphabetically.
Rights and permissions
About this article
Cite this article
Owens, J.S., Lyon, A.R., Brandt, N.E. et al. Implementation Science in School Mental Health: Key Constructs in a Developing Research Agenda. School Mental Health 6, 99–111 (2014). https://doi.org/10.1007/s12310-013-9115-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12310-013-9115-3