Exploring Provider Use of a Digital Implementation Support System for School Mental Health: A Pilot Study
- 75 Downloads
This pilot study explored provider use of an online system, Centervention, to support the delivery of empirically supported school-based mental health interventions (ESIs); and associations between components of this system [resources, training, technical assistance (TA), feedback loops], implementation indicators, and student outcomes. Multilevel modeling data were collected from 39 providers implementing ESIs with 758 students. Training, TA, and progress monitoring predicted ESI adherence, and perceived value of resources and TA influenced student responsiveness. Greater adherence was predictive of better socio-emotional outcomes. Interviews with 15 providers illuminated how they used these four Centervention support strategies. Implications for digital implementation support research are discussed.
KeywordsImplementation Outcomes EBSIS Technology Implementation strategies
This study received support from a grant from the National Institute of Mental Health (2R44MH084375-03).
Compliance with Ethical Standards
Conflict of interest
Five of the six authors who conducted this study were employees of the 3C Institute at the time of the study. Both the software and the ESIs tested in this study are commercially available. In order to minimize conflict of interest concerns, analyses were performed by author K.S., who had no competing financial interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
- 3C Institute. Centervention. Retrieved October 31, 2014, from https://centervention.org.
- Baron, M., & Graham, C. (2007). Identifying key factors in the adoption of innovative practices. Technology and Teacher Education Annual, 18(2), 772.Google Scholar
- Bickman, L., Douglas, S. R., De Andrade, A. R. V., Tomlinson, M., Gleacher, A., Olin, S., & Hoagwood, K. (2016). Implementing a measurement feedback system: A tale of two sites. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 410–425. doi: 10.1007/s10488-015-0647-8.CrossRefPubMedPubMedCentralGoogle Scholar
- Bickman, L., Lyon, A. R., & Wolpert, M. (2016). Achieving precision mental health through effective assessment, monitoring, and feedback processes: Introduction to the special issue. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 271–276. doi: 10.1007/s10488-016-0718-5.CrossRefPubMedPubMedCentralGoogle Scholar
- Brightwood, L. H., & DeRosier, M. E. (2007). Lifestories for kids: Enhancing character development and social skills through storytelling (Grades 3–5). Cary, NC: 3C Institute for Social Development.Google Scholar
- Brightwood, L. H., DeRosier, M. E., & Maschauer, E. L. (2012). Adventures in emotional literacy (AEL): Building social relationships through enhanced communication, cooperation, and confidence (Grade 3). Cary, NC: 3C Institute for Social Development.Google Scholar
- Brightwood, L. H., DeRosier, M. E., Maschauer, E. L., & Wilson, M. E. (2009). Adventures in emotional literacy (AEL): Building social relationships through enhanced communication, cooperation, and confidence (Grades K-2). Cary, NC: 3C Institute for Social Development.Google Scholar
- Craig, A. B., Leary, K. A., Parker, J., McMillen, J. S., & DeRosier, M. E. (2015). Effectiveness of an emotional literacy intervention for the elementary classroom. (Unpublished manuscript).Google Scholar
- Crosse, S., Williams, B., Hagen, C. A., Harmon, M., Ristow, L., DiGaetano, R., … & Derzon, J. H. (2011). Prevalence and implementation fidelity of research-based prevention programs in public schools (Final Report). Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service. Retrieved from http://files.eric.ed.gov/fulltext/ED529062.pdf.
- DeRosier, M. E. (2007). Social Skills Group Intervention (S.S.GRIN): Group interventions and exercises for enhancing children’s communication, cooperation, and confidence (Grades K-5). Cary, NC: 3C Institute for Social Development.Google Scholar
- DeRosier, M. E., & Brightwood, L. H. (2007). Lifestories for kids: Enhancing character development and social skills through storytelling (Grades K-2). Cary, NC: 3C Institute for Social Development.Google Scholar
- DeRosier, M. E., & Mercer, S. (2007). Improving student social behavior: The effectiveness of a school-based character education program. Journal of Research in Character Education, 5, 131–148.Google Scholar
- Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., … & Ialongo, N. S. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1, 6–28. doi: 10.1080/1754730X.2008.9715730.CrossRefPubMedPubMedCentralGoogle Scholar
- Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.Google Scholar
- Gresham, F. M., Cook, C. R., Collins, T., Dart, E., Rasetshwane, K., Truelson, E., & Grant, S. (2010). Developing a change-sensitive brief behavior rating scale as a progress monitoring tool for social behavior: An example using the Social Skills Rating System—Teacher Form. School Psychology Review, 39, 364–379.Google Scholar
- Henderson, J. L., MacKay, S., & Peterson-Badali, M. (2006). Closing the research-practice gap: Factors affecting adoption and implementation of a children’s mental health program. Journal of Clinical Child and Adolescent Psychology, 35, 2–12. doi: 10.1207/s15374424jccp3501_1.CrossRefPubMedGoogle Scholar
- Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466. doi: 10.1016/j.cpr.2010.02.005.CrossRefPubMedPubMedCentralGoogle Scholar
- Langberg, J. M., & Smith, B. H. (2006). Developing evidence-based interventions for deployment into school settings: A case example highlighting key issues of efficacy and effectiveness. Evaluation and Program Planning, 29, 323–334. doi: 10.1016/j.evalprogplan.2006.02.002.CrossRefPubMedGoogle Scholar
- Leeman, J., Calancie, L., Hartman, M. A., Escoffery, C. T., Herrmann, A. K., Tague, L. E., … & Samuel-Hodge, C. (2015). What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective? A systematic review. Implementation Science, 10, 80. doi: 10.1186/s13012-015-0272-7.CrossRefPubMedPubMedCentralGoogle Scholar
- Lyon, A. R., & Lewis, C. C. (2016). Designing health information technologies for uptake: Development and implementation of measurement feedback systems in mental health service delivery. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 344–349. doi: 10.1007/s10488-015-0704-3.CrossRefPubMedPubMedCentralGoogle Scholar
- Lyon, A. R., Lewis, C. C., Boyd, M. R., Hendrix, E., & Liu, F. (2016a). Capabilities and characteristics of digital measurement feedback systems: Results from a comprehensive review. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 441–466. doi: 10.1007/s10488-016-0719-4.CrossRefPubMedPubMedCentralGoogle Scholar
- Lyon, A. R., Ludwig, K., Wasse, J. K., Bergstrom, A., Hendrix, E., & McCauley, E. (2016b). Determinants and functions of standardized assessment use among school mental health clinicians: A mixed methods evaluation. Administration and Policy in Mental Health and Mental Health Services Research, 43(1), 122–134. doi: 10.1007/s10488-015-0626-0.CrossRefPubMedPubMedCentralGoogle Scholar
- Lyon, A. R., Pullmann, M. D., Whitaker, K., Ludwig, K., Wasse, J. K., & McCauley, E. (2017). A digital feedback system to support implementation of measurement-based care by school-based mental health clinicians. Journal of Clinical Child and Adolescent Psychology, 00, 1–12. doi: 10.1080/15374416.2017.1280808.CrossRefGoogle Scholar
- Lyon, A. R., Wasse, J. K., Ludwig, K., Zachry, M., Bruns, E. J., Unützer, J., & McCauley, E. (2016c). The contextualized technology adaptation process (CTAP): Optimizing health information technology to improve mental health systems. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 394–409. doi: 10.1007/s10488-015-0637-x.CrossRefPubMedPubMedCentralGoogle Scholar
- McIntyre, L. L., Gresham, F. M., Digennaro, F. D., & Reed, D. D. (2007). Treatment integrity of school-based interventions with children in Journal of Applied Behavioral Analysis studies from 1991 to 2005. Journal of Applied Behavior Analysis, 40, 659–672. doi: 10.1901/jaba.2007.659-672.CrossRefPubMedGoogle Scholar
- Nadeem, E., Cappella, E., Holland, S., Coccaro, C., & Crisonino, G. (2016). Development and piloting of a classroom-focused measurement feedback system. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 379–393. doi: 10.1007/s10488-015-0651-z.CrossRefPubMedPubMedCentralGoogle Scholar
- Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53. doi: 10.1007/s10488-010-0314-z.CrossRefPubMedGoogle Scholar
- Proctor, E. K., & Brownson, R. C. (2012). Measurement issues in dissemination and implementation research. In R. C. Brownson, G. A. Colditz & E. K. Proctor (Eds.), Dissemination and implementation research in health: Translating science to practice (pp. 261–280). New York, NY: Oxford University Press, Inc. doi: 10.1093/acprof:oso/9780199751877.003.0013.CrossRefGoogle Scholar
- Sopko, K. M. (2006). School mental health services in the United States. InForum, 1–9. Available from National Association of State Directors of Special Education website: http://nasdse.org/DesktopModules/DNNspot-Store/ProductFiles/180_a75a2595-fb39-4f38-ac15-46b4d9bc45b6.pdf.
- Todd, N. R., Allen, N. E., & Javdani, S. (2012). Multilevel modeling: Method and application for community-based research. In L. A. Jason & D. S. Glenwick (Eds.), Methodological approaches to community-based research (pp. 167–186). Washington D.C.: American Psychological Association. 10.1037/13492-010.CrossRefGoogle Scholar
- Wandersman, A., Chien, V. H., & Katz, J. (2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology, 50, 445–459. doi: 10.1007/s10464-012-9509-7.CrossRefPubMedGoogle Scholar
- Weiner, B. J., Amick, H., & Lee, S.-Y. D. (2008). Conceptualization and measurement of organizational readiness for change: A review of the literature in health services research and other fields. Medical Care Research and Review, 65(4), 379–436. doi: 10.1177/1077558708317802.CrossRefPubMedGoogle Scholar