Abstract
Adherence to intervention content and delivery protocols is vital in establishing the efficacy of treatment programs for mental illnesses. Using a fidelity tool during interventions can substantially increase the likelihood of clients receiving the most scientifically rigorous treatment. This article outlines the steps taken to develop a fidelity checklist to measure treatment adherence of a two-part intervention delivered in a prison setting. Researchers followed the five-step guide by Feely et al. (Child and Adolescent Social Work Journal, 35(2), 139–152: 2018) and describe the process of developing a fidelity tool to measure treatment adherence to a newly adapted CBT-based intervention designed to maximize uptake for participants with serious mental illnesses. Key decision points are discussed, along with final decisions and contextual considerations. A 26-item checklist was developed to measure treatment adherence related to process, content, and adaptations of the intervention. The checklist follows the structure of the CBT intervention, as well as provides flexibility for the delivery adaptations. Pilot testing of the checklist revealed all sessions were implemented with at least 85% fidelity, and 90% of sessions were implemented with at least 90% fidelity. Raters agreed on the fidelity of a session in 99.6% of sessions. Contextual considerations included the highly secure study setting, reconciling the constant monitoring of a group and creating a treatment environment, the flexibility mandated by the intervention, the relative newness of the intervention, and the limitations based on study aims and resources. These results illustrate how study specific considerations and challenges can be successfully navigated in the development and deployment of a fidelity tool in a real-world setting. The fidelity checklist achieved our goal of measuring treatment adherence for this intervention. In the development of a fidelity tool, we recommend leaving space for raters to note specific considerations that disrupt facilitators’ ability to deploy the intervention precisely. Measuring fidelity is imperative for mental health interventions to ensure that the treatment is responsible for the changes observed in clients.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
02 March 2021
A Correction to this paper has been published: https://doi.org/10.1007/s10597-021-00804-x
References
Alexander, M. (2011). Applying implementation research to improve community corrections Making sure that “new” thing sticks! Federal Probation, 75(2), 47–51 99. http://libproxy.lib.unc.edu/login?url=https://www.proquest.com/trade-journals/applying-implementation-research-improve/docview/902184915/se-2?accountid=14244.
Blank Wilson, A., Farkas, K., Bonfine, N., & Duda, J. (2018). Criminogenic interventions for justice involved persons with serious mental illnesses: A targeted service delivery approach. International Journal of Offender Therapy and Comparative Criminology, 62(14), 4677–4693. https://doi.org/10.1177/0306624X18759242.
Bond, G. R., Drake, R. E., McHugo, G. J., Rapp, C. A., & Whitley, R. (2009). Strategies for improving fidelity in the national evidence-based practices project. Research on Social Work Practice, 19(5), 569–581. https://doi.org/10.1177/1049731509335531.
Bonta, J., & Andrews, D. A. (2017). The psychology of criminal conduct (6th ed.). London: Routledge. https://doi.org/10.1111/bjop.12254.
Bush, J., Glick, B., Taymans, J., & Guevara, M. (2011). Thinking for a change: Integrated cognitive behavior change program (3.1 ver.). Retrieved from National Institute of corrections, U.S. Department of Justice website: http://nicic.gov/t4c
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104.
Corley, N. A., & Kim, I. (2016). An assessment of intervention fidelity in published social work intervention research studies. Research on Social Work Practice, 26(1), 53–60. https://doi.org/10.1177/1049731515579419.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3), 327–350. https://doi.org/10.1007/s10464-008-9165-0.
Feely, M., Seay, K. D., Lanier, P., Auslander, W., & Kohl, P. L. (2018). Measuring fidelity in research studies: A field guide to developing a comprehensive fidelity measurement system. Child and Adolescent Social Work Journal, 35(2), 139–152. https://doi.org/10.1007/s10560-017-0512-6.
Fraser, M. W., Richman, J. M., Galinsky, M. J., & Day, S. H. (2009). Intervention research: Developing social programs. Oxford: Oxford University Press.
Gearing, R. E., El-Bassel, N., Ghesquiere, A., Baldwin, S., Gillies, J., & Ngeow, E. (2011). Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review, 31(1), 79–88. https://doi.org/10.1016/j.cpr.2010.09.007.
Glasgow, R. E., & Chambers, D. (2012). Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clinical and Translational Science, 5(1), 48–55. https://doi.org/10.1111/j.1752-8062.2011.00383.x.
Glasgow, R. E., Vinson, C., Chambers, D., Khoury, M. J., Kaplan, R. M., & Hunter, C. (2012). National Institutes of Health approaches to dissemination and implementation science: Current and future directions. American Journal of Public Health, 102(7), 1274–1281. https://doi.org/10.2105/AJPH.2012.300755.
Kelly, P. J., Emerson, A., Fair, C., & Ramaswamy, M. (2018). Assessing fidelity: Balancing methodology and reality in jail interventions. BMC Women’s Health, 18 http://dx.doi.org.libproxy.lib.unc.edu/10.1186/s12905-018-0617-x.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310.
Lipsey, M. W., Howell, J. C., Kelly, M. R., Chapman, G., & Carver, D. (2010). Improving the effectiveness of juvenile justice programs: A new perspective on evidence-based practice. https://cjjr.georgetown.edu/wp-content/uploads/2014/12/ebppaper.pdf
Palinkas, L. A., Schoenwald, S. K., Hoagwood, K., Landsverk, J., Chorpita, B. F., & Weiss, J. R. (2008). An ethnographic study of implementation of evidence-based treatments in child mental health: First steps. Psychiatric Services, 59(7), 738–746. https://doi.org/10.1176/ps.2008.59.7.738.
Paulauskaite, L., Hassiotis, A., & Ali, A. (2019). A systematic review of fidelity measurements in complex interventions for people with intellectual disabilities and behaviours that challenge. Advances in Mental Health and Intellectual Disabilities, 13(3/4), 158–172. https://doi.org/10.1108/AMHID-09-2018-0040.
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health, 36(1), 1–17. https://doi.org/10.1007/s10488-008-0197-4.
Schoenwald, S. K., & Henggeler, S. W. (2004). A public health perspective on the transport of evidence-based practices. Clinical Psychology: Science and Practice, 11(4), 360–363. https://doi.org/10.1093/clipsy/bph092.
Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. https://doi.org/10.1007/s10488-010-0321-0.
StataCorp. 2017. Stata statistical software: Release 15. StataCorp.
Swindle, T., Selig, J. P., Rutledge, J. M., Whiteside-Mansell, L., & Curran, G. (2018). Fidelity monitoring in complex interventions: A case study of the WISE intervention. Archives of Public Health, 76. https://doi.org/10.1186/s13690-018-0292-2.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors report no conflicts of interest in the conducting or reporting of this research.
Ethical approval
All procedures and data collection for this study were reviewed and approved by the Institutional Review Board at the University of North Carolina-Chapel Hill. These procedures were performed in line with the tenets of the Declaration of Helsinki. Informed consent was obtained from all study participants. The data used in this analysis does not include information collected from or about participants in this research study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number R34M111855. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
The original online version of this article was revised: the last sentence in “Ethical approval” section has been corrected.
Rights and permissions
About this article
Cite this article
Givens, A., Francis, A.M., Wilson, A.B. et al. Accountability in Intervention Research: Developing a Fidelity Checklist of a Mental Health Intervention in Prisons. Community Ment Health J 57, 1288–1299 (2021). https://doi.org/10.1007/s10597-021-00777-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10597-021-00777-x