Toward the Effective and Efficient Measurement of Implementation Fidelity
Implementation science in mental health is informed by other academic disciplines and industries. Conceptual and methodological territory charted in psychotherapy research is pertinent to two elements of the conceptual model of implementation posited by Aarons and colleagues (2010)—implementation fidelity and innovation feedback systems. Key characteristics of scientifically validated fidelity instruments, and of the feasibility of their use in routine care, are presented. The challenges of ensuring fidelity measurement methods are both effective (scientifically validated) and efficient (feasible and useful in routine care) are identified as are examples of implementation research attempting to balance these attributes of fidelity measurement.
KeywordsImplementation fidelity Fidelity measurement methods Adherence
- Aarons, G. A., Sommerfeld, D., Hecht, D. B., Silovsky, J. F., & Chaffin, M. J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77, 270–280.CrossRefPubMedGoogle Scholar
- Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2010). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research.Google Scholar
- AERA, APA, & NCME. (1999). Standards for educational and psychological testing. Washington, DC: AERA.Google Scholar
- Bearsley-Smith, C., Sellick, K., Chesters, J., Francis, K., & Gippsland Adolescent Depression Research Group. (2008). Treatment content in child and adolescent mental health services: Development of the treatment recording sheet. Administration and Policy Mental Health, 35, 423–435.CrossRefGoogle Scholar
- Chamberlain, P., Sprengelmeyer, P., Saldana, L., & Padget, C. (2010) Web-based observations to monitor treatment fidelity. Manuscript in preparation.Google Scholar
- Child and Adolescent Mental Health Division (2003). Instructions and codebook for provider monthly summaries. Honolulu, HI: Hawaii Department of Health Child and Adolescent Mental Health Division. www.hawaii.gov/health/mental-health/camhd/resources/prov-agency/library/pdf/paf/paf-001.pdf. Accessed 23 March 2010.
- Fixsen, D. L, Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).Google Scholar
- Garland, A. F., Hawley, K. M., Brookman-Frazee, L., & Hurlburt, M. S. (2008a). Identifying common elements of evidence-based psychosocial treatments for children’s disruptive behavior disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 505–514.CrossRefPubMedGoogle Scholar
- Hayes, S. C. (1998). Market-driven treatment development. The Behavior Therapist, 21, 32–33.Google Scholar
- Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., Rowland, M. D., & Cunningham, P. B. (2009). Multisystemic therapy for antisocial behavior in children and adolescents (2nd ed.). New York: The Guilford Press.Google Scholar
- Hepner, K. A., Greenwood, G. L., Azocar, F., Miranda, J., & Burnam, M. A. (2009). Usual care psychotherapy for depression in a large managed behavioral health organization. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-009-0247-6.
- Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy, 33, 332–345.Google Scholar
- Kelley, S. D., Vides de Andrade, A. R., Sheffer, E., & Bickman, L. (2010). Exploring the black box: Measuring youth treatment process and progress in usual care. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-010-0298-8.
- Malik, M. L., Beutler, L. E., Alimohamed, S., Gallagher-Thompson, D., & Thompson, L. (2003). Are all cognitive therapies alike? A comparison of cognitive and noncognitive therapy process and implications for the application of empirically supported treatments. Journal of Consulting and Clinical Psychology, 71, 150–158.CrossRefPubMedGoogle Scholar
- Mihalic, S. (2004). The importance of implementation fidelity. Emotional and Behavioral Disorders in Youth, 4(83–86), 99–105.Google Scholar
- Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.Google Scholar
- National Institute of Mental Health. (2006). The road ahead: Research partnerships to transform services. A report by the National Advisory Mental Health Council’s Workgroup on Services and Clinical Epidemiology Research. Rockville, MD: National Institutes of Health, Department of Health and Human Services.Google Scholar
- Orimoto, T. E., Higa-McMillan, C. K., Mueller, C., & Tolman, R. T. (2009, February). Organization of therapeutic practices in treatment as usual. Paper presented at the 22nd annual research conference of the Research and Training Center for Children’s Mental Health, Tampa, FL.Google Scholar
- Schoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., et al. (2008). A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs). Administration and Policy in Mental Health and Mental Health Services Research, 35, 84–97.CrossRefPubMedGoogle Scholar
- Weersing, V. R., Weisz, J. R., & Donenberg, G. R. (2002). Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child Psychology, 31, 168–180.Google Scholar