Skip to main content
SpringerLink
Log in

Integrity of Evidence-Based Practice: Are Providers Modifying Practice Content or Practice Sequencing?

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

This study examined patterns of evidence-based treatment (EBT) implementation within community settings by evaluating integrity along separate dimensions of practice content (PC; a session included the prescribed procedure) and practice sequencing (a session occurred in the prescribed sequence) within a recent randomized effectiveness trial. We measured whether sessions showed integrity to PC and to flexible or linear practice sequences. Findings revealed that providers tended to incorporate content from the EBT protocol in most treatment sessions, but that the sequencing of the sessions was often modified, suggesting that providers are amenable to evidence-based procedures, but not necessarily their prescribed arrangement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • American Psychiatric Association. (2000). American Psychiatric Association Diagnostic and statistical manual of mental disorders: DSM-IV-TR. Washington, DC: American Psychiatric Association.

    Google Scholar 

  • Barkley, R. A. (1997). Defiant children: A provider’s manual for assessment and parent training (2nd ed.). New York: Guilford Press, New York, NY.

    Google Scholar 

  • Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114–1119. doi:10.1097/CHI.0b013e3181825af8.

    Article  PubMed Central  PubMed  Google Scholar 

  • Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to providers on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62(12), 1423–1429. doi:10.1176/appi.ps.002052011.

    Article  PubMed  Google Scholar 

  • Calhoun, K. S., Moras, K., Pilkonis, P. A., & Rehm, L. P. (1998). Empirically supported treatments: Implications for training. Journal of Consulting and Clinical Psychology, 66(1), 151–162. doi:10.1037/0022-006X.66.1.151.

    Article  CAS  PubMed  Google Scholar 

  • Carroll, K. M., & Rounsaville, B. J. (2007). A vision of the next generation of behavioral therapies research in the addictions. Addiction, 102(6), 850–862.

    Article  PubMed Central  PubMed  Google Scholar 

  • Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18. doi:10.1037/0022-006X.66.1.7.

    Article  CAS  PubMed  Google Scholar 

  • Chapman, J. E., McCart, M. R., Letourneau, E. J., & Sheidow, A. J. (2013). Comparison of youth, caregiver, therapist, trained, and treatment expert raters of therapist adherence to a substance abuse treatment protocol. Journal of Consulting and Clinical Psychology, 81(4), 674–680. doi:10.1037/a0033021.

    Article  PubMed Central  PubMed  Google Scholar 

  • Chorpita, B. F., & Daleiden, E. L. (2014). Structuring the collaboration of science and service in pursuit of a shared vision. Journal of Clinical Child and Adolescent Psychology, 43(2), 323–338. doi:10.1080/15374416.2013.828297.

  • Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005). Modularity in the design and application of therapeutic interventions. Applied and Preventive Psychology, 11(3), 141–156. doi:10.1016/j.appsy.2005.05.002.

    Article  Google Scholar 

  • Chorpita, B. F., & Southam-Gerow, M. (2006). Fears and anxieties. New York: Guilford Press.

    Google Scholar 

  • Chorpita, B. F., & Weisz, J. R. (2005). Modular approach to therapy for children with anxiety, depression, or conduct problems. Honolulu, HI: University of Hawaii at Manoa; Boston: Judge Baker Children’s Center; Harvard Medical School.

  • Chorpita, B. F., Weisz, J. R., Daleiden, E. L., Schoenwald, S. K., Palinkas, L. A., Miranda, J., et al. (2013). Long-term outcomes for the child STEPs randomized effectiveness trial: A comparison of modular and standard treatment designs with usual care. Journal of Consulting and Clinical Psychology, 81(6), 999–1009. doi:10.1037/a0034200.

    Article  PubMed  Google Scholar 

  • Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating integrity: Predictive validity for a measure of competent adherence to the Oregon Model of Parent Management Training. Behavior Therapy, 36(1), 3–13. doi:10.1037/0022-006X.67.5.711, 10.1037//0022-006X.65.5.821.

  • Hoagwood, K. E., Vogel, J. M., Levitt, J. M., D’Amico, P. J., Paisner, W. I., & Kaplan, S. J. (2007). Implementing an evidence-based trauma treatment in a state system after September 11: The CATS Project. Journal of the American Academy of Child and Adolescent Psychiatry, 46(6), 773–779. doi:10.1097/chi.0b013e3180413def.

    Article  PubMed  Google Scholar 

  • Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 230–244. doi:10.1007/s10488-009-0251-x.

    Article  PubMed Central  PubMed  Google Scholar 

  • Kendall, P. C., & Beidas, R. S. (2007). Smoothing the trail for dissemination of evidence-based practices for youth: Flexibility within fidelity. Professional Psychology: Research and Practice, 38(1), 13–20. doi:10.1037/0735-7028.38.1.13.

    Article  Google Scholar 

  • Kendall, P. C., Gosch, E., Furr, J. M., & Sood, E. (2008). Flexibility within fidelity. Journal of the American Academy of Child and Adolescent Psychiatry, 47(9), 987–993. doi:10.1097/CHI.0b013e31817eed2f.

    Article  PubMed  Google Scholar 

  • Kendall, P. C., Kane, M., Howard, B., & Siqueland, L. (1990). Cognitive-behavioral treatment of anxious children. Ardmore, PA: Workbook Publishing.

    Google Scholar 

  • Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions. Journal of Clinical Psychology, 61(2), 165–174.

    Article  PubMed  Google Scholar 

  • Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001). The effects of providing providers with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11(1), 49–68. doi:10.1093/ptr/11.1.49.

    Article  Google Scholar 

  • Lonigan, C. J., Elbert, J. C., & Johnson, S. B. (1998). Empirically supported psychosocial interventions for children: An overview. Journal of Clinical Child Psychology, 27(2), 138–145.

    Article  CAS  PubMed  Google Scholar 

  • McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodríguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20(1), 14–32. doi:10.1111/cpsp.12020.

    Google Scholar 

  • National Advisory Mental Health Council Workgroup on Child and Adolescent Mental Health Intervention and Deployment. (2001). Blueprint for change: Research on child and adolescent mental health. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health.

    Google Scholar 

  • Office of the Surgeon General. (1999). Mental health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services.

    Google Scholar 

  • Office of the Surgeon General. (2004). Report of the Surgeon General’s Conference on children’s mental health: A national action agenda. Rockville, MD: U.S. Department of Health and Human Services.

    Google Scholar 

  • Palinkas, L. A., Aarons, G. A., Chorpita, B. F., Hoagwood, K., Landsverk, J., & Weisz, J. R. (2009). Cultural exchange and the implementation of evidence-based practices: Two case studies. Research on Social Work Practice, 19(5), 602–612. doi:10.1177/1049731509335529.

    Article  Google Scholar 

  • Palinkas, L. A., Weisz, J. R., Chorpita, B. F., Garland, A., Hoagwood, K. E., Landsverk, J., et al. (2013). Use of evidence-based treatments for youth mental health subsequent to a randomized controlled effectiveness trial: A qualitative study. Psychiatric Services, 64(11), 1110–1118. doi:10.1176/appi.ps.004682012.

    Article  PubMed  Google Scholar 

  • Perepletchikova, F., Hilt, L. M., Chereji, E., & Kazdin, A. E. (2009). Barriers to implementing treatment integrity procedures: Survey of treatment outcome researchers. Journal of Consulting and Clinical Psychology, 77(2), 212–218. doi:10.1037/a0015232.

    Article  PubMed  Google Scholar 

  • Perepletchikova, F., & Kazdin, A. E. (2009). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12(4), 365–383. doi:10.1093/clipsy/bpi045.

    Google Scholar 

  • President’s New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America: Final report. DHHS Publication number SMA-03-3832. Rockville, MD.

  • Proctor, E. K. (2004). Leverage points for the implementation of evidence-based practice. Brief Treatment and Crisis Intervention, 4(3), 227. doi:10.1093/brief-treatment/mhh020.

    Article  Google Scholar 

  • Regan, J., Daleiden, E. L., & Chorpita, B. F. (2013). Integrity in mental health systems: An expanded framework for managing uncertainty in clinical care. Clinical Psychology: Science and Practice, 20(1), 78–98. doi:10.1111/cpsp.12024.

    Google Scholar 

  • Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. doi:10.1007/s10488-010-0321-0.

    Article  PubMed Central  PubMed  Google Scholar 

  • Schoenwald, S. K., Kelleher, K., & Weisz, J. R. (2008). Building bridges to evidence-based practice: The MacArthur Foundation Child System and Treatment Enhancement Projects (Child STEPs). Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 66–72. doi:10.1007/s10488-006-0047-1.2006-08150-01010.1007/s10488-006-0047-1.

    Article  PubMed  Google Scholar 

  • Southam-Gerow, M. A., Chorpita, B. F., Miller, L. M., & Gleacher, A. A. (2008). Are children with anxiety disorders self-referred to a university clinic like those from the public mental health system? Administration and Policy in Mental Health and Mental Health Services Research, 35, 168–180. doi:10.1007/s10488-007-0154-7.

    Article  PubMed  Google Scholar 

  • Southam-Gerow, M. A., & McLeod, B. D. (2013). Advances in applying treatment integrity research for dissemination and implementation science: Introduction to special issue. Clinical Psychology: Science and Practice, 20, 1–13.

    Google Scholar 

  • Southam-Gerow, M. A., Weisz, J. R., Chu, B. C., McLeod, B. D., Gordis, E. B., & Connor-Smith, J. K. (2010). Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child and Adolescent Psychiatry, 49(10), 1043–1052.

    Article  PubMed Central  PubMed  Google Scholar 

  • Stirman, S. W., Calloway, A., Toder, K., Miller, C. J., DeVito, A. K., Meisel, S. N., et al. (2013a). Community mental health provider modifications to cognitive therapy: Implications for sustainability. Psychiatric Services, 64(10), 1056–1059. doi:10.1176/appi.ps.201200456.

    Article  Google Scholar 

  • Stirman, S. W., Miller, C. J., Toder, K., & Calloway, A. (2013b). Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science, 8(1), 65. doi:10.1186/1748-5908-8-65.

    Article  PubMed Central  PubMed  Google Scholar 

  • Travis, J. K., & Brestan-Knight, E. (2013). A pilot study examining trainee treatment session fidelity when parent–child interaction therapy (PCIT) is implemented in community settings. The Journal of Behavioral Health Services and Research, 40(3), 342–354. doi:10.1007/s11414-013-9326-2.

    Article  PubMed  Google Scholar 

  • Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., et al. (2013). Tracking evidence based practice with youth: Validity of the match and standard manual consultation records. Journal of Clinical Child and Adolescent Psychology, 42(1), 44–55. doi:10.1080/15374416.2012.700505.

    Article  PubMed  Google Scholar 

  • Weersing, V. R., Weisz, J. R., & Donenberg, G. R. (2002). Development of the therapy procedures checklist: A provider-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology, 31(2), 168–180. doi:10.1037//0003-066X.51.7.689.

    Article  PubMed  Google Scholar 

  • Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S., et al. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. doi:10.1007/s10488-007-0151-x.2008-00915-01110.1007/s10488-007-0151-x.

    Article  PubMed  Google Scholar 

  • Weisz, J. R., & Gray, J. S. (2008). Evidence-based psychotherapy for children and adolescents: Data from the present and a model for the future. Child and Adolescent Mental Health, 13(2), 54–65. doi:10.1111/j.1475-3588.2007.00475.x.

    Article  Google Scholar 

  • Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61(7), 671–689. doi:10.1207/s15374424jccp3401_11.2005-01144-01110.1207/s15374424jccp3401_11.

    Article  PubMed  Google Scholar 

  • Weisz, J. R., & Kazdin, A. E. (2010). The present and future of evidence-based psychotherapies for children and adolescents (pp. 557–572). New York: Guilford Press.

    Google Scholar 

  • Weisz, J. R., Moore, P. S., Southam-Gerow, M. A., Weersing, V. R., Valeri, S. M., & McCarty, C. A. (2005). Provider’s manual PASCET: Primary and secondary control enhancement training program (3rd ed.). Los Angeles: University of California.

    Google Scholar 

  • Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects of psychotherapy with children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin, 117(3), 450–468.

    Article  CAS  PubMed  Google Scholar 

  • Weller, E. B., Weller, R. A., Rooney, M. T., & Fristad, M. A. (1999a). Children’s interview for psychiatric syndromes—Parent version (P-ChIPS). Arlington, VA: American Psychiatric Association, Arlington, VA.

    Google Scholar 

  • Weller, E. B., Weller, R. A., Rooney, M. T., & Fristad, M. A. (1999b). Children’s interview for psychiatric syndromes: ChIPS. Arlington, VA: American Psychiatric Association, Arlington, VA.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of California, Los Angeles, 1285 Franz Hall, Box 951563, Los Angeles, CA, 90095, USA

    Alayna L. Park, Bruce F. Chorpita & Jennifer Regan

  2. Department of Psychology, Harvard University, William James Hall, 33 Kirkland Street, Cambridge, MA, 02138, USA

    John R. Weisz

Authors
  1. Alayna L. Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruce F. Chorpita
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jennifer Regan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John R. Weisz
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

The Research Network on Youth Mental Health

Corresponding author

Correspondence to Alayna L. Park.

Additional information

During the time of this study, the Research Network on Youth Mental Health included Bruce F. Chorpita, Ann Garland, Robert Gibbons, Charles Glisson, Evelyn Polk Green, Kimberly Hoagwood, Kelly Kelleher, John Landsverk, Stephen Mayberg, Jeanne Miranda, Lawrence A. Palinkas, Sonja K. Schoenwald, and John R. Weisz (Network Director).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, A.L., Chorpita, B.F., Regan, J. et al. Integrity of Evidence-Based Practice: Are Providers Modifying Practice Content or Practice Sequencing?. Adm Policy Ment Health 42, 186–196 (2015). https://doi.org/10.1007/s10488-014-0559-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-014-0559-z

Keywords

Search

Navigation