Skip to main content

Advertisement

SpringerLink
Log in

A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Despite the central role of training and consultation in the implementation of evidence-based psychological interventions (EBPIs), comprehensive reviews of research on training have highlighted serious gaps in knowledge regarding best practices. Consultation after initial didactic training appears to be of critical importance, but there has been very little research to determine optimal consultation format or interventions. This observational study compared two consultation formats that included review of session audio and feedback in the context of a program to train clinicians (n = 85) in community mental health clinics to deliver cognitive therapy (CT). A “gold standard” condition in which clinicians received individual feedback after expert consultants reviewed full sessions was compared to a group consultation format in which short segments of session audio were reviewed by a group of clinicians and an expert consultant. After adjusting for potential baseline differences between individuals in the two consultation conditions, few differences were found in terms of successful completion of the consultation phase or in terms of competence in CT at the end of consultation or after a 2 year follow-up. However, analyses did not support hypotheses regarding non-inferiority of the group consultation condition. While both groups largely maintained competence, clinicians in the group consultation condition demonstrated increases in competence over the follow-up period, while a sub-group of those in the individual condition experienced decreases. These findings, if replicated, have important implications for EBP implementation programs, as they suggest that observation and feedback is feasible in community mental health setting, and that employing this method in a group format is an effective and efficient consultation strategy that may enhance the implementation and sustainability of evidence-based psychotherapies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Baer, J. S., Rosengren, D. B., Dunn, C. W., Wells, E. A., Ogle, R. L., & Hartzler, B. (2004). An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug and Alcohol Dependence, 73(1), 99–106. doi:10.1016/j.drugalcdep.2003.10.001.

    Article  PubMed  Google Scholar 

  • Beck, A. T. (2005). The current state of cognitive therapy: A 40-year retrospective. Archives of General Psychiatry, 62(9), 953–959. doi:10.1001/archpsyc.62.9.953.

    Article  PubMed  Google Scholar 

  • Beidas, R., Edmunds, J., Cannuscio, C., Gallagher, M., Downey, M. M., & Kendall, P. (2013). Therapists perspectives on the effective elements of consultation following training. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 507–517. doi:10.1007/s10488-013-0475-7.

    Article  PubMed  PubMed Central  Google Scholar 

  • Beidas, R., Edmunds, J., Marcus, S., & Kendall, P. (2012). Training and consultation to promote implementation of an empirically supported treatment: A randomized trial. Psychiatric Services, 63(7), 660–665. doi:10.1176/appi.ps.201100401.

    Article  PubMed  PubMed Central  Google Scholar 

  • Beidas, R., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1. doi:10.1111/j.1468-2850.2009.01187.x.

    Google Scholar 

  • Blackwelder, W. (2004). Current issues in clinical equivalence trials. Journal of Dental Research, 83(suppl 1), C113–C115.

    Article  PubMed  Google Scholar 

  • Branson, A., Shafran, R., & Myles, P. (2015). Investigating the relationship between competence and patient outcome with CBT. Behaviour Research and Therapy, 68, 19–26. doi:10.1016/j.brat.2015.03.002.

    Article  PubMed  Google Scholar 

  • Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self evaluation of cognitive therapy performance: Do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36(5), 581. doi:10.1017/S1352465808004438.

    Article  Google Scholar 

  • Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications Inc.

    Google Scholar 

  • Carroll, K. M., Martino, S., & Rounsaville, B. J. (2010). No train, no gain? Clinical Psychology: Science and Practice, 17(1), 36–40. doi:10.1111/j.1468-2850.2009.01190.x.

    Google Scholar 

  • Chorpita, B. F., & Regan, J. (2009). Dissemination of effective mental health treatment procedures: Maximizing the return on a significant investment. Behaviour Research and Therapy, 47(11), 990–993. doi:10.1016/j.brat.2009.07.002.

    Article  PubMed  Google Scholar 

  • Cichetti, D. (1994). Guidelines, criteria, and rules of thumb for normed and standardized instruments in psychology. Psychological Assessment, 6(4), 284–290.

    Article  Google Scholar 

  • Creed, T. A., Stirman, S. W., Evans, A. C., & Beck, A. T. (2014). A model for implementation of cognitive therapy in community mental health: The Beck Initiative. The Behavior Therapist, 37(3).

  • D’Agostino, R. B., Massaro, J. M., & Sullivan, L. M. (2003). Non-inferiority trials: Design concepts and issues-the encounters of academic consultants in statistics. Statistics in Medicine, 22(2), 169–186.

    Article  PubMed  Google Scholar 

  • Eckardt, P. (2012). Propensity score estimates in multilevel models for causal inference. Nursing Research, 61(3), 213–223.

    Article  PubMed  Google Scholar 

  • Edmunds, J. M., Beidas, R. S., & Kendall, P. C. (2013). Dissemination and implementation of evidence-based practices: Training and consultation as implementation strategies. Clinical Psychology: Science and Practice, 20(2), 152–165. doi:10.1111/cpsp.12031.

    Google Scholar 

  • Feldstein, A. C., Glasgow, R. E., & Smith, D. H. (2008). A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Joint Commission Journal on Quality and Patient Safety, 34(4), 228–243. doi:10.2165/00148365-200806040-00001.

    Article  PubMed  Google Scholar 

  • Fixsen, D. L., Naoom, S. F., Blase, K. A., Friendman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: The National Implementation Research Network, Louis de la Parte Florida Mental Health Institute, University of South Florida.

    Google Scholar 

  • Fleming, L. M., Glass, J. A., Fujisaki, S., & Toner, S. L. (2010). Group process and learning: A grounded theory model of group supervision. Training and Education in Professional Psychology, 4(3), 194.

    Article  Google Scholar 

  • Franklin, M. E., Abramowitz, J. S., Kozak, M. J., Levitt, J. T., & Foa, E. B. (2000). Effectiveness of exposure and ritual prevention for obsessive-compulsive disorder: Randomized compared with nonrandomized samples. Journal of Consulting and Clinical Psychology, 68(4), 594. doi:10.1037/0022-006X.68.4.594.

    Article  CAS  PubMed  Google Scholar 

  • Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15(3), 234–249. doi:10.1037/a0019623.

    Article  PubMed  PubMed Central  Google Scholar 

  • Henggeler, S. W., Chapman, J. E., Rowland, M. D., Sheidow, A. J., & Cunningham, P. B. (2013). Evaluating training methods for transporting contingency management to therapists. Journal of Substance Abuse Treatment, 45(5), 466–474.

    Article  PubMed  PubMed Central  Google Scholar 

  • Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. doi:10.1016/j.cpr.2010.02.005.

    Article  PubMed  PubMed Central  Google Scholar 

  • Juster, H. R., Heimberg, R. G., & Engelberg, B. (1995). Self selection and sample selection in a treatment study of social phobia. Behavioral Research Therapy, 33(3), 321–324.

    Article  CAS  Google Scholar 

  • Karlin, B. E., Ruzek, J. I., Chard, K. M., Eftekhari, A., Monson, C. M., Hembree, E. A., et al. (2010). Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the veterans health administration. Journal of Traumatic Stress, 23(6), 663–673. doi:10.1002/jts.20588.

    Article  PubMed  Google Scholar 

  • Kolko, D. J., Baumann, B. L., Herschell, A. D., Hart, J. A., Holden, E. A., & Wisniewski, S. R. (2012). Implementation of AF-CBT by community practitioners serving child welfare and mental health: A randomized trial. Child maltreatment. doi:10.1177/1077559511427346.

    PubMed Central  Google Scholar 

  • Lewandowsky, S., & Thomas, J. L. (2009). Expertise: Acquisition, limitations, and control. Reviews of human factors and ergonomics, 5(1), 140–165.

    Article  Google Scholar 

  • McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist, 65, 73–84. doi:10.1037/a0018121.

    Article  PubMed  Google Scholar 

  • Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72(6), 1050–1062. doi:10.1037/0022-006X.72.6.1050.

    Article  PubMed  Google Scholar 

  • Milne, D. (2014). Beyond the “Acid Test”: A conceptual review and reformulation of outcome evaluation in clinical supervision. American Journal of Psychotherapy, 68(2), 213–230.

    PubMed  Google Scholar 

  • Nacasch, N., Huppert, J. D., Su, Y.-J., Kivity, Y., Dinshtein, Y., Yeh, R., & Foa, E. B. (2014). Are 60-minute prolonged exposure sessions with 20-minute imaginal exposure to traumatic memories sufficient to successfully treat PTSD? A randomized noninferiority clinical trial. Behavior Therapy, 46, 328–341.

    Article  PubMed  Google Scholar 

  • Ögren, M.-L. E. C. (2009). Group supervision in psychotherapy. Main findings from a Swedish research project on psychotherapy supervision in a group format. British Journal of Guidance & Counselling, 37(2), 129–139. doi:10.1080/03069880902728614.

    Article  Google Scholar 

  • Rakovshik, S. G., & McManus, F. (2010). Establishing evidence-based training in cognitive behavioral therapy: A review of current empirical findings and theoretical guidance. Clinical Psychology Review, 30(5), 496–516. doi:10.1016/j.cpr.2010.03.004.

    Article  PubMed  Google Scholar 

  • Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173.

    Article  Google Scholar 

  • Ruzek, J. I., & Rosen, R. C. (2009). Disseminating evidence-based treatments for PTSD in organizational settings: A high priority focus area. Behaviour Research and Therapy, 47(11), 980–989. doi:10.1016/j.brat.2009.07.008.

    Article  PubMed  Google Scholar 

  • Schafer, J. L., & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13(4), 279–313. doi:10.1037/a0014268.

    Article  PubMed  Google Scholar 

  • Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-010-0321-0.

    PubMed  PubMed Central  Google Scholar 

  • Shaw, B. F., Elkin, I., Yamaguchi, J., Olmsted, M., Vallis, T. M., Dobson, K. S., et al. (1999). Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology, 67(6), 837.

    Article  CAS  PubMed  Google Scholar 

  • Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S. A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73(1), 106–115. doi:10.1037/0022-006X.73.1.106.

    Article  PubMed  PubMed Central  Google Scholar 

  • Simons, A. D., Padesky, C. A., Montemarano, J., Lewis, C. C., Murakami, J., Lamb, K., et al. (2010). Training and dissemination of cognitive behavior therapy for depression in adults: A preliminary examination of therapist competence and client outcomes. Journal of Consulting and Clinical Psychology, 78(5), 751–756. doi:10.1037/a0020569.

    Article  PubMed  Google Scholar 

  • Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. London: Oxford University Press.

    Book  Google Scholar 

  • Stirman, S. W., Bhar, S. S., Spokas, M., Brown, G. K., Creed, T. A., Perivoliotis, D., et al. (2010). Training and consultation in evidence-based psychosocial treatments in public mental health settings: The ACCESS model. Professional Psychology: Research and Practice, 41(1), 48–56. doi:10.1037/a0018099.

    Article  Google Scholar 

  • Stirman, S. W., Buchhofer, R., McLaulin, J. B., Evans, A. C., & Beck, A. T. (2009). The Beck initiative: A partnership to implement cognitive therapy in a community behavioral health system. Psychiatric Services, 60, 1302–1310. doi:10.1176/appi.ps.60.10.1302.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stirman, S. W., Crits-Christoph, P., & DeRubeis, R. J. (2004). Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clinical Psychology: Science and Practice, 11(4), 343–359. doi:10.1093/clipsy/bph091.

    Google Scholar 

  • Stirman, S. W., Gutierrez-Colina, A., Toder, K., Castro, F., Esposito, G., Barg, F., et al. (2013). Clinicians’ perspectives on cognitive therapy in community mental health settings: Implications for training and implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40(4), 274–285. doi:10.1007/s10488-012-0418-8.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stirman, S. W., Miller, C. J., Toder, K., Calloway, A., Crits-Christoph, P., & Beck, A. T. (2012). Perspectives on cognitive therapy training within community mental health settings: Implications for clinician satisfaction and skill development. Depression Research and Treatment, 2012, 1–11. doi:10.1155/2012/391084.

    Article  Google Scholar 

  • Swales, M. A., Taylor, B., & Hibbs, R. A. (2012). Implementing dialectical behaviour therapy: Programme survival in routine healthcare settings. Journal of Mental Health, 21(6), 548–555.

    Article  PubMed  Google Scholar 

  • Tracey, T. J., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69(3), 218.

    Article  PubMed  Google Scholar 

  • Weissman, M. M., Verdeli, H., Gameroff, M. J., Bledsoe, S. E., Betts, K., Mufson, L., et al. (2006). National survey of psychotherapy training in psychiatry, psychology, and social work. Archives of General Psychiatry, 63(8), 925–934.

    Article  PubMed  Google Scholar 

  • Young, J., & Beck, A. T. (1980). Cognitive therapy scale: Rating manual. Unpublished manuscript.

Download references

Acknowledgments

A portion of this work was supported by funding from the National Institute of Mental Health (R00 MH-080100; PI: Stirman). The authors wish to thanks Drs. Michael Suvak and Robert Gallop for their feedback on the statistical analyses in this paper.

Author information

Authors and Affiliations

  1. Dissemination and Training Division, National Center for PTSD and Department of Psychiatry and Behavioral Sciences, VA Palo Alto Healthcare System and Stanford University, 795 Willow Road (NC-PTSD), Menlo Park, CA, 94025, USA

    Shannon Wiltsey Stirman

  2. Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market St., Philadelphia, PA, 19104, USA

    Kristin Pontoski, Torrey Creed, Aaron T. Beck & Paul Crits-Christoph

  3. Department of Behavioral Health and Intellectual DisAbility Services, 801 Market Street, 7th Floor, Philadelphia, PA, 19107, USA

    Regina Xhezo & Arthur C. Evans

Authors
  1. Shannon Wiltsey Stirman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristin Pontoski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Torrey Creed
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Regina Xhezo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arthur C. Evans
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Aaron T. Beck
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Paul Crits-Christoph
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shannon Wiltsey Stirman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stirman, S.W., Pontoski, K., Creed, T. et al. A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy. Adm Policy Ment Health 44, 55–66 (2017). https://doi.org/10.1007/s10488-015-0700-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-015-0700-7

Keywords

Search

Navigation