Skip to main content

Advertisement

SpringerLink
Log in

Clinician Use of Standardized Assessments Following a Common Elements Psychotherapy Training and Consultation Program

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Despite increasing emphasis on the implementation of evidence-based treatments in community service settings, little attention has been paid to supporting the use of evidence-based assessment (EBA) methods and processes, a parallel component of evidence-based practice. Standardized assessment (SA) tools represent a key aspect of EBA and are central to data-driven clinical decision making. The current study evaluated the impact of a statewide training and consultation program in a common elements approach to psychotherapy. Practitioner attitudes toward, skill applying, and use of SA tools across four time points (pre-training, post-training, post-consultation, and follow-up) were assessed. Results indicated early increases in positive SA attitudes, with more gradual increases in self-reported SA skill and use. Implications for supporting the sustained use of SA tools are discussed, including the use of measurement feedback systems, reminders, and SA-supportive supervision practices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.

    Article  PubMed Central  PubMed  Google Scholar 

  • Angold, A., & Costello, E. J. (1987). Mood and feelings questionnaire (MFQ). Durham, NC: Developmental Epidemiology Program, Duke University.

    Google Scholar 

  • Barrington, J., Prior, M., Richardson, M., & Allen, K. (2005). Effectiveness of CBT versus standard treatment for childhood anxiety disorders in a community clinic setting. Behaviour Change, 22, 29–43.

    Article  Google Scholar 

  • Beidas, R. S., & Kendall, P. C. (2010). Training providers in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30.

    Google Scholar 

  • Berliner, L., Dorsey, S., Merchant, L., Jungbluth, N., Sedlar, G. (2013). Practical Guide for EBP Implementation in Public Health. Washington State Division of Behavior Health and Recovery, Harborview Center for Sexual Assault and Traumatic Stress and University of Washington, School of Medicine, Public Behavior Health and Justice Policy.

  • Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 1114–1119.

    Article  PubMed Central  PubMed  Google Scholar 

  • Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62, 1423–1429.

    Article  PubMed  Google Scholar 

  • Birmaher, B., Khetarpal, S., Brent, D., & Cully, M. (1997). The Screen for Child Anxiety Related Emotional Disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child and Adolescent Psychiatry, 36(4), 545–553.

    Article  CAS  PubMed  Google Scholar 

  • Borntrager, C. F., Chorpita, B. F., Higa-McMillan, C., & Weisz, J. R. (2009). Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals. Psychiatric Services, 60, 677–681.

    Article  PubMed  Google Scholar 

  • Brody, B. B., Cuffel, B., McColloch, J., Tani, S., Maruish, M., & Unützer, J. (2005). The acceptability and effectiveness of patient-reported assessments and feedback in a managed behavioral healthcare setting. The American Journal of Managed Care, 11(12), 774–780.

    Google Scholar 

  • Carlier, I. V., Meuldijk, D., Van Vleit, I. M., Van Fenema, E., Van der Wee, N. J., & Zitman, F. G. (2012). Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. Journal of Evaluating in Clinical Practice, 18(1), 104–110.

    Article  Google Scholar 

  • Chorpita, B. F., Bernstein, A., Daleiden, E. L., & The Research Network on Youth Mental Health. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35, 114–123.

    Article  PubMed  Google Scholar 

  • Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005a). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7, 5–20.

    Article  PubMed  Google Scholar 

  • Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005b). Modularity in the design and application of therapeutic interventions. Applied and Preventive Psychology, 11(3), 141–156.

    Article  Google Scholar 

  • Clark, D. M., Layard, R., Smithies, R., Richard, D. A., Suckling, R., & Wright, B. (2009). Improving access to psychological therapy: Initial evaluation of two UK demonstration sites. Behaviour Research and Therapy, 47, 910–920.

    Article  PubMed Central  PubMed  Google Scholar 

  • Close-Goedjen, J., & Saunders, S. M. (2002). The effect of technical support on clinician attitudes toward an outcome assessment instrument. Journal of Behavioral Health Services and Research, 29, 99–108.

    Article  PubMed  Google Scholar 

  • Cohen, J. A., Mannarino, A. P., & Deblinger, E. (2006). Treating trauma and traumatic grief in children and adolescents. New York: Guilford Press.

    Google Scholar 

  • Daleiden, E., & Chorpita, B. F. (2005). From data to wisdom: Quality improvement strategies supporting large-scale implementation of evidence based services. Child and Adolescent Psychiatric Clinics of North America, 14, 329–349.

    Article  PubMed  Google Scholar 

  • de Rond, M., de Wit, R., & van Dam, F. (2001). The implementation of a pain monitoring programme for nurses in daily clinical practice: Results of a follow-up study in five hospitals. Journal of Advanced Nursing, 35, 590–598.

    Article  PubMed  Google Scholar 

  • Dorsey, S., Berliner, L., Lyon, A. R., Pullmann, M., & Murray, L. K. (under review). A state-wide common elements initiative for children’s mental health.

  • Dorsey, S., Pullmann, M. D., Deblinger, E., Berliner, L., Kerns, S. E., Thompson, K., et al. (2013). Improving practice in community-based settings: a randomized trial of supervision–study protocol. Implementation Science, 8, 89.

    Article  PubMed Central  PubMed  Google Scholar 

  • Ell, K., Unützer, J., Aranda, M., Sanchez, K., & Lee, P. J. (2006). Routine PHQ-9 depression screening in home health care: Depression prevalence, clinical treatment characteristics, and screening implementation. Home Health Care Services Quarterly, 24(4), 1–19.

    Article  Google Scholar 

  • Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature (FMHI Publication No. 231). Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.

  • Foa, E. B., Johnson, K. M., Feeny, N. C., & Treadwell, K. H. (2001). The Child PTSD Symptom Scale: A preliminary examination of its psychometric properties. Journal of Clinical Child Psychology, 30(3), 376–384.

    Article  CAS  PubMed  Google Scholar 

  • Gardner, W., Murphy, M., Childs, G., Kelleher, K., Pagano, M., Jellinek, M., et al. (1999). The PSC-17: A brief pediatric symptom checklist with psychosocial problem subscales. A report from PROS and ASPN. Ambulatory Child Health, 5, 225–236.

    Google Scholar 

  • Garland, A. F., Haine-Schlagel, R., Accurso, E. C., Baker-Ericzen, M. J., & Brookman-Frazee, L. (2012). Exploring the effect of therapists’ treatment practices on client attendance in community-based care for children. Psychological Services, 9, 74–88.

    Article  PubMed Central  PubMed  Google Scholar 

  • Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley- King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health and Mental Health Services Research, 40, 6–22.

    Article  PubMed Central  PubMed  Google Scholar 

  • Garland, A. F., Kruse, M., & Aarons, G. A. (2003). Clinicians and outcome measurement: What’s the use? Journal of Behavioral Health Services and Research, 30, 393–405.

    Article  PubMed  Google Scholar 

  • Gilbody, S. M., House, A. O., & Sheldon, T. A. (2002). Psychiatrists in the UK do not use outcome measures. The British Journal of Psychiatry, 180, 101–103.

    Article  PubMed  Google Scholar 

  • Graham, I. D., & Tetroe, J. (2009). Learning from the US Department of Veterans Affairs Quality Enhancement Research Initiative: QUERI Series. Implementation Science, 4(1), 13.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., et al. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163.

    Article  PubMed  Google Scholar 

  • Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35, 485–491.

    Article  Google Scholar 

  • Hatfield, D. R., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others do not. Administration and Policy in Mental Health and Mental Health Services Research, 34, 283–291.

    Article  PubMed  Google Scholar 

  • Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A. H., Dellinger, E. P., et al. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine, 360, 491–499.

    Article  CAS  PubMed  Google Scholar 

  • Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466.

    Article  PubMed Central  PubMed  Google Scholar 

  • Higa-McMillan, C. K., Powell, C. K., Daleiden, E. L., & Mueller, C. W. (2011). Pursuing an evidence-based culture through contextualized feedback: Aligning youth outcomes and provider practices. Professional Psychology: Research and Practice, 42, 137–144.

    Article  Google Scholar 

  • Huffman, L. C., Martin, J., Botcheva, L., Williams, S. E., & Dyer-Friedman, J. (2004). Practitioners’ attitudes toward the use of treatment progress and outcomes data in child mental health services. Evaluation and the Health Professions, 27(2), 165–188.

    Article  PubMed  Google Scholar 

  • Hunsley, J., & Mash, E. J. (2005). Introduction to the special section on developing guidelines for the evidence-based assessment (EBA) of adult disorders. Psychological Assessment, 17, 251–255.

    Article  PubMed  Google Scholar 

  • Ivey, S. L., Scheffler, R., & Zazzali, J. L. (1998). Supply dynamics of the mental health workforce: Implications for health policy. The Milbank Quarterly, 76, 25–58.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Jensen-Doss, A. (2011). Practice involves more than treatment: How can evidence-based assessment catch up to evidence-based treatment? Clinical Psychology: Science and Practice, 18(2), 173–177.

    Google Scholar 

  • Jensen-Doss, A., & Hawley, K. M. (2010). Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child and Adolescent Psychology, 39, 885–896.

    Article  PubMed Central  PubMed  Google Scholar 

  • Jensen-Doss, A., & Hawley, K. M. (2011). Understanding clinicians’ diagnostic practices: Attitudes toward the utility of diagnosis and standardized diagnostic tools. Administration and Policy in Mental Health and Mental Health Services Research, 38(6), 476–485.

    Article  PubMed  Google Scholar 

  • Joyce, B., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

    Google Scholar 

  • Koerner, K. (2013, May). PracticeGround: An online platform to help therapists learn, implement, and measure the impact of EBPs. Paper presented at the Seattle Implementation Research Collaborative National Conference, Seattle, WA.

  • Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10, 288–301.

    Google Scholar 

  • Lyon, A. R., Borntrager, C., Nakamura, B., & Higa-McMillan, C. (2013). From distal to proximal: Routine educational data monitoring in school-based mental health. Advances in School Mental Health Promotion, 6(4), 263–279.

    Article  Google Scholar 

  • Lyon, A. R., Charlesworth-Attie, S., Vander Stoep, A., & McCauley, E. (2011a). Modular psychotherapy for youth with internalizing problems: Implementation with therapists in school-based health centers. School Psychology Review, 40, 569–581.

    Google Scholar 

  • Lyon, A. R., & Dorsey, S. (2010). The Current Assessment Practice Evaluation. Unpublished measure, University of Washington.

  • Lyon, A. R., Stirman, S. W., Kerns, S. E. U., & Bruns, E. J. (2011b). Developing the mental health workforce: Review and application of training strategies from multiple disciplines. Administration and Policy in Mental Health and Mental Health Services Research, 38, 238–253.

    Article  PubMed Central  PubMed  Google Scholar 

  • Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362–379.

    Article  PubMed  Google Scholar 

  • McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. American Psychologist, 65, 73–84.

    Article  PubMed  Google Scholar 

  • Muris, P., Merckelbach, H., Korver, P., & Meesters, C. (2000). Screening for trauma in children and adolescents: The validity of the Traumatic Stress Disorder scale of the Screen for Child Anxiety Related Emotional Disorders. Journal of Clinical Child Psychology, 29, 406–413.

    Article  CAS  PubMed  Google Scholar 

  • Nakamura, B. J., Mueller, C. W., Higa-McMillan, C., Okamura, K. H., Chang, J. P., Slavin, L., et al. (2013). Engineering youth service system infrastructure: Hawaii’s continued efforts at large-scale implementation through knowledge management strategies. Journal of Clinical Child and Adolescent Psychology. doi:10.1080/15374416.2013.812039.

    PubMed  Google Scholar 

  • Palmiter, D. J. (2004). A survey of the assessment practices of child and adolescent clinicians. American Journal of Orthopsychiatry, 74, 122–128.

    Article  PubMed  Google Scholar 

  • Pearsons, J. B., Thomas, C., & Liu, H. (2012, November). Can we train psychotherapists to monitor their clients’ progress at every therapy session? Initial findings. Paper presented at the Association for Behavioral and Cognitive Therapies Annual Convention, National Harbor, MD.

  • Poston, J. M., & Hanson, W. E. (2010). Meta-analysis of psychological assessment as a therapeutic intervention. Psychological Assessment, 22(2), 203–212.

    Article  PubMed  Google Scholar 

  • Robiner, W. N. (2006). The mental health professions: Workforce supply and demand, issues, and challenges. Clinical Psychology Review, 26(5), 600–625.

    Article  PubMed  Google Scholar 

  • Sburlati, E. S., Schniering, C. A., Lyneham, H. J., & Rapee, R. M. (2011). A model of therapist competencies for the empirically supported cognitive behavioral treatment of child and adolescent anxiety and depressive disorders. Clinical Child and Family Psychology Review, 14(1), 89–109.

    Article  PubMed  Google Scholar 

  • Shea, S., DuMouchel, W., & Bahamonde, L. (1996). A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. Journal of the American Medical Informatics Association, 3, 399–409.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Shojania, K. G., Jennings, A., Mayhew, A., Ramsay, C., Eccles, M., & Grimshaw, J. (2010). Effect of point-of-care computer reminders on physician behavior: A systematic review. Canadian Medical Association Journal, 182, E216–E225.

    Article  PubMed Central  PubMed  Google Scholar 

  • Southam-Gerow, M. A., Daleiden, E. L., Chorpita, B. F., Bae, C. Mitchell, C., Faye, M., et al. (in press). MAPping Los Angeles County: Taking an evidence-informed model of mental health care to scale. Journal of Clinical Child and Adolescent Psychology. doi:10.1080/15374416.2013.833098.

  • Southam-Gerow, M. A., Weisz, J. R., Chu, B. C., McLeod, B. D., Gordis, E. B., & Connor-Smith, J. K. (2010). Does CBT for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child and Adolescent Psychiatry, 49, 1043–1052.

    Article  PubMed Central  PubMed  Google Scholar 

  • Spielmans, G., Gatlin, E., & McFall, E. (2010). The efficacy of evidence-based psychotherapies versus usual care for youths: Controlling confounds in a meta-reanalysis. Psychotherapy Research, 20, 234–246.

    Article  PubMed  Google Scholar 

  • Stirman, S. W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7(17), 1–19.

    Google Scholar 

  • Substance Abuse and Mental Health Services Administration. (2012, January). Partners for Change Outcome Management System (PCOMS): International Center for Clinical Excellence. Retrieved from the National Registry of Evidence-based Programs and Practices Web site, http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=249. Accessed 27 Feb 2014.

  • Tabachnick, B. C., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Pearson Education, Inc.

    Google Scholar 

  • Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., et al. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69, 274–282.

    Article  PubMed  Google Scholar 

  • Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61, 671–689.

    Article  PubMed  Google Scholar 

  • Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70(7), 750–761.

    Article  CAS  PubMed  Google Scholar 

  • Weisz, J. R., Southam-Gerow, M. A., Gordis, E. B., Connor-Smith, J. K., Chu, B. C., Langer, D. A., et al. (2009). Cognitive-behavioral therapy versus usual clinical care for youth depression: An initial test of transportability to community clinics and clinicians. Journal of Consulting and Clinical Psychology, 77, 383–396.

    Article  PubMed Central  PubMed  Google Scholar 

  • Wolpert, M. (2014). Uses and abuses of patient reported outcome measures (PROMs): Potential iatrogenic impact of PROMs implementation and how it can be mitigated. Administration and Policy in Mental Health and Mental Health Services Research, 41(2), 141–145.

    Article  PubMed Central  PubMed  Google Scholar 

  • Youngstrom, E. A. (2008). Evidence based assessment is not evidence based medicine: Commentary on evidence-based assessment of cognitive functioning in pediatric psychology. Journal of Pediatric Psychology, 33, 1015–1020.

    Article  PubMed Central  Google Scholar 

Download references

Acknowledgments

This publication was made possible in part by funding from the Washington State Department of Social and Health Services, Division of Behavioral Health Recovery and from Grant Nos. F32 MH086978, K08 MH095939, and R01 MH095749, awarded from the National Institute of Mental Health (NIMH). Drs. Lyon and Dorsey are investigators with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI).

Author information

Authors and Affiliations

  1. University of Washington, 6200 NE 74th St., Suite 100, Seattle, WA, 98115, USA

    Aaron R. Lyon, Shannon Dorsey, Michael Pullmann, Jessica Silbaugh-Cowdin & Lucy Berliner

Authors
  1. Aaron R. Lyon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shannon Dorsey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Pullmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jessica Silbaugh-Cowdin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lucy Berliner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron R. Lyon.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lyon, A.R., Dorsey, S., Pullmann, M. et al. Clinician Use of Standardized Assessments Following a Common Elements Psychotherapy Training and Consultation Program. Adm Policy Ment Health 42, 47–60 (2015). https://doi.org/10.1007/s10488-014-0543-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-014-0543-7

Keywords

Search

Navigation