Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Clinician Use of Standardized Assessments Following a Common Elements Psychotherapy Training and Consultation Program

Abstract

Despite increasing emphasis on the implementation of evidence-based treatments in community service settings, little attention has been paid to supporting the use of evidence-based assessment (EBA) methods and processes, a parallel component of evidence-based practice. Standardized assessment (SA) tools represent a key aspect of EBA and are central to data-driven clinical decision making. The current study evaluated the impact of a statewide training and consultation program in a common elements approach to psychotherapy. Practitioner attitudes toward, skill applying, and use of SA tools across four time points (pre-training, post-training, post-consultation, and follow-up) were assessed. Results indicated early increases in positive SA attitudes, with more gradual increases in self-reported SA skill and use. Implications for supporting the sustained use of SA tools are discussed, including the use of measurement feedback systems, reminders, and SA-supportive supervision practices.

This is a preview of subscription content, log in to check access.

Fig. 1

References

  1. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.

  2. Angold, A., & Costello, E. J. (1987). Mood and feelings questionnaire (MFQ). Durham, NC: Developmental Epidemiology Program, Duke University.

  3. Barrington, J., Prior, M., Richardson, M., & Allen, K. (2005). Effectiveness of CBT versus standard treatment for childhood anxiety disorders in a community clinic setting. Behaviour Change, 22, 29–43.

  4. Beidas, R. S., & Kendall, P. C. (2010). Training providers in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30.

  5. Berliner, L., Dorsey, S., Merchant, L., Jungbluth, N., Sedlar, G. (2013). Practical Guide for EBP Implementation in Public Health. Washington State Division of Behavior Health and Recovery, Harborview Center for Sexual Assault and Traumatic Stress and University of Washington, School of Medicine, Public Behavior Health and Justice Policy.

  6. Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 1114–1119.

  7. Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62, 1423–1429.

  8. Birmaher, B., Khetarpal, S., Brent, D., & Cully, M. (1997). The Screen for Child Anxiety Related Emotional Disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child and Adolescent Psychiatry, 36(4), 545–553.

  9. Borntrager, C. F., Chorpita, B. F., Higa-McMillan, C., & Weisz, J. R. (2009). Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals. Psychiatric Services, 60, 677–681.

  10. Brody, B. B., Cuffel, B., McColloch, J., Tani, S., Maruish, M., & Unützer, J. (2005). The acceptability and effectiveness of patient-reported assessments and feedback in a managed behavioral healthcare setting. The American Journal of Managed Care, 11(12), 774–780.

  11. Carlier, I. V., Meuldijk, D., Van Vleit, I. M., Van Fenema, E., Van der Wee, N. J., & Zitman, F. G. (2012). Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. Journal of Evaluating in Clinical Practice, 18(1), 104–110.

  12. Chorpita, B. F., Bernstein, A., Daleiden, E. L., & The Research Network on Youth Mental Health. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35, 114–123.

  13. Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005a). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7, 5–20.

  14. Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005b). Modularity in the design and application of therapeutic interventions. Applied and Preventive Psychology, 11(3), 141–156.

  15. Clark, D. M., Layard, R., Smithies, R., Richard, D. A., Suckling, R., & Wright, B. (2009). Improving access to psychological therapy: Initial evaluation of two UK demonstration sites. Behaviour Research and Therapy, 47, 910–920.

  16. Close-Goedjen, J., & Saunders, S. M. (2002). The effect of technical support on clinician attitudes toward an outcome assessment instrument. Journal of Behavioral Health Services and Research, 29, 99–108.

  17. Cohen, J. A., Mannarino, A. P., & Deblinger, E. (2006). Treating trauma and traumatic grief in children and adolescents. New York: Guilford Press.

  18. Daleiden, E., & Chorpita, B. F. (2005). From data to wisdom: Quality improvement strategies supporting large-scale implementation of evidence based services. Child and Adolescent Psychiatric Clinics of North America, 14, 329–349.

  19. de Rond, M., de Wit, R., & van Dam, F. (2001). The implementation of a pain monitoring programme for nurses in daily clinical practice: Results of a follow-up study in five hospitals. Journal of Advanced Nursing, 35, 590–598.

  20. Dorsey, S., Berliner, L., Lyon, A. R., Pullmann, M., & Murray, L. K. (under review). A state-wide common elements initiative for children’s mental health.

  21. Dorsey, S., Pullmann, M. D., Deblinger, E., Berliner, L., Kerns, S. E., Thompson, K., et al. (2013). Improving practice in community-based settings: a randomized trial of supervision–study protocol. Implementation Science, 8, 89.

  22. Ell, K., Unützer, J., Aranda, M., Sanchez, K., & Lee, P. J. (2006). Routine PHQ-9 depression screening in home health care: Depression prevalence, clinical treatment characteristics, and screening implementation. Home Health Care Services Quarterly, 24(4), 1–19.

  23. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature (FMHI Publication No. 231). Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.

  24. Foa, E. B., Johnson, K. M., Feeny, N. C., & Treadwell, K. H. (2001). The Child PTSD Symptom Scale: A preliminary examination of its psychometric properties. Journal of Clinical Child Psychology, 30(3), 376–384.

  25. Gardner, W., Murphy, M., Childs, G., Kelleher, K., Pagano, M., Jellinek, M., et al. (1999). The PSC-17: A brief pediatric symptom checklist with psychosocial problem subscales. A report from PROS and ASPN. Ambulatory Child Health, 5, 225–236.

  26. Garland, A. F., Haine-Schlagel, R., Accurso, E. C., Baker-Ericzen, M. J., & Brookman-Frazee, L. (2012). Exploring the effect of therapists’ treatment practices on client attendance in community-based care for children. Psychological Services, 9, 74–88.

  27. Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley- King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health and Mental Health Services Research, 40, 6–22.

  28. Garland, A. F., Kruse, M., & Aarons, G. A. (2003). Clinicians and outcome measurement: What’s the use? Journal of Behavioral Health Services and Research, 30, 393–405.

  29. Gilbody, S. M., House, A. O., & Sheldon, T. A. (2002). Psychiatrists in the UK do not use outcome measures. The British Journal of Psychiatry, 180, 101–103.

  30. Graham, I. D., & Tetroe, J. (2009). Learning from the US Department of Veterans Affairs Quality Enhancement Research Initiative: QUERI Series. Implementation Science, 4(1), 13.

  31. Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., et al. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163.

  32. Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35, 485–491.

  33. Hatfield, D. R., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others do not. Administration and Policy in Mental Health and Mental Health Services Research, 34, 283–291.

  34. Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A. H., Dellinger, E. P., et al. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine, 360, 491–499.

  35. Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466.

  36. Higa-McMillan, C. K., Powell, C. K., Daleiden, E. L., & Mueller, C. W. (2011). Pursuing an evidence-based culture through contextualized feedback: Aligning youth outcomes and provider practices. Professional Psychology: Research and Practice, 42, 137–144.

  37. Huffman, L. C., Martin, J., Botcheva, L., Williams, S. E., & Dyer-Friedman, J. (2004). Practitioners’ attitudes toward the use of treatment progress and outcomes data in child mental health services. Evaluation and the Health Professions, 27(2), 165–188.

  38. Hunsley, J., & Mash, E. J. (2005). Introduction to the special section on developing guidelines for the evidence-based assessment (EBA) of adult disorders. Psychological Assessment, 17, 251–255.

  39. Ivey, S. L., Scheffler, R., & Zazzali, J. L. (1998). Supply dynamics of the mental health workforce: Implications for health policy. The Milbank Quarterly, 76, 25–58.

  40. Jensen-Doss, A. (2011). Practice involves more than treatment: How can evidence-based assessment catch up to evidence-based treatment? Clinical Psychology: Science and Practice, 18(2), 173–177.

  41. Jensen-Doss, A., & Hawley, K. M. (2010). Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child and Adolescent Psychology, 39, 885–896.

  42. Jensen-Doss, A., & Hawley, K. M. (2011). Understanding clinicians’ diagnostic practices: Attitudes toward the utility of diagnosis and standardized diagnostic tools. Administration and Policy in Mental Health and Mental Health Services Research, 38(6), 476–485.

  43. Joyce, B., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

  44. Koerner, K. (2013, May). PracticeGround: An online platform to help therapists learn, implement, and measure the impact of EBPs. Paper presented at the Seattle Implementation Research Collaborative National Conference, Seattle, WA.

  45. Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10, 288–301.

  46. Lyon, A. R., Borntrager, C., Nakamura, B., & Higa-McMillan, C. (2013). From distal to proximal: Routine educational data monitoring in school-based mental health. Advances in School Mental Health Promotion, 6(4), 263–279.

  47. Lyon, A. R., Charlesworth-Attie, S., Vander Stoep, A., & McCauley, E. (2011a). Modular psychotherapy for youth with internalizing problems: Implementation with therapists in school-based health centers. School Psychology Review, 40, 569–581.

  48. Lyon, A. R., & Dorsey, S. (2010). The Current Assessment Practice Evaluation. Unpublished measure, University of Washington.

  49. Lyon, A. R., Stirman, S. W., Kerns, S. E. U., & Bruns, E. J. (2011b). Developing the mental health workforce: Review and application of training strategies from multiple disciplines. Administration and Policy in Mental Health and Mental Health Services Research, 38, 238–253.

  50. Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362–379.

  51. McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. American Psychologist, 65, 73–84.

  52. Muris, P., Merckelbach, H., Korver, P., & Meesters, C. (2000). Screening for trauma in children and adolescents: The validity of the Traumatic Stress Disorder scale of the Screen for Child Anxiety Related Emotional Disorders. Journal of Clinical Child Psychology, 29, 406–413.

  53. Nakamura, B. J., Mueller, C. W., Higa-McMillan, C., Okamura, K. H., Chang, J. P., Slavin, L., et al. (2013). Engineering youth service system infrastructure: Hawaii’s continued efforts at large-scale implementation through knowledge management strategies. Journal of Clinical Child and Adolescent Psychology. doi:10.1080/15374416.2013.812039.

  54. Palmiter, D. J. (2004). A survey of the assessment practices of child and adolescent clinicians. American Journal of Orthopsychiatry, 74, 122–128.

  55. Pearsons, J. B., Thomas, C., & Liu, H. (2012, November). Can we train psychotherapists to monitor their clients’ progress at every therapy session? Initial findings. Paper presented at the Association for Behavioral and Cognitive Therapies Annual Convention, National Harbor, MD.

  56. Poston, J. M., & Hanson, W. E. (2010). Meta-analysis of psychological assessment as a therapeutic intervention. Psychological Assessment, 22(2), 203–212.

  57. Robiner, W. N. (2006). The mental health professions: Workforce supply and demand, issues, and challenges. Clinical Psychology Review, 26(5), 600–625.

  58. Sburlati, E. S., Schniering, C. A., Lyneham, H. J., & Rapee, R. M. (2011). A model of therapist competencies for the empirically supported cognitive behavioral treatment of child and adolescent anxiety and depressive disorders. Clinical Child and Family Psychology Review, 14(1), 89–109.

  59. Shea, S., DuMouchel, W., & Bahamonde, L. (1996). A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. Journal of the American Medical Informatics Association, 3, 399–409.

  60. Shojania, K. G., Jennings, A., Mayhew, A., Ramsay, C., Eccles, M., & Grimshaw, J. (2010). Effect of point-of-care computer reminders on physician behavior: A systematic review. Canadian Medical Association Journal, 182, E216–E225.

  61. Southam-Gerow, M. A., Daleiden, E. L., Chorpita, B. F., Bae, C. Mitchell, C., Faye, M., et al. (in press). MAPping Los Angeles County: Taking an evidence-informed model of mental health care to scale. Journal of Clinical Child and Adolescent Psychology. doi:10.1080/15374416.2013.833098.

  62. Southam-Gerow, M. A., Weisz, J. R., Chu, B. C., McLeod, B. D., Gordis, E. B., & Connor-Smith, J. K. (2010). Does CBT for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child and Adolescent Psychiatry, 49, 1043–1052.

  63. Spielmans, G., Gatlin, E., & McFall, E. (2010). The efficacy of evidence-based psychotherapies versus usual care for youths: Controlling confounds in a meta-reanalysis. Psychotherapy Research, 20, 234–246.

  64. Stirman, S. W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7(17), 1–19.

  65. Substance Abuse and Mental Health Services Administration. (2012, January). Partners for Change Outcome Management System (PCOMS): International Center for Clinical Excellence. Retrieved from the National Registry of Evidence-based Programs and Practices Web site, http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=249. Accessed 27 Feb 2014.

  66. Tabachnick, B. C., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Pearson Education, Inc.

  67. Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., et al. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69, 274–282.

  68. Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61, 671–689.

  69. Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70(7), 750–761.

  70. Weisz, J. R., Southam-Gerow, M. A., Gordis, E. B., Connor-Smith, J. K., Chu, B. C., Langer, D. A., et al. (2009). Cognitive-behavioral therapy versus usual clinical care for youth depression: An initial test of transportability to community clinics and clinicians. Journal of Consulting and Clinical Psychology, 77, 383–396.

  71. Wolpert, M. (2014). Uses and abuses of patient reported outcome measures (PROMs): Potential iatrogenic impact of PROMs implementation and how it can be mitigated. Administration and Policy in Mental Health and Mental Health Services Research, 41(2), 141–145.

  72. Youngstrom, E. A. (2008). Evidence based assessment is not evidence based medicine: Commentary on evidence-based assessment of cognitive functioning in pediatric psychology. Journal of Pediatric Psychology, 33, 1015–1020.

Download references

Acknowledgments

This publication was made possible in part by funding from the Washington State Department of Social and Health Services, Division of Behavioral Health Recovery and from Grant Nos. F32 MH086978, K08 MH095939, and R01 MH095749, awarded from the National Institute of Mental Health (NIMH). Drs. Lyon and Dorsey are investigators with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI).

Author information

Correspondence to Aaron R. Lyon.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lyon, A.R., Dorsey, S., Pullmann, M. et al. Clinician Use of Standardized Assessments Following a Common Elements Psychotherapy Training and Consultation Program. Adm Policy Ment Health 42, 47–60 (2015). https://doi.org/10.1007/s10488-014-0543-7

Download citation

Keywords

  • Evidence-based assessment
  • Standardized assessment
  • Community mental health
  • Implementation
  • Training