Skip to main content
SpringerLink
Log in

Monitoring Treatment Progress and Providing Feedback is Viewed Favorably but Rarely Used in Practice

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Numerous trials demonstrate that monitoring client progress and using feedback for clinical decision-making enhances treatment outcomes, but available data suggest these practices are rare in clinical settings and no psychometrically validated measures exist for assessing attitudinal barriers to these practices. This national survey of 504 clinicians collected data on attitudes toward and use of monitoring and feedback. Two new measures were developed and subjected to factor analysis: The monitoring and feedback attitudes scale (MFA), measuring general attitudes toward monitoring and feedback, and the attitudes toward standardized assessment scales-monitoring and feedback (ASA-MF), measuring attitudes toward standardized progress tools. Both measures showed good fit to their final factor solutions, with excellent internal consistency for all subscales. Scores on the MFA subscales (Benefit, Harm) indicated that clinicians hold generally positive attitudes toward monitoring and feedback, but scores on the ASA-MF subscales (Clinical Utility, Treatment Planning, Practicality) were relatively neutral. Providers with cognitive-behavioral theoretical orientations held more positive attitudes. Only 13.9 % of clinicians reported using standardized progress measures at least monthly and 61.5 % never used them. Providers with more positive attitudes reported higher use, providing initial support for the predictive validity of the ASA-MF and MFA. Thus, while clinicians report generally positive attitudes toward monitoring and feedback, routine collection of standardized progress measures remains uncommon. Implications for the dissemination and implementation of monitoring and feedback systems are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Providers could indicate multiple work settings and multiple orientations. Providers who indicated they spent any time working in private practice were included in the Private Practice group, those who listed cognitive or behavioral as part of their orientations were counted in the Cognitive-Behavioral group, and those who listed psychodynamic as part of their orientations were counted in the Psychodynamic group.

  2. These definitions are included in the instructions for the final versions of the measures, which are included as supplemental material to this article.

  3. Degree information was not available for AMHCA members.

References

  • Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The evidence-based practice attitude scale (EBPAS). Mental Health Services Research, 6(2), 61–74.

    Article  PubMed  PubMed Central  Google Scholar 

  • Aarons, G. A., Horowitz, J., Dlugosz, L., & Ehrhart, M. (2012). The role of organizational processes in dissemination and implementation research. Dissemination and Implementation Research in Health: Translating Science to Practice, 128–153.

  • Aarons, G. A., & Sawitzky, A. C. (2006). Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services, 3(1), 61.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior & Human Decision Processes, 50, 179–211.

    Article  Google Scholar 

  • APA Presidential Task FORCE on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285.

    Article  Google Scholar 

  • Batty, M. J., Moldavsky, M., Foroushani, P. S., Pass, S., Marriott, M., Sayal, K., & Hollis, C. (2013). Implementing routine outcome measures in child and adolescent mental health services: From present to future practice. Child and Adolescent Mental Health, 18(2), 82–87.

    Article  Google Scholar 

  • Becker, E. M., & Jensen-Doss, A. (2013). Computer-assisted therapies: Examination of therapist-level barriers to their use. Behavior Therapy, 44(4), 614–624.

    Article  PubMed  Google Scholar 

  • Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child & Adolescent Psychiatry, 47(10), 1114–1119.

    Article  Google Scholar 

  • Bickman, L., Douglas, S. R., De Andrade, A. R. V., Tomlinson, M., Gleacher, A., Olin, S., & Hoagwood, K. (2016). Implementing a measurement feedback system: A tale of two sites. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 410–425.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bickman, L., Kelley, S. D., Breda, C., De Andrade, A. R. V., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62(12), 1423–1429.

    Article  PubMed  Google Scholar 

  • Borntrager, C. F., & Lyon, A. R. (2015). Client progress monitoring and feedback in school-based mental health. Cognitive and Behavioral Practice, 22(1), 74–86.

    Article  PubMed  Google Scholar 

  • Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2015). Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research, 25(1), 6–19.

    Article  PubMed  Google Scholar 

  • Chorpita, B. F., Bernstein, A., Daleiden, E. L., & Research Network on Youth Mental Health. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 114–123.

    Article  PubMed  Google Scholar 

  • Creed, T. A., Wolk, C. B., Feinberg, B., Evans, A. C., & Beck, A. T. (2016). Beyond the label: Relationship between community therapists’ self-report of a cognitive behavioral therapy orientation and observed skills. Administration and Policy in Mental Health and Mental Health Services Research, 43(1), 36–43.

    Article  PubMed  Google Scholar 

  • Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method. (3rd ed.). Hoboken: Wiley.

    Google Scholar 

  • Dozois, D. J., Mikail, S. F., Alden, L. E., Bieling, P. J., Bourgon, G., Clark, D. A., Drapeau, M., Gallson, D., Greenberg, L., Hunsley, J. (2014). The CPA presidential task force on evidence-based practice of psychological treatments. Canadian Psychology/Psychologie canadienne, 55(3), 153.

    Article  Google Scholar 

  • Ebesutani, C., & Shin, S. H. (2014). Knowledge, attitudes, and usage of evidence-based assessment and treatment practices in the Korean mental health system: Current status and future directions. The Korean Journal of Clinical Psychology, 33(4), 891–917.

    Article  Google Scholar 

  • Elliott, R., Wagner, J., Sales, C. M. D., Rodgers, B., Alves, P., & Café, M. J. (2016). Psychometrics of the personal questionnaire: A client-generated outcome measure. Psychological Assessment, 28(3), 263–278.

    Article  PubMed  Google Scholar 

  • Frauenhoffer, D., Ross, M. J., Gfeller, J., Searight, H. R., & Piotrowski, C. (1998). Psychological test usage among licensed mental health practitioners: A multidisciplinary survey. Journal of Psychological Practice, 4(1), 28–33.

    Google Scholar 

  • Garland, A. F., Bickman, L., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research, 37(1–2), 15–26.

    Article  PubMed  PubMed Central  Google Scholar 

  • Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R. J., Haine-Schlagel, R., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61(8), 788–795.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gilbody, S. M., House, A. O., & Sheldon, T. A. (2002). Psychiatrists in the UK do not use outcomes measures: National survey. British Journal of Psychiatry, 180(2), 101–103.

    Article  PubMed  Google Scholar 

  • Gleacher, A. A., Olin, S. S., Nadeem, E., Pollock, M., Ringle, V., Bickman, L., Douglas, S., Hoagwood, K., et al. (2016).. Administration and Policy in Mental Health and Mental Health Services Research, 43, 426–440.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hall, C., Moldavsky, M., Taylor, J., Sayal, K., Marriott, M., Batty, M., Pass, S., Hollis, C., et al. (2014). Implementation of routine outcome measurement in child and adolescent mental health services in the United Kingdom: A critical perspective. European Child & Adolescent Psychiatry, 23(4), 239–242.

    Article  CAS  Google Scholar 

  • Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35(5), 485–491.

    Article  Google Scholar 

  • Hatfield, D. R., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others do not. Administration and Policy in Mental Health and Mental Health Services Research, 34(3), 283–291.

    Article  PubMed  Google Scholar 

  • Hawley, K. M., Cook, J. R., & Jensen-Doss, A. (2009). Do noncontingent incentives increase survey response rates among mental health providers? A randomized trial comparison. Administration and Policy in Mental Health, 36(5), 343–348.

    Article  PubMed  PubMed Central  Google Scholar 

  • Herschell, A. D., Lindhiem, O. J., Kogan, J. N., Celedonia, K. L., & Stein, B. D. (2014). Evaluation of an implementation initiative for embedding dialectical behavior therapy in community settings. Evaluation and program planning, 43, 55–63.

    Article  PubMed  Google Scholar 

  • Higa-McMillan, C. K., Powell, C. K. i. K., Daleiden, E. L., & Mueller, C. W. (2011). Pursuing an evidence-based culture through contextualized feedback: Aligning youth outcomes and practices. Professional Psychology: Research and Practice, 42(2), 137–144.

    Article  Google Scholar 

  • Hu, L. t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

    Article  Google Scholar 

  • Ionita, G., & Fitzpatrick, M. (2014). Bringing science to clinical practice: A Canadian survey of psychological practice and usage of progress monitoring measures. Canadian Psychology/Psychologie Canadienne, 55(3), 187–196.

    Article  Google Scholar 

  • Jensen-Doss, A., & Hawley, K. M. (2010). Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child & Adolescent Psychology, 39(6), 885–896.

    Article  Google Scholar 

  • Jensen-Doss, A., & Hawley, K. M. (2011). Understanding clinicians’ diagnostic practices: Attitudes toward the utility of diagnosis and standardized diagnostic tools. Administration and Policy in Mental Health and Mental Health Services Research, 38(6), 476–485.

    Article  PubMed  Google Scholar 

  • Jensen-Doss, A., Hawley, K. M., Lopez, M., & Osterberg, L. D. (2009). Using evidence-based treatments: The experiences of youth providers working under a mandate. Professional Psychology: Research and Practice, 40(4), 417.

    Article  Google Scholar 

  • Johnston, C., & Gowers, S. (2005). Routine outcome measurement: a survey of UK child and adolescent mental health services. Child and Adolescent Mental Health, 10(3), 133–139.

    Article  Google Scholar 

  • Koerner, K., & Castonguay, L. G. (2015). Practice-oriented research: What it takes to do collaborative research in private practice. Psychotherapy Research, 25(1), 67–83.

    Article  PubMed  Google Scholar 

  • Kotte, A., Hill, K. A., Mah, A. C., Korathu-Larson, P. A., Au, J. R., Izmirian, S., … Higa-McMillan, C. K., et al. (2016). Facilitators and barriers of implementing a measurement feedback system in public youth mental health. Administration and Policy in Mental Health and Mental Health Services Research, 1–18.

  • Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10(3), 288–301.

    Google Scholar 

  • Landes, S. J., Carlson, E. B., Ruzek, J. I., Wang, D., Hugo, E., DeGaetano, N., … Lindley, S. E., et al. (2015). Provider-driven development of a measurement feedback system to enhance measurement-based care in va mental health. Cognitive and Behavioral Practice, 22(1), 87–100.

    Article  Google Scholar 

  • Little, R. J. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202.

  • Lyon, A. R., Dorsey, S., Pullmann, M., Silbaugh-Cowdin, J., & Berliner, L. (2015). Clinician use of standardized assessments following a common elements psychotherapy training and consultation program. Administration and Policy in Mental Health and Mental Health Services Research, 42(1), 47–60.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lyon, A. R., Ludwig, K., Wasse, J. K., Bergstrom, A., Hendrix, E., & McCauley, E. (2016). Determinants and functions of standardized assessment use among school mental health clinicians: A mixed methods evaluation. Administration and Policy in Mental Health and Mental Health Services Research, 43(1), 122–134.

    Article  PubMed  PubMed Central  Google Scholar 

  • Martinez, R. G., Lewis, C. C., & Weiner, B. J. (2014). Instrumentation issues in implementation science. Implementation Science, 9, 118.

  • Meehan, T., McCombes, S., Hatzipetrou, L., & Catchpoole, R. (2006). Introduction of routine outcome measures: Staff reactions and issues for consideration. Journal of Psychiatric and Mental Health Nursing, 13(5), 581–587.

    Article  CAS  PubMed  Google Scholar 

  • Muthén, L., & Muthén, B. (1998–2011). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.

  • Overington, L., Fitzpatrick, M., Hunsley, J., & Drapeau, M. (2015). Trainees’ experiences using progress monitoring measures. Training and Education in Professional Psychology, 9(3), 202–209.

    Article  Google Scholar 

  • Palmiter, D. J. Jr. (2004). A survey of the assessment practices of child and adolescent clinicians. American Journal of Orthopsychiatry, 74(2), 122–128.

    Article  PubMed  Google Scholar 

  • Persons, J. B. (2006). Case formulation–driven psychotherapy. Clinical Psychology: Science and Practice, 13(2), 167–170.

    Google Scholar 

  • Reese, R. J., Norsworthy, L. A., & Rowlands, S. R. (2009). Does a continuous feedback system improve psychotherapy outcome? Psychotherapy: Theory, Research, Practice, Training, 46(4), 418–431.

    Article  Google Scholar 

  • Unsworth, G., Cowie, H., & Green, A. (2012). Therapists’ and clients’ perceptions of routine outcome measurement in the NHS: A qualitative study. Counselling and Psychotherapy Research, 12(1), 71–80.

    Article  Google Scholar 

  • Ventimiglia, J. A., Marschke, J., Carmichael, P., & Loew, R. (2000). How do clinicians evaluate their practice effectiveness? A survey of clinical social workers. Smith College Studies in Social Work, 70(2), 287–306.

    Article  Google Scholar 

  • Weisz, J. R., Chorpita, B. F., Frye, A., Ng, M. Y., Lau, N., Bearman, S. K., Ugueto, A. M., Langer, D. A., Hoagwood, K. E., et al. (2011). Youth top problems: Using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. Journal of Consulting and Clinical Psychology, 79(3), 369.

    Article  PubMed  Google Scholar 

Download references

Funding

This research was supported by an award from the University of Miami’s Provost Resaerch Award program to Dr. Jensen-Doss. Dr. Lewis’ work on this project was supported by the National Institute Of Mental Health of the National Institutes of Health (NIH) under Award Number R01MH103310 and Dr. Lyon’s work by NIH award K08MH095939.

Author information

Authors and Affiliations

  1. Child Division, Department of Psychology, University of Miami, P.O. Box 248185, Coral Gables, FL, 33124-0751, USA

    Amanda Jensen-Doss, Emily M. Becker Haimes & Ashley M. Smith

  2. University of Washington, Seattle, USA

    Aaron R. Lyon & Cara C. Lewis

  3. Indiana University, Bloomington, USA

    Cara C. Lewis

  4. University of Montana, Missoula, USA

    Cameo F. Stanick

  5. University of Missouri, Columbia, USA

    Kristin M. Hawley

Authors
  1. Amanda Jensen-Doss
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emily M. Becker Haimes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ashley M. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aaron R. Lyon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cara C. Lewis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cameo F. Stanick
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kristin M. Hawley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amanda Jensen-Doss.

Ethics declarations

Conflict of Interest

None of the authors have conflicts of interest to declare.

Ethical Approval

All procedures performed in this study were in accordance with the ethical standards of the University of Miami Institutional Review Board and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

This study was approved for a waiver of signed consent; all participants were provided with a consent statement.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 23 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jensen-Doss, A., Haimes, E.M.B., Smith, A.M. et al. Monitoring Treatment Progress and Providing Feedback is Viewed Favorably but Rarely Used in Practice. Adm Policy Ment Health 45, 48–61 (2018). https://doi.org/10.1007/s10488-016-0763-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-016-0763-0

Keywords

Search

Navigation