Advertisement

Individualized Progress Measures Are More Acceptable to Clinicians Than Standardized Measures: Results of a National Survey

  • Amanda Jensen-Doss
  • Ashley M. Smith
  • Emily M. Becker-Haimes
  • Vanesa Mora Ringle
  • Lucia M. Walsh
  • Monica Nanda
  • Samantha L. Walsh
  • Colleen A. Maxwell
  • Aaron R. Lyon
Original Article

Abstract

Despite research supporting measurement-based care grounded in standardized progress measures, such measures are underutilized by clinicians. Individualized measures of client-specific targets present an alternative, but little is known about their acceptability or use. We compared attitudes toward and use of standardized and individualized progress measures in a national sample of 504 clinicians. Clinicians reported neutral to positive attitudes toward both types of measures, but strongly preferred and were more likely to use individualized measures. Clinician attitudes, theoretical orientation, and work setting predicted assessment preferences and practices. Implications for dissemination and implementation of measurement-based care are discussed.

Keywords

Measurement-based care Standardized assessment Individualized assessment evidence based practice Therapists 

Notes

Funding

This research was supported by an award from the University of Miami’s Provost Resaerch Award program to Dr. Jensen-Doss. Dr. Lyon’s work on this project was supported by the National Institute Of Mental Health of the National Institutes of Health (NIH) under Award K08MH095939.

Compliance with Ethical Standards

Conflict of interest

None of the authors have conflicts of interest to declare.

Ethical Approval

All procedures performed in this study were in accordance with the ethical standards of the University of Miami Institutional Review Board and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

This study was approved for a waiver of signed consent; all participants were provided with a consent statement.

References

  1. Ashworth, M., Evans, C., & Clement, S. (2009). Measuring psychological outcomes after cognitive behaviour therapy in primary care: A comparison between a new patient-generated measure “PSYCHLOPS” (Psychological Outcome Profiles) and “HADS” (Hospital Anxiety and Depression Scale). Journal of Mental Health, 18, 169–177.CrossRefGoogle Scholar
  2. Batty, M. J., Moldavsky, M., Foroushani, P. S., Pass, S., Marriott, M., Sayal, K., & Hollis, C. (2013). Implementing routine outcome measures in child and adolescent mental health services: From present to future practice. Child and Adolescent Mental Health, 18, 82–87.CrossRefGoogle Scholar
  3. Bickman, L., Douglas, S. R., De Andrade, A. R. V., Tomlinson, M., Gleacher, A., Olin, S., & Hoagwood, K. (2016). Implementing a measurement feedback system: A tale of two sites. Administration and Policy in Mental Health and Mental Health Services Research, 43, 410–425.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Borntrager, C. F., Chorpita, B. F., Higa-McMillan, C., & Weisz, J. R. (2009). Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals? Psychiatric Services, 60, 677–681.CrossRefPubMedGoogle Scholar
  5. Borntrager, C. F., & Lyon, A. R. (2015). Client progress monitoring and feedback in school-based mental health. Cognitive and Behavioral Practice, 22, 74–86.CrossRefPubMedGoogle Scholar
  6. Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2015). Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research, 25, 6–19.CrossRefPubMedGoogle Scholar
  7. Claiborn, C. D., & Goodyear, R. K. (2005). Feedback in psychotherapy. Journal of Clinical Psychology, 61, 209–217.CrossRefPubMedGoogle Scholar
  8. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Lawrence Earlbaum Associates, Inc.Google Scholar
  9. Cone, J. D. (1986). Idiographic, nomothetic, and related perspectives in behavioral assessment. In R. O. Nelson & S. C. Hayes (Eds.), Conceptual foundations of behavioral assessment (pp. 111–128). New York: The Guilford Press.Google Scholar
  10. Connors, E. H., Arora, P., Curtis, L., & Stephan, S. H. (2015). Evidence-based assessment in school mental health. Cognitive and Behavioral Practice, 22, 60–73.CrossRefGoogle Scholar
  11. Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). Wiley.Google Scholar
  12. Doss, B. D., Thum, Y. M., Sevier, M., Atkins, D. C., & Christensen, A. (2005). Improving relationships: Mechanisms of change in couple therapy. Journal of Consulting and Clinical Psychology, 73, 624–633.CrossRefPubMedGoogle Scholar
  13. Dozois, D. J., Mikail, S. F., Alden, L. E., Bieling, P. J., Bourgon, G., Clark, D. A., … Hunsley, J. (2014). The CPA presidential task force on evidence-based practice of psychological treatments. Canadian Psychology/Psychologie Canadienne, 55, 153–160.CrossRefGoogle Scholar
  14. Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1, 170–177.CrossRefGoogle Scholar
  15. Duong, M. T., Lyon, A. R., Ludwig, K., Wasse, J. K., & McCauley, E. (2016). Student perceptions of the acceptability and utility of standardized and idiographic assessment in school mental health. International Journal of Mental Health Promotion, 18, 49–63.CrossRefGoogle Scholar
  16. Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R. J., Haine-Schlagel, R., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61, 788–795.CrossRefPubMedPubMedCentralGoogle Scholar
  17. Gilbody, S. M., House, A. O., & Sheldon, T. A. (2002). Psychiatrists in the UK do not use outcomes measures: National survey. British Journal of Psychiatry, 180, 101–103.CrossRefPubMedGoogle Scholar
  18. Gleacher, A. A., Olin, S. S., Nadeem, E., Pollock, M., Ringle, V., Bickman, L., … Hoagwood, K. (2016). Implementing a measurement feedback system in community mental health clinics: A case study of multilevel barriers and facilitators. Administration and Policy in Mental Health and Mental Health Services Research, 43, 426–440.CrossRefPubMedPubMedCentralGoogle Scholar
  19. Gondek, D., Edbrooke-Childs, J., Fink, E., Deighton, J., & Wolpert, M. (2016). Feedback from outcome measures and treatment effectiveness, treatment efficiency, and collaborative practice: A systematic review. Administration and Policy in Mental Health and Mental Health Services Research, 43, 325–343.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Green, D. (2016). Making the case for using personalised outcome measures to track progress in psychotherapy. European Journal of Psychotherapy and Counselling, 18, 39–57.CrossRefGoogle Scholar
  21. Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61, 155–163.CrossRefPubMedGoogle Scholar
  22. Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35, 485–491.CrossRefGoogle Scholar
  23. Hatfield, D. R., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others do not. Administration and Policy in Mental Health and Mental Health Services Research, 34, 283–291.CrossRefPubMedGoogle Scholar
  24. Herschell, A. D., Lindhiem, O. J., Kogan, J. N., Celedonia, K. L., & Stein, B. D. (2014). Evaluation of an implementation initiative for embedding dialectical behavior therapy in community settings. Evaluation and Program Planning, 43, 55–63.CrossRefPubMedGoogle Scholar
  25. Hu, L. t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.CrossRefGoogle Scholar
  26. Ionita, G., & Fitzpatrick, M. (2014). Bringing science to clinical practice: A Canadian survey of psychological practice and usage of progress monitoring measures. Canadian Psychology/Psychologie Canadienne, 55, 187–196.CrossRefGoogle Scholar
  27. Jensen-Doss, A., Becker, E. M., Smith, A. M., Lyon, A. R., Lewis, C. C., Stanick, C. F., & Hawley, K. M. (in press). Monitoring treatment progress and providing feedback is viewed favorably but rarely used in practice. Administration and Policy in Mental Health and Mental Health Services Research.Google Scholar
  28. Jensen-Doss, A., & Hawley, K. M. (2010). Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child & Adolescent Psychology, 39, 885–896.CrossRefGoogle Scholar
  29. Johnston, C., & Gowers, S. (2005). Routine outcome measurement: A survey of UK child and adolescent mental health services. Child and Adolescent Mental Health, 10, 133–139.CrossRefGoogle Scholar
  30. Kelley, S. D., & Bickman, L. (2009). Beyond outcomes monitoring: Measurement feedback systems in child and adolescent clinical practice. Current Opinion in Psychiatry, 22, 363–368.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Kotte, A., Hill, K. A., Mah, A. C., Korathu-Larson, P. A., Au, J. R., Izmirian, S., … Higa-McMillan, C. K. (2016). Facilitators and barriers of implementing a measurement feedback system in public youth mental health. Administration and Policy in Mental Health and Mental Health Services Research, 43, 861–878.CrossRefPubMedGoogle Scholar
  32. Krägeloh, C. U., Czuba, K. J., Billington, D. R., Kersten, P., & Siegert, R. J. (2015). Using feedback from patient-reported outcome measures in mental health services: A scoping study and typology. Psychiatric Services, 66, 224–241.CrossRefPubMedGoogle Scholar
  33. Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69, 159–172.CrossRefPubMedGoogle Scholar
  34. Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions. Journal of Clinical Psychology, 61, 165–174.CrossRefPubMedGoogle Scholar
  35. Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72–79.CrossRefPubMedGoogle Scholar
  36. Landes, S. J., Carlson, E. B., Ruzek, J. I., Wang, D., Hugo, E., DeGaetano, N., … Lindley, S. E. (2015). Provider-driven development of a measurement feedback system to enhance measurement-based care in VA mental health. Cognitive and Behavioral Practice, 22, 87–100.CrossRefGoogle Scholar
  37. Lewis, C. C., Scott, K., Marti, C. N., Marriott, B. R., Kroenke, K., Putz, J. W., … Rutkowski, D (2015). Implementing measurement-based care (iMBC) for depression in community mental health: A dynamic cluster randomized trial study protocol. Implementation Science, 10, 127.CrossRefPubMedPubMedCentralGoogle Scholar
  38. Lindhiem, O., Bennett, C. B., Orimoto, T. E., & Kolko, D. J. (2016). A meta-analysis of personalized treatment goals in psychotherapy: A preliminary report and call for more studies. Clinical Psychology: Science and Practice, 23, 165–176.Google Scholar
  39. Little, R. J. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198–1202.CrossRefGoogle Scholar
  40. Lyon, A. R., Lewis, C. C., Boyd, M. R., Hendrix, E., & Liu, F. (2016a). Capabilities and characteristics of digital measurement feedback systems: Results from a comprehensive review. Administration and Policy in Mental Health and Mental Health Services Research, 43, 441–466.CrossRefPubMedPubMedCentralGoogle Scholar
  41. Lyon, A. R., Ludwig, K., Wasse, J. K., Bergstrom, A., Hendrix, E., & McCauley, E. (2016b). Determinants and functions of standardized assessment use among school mental health clinicians: A mixed methods evaluation. Administration and Policy in Mental Health and Mental Health Services Research, 43, 122–134.CrossRefPubMedPubMedCentralGoogle Scholar
  42. McGuire, J. F., Sukhodolsky, D. G., Bearss, K., Grantz, H., Pachler, M., Lombroso, P. J., & Scahill, L. (2014). Individualized assessments in treatment research: An examination of parent-nominated target problems in the treatment of disruptive behaviors in youth with Tourette syndrome. Child Psychiatry and Human Development, 45, 686–694.CrossRefPubMedGoogle Scholar
  43. Overington, L., Fitzpatrick, M., Hunsley, J., & Drapeau, M. (2015). Trainees’ experiences using progress monitoring measures. Training and Education in Professional Psychology, 9, 202–209.CrossRefGoogle Scholar
  44. Persons, J. B. (2006). Case formulation-drive psychotherapy. Clinical Psychology: Science and Practice, 13, 167–170.Google Scholar
  45. Ringle, V. A., Read, K. L., Edmunds, J. M., Brodman, D. M., Kendall, P. C., Barg, F., & Beidas, R. S. (2015). Barriers to and facilitators in the implementation of cognitive-behavioral therapy for youth anxiety in the community. Psychiatric Services, 66, 938–945.CrossRefPubMedPubMedCentralGoogle Scholar
  46. Sales, C., & Alves, P. C. (2012). Individualized patient-progress systems: Why we need to move towards a personalized evaluation of psychological treatments. Canadian Psychology/Psychologie Canadienne, 53, 115–121.CrossRefGoogle Scholar
  47. Sales, C., Goncalves, S., Fragoeiro, A., Noronha, S., & Elliott, R. (2007). Psychotherapists openness to routine naturalistic idiographic research? Mental Health and Learning Disabilities Research and Practice, 4, 145–161.CrossRefGoogle Scholar
  48. Scott, K., & Lewis, C. C. (2015). Using measurement-based care to enhance any treatment. Cognitive and Behavioral Practice, 22, 49–59.CrossRefPubMedPubMedCentralGoogle Scholar
  49. Stirman, S. W., Gutiérrez-Colina, A., Toder, K., Esposito, G., Barg, F., Castro, F., … Crits-Christoph, P. (2013). Clinicians’ perspectives on cognitive therapy in community mental health settings: Implications for training and implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40, 274–285.CrossRefPubMedPubMedCentralGoogle Scholar
  50. Walfish, S., McAlister, B., O’Donnell, P., & Lambert, M. J. (2012). An investigation of self-assessment bias in mental health providers. Psychological Reports, 110, 639–644.CrossRefPubMedGoogle Scholar
  51. Weisz, J. R., Chorpita, B. F., Frye, A., Ng, M. Y., Lau, N., Bearman, S. K., … Hoagwood, K. E. (2011). Youth top problems: Using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. Journal of Consulting and Clinical Psychology, 79, 369–380.CrossRefPubMedGoogle Scholar
  52. Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003). Improving the effects of psychotherapy: The use of early identification of treatment and problem-solving strategies in routine practice. Journal of Counseling Psychology, 50, 59–68.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Amanda Jensen-Doss
    • 1
  • Ashley M. Smith
    • 1
    • 5
  • Emily M. Becker-Haimes
    • 2
  • Vanesa Mora Ringle
    • 1
  • Lucia M. Walsh
    • 1
  • Monica Nanda
    • 1
  • Samantha L. Walsh
    • 3
  • Colleen A. Maxwell
    • 1
  • Aaron R. Lyon
    • 4
  1. 1.Department of PsychologyUniversity of MiamiCoral GablesUSA
  2. 2.Perelman School of MedicineUniversity of PennsylvaniaPhiladelphiaUSA
  3. 3.Massachusetts General HospitalBostonUSA
  4. 4.Department of Psychiatry and Behavioral SciencesUniversity of WashingtonSeattleUSA
  5. 5.University of CaliforniaLos AngelesUSA

Personalised recommendations