Abstract
This paper describes the successful implementation of Measurement-Based Care (MBC) within a Partial Hospitalization Program (PHP) for children and adolescents. Measurement-based care (MBC), the practice of using patient-reported measures routinely to inform decision-making, is associated with improved clinical outcomes for behavioral health patients (Jong et al., Clinical Psychology Review 85, 2021; Fortney & Sladek, 2015). MBC holds great promise in partial hospital programs (PHP) to improve outcomes, yet implementation strategies are as complex as the setting itself. This paper provides a case study example of MBC implementation in a PHP for youth. Over the course of 18 months, an interdisciplinary staff of approximately 20 behavioral health professionals provided partial hospitalization level of care to 633 (39% in-person, 61% telehealth) youth from ages 5 to 18 years old. MBC in this setting incorporated daily patient self-report and weekly caregiver-reported measurements. This descriptive reconstruction, which includes examples of the data that were used during the implementation project, illustrates specific barriers and facilitators in a successful implementation in the local PHP setting. Implementation strategies to address workflow integration, leadership and supervision, and coaching are described, including evolution of these strategies over the course of implementation. Practical considerations for implementing MBC in youth PHP settings are discussed. Finally, the authors explore the potential relationships between the data-driven MBC model of decision making and the development of resilient organizations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Bickman et al. (2016) calculated the implementation index score based on two primary metrics: rates of questionnaire completion and rates of feedback viewing.
The Peabody Treatment Progress Battery (PTPB), which includes the SFSS, TAQ, and CGSQ-SF, was normed for youths aged 11 to 18 years in intensive in-home and outpatient treatment and their caregivers. After review of the items by clinical leaders at the agency, the decision was made to extend the age down to 10 years for youth self-report. The PTPB was not used in its entirety due to the length of the measure and the recommended cadence of measurement, which did not match with the very brief treatment model of the CPHP program.
The original TAQ item stems refer to individual treatment (e.g., “your therapist”) and discrete therapy sessions. Given the team-style nature of the CPHP, the items were modified to refer to the team (e.g., “your treatment team”). In addition, references to “the session” were replaced with “the treatment day.”.
Most common ad hoc measures included the NICHQ Vanderbilt Assessment Scale (Wolraich et al., 2003) for ADHD, the Center for Epidemiological Studies Depression Scale for Children (CES-DC) (Weissman et al., 1980) for depression, and the Screen for Child Anxiety Related Disorders (SCARED) (Birmaher et al., 1997) for anxiety.
While staff training and coaching in MBC also emphasized the additional steps of sharing results with the patient (and/or caregiver) and using this information and conversation to guide treatment planning, the technology platform did not collect data for these components of implementation.
The three phases identified in this case study are presented to structure this case study rather than to suggest a framework that will apply to all MBC implementations.
This clinical anecdote described by Chandler, et al. (2020) was first identified during a weekly clinical team meeting, and details were verified through review of the client’s medical records including MBC results.
This story was identified during the reconstructive process of this study. Details were verified through review of internal email communications between program supervisors, staff, and behavioral health leadership.
The data set reviewed for this study ended in June 2021. Program leadership continued to monitor implementation metrics through June of 2022, and observed that the rates of measure completion and staff feedback remained roughly consistent over that time frame.
This story was identified during the reconstructive process for this study. Details were verified through review of internal email communications between program supervisors, staff, and behavioral health leadership.
This story was identified during the reconstructive process for this study. Details were verified through review of internal email communications between program supervisors, staff, and behavioral health leadership.
References
Aboraya, A., Nasrallah, H. A., Elswick, D. E., Ahmed, E., Estephan, N., Aboraya, D., Berzingi, S., Chumbers, J., Berzingi, S., Justice, J., Zafar, J., & Dohar, S. (2018). Measurement-based care in psychiatry—past, present, and future. Innovations in Clinical Neuroscience, 15(11–12), 13–26.
Athay, M. M., Riemer, M., & Bickman, L. (2012). The Symptoms and Functioning Severity Scale (SFSS): Psychometric Evaluation and discrepancies among youth, caregiver, and clinician ratings over time. Administration and Policy in Mental Health, 39(1–2), 13–29. https://doi.org/10.1007/s10488-012-0403-2
Bargmann, S. (2017). Achieving excellence through feedback-informed supervision. In D. S. Prescott, C. L. Maeschalck, & S. D. Miller (Eds.), Feedback-informed treatment in clinical practice: reaching for excellence (pp. 79–100)
Barnes, S. E., de Steiguer, A., Gelman, D., & Parade, S. (2020). Implementation of measurement-based care in a child partial hospital program. The Brown University Child and Adolescent Behavior Letter, 36(7), 1–7. https://doi.org/10.1002/cbl.30475
Baruch, G., & Vrouva, I. (2010). Collecting routine outcome data in a psychotherapy community clinic for young people: findings from an ongoing study. Child and Adolescent Mental Health, 15(1), 30–36. https://doi.org/10.1111/j.1475-3588.2009.00531.x
Bertram, R. M., Blase, K. A., & Fixsen, D. L. (2015). Improving Programs and outcomes: Implementation frameworks and organization change. Research on Social Work Practice, 25(4), 477–487. https://doi.org/10.1177/1049731514537687
Bickman, L., Athay, M., Riemer, M., Lambert, E. W., Douglas Kelley, S., Breda, C., Tempesti, T., Dew-Reeves, S. E., Brannan, A. M., & Vides de Andrade, A. R. (2010). Manual of the peabody treatment progress battery. Vanderbilt University.
Bickman, L., Douglas Kelley, S., Breda, C., De Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on youth mental health outcomes: A randomized cluster design. Psychiatric Services, 62, 1423–1429. https://doi.org/10.1176/appi.ps.002052011
Bickman, L., Douglas, S. R., Andrade, A. R. V. D., Tomlinson, M., Gleacher, A., Olin, S., & Hoagwood, K. (2016). Implementing a measurement feedback system: A tale of two sites. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 410–425. https://doi.org/10.1007/s10488-015-0647-8
Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., & Neer, S. M. (1997). The Screen for Child Anxiety Related Emotional Disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child and Adolescent Psychiatry, 36(4), 545–553. https://doi.org/10.1097/00004583-199704000-00018
Brannan, A. M., Athay, M. M., & de Andrade, A. R. V. (2012). Measurement quality of the caregiver strain questionnaire-short form 7 (CGSQ-SF7). Administration and Policy in Mental Health, 39(1–2), 51–59. https://doi.org/10.1007/s10488-012-0412-1
Burlingame, G. M., Alldredge, C. T., & Arnold, R. A. (2021). Alliance rupture detection and repair in group therapy: Using the group questionnaire–GQ. International Journal of Group Psychotherapy, 71(2), 338–370. https://doi.org/10.1080/00207284.2020.1844010
Burlingame, G. M., Whitcomb, K. E., Woodland, S. C., Olsen, J. A., Beecher, M., & Gleave, R. (2018). The effects of relationship and progress feedback in group psychotherapy using the group questionnaire and outcome questionnaire-45: A randomized clinical trial. Psychotherapy, 55(2), 116–131. https://doi.org/10.1037/pst0000133
Chandler, B., Donelan, J., Casey, S., & Douglas, S. (2020). A leap to telehealth group programs: How measurement-based care can enhance patient engagement and communication. Association of Ambulatory Behavioral Health Care Annual Meeting.
Childs, A. W., & Connors, E. H. (2021). A roadmap for measurement-based care implementation in intensive outpatient treatment settings for children and adolescents. Evidence-Based Practice in Child and Adolescent Mental Health. https://doi.org/10.1080/23794925.2021.1975518
Childs, A. W., Unger, A., & Li, L. (2020). Rapid design and deployment of intensive outpatient, group-based psychiatric care using telehealth during coronavirus disease 2019 (COVID-19). Journal of the American Medical Informatics Association, 27(9), 1420–1424. https://doi.org/10.1093/jamia/ocaa138
Connors, E. H., Douglas, S., Jensen-Doss, A., Landes, S. J., Lewis, C. C., McLeod, B. D., Stanick, C., & Lyon, A. R. (2021). What gets measured gets done: How mental health agencies can leverage measurement-based care for better patient care, clinician supports, and organizational goals. Administration and Policy in Mental Health and Mental Health Services Research, 48(2), 250–265. https://doi.org/10.1007/s10488-020-01063-w
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. https://doi.org/10.1186/1748-5908-4-50
de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021). Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis. Clinical Psychology Review, 85, 102002. https://doi.org/10.1016/j.cpr.2021.102002
Dollar, K. M., Kirchner, J. E., DePhilippis, D., Ritchie, M. J., McGee-Vincent, P., Burden, J. L., & Resnick, S. G. (2020). Steps for implementing measurement-based care: Implementation planning guide development and use in quality improvement. Psychological Services, 17(3), 247–261. https://doi.org/10.1037/ser0000368
Douglas, S., Button, S., & Casey, S. E. (2016). Implementing for sustainability: Promoting use of a measurement feedback system for innovation and quality improvement. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 286–291. https://doi.org/10.1007/s10488-014-0607-8
Douglas, S., Jensen-Doss, A., Ordorica, C., & Comer, J. S. (2020). Strategies to enhance communication with telemental health measurement-based care (tMBC). Practice Innovations, 5(2), 143–149. https://doi.org/10.1037/pri0000119
Duchek, S. (2020). Organizational resilience: A capability-based conceptualization. Business Research, 13(1), 215–246. https://doi.org/10.1007/s40685-019-0085-7
Emanuel, R., Catty, J., Anscombe, E., Cantle, A., & Muller, H. (2014). Implementing an aim-based outcome measure in a psychoanalytic child psychotherapy service: Insights, experiences and evidence. Clinical Child Psychology and Psychiatry, 19(2), 169–183. https://doi.org/10.1177/1359104513485081
Fortney, J. C., & Sladek, R. (2015). Fixing Behavioral Health Care in America: A National Call for Measurement-Based Care in the Delivery of Behavioral Health Services. The Kennedy Forum.
Fortney, J. C., Unützer, J., Wrenn, G., Pyne, J. M., Smith, G. R., Schoenbaum, M., & Harbin, H. T. (2017). A tipping point for measurement-based care. Psychiatric Services, 68(2), 179–188. https://doi.org/10.1176/appi.ps.201500439
Gianukakis, A., & Parent, J. (2022). Implementing weekly measurement-based care in child intensive treatment programs. The Brown University Child and Adolescent Behavior Letter, 38(3), 1–4. https://doi.org/10.1002/cbl.30610
Goldberg, S. B., Babins-Wagner, R., Rousmaniere, T., Berzins, S., Hoyt, W. T., Whipple, J. L., Miller, S. D., & Wampold, B. E. (2016). Creating a climate for therapist improvement: A case study of an agency focused on outcomes and deliberate practice. Psychotherapy (chicago, Ill), 53(3), 367–375. https://doi.org/10.1037/pst0000060
Jellinek, M. S., Murphy, J. M., Robinson, J., Feins, A., Lamb, S., & \Fenton, T. (1988). Pediatric Symptom Checklist: Screening school-age children for psychosocial dysfunction. The Journal of Pediatrics, 112(2), 201–209. https://doi.org/10.1016/S0022-3476(88)80056-8.10.1016/S0022-3476(88)80056-8
Jensen-Doss, A., Douglas, S., Phillips, D. A., Gencdur, O., Zalman, A., & Gomez, N. E. (2020). Measurement-based care as a practice improvement tool: Clinical and organizational applications in youth mental health. Evidence-Based Practice in Child and Adolescent Mental Health, 5(3), 233–250. https://doi.org/10.1080/23794925.2020.1784062
Koementas-de Vos, M. M. W., van Dijk, M., Tiemens, B., de Jong, K., Witteman, C. L. M., & Nugter, M. A. (2022). Feedback-informed group treatment: A qualitative study of the experiences and needs of patients and therapists. International Journal of Group Psychotherapy, 72(3), 193–227. https://doi.org/10.1080/00207284.2022.2086557
Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69(2), 159–172.
Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions. Journal of Clinical Psychology, 61(2), 165–174. https://doi.org/10.1002/jclp.20113
Lambert, M. J., & Hawkins, E. J. (2001). Using Information about patient progress in supervision: Are outcomes enhanced? Australian Psychologist, 36(2), 131–138. https://doi.org/10.1080/00050060108259645
Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10(3), 288–301. https://doi.org/10.1093/clipsy.bpg025
Lamers, A., Van Nieuwenhuizen, C., Siebelink, B., Blaauw, T., & Vermeiren, R. (2015). The use of routine outcome monitoring in child semi-residential psychiatry: Predicting parents’ completion rates. Child and Adolescent Psychiatry and Mental Health, 9(1), 18. https://doi.org/10.1186/s13034-015-0049-4
Lavender, J., Benningfield, M. M., Merritt, J. A., Gibson, R. L., & Bettis, A. H. (2021). Measurement-based care in the adolescent partial hospital setting: implementation, challenges, & future directions. Evidence-Based Practice in Child and Adolescent Mental Health. https://doi.org/10.1080/23794925.2021.1981178
Lewis, C. C., Boyd, M., Puspitasari, A., Navarro, E., Howard, J., Kassab, H., Hoffman, M., Scott, K., Lyon, A., Douglas, S., Simon, G., & Kroenke, K. (2019). Implementing measurement-based care in behavioral health: A review. JAMA Psychiatry, 76(3), 324–335. https://doi.org/10.1001/jamapsychiatry.2018.3329
Lewis, C. C., Scott, K., Marti, C. N., Marriott, B. R., Kroenke, K., Putz, J. W., Mendel, P., & Rutkowski, D. (2015). Implementing measurement-based care (iMBC) for depression in community mental health: A dynamic cluster randomized trial study protocol. Implementation Science, 10(1), 127. https://doi.org/10.1186/s13012-015-0313-2
Moran, P., Kelesidi, K., Guglani, S., Davidson, S., & Ford, T. (2012). What do parents and carers think about routine outcome measures and their use? A focus group study of CAMHS attenders. Clinical Child Psychology and Psychiatry, 17(1), 65–79. https://doi.org/10.1177/1359104510391859
NIRN. (2020). Implementation Stages Planning Tool.
Posner, K. (2008). Columbia-Suicide Severity Rating Scale Screener Basic Since Last Contact. The Research Foundation for Mental Hygiene, Inc. https://cssrs.columbia.edu/wp-content/uploads/C-SSRS-Screener-recent-with-triage-colors-2021.docx
Posner, K., Brent, D., Lucas, C., Gould, M., Stanley, B., Brown, G., Fisher, P., Zelazny, J., Burke, A., Oquendo, M., & Mann, J. (2010). Columbia-Suicide Severity Rating Scale (C-SSRS) Children’s Baseline Version 6/23/10. The Research Foundation for Mental Hygiene, Inc. https://cssrs.columbia.edu/wp-content/uploads/C-SSRS_ChildBaseline_11.14.16.pdf
Powell, B. J., Fernandez, M. E., Williams, N. J., Aarons, G. A., Beidas, R. S., Lewis, C. C., McHugh, S. M., & Weiner, B. J. (2019). Enhancing the impact of implementation strategies in healthcare: A research agenda. Frontiers in Public Health. https://doi.org/10.3389/fpubh.2019.00003
Powell, B. J., Mandell, D. S., Hadley, T. R., Rubin, R. M., Evans, A. C., Hurford, M. O., & Beidas, R. S. (2017). Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A Cross-Sectional Observational Study. Implementation Science, 12(1), 64. https://doi.org/10.1186/s13012-017-0593-9
Pullmann, M. D., Lucid, L., Harrison, J. P., Martin, P., Deblinger, E., Benjamin, K. S., & Dorsey, S. (2018). Implementation climate and time predict intensity of supervision content related to evidence-based treatment. Frontiers in Public Health. https://doi.org/10.3389/fpubh.2018.00280
Resnick, S. G., & Hoff, R. A. (2020). Observations from the national implementation of measurement based care in mental health in the department of veterans affairs. Psychological Services, 17, 238–246. https://doi.org/10.1037/ser0000351
Roser, J., & Michael, S. (2021). Standards and Guidelines for Partial Hospitalization Programs and Intensive Outpatient Programs. Association for Ambulatory Behavioral Healthcare. https://aabh.org/wp-content/uploads/2022/03/ASG20F21A.pdf
Saldana, L., Chamberlain, P., Wang, W., & Hendricks Brown, C. (2012). Predicting program start-up using the stages of implementation measure. Administration and Policy in Mental Health and Mental Health Services Research, 39(6), 419–425. https://doi.org/10.1007/s10488-011-0363-y
Shechtman, Z., & Sarig, O. (2016). The effect of client progress feedback on child/adolescent’s group-counseling outcomes. The Journal for Specialists in Group Work, 41(4), 334–349. https://doi.org/10.1080/01933922.2016.1232323
Slavin, R. E., Cheung, A., Holmes, G., Madden, N. A., & Chamberlain, A. (2013). Effects of a data-driven district reform model on state assessment outcomes. American Educational Research Journal, 50(2), 371–396. https://doi.org/10.3102/0002831212466909
Svien, H., Burlingame, G. M., Griner, D., Beecher, M. E., & Alldredge, C. T. (2021). Group therapeutic relationship change: Using routine outcome monitoring to detect the effect of single versus multiple ruptures. American Psychological Association, 25(1), 45–58. https://doi.org/10.1037/gdn0000148
The Joint Commission. (2018). Complying with standard CTS.03.01.09 behavioral health care accreditation program. Oakbrook Terrace: The Joint Commission.
van Sonsbeek, M. A. M. S., Hutschemaekers, G. J. M., Veerman, J. W., Vermulst, A., Kleinjan, M., & Tiemens, B. G. (2021). Challenges in investigating the effective components of feedback from routine outcome monitoring (ROM) in youth mental health care. Child & Youth Care Forum, 50(2), 307–332. https://doi.org/10.1007/s10566-020-09574-1
Vito, R., Schmidt Hanbidge, A., & Brunskill, L. (2023). Leadership and organizational challenges, opportunities, resilience, and supports during the COVID-19 pandemic. Human Service Organizations: Management, Leadership & Governance, 47(2), 83–98. https://doi.org/10.1080/23303131.2022.2157355
Weissman, M. M., Orvaschel, H., & Padian, N. (1980). Children’s symptom and social functioning self-report scales comparison of mothers’ and children’s reports. The Journal of Nervous and Mental Disease, 168(12), 736.
Wolraich, M. L., Lambert, W., Doffing, M. A., Bickman, L., Simmons, T., & Worley, K. (2003). Psychometric properties of the vanderbilt ADHD diagnostic parent rating scale in a referred population. Journal of Pediatric Psychology, 28(8), 559–567. https://doi.org/10.1093/jpepsy/jsg046
Worthen, V. E., & Lambert, M. J. (2007). Outcome oriented supervision: Advantages of adding systematic client tracking to supportive consultations. Counselling and Psychotherapy Research, 7(1), 48–53. https://doi.org/10.1080/14733140601140873
Yin, R. K. (2009). Case study research: Design and methods (4th ed). Sage Publications.Jensen-Doss, A., Douglas, S., Phillips, D. A., Gencdur, O., Zalman, A., Gomez, N. E. (2020). Measurement-based care as a practice improvement tool: Clinical and organizational applications in youth mental health. Evidence-Based Practice in Child and Adolescent Mental Health, 5(3):233–250. https://doi.org/10.1080/23794925.2020.1784062
Funding
No funding, grants, or other support was received to assist with the preparation of this manuscript.
Author information
Authors and Affiliations
Contributions
JD and SD: were responsible for the initial conceptualization and broad outline of the article and overseeing the revision process. JD, TL, and SD: assembled and reviewed qualitative data and wrote initial drafts. JD, SD, and AW: organized and formatted the drafted sections, and completed the final draft. All authors reviewed the final draft of the document and have approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
At the time of this publication, Jill Donelan is employed by Mirah, a Measurement-Based Care (MBC) technology company, and therefore declares a potential conflict of interest. The project was designed and all data collection, review, and analysis were completed between 2021 and 2023, while Jill Donelan was employed as a clinical supervisor for the Child Partial Hospitalization Program in the department of Psychiatry at Baystate Health, with no financial relationship with Mirah. Vanderbilt University and Susan Douglas receive compensation related to the Peabody Treatment Progress Battery; and Susan Douglas has a financial relationship with MIRAH, and both are Measurement -Based Care (MBC) tools. The author declares a potential conflict of interest. There is a management plan in place at Vanderbilt University to monitor that this potential conflict does not jeopardize the objectivity of Dr. Douglas’ research.
IRB Approval
This study was reviewed by the Baystate Medical Center Institutional Board, which determined that this project met federal criteria for exemption and provided a waiver of the requirement for HIPAA authorization.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Donelan, J., Douglas, S., Willson, A. et al. Implementing Measurement-Based Care in a Youth Partial Hospital Setting: Leveraging Feedback for Sustainability. Adm Policy Ment Health (2024). https://doi.org/10.1007/s10488-024-01358-2
Accepted:
Published:
DOI: https://doi.org/10.1007/s10488-024-01358-2