Introduction

Residential treatment (RT) provides a structured form of care to individuals in a non-hospital setting. Residential facilities aim to address the specific needs of clients and improve independent functioning while helping them manage medications. Those who may benefit from residential care include individuals who need safe housing due to a substance use disorder or a comorbid mental illness (Reif et al., 2014). These environments are staffed up to 24 h per day by trained staff who work together on clients achieving treatment goals. Residents are supported in their efforts to develop key life skills (budgeting, grocery shopping, etc.), improve their interpersonal skills, and manage their illness and medications.

The unique setting of residential treatment is equipped with the resources and environment to sustain this level of care. As indicated by Heredia and colleagues (2022), this is particularly important for individuals who suffer from a severe mental illness (SMI), such as severe bipolar disorder or chronic suicidal ideality/intent. Treatment centers that utilize a multidisciplinary team — which might include human service aids, licensed professional counselors, nurses, case workers/managers, psychiatrists and psychologists, and employment specialists — can better address the unique needs of patients with an SMI (Heredia et al., 2022). Community participation and vocational activities are encouraged, giving the resident opportunities to function independently within the community.

The current pilot study assesses the impact of the Transitional Living Center (TLC), a residential treatment program that serves Lehigh County, Pennsylvania, residents aged 18 years and over who have a mental health diagnosis and need housing. Clients have an average stay of 9 to 12 months in garden-style apartments, where they receive support, supervision, and instruction in basic living skills. The objective of this study is to better understand the impact the clinical programing of TLC has on clients who successfully complete the program and reintegrate into the community, which will provide valuable insight into population-based health strategies. A secondary research objective is to gain insights to inform improvements to the quality of care the residents receive by analyzing the challenges and obstacles they face after discharge. Understanding how and whether individuals readjust to the community without TLC’s support structure is important in determining the best way to provide care. This longitudinal pilot study provides a base for further research in this specific area.

Methods

Participants

TLC clients who were successfully discharged from the program between 2019 and 2022 were invited to participate. Participants were compensated for their time and travel.

Measures

There were three primary assessments administered to the study clients by trained TLC staff given to the successfully discharged clients by scheduled intervals after program discharge. These assessments were the following: the World Health Organization Disability Assessment Schedule 2.0 (WHODAS-2.0), the Outcome Rating Scale (ORS), and a homelessness screening tool developed by Feldman and colleagues (2017).

The WHODAS-2.0 has been validated to measure health and disability across all disease categories (Federici et al., 2017), including mental, neurological, and addictive disorders over six domains of functioning (cognition, mobility, self-care, getting along, life activities, and community participation). The WHODAS-2.0 complex scoring template was used, which requires tabulating the scores from all items within each domain into a domain score. The domain score is then represented as a percentage (0–100) of the instrument’s 144 possible points. A lower score indicates lower level of disability.

The second assessment used was the Outcome Rating Scale (ORS), a validated four-item tool designed to measure areas of life functioning (individual, interpersonal, social, and overall) (Bringhurst et al., 2006). Scores in each domain range from 0 to 10, with higher scores indicating greater functioning. The scores from each of the four domains are totaled. This scale has been shown to be a useful tool for assessing well-being longitudinally throughout treatment (Manthei & Tuck, 2021; Dixon et al., 2022).

The third measure used was a five-question dichotomous scale (yes/no) that assesses risk for homelessness and determines housing status within the past 60 days (Feldman et al., 2017). Responses were coded as 1 for “Yes” and 0 for “No.” Respondents were considered “at risk” if they responded yes to the first question (“In the last 60 days have you been concerned about losing your housing”) and were considered “homeless” if they responded yes to any of the other four questions, which are based on definitions of homelessness from the U.S. Department of Housing and Urban Development, U.S. Department of Health and Huan Services, and the Veterans Administration (Feldman et al., 2017).

Procedures

Assessments were administered by trained TLC staff members in the clients’ apartments or the community after discharge. The WHODAS-2.0 was self-administered and completed under staff supervision. Staff members administered the ORS and homelessness scales verbally by asking for oral ratings or responses, which were recorded by the staff member and then deidentified prior to statistical analysis.

Each scale was administered at specific timepoints post-discharge (at discharge, 6 weeks, 3 months, 6 months, and 12 months). The IRB at our health network reviewed the study protocol determined that it was not human subjects research.

Results

Out of 124 total discharges from the TLC program, there were 32 successful discharges (graduation). Eleven clients agreed to participate, and six participants completed all tools at all timepoints (Fig. 1). Demographics for the six completed participants are shown in Table 1. Descriptive statistics were used for analysis due to the limited sample size. Scores from these six participants were aggregated for each tool at each timepoint (Tables 2, 3 and 4).

Fig. 1
figure 1

Study population and sampling breakdown

Full size image
Table 1 Demographics of the six study participants are shown below
Full size table
Table 2 World Health Organization Disability Assessment Schedule 2.0 (WHODAS-2.0): Discharge to 12 months post-discharge
Full size table
Table 3 Outcome rating scale (ORS) responses: discharge to 12 months post-discharge
Full size table
Table 4 Homelessness scale scoring: discharge to 12 months post-discharge
Full size table

Each of the six clients’ complex WHODAS-2.0 scores was calculated at each timepoint (Fig. 2). All six clients remained stable until the 6-month mark, at which point one client became unstable. At the 12-month mark, three clients reported significantly higher WHODAS-2.0 complex scores. Clients 3, 4, and 6 stood out as having the steepest increase in scores by 12 months post-discharge. Clients 3 and 6 scored the lowest at 3 months and then jumped significantly higher. Client 4 had the most variability in scores between time points but scored significantly higher in every assessment period except for month three. Clients 1, 2, and 5 had consistently low scores throughout the study period.

Fig. 2
figure 2

Individual patient WHODAS score trajectory: Discharge to 12 months post-discharge

Full size image

At discharge, all participants fell below the 85th percentile of the general population normative scores curve (Fig. 3). At 12 months post-discharge, clients 1, 2, and 5 were within approximately the 70th percentile compared with the general population, while clients 3, 4, and 6 sat roughly between the 88th and 95th percentiles (Fig. 4).

Fig. 3
figure 3

Participant WHODAS complex score at discharge: plotted on general population scoring curve

Full size image
Fig. 4
figure 4

Participant WHODAS complex score at 12 months post-discharge plotted on general population scoring

Full size image

The aggregated median and interquartile ranges (IQR) for the total WHODAS-2.0 scores, as well as each individual domain also were calculated (Table 1). At 12 months, the three highest-scoring domains across all clients were mobility (31.3), household activities (22.9), and cognition (20.0) (Table 1). There is no clear pattern, but we observed that most clients reported consistently low WHODAS-2.0 scores at discharge.

The median ORS scores remained rather consistent throughout the study period (Table 2). The most notable dip in median scores was at month 12 (7.6) in the “social” domain. The IQR of the total ORS scores provide a much larger range and have nearly identical values at discharge and 12 months (Table 2).

The homelessness scale results showed one client scored “homeless” at discharge, one client was “at risk” for homelessness at the 6-month mark, and two were “at risk” at the 12-month mark.

Discussion

The purpose of our project was to assess the impact the TLC had on program graduates as they reintegrated into the community. This pilot study is the first attempt to evaluate the TLC program and an important first step in establishing a framework for future study. A secondary objective was to gain insight into improvements we can make at TLC to better prepare clients for encountering stressful life events, maintaining their independence, and functioning without the TLC support structure. The data indicate that clients who scored the highest on the WHODAS-2.0 had increased difficulty in functioning during the three- to 12- month period. Those who scored the lowest had little difficulty throughout the entire period of assessment.

WHODAS-2.0 scores were compared with the results from the ORS scale to determine whether they reflected similar score changes. The increased scores seen at months six and 12 on the WHODAS-2.0 were not nearly as pronounced in the ORS data set (Table 2). Similarly, the homelessness scale (Table 3) did not reflect any of the fluctuations captured in the ORS and WHODAS-2.0 data sets. This measure provided little insight and was instead used to supplement the data collected by the assessments to provide an additional lens to potential stressors contributing to the scores. On an individual basis, being at risk for homelessness, or being considered homeless, can jeopardize well-being. However, the status of being at risk or homeless is not necessarily shown in the WHODAS-2.0 or ORS scales. While there are 2 clients who are at risk for homelessness at the 12-month benchmark, this may not explain the increase in WHODAS-2.0 scores alone. There are likely many other factors that contribute to the decrease in psychosocial functioning but being at risk of losing housing, or having lost housing, can be a major stressor and should be considered for each client.

Study Limitations

A limitation of the study was the small number of participants and high dropout rate (Fig. 1). Understanding why these participants were not able to be retained should be a critical step in improving research methods for a follow-up study. Despite the small N, this pilot study starts the conversation and provides initial evidence toward closing a gap in the literature related to measuring outcomes of residential programs.

It also is worth noting the impact of the COVID-19 pandemic on TLC operations and the gathering of research data. Day-to-day operations were heavily modified to fit safety standards and decrease exposure to the virus. Structured group activities were ceased, services were limited, and residents were isolated in their apartments until virtual services could be set up. Referral sources were significantly impacted, with many sources no longer working and bed numbers declining for roughly 18 months following the beginning of the pandemic. While there is no direct evidence collected regarding the impact of COVID-19 on mental health in this study, social isolation for 12 to 18 months likely contributed to increased difficulty in functioning and influenced their scores on the study instruments.

The pandemic also impacted data collection because some clients were not comfortable with meeting face to face. Some clients lost their jobs and no longer had the ability to pay for their phone service, which made it difficult for study personnel to contact them.

Finally, we experienced an unexpected disruption in data analysis when we discovered — after data collection was complete — that the WHO no longer provided the WHODAS-2.0 score calculator on its website, and we had significant difficulty locating the complex scoring formula to complete our analysis.

Application to Practice

Graduation from TLC is achieved when a client shows progress suggesting they are prepared to reintegrate into society and function normally without a residential support structure. These benchmarks include having a stable income, structured programming/work, managing medications and outside appointments independently, consistent demonstration of independent living skills (cleaning, cooking, grocery shopping). There is some subjectivity since every resident is different. Therefore, it will be important to assess well-being and difficulties in psychosocial functioning throughout the duration of a client’s stay at TLC in addition to tracking clients post-graduation. We should consider adding a more consistent assessment schedule (i.e., measures taken at intake, every three months during treatment, and every three months for one year after discharge) for all TLC clients.

Intensifying outpatient care as clients return to independent living should be seriously considered due to the transition period posing additional difficulties to functioning (de Mooij et al., 2016). Residential and care instability was found, with 5% of their participants (N = 262) making a successful transition from residential housing to independent living, 33% moving from independent living to a psychiatric hospital, and half of the participants failing to achieve care stability.Our findings indicate a decrease in functioning 6–12 months after discharge, which further supports the need for intensified outpatient follow-up. Other studies have observed a discontinuation of outpatient care after RT due to variable follow-up rates, which may help explain the difficulty in functioning after clients are discharged. Transportation and complexity of the patient have been identified as two barriers in continuing patient care, as clients with opioid addictions and clients with complex needs were difficult to keep engaged in treatment (Cole et al., 2022). Disparities in continued care were also documented, with non-White, Hispanic, and rural enrollees being significantly less likely to receive follow-ups (Acevedo et al., 2018). Additionally, both Cole et al. and Acevedo et al. found highly variable follow-up rates and a large proportion of patients not continuing care within 30 days after being discharged.

It also would be valuable to include more outside perspectives of clients, such as from family members, close friends, primary care providers, or caseworkers. These opinions can provide more insights about a client’s ability to function outside of the residential care environment. Incorporating this into our program may allow us to better track our clients’ progress and anticipate issues that may arise once outside of the TLC support structure.

Future Directions for Study

The aforementioned outside perspectives also could be used as an assessment in a follow-up study to see whether they help to detect decreases in functioning three to 12 months post-discharge. Also, upon intake into the program, clients should be assessed for complex treatment plans and/or needs that should be addressed. The residential care environment can provide multiple levels of support and intervention, so knowing what kind of structure would best benefit our clients may improve our quality of care. Thus, including a measure that specifies and identifies areas of difficulty can provide valuable data to track progress, develop treatment plans post-discharge, and improve our effectiveness in treating clients. A few examples could be social determinants of health (environment, access to resources, health care quality) and which services at TLC yielded the best results for clients.

TLC also provides safe housing to individuals who have a substance use disorder (SUD). A decline in seeking care after discharge from RT without consistent follow-ups has also been identified in individuals with SUDs (Rubinsky et al., 2017). Rubinsky et al. have identified strong patient relationships, predischarge introductions to continuing care providers, predischarge scheduling, accountability of program staff, accessibility, and persistent emphasis to be implicated in continuing care. Future studies at TLC should include follow-ups to establish outpatient care to improve long-term functioning after discharge.

Further research will be conducted using an improved framework for data collection. As part of this new framework, we will be establishing a REDcap (Research Electronic Data Capture) database of TLC client data.

Conclusion

The most important finding of this pilot study is the vulnerability of clients as they transition between the six- and 12-month period after discharge of residential treatment. Further research into programming and outcomes can provide insight into how to better prepare clients for discharge. In our efforts to better understand how TLC can improve lives, we will contribute to the growing body of research on residential care by advancing the methodologies of data collection, offering new and original research, and better understanding how TLC prepares clients for independent living.