Comparative Effectiveness of Integrated Peer Support and Clinical Staffing Models for Community-Based Residential Mental Health Rehabilitation: A Prospective Observational Study

This observational study compared the outcomes of consumers receiving community-based residential mental health rehabilitation support in Australia under a clinical staffing model and an integrated staffing model where Peer Support Workers are the majority component of the staffing profile. Reliable and clinically significant (RCS) change between admission and discharge in functional and clinical assessment measures were compared for consumers receiving care under the clinical (n = 52) and integrated (n = 93) staffing models. Covariate analyses examined the impact of known confounders on the outcomes of the staffing model groups. No statistically significant differences in RCS improvement were identified between the staffing models. However, logistic regression modelling showed that consumers admitted under the integrated staffing model were more likely to experience reliable improvement in general psychiatric symptoms and social functioning. The findings support the clinical and integrated staffing models achieving at least equivalent outcomes for community-based residential rehabilitation services consumers. Supplementary Information The online version contains supplementary material available at 10.1007/s10597-022-01023-8.


Introduction
Contemporary community-based mental health residential rehabilitation services combine medium-to-long term accommodation with intensive rehabilitation and psychosocial support (Parker et al., 2019a(Parker et al., , 2019b(Parker et al., , 2019c. These services provide transitional residential rehabilitation (TRR) to people experiencing severe and persistent mental illness to enable them to live more independently in the community.
Most people accessing these services are diagnosed with schizophrenia and have complex care needs (Dalton-Locke et al., 2020;Parker et al., 2019aParker et al., , 2019bParker et al., , 2019c. Providing intensive support over an extended duration in a residential setting incurs high costs per episode of care . Despite the costs associated with these services, there have been limited comparative studies to establish their effectiveness (Dalton-Locke et al., 2020;Parker et al., 2019aParker et al., , 2019bParker et al., , 2019c, and TRR service capacity has expanded considerably in Australia over the last decade (Karan et al., 2022).
Australian TRR-type services have been adapted based on changing policy agendas and local service priorities. For example, services have generally shifted from a focus on providing a permanent residence to transitional support (Gerrand et al., 2007), increasingly emphasize recoveryoriented practice (McKenna et al., 2016), and are exploring novel staffing configurations with reduced emphasis on clinical roles (Karan et al., 2022;Parker et al., 2016;Saraf & Newton, 2017). Including staff with a lived experience of mental illness (Peer Support Workers, PSWs) in traditional clinical mental health services is increasingly encouraged in Australia (Saraf & Newton, 2017;State of Victoria, 2021).
One such approach is the 'integrated staffing model' (Karan et al., 2022). Under this model, PSWs, rather than mental health nurses, represent the majority staffing component and draw on their personal recovery experiences to support consumers. The integrated staffing model was not intended to alter the principles and objectives of TRR care (Parker et al., 2016).
While there has been strong advocacy for the benefits of incorporating PSW roles in mental health services, the supporting evidence is mainly qualitative, with largely equivocal findings emerging from quantitative studies (Lloyd-Evans et al., 2014;O'Connor et al., 2017). Furthermore, there are concerns about how efforts to integrate paid PSWs within routine mental health care might undermine the value derived from 'real-world interactions between people supporting each other with their emotional distress' (p342) (Gillard, 2019).
Qualitative research conducted at TRR units trialing the integrated staffing model found that consumers and staff held positive expectations of this approach Parker et al., 2018Parker et al., , 2019aParker et al., , 2019bParker et al., , 2019c. Furthermore, 12-18 months following service entry, consumers supported under this staffing approach emphasized the value of PSW availability . A recent cross-sectional study in Queensland found that the integrated staffing model was associated with lower levels of restrictive practice, pharmacological treatment, and greater staff-rated consumer engagement than the clinical staffing model (Karan et al., 2022). The authors of this study emphasized the need for findings to be interpreted with caution as no inferences could be drawn as to whether the differential treatment was appropriate or preferable. No available research compares care outcomes between the integrated and clinical staffing models. Despite the limited quantitative evidence, the integrated staffing model was identified as 'show[ing] promise in supporting recovery-oriented practice and maximizing consumer choice and control' (p37) in a report commissioned by the 2020 Victorian Royal Commission into Mental Health (Harvey & Brophy, 2020). Before the broader dissemination of such an approach, it is critical to demonstrate that a substantial reduction in clinical staff within TRR services to accommodate PSW availability does not degrade the clinical and functional outcomes achieved.

Aims
This study considers whether clinical and functional outcomes differ between consumers admitted to community rehabilitation units operating the integrated and clinical staffing models. Given that the integrated staffing model was not intended to alter the core function of the service, we hypothesized that significant differences in outcomes between the staffing approaches would not emerge (Parker et al., 2016). However, if differences are present, this would affect the optimal staffing approach for future services.

Methods
Data were collected as part of a mixed-methods evaluation of the comparative effectiveness of integrated and clinical staffing models for Community Care Units (CCUs; ethics approval HREC/14/QPAH/62) (Parker et al., 2016). A prospective observational design was used due to the ethical and clinical inappropriateness of randomized site allocation or waitlist control. Focusing on 'comparative effectiveness' followed the assumption of clinical equipoise at the policy level. Individualized change rather than group-level comparisons were chosen based on advocacy for this approach (Trauer, 2010) and its increasing use in similar contexts (Barbato et al., 2007;Gonda et al., 2012;Maxwell, Tsoutsoulis, Menon Tarur Padinjareveettil, Zivkovic, & Rogers, 2019;Murugesan et al., 2007). The ISPOR Task Force Report for comparative effectiveness research (Berger et al., 2012) andSTROBE statement (von Elm et al., 2007) guided study reporting. Publications based on related data are available, including cohort description (Parker et al., 2019a(Parker et al., , 2019b(Parker et al., , 2019c, modelling predictors of unplanned discharge (Arnautovska et al., 2021), and qualitative research of stakeholder perspectives Parker et al., 2018Parker et al., , 2019Parker et al., , 2021.

Study Context
CCUs are the dominant community-based TRR service-type operated by public mental health services in Australia. These units provide living skills development and community integration support to consumers residing in independent living units (in a cluster housing configuration) over 6-to-24-months. Staff support is available 24-h a day.
This study considered data from consumers admitted across three CCUs in Queensland over 3-years (12/2014to-12/2017). One site operated the clinical staffing model, and two operated the integrated staffing model. Under the clinical staffing model, nursing staff reflect most staff roles; there are also senior allied health practitioners and medical staff. Under the integrated staffing model, PSWs reflect the majority team component (> 50%), with a reduction in the number of nursing roles. PSW staff have their own leadership structure and draw on their lived recovery experience to support consumers and guide clinical staff toward recovery-oriented practices. There is no specified interventional framework for PSWs at the CCUs. However, a qualitative study provides a rich description of what PSWs perceived their roles to be, emphasizing self-disclosure and connection through 'shared engagement in everyday activities… providing authentic opportunities to support residents deal with their experiences and fears… [building] relationships and trust… [and] reducing shame and isolation' (p5) (Wyder et al., 2020).

Participants
Consumers were admitted to the CCU closest to their mostrecent principal residence and were included in the cohort if they provided consent and stayed beyond the assessment period (6-weeks, n = 145/161). Ninety-one percent and 89% of eligible clinical and integrated staffing model site consumers consented. Recruitment exceeded the target to achieve > 80% power to detect a 15% difference in the Health of the Nation Outcome Scales (HoNOS) (Wing et al., 1998) set for the parent evaluation (n ≥ 100) that was based on the treatment change observed in an Australian community residential step-up/down service (Siskind et al., 2013).

Data Collection and Measures
Unblinded trained multi-disciplinary team members completed an assessment battery on admission and discharge. Diagnostic and demographic information was collected at admission, and treatment-related variables were collected at admission and discharge (see Table 1). Based on the literature, a range of known confounders relevant to understanding rehabilitation outcomes were available in our data set (see Supplementary Materials 1).

Defining Reliable and Clinically Significant Change
Difference scores (admission-discharge) and subsequent analyses were conducted on measures with > 50% of paired data. Where relevant, scores were transformed so that positive differences reflected improvement on all measures.
The reliable change index (RCI) was calculated using the Christensen and Mendoza formula (Christensen & Mendoza, 1986). The clinical significance of an individual discharge score was operationalized based on three cut-off methods (N. S. Jacobson & Truax, 1991): [Cut-off 1] More than 2 SDs from the dysfunctional population mean (i.e., cohort mean at admission); [Cut-off 2] Within 2 SDs of the functional population mean (i.e., normative data); and [Cut-off 3] Closer to the functional population mean than the dysfunctional population mean.
Normative data for Cut-offs 2 and 3 for HoNOS and LSP-16 came from a study of Australian individuals with a psychotic disorder accessing mental health services who had experienced at least one inpatient or emergency department admission within five years but none within six months (n = 114) (Maxwell et al., 2019). For BPRS-18 and SANS, normative data came from a study of community-dwelling individuals with clinically stable 'chronic schizophrenia' without admissions in the previous six months (n = 120) (Baynes et al., 2000). Relevant functional population data was not identifiable for the other measures; thus, only Cutoff 1 could be applied. Where skewed data limited the ability to interpret RCS based on Cut-off 1 meaningfully, RCS improvement was not considered (N. Jacobson et al., 1988).
Reliable and clinically significant (RCS) change was assumed where the change between admission and discharge score exceeded the RCI and crossed a clinical significance threshold (i.e., RCS improvement or RCS deterioration).

Analysis
Analyses were completed in IBM SPSS Statistics Version 27 (SPSS, 2017). Comparability of the three sites was supported by examining measures at admission using the Kruskal-Wallis test (Supplementary Materials 2). Data from integrated staffing model sites were merged for subsequent analyses.
Individual difference scores were categorized as 'reliable improvement 'or' no reliable improvement' (i.e., stable/deterioration) based on the RCI. The RCS improvement cutoff producing the largest proportion of improved consumers (Gonda et al., 2012) was used to categorize scores into 'improvement' and 'no improvement' for RCS change. Outcomes for the staffing model groups were compared using Chi-Square/Fisher's Exact tests, with effect size estimated using Cramer's V (Kim, 2017).
The impact of known confounders on the relationship between staffing model and reliable improvement was explored using binomial logistic regression modelling (see Supplementary Materials 1). Modelling was not undertaken for RCS improvement due to the low event rates. Independent variables (IVs) additional to the 'Integrated staffing model' entered in the final models were 1 3 rationalized based on a threshold of p < 0.200 (Mickey & Greenland, 1989). All IVs were entered simultaneously. Where time-related covariates were included, interactions with staffing model were explored. Events-to-IVs ratios in the final models exceeded the acceptable minimum threshold (5:1) in assessing confounders (Vittinghoff & McCulloch, 2007).

Results
The sample included 145 consumers aged 18-59 years (M = 31.4, SD = 9.0, see Table 2). The median duration of CCU care was 303-days. Although there were no differences in the frequency of involuntary treatment on  (Table 2).
In unadjusted analyses, only BPRS-18 scores differed between the two staffing models. A higher proportion of consumers admitted under the integrated staffing approach were reliably improved compared to the clinical staffing   Table 5  Table 4.

Discussion
This study considered whether consumers receiving TRR support under integrated and clinical staffing models achieve equivalent functional and clinical outcomes at discharge. Regardless of staffing model, most consumers (50.4%-72.5%) showed reliable improvements in negative psychotic symptoms (SANS), psychological wellbeing and distress (MHI-38), social functioning (SFS), disability (LSP-16), and mental health and social functioning (HoNOS). The unadjusted odds of reliable improvement were equivalent between the staffing model groups on all measures, except in general psychiatric symptoms (BPRS-18, favoring those in the integrated staffing model). No significant differences emerged in the likelihood of RCS improvement between the staffing model groups. Covariate analyses suggested that consumers admitted under the integrated staffing model were more likely to experience reliable improvement than in the clinical staffing model on two outcomes (BPRS-18 and SFS). Additional predictors of reliable improvement on the BPRS-18 were having a primary diagnosis of schizophrenia, higher HoNOS total score on admission, and lower levels of education. Additional predictors of reliable improvement in SFS emerging through the covariate analyses were the presence of comorbid substance use, involuntary mental health act status, and lower LSP-16 total scores on admission. The gains in clinical and functional outcomes are consistent with the literature supporting the positive impact of mental health rehabilitation (Chan et al., 2021;Dalton-Locke et al., 2020). Unlike a recent retrospective cohort study that included only clinical staffing model sites , reliable improvements in disability occurred for most consumers. However, our study focused on admissiondischarge outcomes rather than pre-admission and post-discharge. Our results also compare favorably to a recent Australian inpatient rehabilitation cohort study (Maxwell et al., 2019), with higher frequencies of reliable improvement in HoNOS and LSP. However, the comparability of these outcomes is limited by our mean length of admission being over three times longer.
The absence of marked differences between the integrated and clinical staffing configurations is consistent with the quantitative literature considering PSWs as care providers in Australian (O'Donnell et al., 1999) and international (Pitt et al., 2013) clinical services. Our findings indicate that the integrated staffing model achieved at least equivalent outcomes and that consumers under this model were more likely to have their involuntary treatment revoked prior to discharge. These findings provide reassurance that reduced restrictive and pharmacologically focused treatment at CCUs under the integrated staffing model (Karan et al., 2022) is not associated with inferior clinical and functional outcomes. Additionally, our findings add weight to the relevance of considering consumer preferences in terms of their emphasis on the value of the availability of PSWs under an integrated staffing model . Figure. 1 Plot of admission and discharge assessment scores for measures where reliable and clinically Significant change was calculable (deterioration / no change / improvement)

Limitations
A key limitation is the absence of process evaluation to identify treatment differences between the approaches. Additionally, inter-rater reliability data was unavailable, and a later admission date was associated with a lower likelihood of reliable improvement on HoNOS and SANS, suggesting possible impacts of processes and staff changes. Unmeasured and unknown confounders may have also impacted the results. For example, service-level factors that impact organizational performance were not considered, such as staffing turnover, shortages, and burnout (Coates & Howe, 2015).
Outcomes were considered without correcting for multiple comparisons, increasing the risk of Type 1 error. The nature of the planned analyses meant that applying such corrections would have limited the ability to draw meaningful conclusions due to inflation of the Type 2 error risk (Armstrong, 2014). Another important consideration is limitations in statistical power to detect small differences between the staffing models. This is a particularly relevant consideration for the outcomes with higher levels of missing data (AUDIT, BPRS, SANS, and SFS). Given this, our findings should be interpreted cautiously.
Unavailability of paired admission data occurred more frequently for consumers who had experienced unplanned discharge. This means the findings may be biased toward consumers who are more likely to have favorable outcomes. Additionally, high rates of missing data prevented comparing the personal recovery measure (STORI-30), an outcome highly relevant to the service focus. Table 4 Comparison of reliable and clinically significant improvement in outcome variables where > 50% of paired admission-to-discharge data was available a Normative population for the calculation of 'Cut-off 2' for HoNOS total and LSP-16 total was obtained from a study by Maxwell et al. (2018), and for the calculation of cut-off 3 for BPRS total, SANS total, and all SANS subscales from a study by Baynes et al. (2000) b For the MHI, only RCS 'Cut-off 1' was calculable, the raw score cut off (215)

Conclusions
Reliable improvements in symptoms and functioning generally occurred between admission to and discharge from community-based residential rehabilitation. Furthermore, most consumers demonstrated clinically significant improvements in negative psychotic symptoms and disability. Under the integrated and clinical staffing models, consumers had at least equivalent clinical and functional outcomes. In the context of other emerging research, our findings further emphasise the promising nature of the integrated approach as an alternative to traditional clinical staffing models. More research in other contexts will enhance the ability for future decisions about mental health rehabilitation services staffing to be evidence-informed.

Declarations
Conflict of interest FD, NK, and SP are employees of the public mental health service where the study was conducted.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.