Introduction

Severe mental illnesses (SMIs) often lead to psychosocial disability, affecting an individual’s capacity to interact with others, work, and lead ‘a contributing life’ [25]. Cognitive impairments associated with SMIs can include difficulties with memory, concentration, organised thinking, planning, problem-solving, and goal setting [13, 29]. On average, people with psychotic disorders score 1.6 standard deviations below the general population in cognitive ability,this has a significant impact on everyday functioning and ability to undertake everyday activities [23].

The majority of people with SMIs want to work and contribute to society [23]. The mental health benefits of work for this population are well established. A systematic meta-review by Modini et al. [22] found that ‘the role work can play in facilitating recovery from an illness and enhancing mental well-being needs to be highlighted and promoted more widely’ (p. 331). In addition to an income, identified mental health benefits of work for people with SMI include: enhanced structured daily activities, improved self-esteem, feeling a useful member of the community, and provision of social opportunities [9, 22]. Work remains elusive for many, however. Australian data show that only one in four people with a psychosocial disability (25.7%) are employed, compared with 80.3% of those without disability [1]. An important contributing factor to this may be that many modern work roles make complex demands on cognitive and social skills, which are often impaired in individuals with SMIs [23]. Psychosocial rehabilitation programs, particularly those focusing on vocational goals, need to address this issue.

The Cognitive Remediation Experts Working Group [7] define cognitive remediation (CR) as an ‘intervention targeting cognitive deficit (attention, memory, executive function, social cognition or meta-cognition) using scientific principles of learning with the ultimate goal of improving functional outcomes. Its effectiveness is enhanced when provided in a context (formal or informal) that provides support and opportunity for extending to everyday functioning'. CR has been shown to be effective in improving cognitive function in people with SMIs [40]. A review of CR interventions in individuals with SMIs found that computer-based CR exercises are equally as effective when delivered via computer or clinician, with computer-based CR more easily standardised [10]. Vocational rehabilitation can improve everyday living and employment-related skills [3, 14, 19] with the suggestion that the future of CR for individuals with SMI lies in ‘developing effective programs that combine CR and psychosocial and vocational rehabilitation’ [10, p. 4].

The EYM program incorporates core features of CR as identified by Bowie et al. [4]: use of qualified cognitive remediation therapists,cognitive exercise aimed at improving cognitive functioning; procedures to develop problem-solving strategies, and procedures to facilitate transfer to real-world functioning. A recent meta-analysis by Lejeune et al. [17] concluded that programs that incorporate bridging sessions and strategy coaching are ‘more cognitively potent’. The EYM program also incorporates lessons gleaned from the evaluation of the EYM pilot program [21], including use of validated assessments to measure cognitive and psychosocial outcomes, collection of vocational outcomes, implementation by a dedicated multi-disciplinary team of mental health professionals and longer-term follow-up of participant outcomes.

The current study aimed to evaluate the effectiveness of EYM in improving cognitive skills and psychosocial outcomes relevant to employment and community engagement in individuals with SMIs.

Methods

Overview and Design

EYM was delivered by WISE Employment, a not-for-profit employment services provider in Victoria, Australia. The EYM team consists of three masters-qualified occupational therapists, a counsellor, Indigenous mental health worker, and a mental health peer support worker. Evaluation was conducted by St Vincent’s Hospital Mental Health Service. Recruitment took place between March 2019 and October 2020. The study was granted ethics approval from the Human Research Ethics Committee at St Vincent’s Hospital, Melbourne, and all procedures were in line with the Declaration of Helsinki. Informed written consent was obtained from all participants.

Participants

Participants were referred to the EYM program by mental health and other community support services. They were eligible to complete the program if they were over the age of 18, had a diagnosis of a SMI (including affective disorders, psychotic disorders, or PTSD), had an employment or other vocational goal, and were able and willing to participate in groups. Exclusion criteria were: (1) unable to attend sessions at the service and (2) lack of basic English literacy.

Program Outline

EYM was originally designed by Fife Employment Access Trust in Scotland. The six-month program involves one to two sessions per week, combining computer-based CR exercises, individual project work, group reflection/ discussion, bridging exercises, and a work experience placement (see Table 1 for program outline). Bridging exercises, as described by Medalia and Bowie [20], are ‘therapist-led verbal discussion and skills-training activities that help cognitive remediation participants apply what is learned on the restorative exercises to cognitive performance in everyday life’ (p. 66). The CR program used was Happy Neuron [12], a computer-based cognitive training module, which includes exercises on spatial memory, verbal memory, visual memory, executive function, visual and auditory attention, and processing speed. COVID-19 restrictions were introduced in Melbourne, Australia, while recruitment was underway, leading to transitioning of the delivery to online,this was readily effected and did not unduly disrupt program delivery.

Table 1 Outline of Employ Your Mind program phases and components
Full size table

Outcome Measures

Basic demographic and clinical information were collected. Participants self-reported their primary mental health diagnosis, which were classified according to the International Classification of Diseases, 10th revision [39]. Assessments of psychosocial functioning (Table 2) and cognition (Table 3) were collected at commencement (baseline), completion, and at 12-month follow-up from program commencement. Vocational data were collected at program commencement and at 12-month follow-up from program commencement.

Table 2 Description of psychosocial outcome measures
Full size table
Table 3 Description of cognitive tests and neurocognitive domain
Full size table

Psychosocial Outcome Measures

Table 2. details the psychosocial outcomes that were collected.

Cognitive Outcome Measures

Cognitive assessments were selected from the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery (MCCB) [27] for which normative data have been established [16]. Eight of the ten MCCB tests were included as sufficient to evaluate five key neurocognitive domains (see Table 3). The excluded tests were the Continuous Performance Test, Identical Pairs (CPT-IP) [8] and the Brief Visuospatial Memory Test, Revised (BVMT-R) [2]. This selection reduced assessment burden on participants and allowed for more extensive assessment of psychosocial outcomes.

Raw scores for each MCCB test and domain were converted to age and gender corrected t-scores using the MCCB scoring program. Testing was conducted by masters-level examiners who received extensive training in administration of the MCCB using video and group training sessions under the guidance of a psychiatrist. Quality assurance included periodic checks on MCCB administration and scoring practices.

Vocational Outcome Measures

Employment and other vocational data for the preceding 12-months were collected to provide an overview of activity over two time periods: the year prior- and the year post-EYM commencement. Participants were asked to provide details of (1) competitive paid employment, (2) informal paid employment (for example, babysitting), (3) voluntary unpaid work, and (4) study and educational engagement over the previous 12-months.

Statistical Analyses

Data analyses were performed using Statistical Package for the Social Sciences (IBM, SPSS Statistics Version 25). Descriptive and frequency analyses were used to explore demographic and clinical characteristics. Data were assessed for normality and paired t-tests were used to compare cognitive and psychosocial parameters between baseline and post-program for the participants who completed the EYM program. Between-group comparisons of vocational outcomes were conducted using Fisher’s exact test. For all analyses, Bonferroni corrections were applied to account for multiple comparisons and significance was set at p < 0.003 for psychosocial outcomes and p < 0.006 for cognitive outcomes. Missing values were excluded on a list-wise basis.

Results

Thirty-two participants commenced the EYM program. Table 4 provides demographic and clinical characteristics of the entire sample at baseline.

Table 4 Demographic and clinical characteristics
Full size table

Twenty-one participants completed the four phases of the EYM program: 15 (71.4%) completed the program in person at WISE employment and 6 (28.6%) completed the program remotely via video teleconferencing due to COVID-19 restrictions. Of the 32 participants recruited, 11 (34.3%) withdrew prior to completion of the program: one (3.1%) due to mental ill health; one (3.1%) relocated interstate; two (6.25%) discontinued due to COVID-19; three (9.4%) were lost to follow-up, and four (12.5%) commenced employment or study. (Note: the two who discontinued due to COVID-19 rejoined the program when restrictions were relaxed but after completion of the evaluation).

Psychosocial Outcomes

Psychosocial and cognitive outcomes are reported for the 21 participants who completed the EYM program. Comparison of psychosocial outcomes between baseline and post-program are reported in Table 5.

Table 5 Psychosocial outcomes pre- and post-EYM program completion
Full size table

Participants reported significant increases in positive mental wellbeing (WEMWBS), mental health QoL and overall QoL, as well as significantly increased self-awareness, interests and values, and enhanced overall perceived working ability (as assessed by the DWA). Improvements were also seen in additional subscales of the AQoL (independent living, relationships, coping) and DWA (physical ability, organisation and problem solving, communication), increased work and social adjustment, as well as increased self-reported recovery, but these did not remain significant after correction for multiple comparisons.

Cognitive Outcomes

Participants completed an average of 13.05 ± 10.66 computer-based CR hours over the course of the EYM program. Cognitive outcomes are reported below in Table 6 for the participants who completed the EYM program. Participants also engaged in group discussion, reflection, strategy coaching, and bridging exercises. It is important to note that while this additional activity is considered instrumental in facilitation of transfer to real-world functioning, the time spent engaged in these activities was not collected as part of the evaluation.

Table 6 Cognitive outcomes pre- and post-EYM program completion
Full size table

A significant increase in speed of processing was observed in the Category Fluency Test between baseline and post-EYM. There were also increases in verbal learning (HVLT-R), working memory (LNS), reasoning and problem solving (Mazes) and social cognition (MSCEIT), yet these did not remain significant after correction for multiple comparisons.

Vocational Outcomes

Fifteen participants completed assessment of vocational outcomes for the preceding year at baseline and 12-months following EYM commencement (see Table 7).

Table 7 Vocational outcomes for year preceding and year following EYM commencement
Full size table

There was no significant change in the proportion of participants engaged in vocational activities between the year prior to and year following EYM commencement. The proportion of participants in competitive employment doubled from four (26.6%) to eight (53.3%) in the year following EYM commencement.

Discussion

This study aimed to evaluate the effectiveness of the Employ Your Mind (EYM) program in improving cognitive skills and psychosocial outcomes relevant to employment and community engagement in individuals with SMIs. The evaluation incorporated the recommendations made in the pilot EYM evaluation [21] including the utilisation of a selection of tests from the MCCB, the inclusion of validated assessments of mental well-being and quality of life, collection of vocational outcomes and longer term follow up. In addition, the program was delivered by a team directly employed by WISE Employment, which allowed for closer adherence to the core features of CR as identified by Bowie et al. [4] through ongoing training, professional development and supervision. Under these more stringent conditions, our study confirmed the positive results of the pilot program evaluation.

There were significant improvements in category fluency—an assessment of speed of processing—and measures of mental well-being, quality of life, and perceived working ability, despite the small sample size and limited statistical power. Moreover, improvements in these outcomes demonstrated medium to large effect sizes. There were trends towards improvement across all psychosocial measures (mental wellbeing, quality of life, recovery, work and social adjustment, perceived working ability) and several cognitive domains including speed of processing, verbal learning, working memory, reasoning and problem solving, and social cognition. The trend towards more competitive employment—despite the impact of a strict and extended COVID-19 lockdown in Melbourne during 2020—is also encouraging. These results are consistent with increasing evidence that CR has valuable and sustained benefits for people with SMI with vocational goals [31, 36, 42].

It is noteworthy that the evaluation results were obtained with an average time of 13.05 ± 10.66 h engaged in computer-based CR (using the Happy Neuron program) over the course of the EYM intervention. This amount of time is significantly less than the recommended hours (20–40 h) previously reported in the literature [4]. The Expert Working Group on Cognitive Remediation advised that 'intensive training is ideal to produce meaningful effects' whilst acknowledging there are as yet no published comparative 'dosing' studies of CR [4, p. 51]. Dosage captured by the Happy Neuron program does not include the valuable time spent engaged in group and individual strategy coaching, bridging, and reflection/ discussion, all of which are key components of CR used in the EYM program. In subsequent evaluations, it will be important to document this time in more detail so that dosage can be more accurately reported. The length of intervention, however, is consistent with other CR programs. A meta-analysis by Wykes et al. [41] found the average length of treatment to be 32.2 h provided across 16.7 weeks. More recently, in their meta-analysis of CR for negative symptoms of schizophrenia, Cella et al. [6] identified effective programs provided 24 to 100 h of therapy time across periods ranging from 12 weeks to two years. EYM provides 63 h of individual and group intervention and up to 20 h of volunteer work experience over the course of six months. These results are consistent with other research that demonstrates improvement in cognitive and psychosocial functioning, when provided in the context of rehabilitation [19, 41]. Further investigation would establish if positive outcomes might be obtained with fewer hours of computer based CR activities integrated into vocational rehabilitation.

Limitations

A particular challenge was the facilitation of EYM during the COVID-19 lockdown restrictions in 2020, which were strict and long-lasting in Melbourne. With recruitment affected for six months and no face-to-face contact, program delivery and the ability of participants to gain employment were all impacted. This was mitigated by adaptation of the program to telepractice (using computer-based CR and videoconferencing). Despite the effect of workplace restrictions, it is notable that a number of participants were nevertheless able to find employment. Future evaluation of the program should include a larger sample size, controlled design, and investigation of the effect of dosage, particularly in the context of a vocational rehabilitation setting.

Conclusion

The results of this evaluation strengthen confidence in the value of the EYM program to improve cognitive and psychosocial outcomes related to vocational and community engagement. A larger trial to improve understanding and effectiveness of the program, especially relating to the CR dose–effect relationship, is warranted.