Introduction

As the demand for mental health treatment among young people increases, barriers to accessing and engaging in mental healthcare services are becoming more visible. Systematic inadequacies in the provision of mental healthcare include long wait times, difficulty accessing (appropriate) treatment, insufficient cooperation and communication between organizations, and discontinuity of care during the transition from Child and Adolescent Mental Health Services (CAMHS) to Adult Mental Health Services (AMHS) [1,2,3,4]. Together, these issues lead to the inability to reach or engage young people in care.

Adolescents struggling with interrelated psychiatric and social problems are particularly vulnerable to the abovementioned service design deficiencies. This subgroup of young people experiences multiple difficulties in everyday life, such as problems with educational and vocational attendance, peer relationships, housing, the legal system, and/or personal finance [5,6,7,8]. Because of the multifaceted care needs required to address these problems, various professionals from different institutions need to be involved. In addition, many of these adolescents have often already had dissatisfactory experiences with mental health services, which leads to decreased trust in the services. In sum, these adolescents find it difficult to engage in traditional office-based mental health services due to the barriers and limitations of the care system [4, 9,10,11].

Several novel integrated youth-friendly care approaches have been developed to tackle the aforementioned service design issues, such as ACCESS Open Minds in Canada and Jigsaw in Ireland [12,13,14,15]. Yet, such initiatives do not specifically address the developmental needs of young people with more severe and enduring mental health presentations. Exceptions are the “early intervention psychosis” teams and the AMYOS (Assertive Mobile Youth Outreach Service) model [16]. In the Netherlands, Youth Flexible Assertive Community Treatment (Flexible ACT) teams have been set up to provide long-term integrated outreach care specifically for young people (up to 24 years of age) that (1) have wide-ranging and interrelated (persistent and enduring) psychiatric and social care needs and (2) do not readily engage in regular office-based mental health services. The multidisciplinary teams work closely with the adolescents, their families, and/or other key support figures, address their age-related developmental needs, and support them in their personal, social, and symptomatic recovery. Since 2014, Youth Flexible ACT teams have been deployed widely throughout the Netherlands. Nowadays, around 80 teams are active or under development [17].

Although Youth Flexible ACT is widespread in the Netherlands and gaining international appraisal, with the first teams being implemented in Norway, only a limited body of research has examined the effects of this integrated service delivery model. Two Dutch pilot studies [6, 18] showed preliminary evidence of a reduction in behavioral problems, family life problems, hallucinations and delusions, attentional problems, emotional symptoms, self-injury, and peer problems. Indications for the effectiveness of Youth Flexible ACT may also be derived from the broader domain of Youth Assertive Community Treatment (Youth ACT) programs. A systematic review of 13 studies revealed that Youth ACT supports improved mental health and general functioning [19]. Other recently published studies have shown encouraging clinical recovery rates in adolescents cared for by ACT teams [16, 20]. In contrast to regular ACT, Flexible ACT can provide dynamic levels of care and encompass a multi-agency approach delivering psychiatric treatment and social support for wide-ranging problems. In short, studies into youth-integrated outreach models show promising treatment outcomes. Yet, evidence regarding outcomes supporting the effectiveness of the Youth Flexible ACT model remains slim.

With our Youth Flexible ACT Multicenter Study, we aimed to investigate the effects of this Dutch client-centered service delivery model and contribute to developing accessible and integrated mental healthcare programs for adolescents with persistent and multiple care needs who find it difficult to engage in traditional mental health services. In this article, we aimed to examine change in symptomatic, social, and personal recovery outcomes throughout Youth Flexible ACT. Based on the extant literature, we hypothesized that adolescents receiving Youth Flexible ACT would show a reduction in the severity of mental health difficulties and improved social functioning over the 18-month care period.

Method

Study design

The Multicenter Youth Flexible ACT Study is a longitudinal observational prospective cohort study of 16 Youth Flexible ACT teams from seven mental healthcare institutions throughout the Netherlands. Adolescents and their mental health workers were asked to complete a baseline and 3 follow-up measurements every 6 months, totaling four measurements (T0, T1, T2, T3; data collection period October 2016–January 2020). See our Study Protocol for an extensive description of the research setting, data collection, and procedure [21].

Study inclusion criteria

The study participants comprised 12–24 years old adolescents who received Youth Flexible ACT at one of the participating mental healthcare organizations. Young people are eligible for Youth Flexible ACT if they: (1) are < 24 years of age; (2) are diagnosed with a mental health disorder (or presumptive diagnosis); (3) experience difficulties in multiple areas of daily life; (4) are not able to attend office-based treatment due to complexity of mental illness or actively refuse contact; (5) face family system problems and/or parenting issues; (6) live in the district of the Youth Flexible ACT team. Additionally, the following research inclusion criteria were used: clients had to be between 12 and 24 years of age, have sufficient knowledge of the Dutch language, and provide written informed consent (along with parent/caregivers’ consent).

Youth Flexible ACT

The Youth Flexible ACT model is an adapted variant of the existing version of the Adult Flexible ACT, which is the standard service delivery model for people with severe mental illness in the Netherlands [22, 23]. Flexible ACT teams incorporate an integral focused care approach, providing care across several domains, including psychiatric, addiction, and supportive care. The teams are composed of employees from different organizations (multi-agency approach) and collaborate closely with professionals from (other) care organizations (e.g., AMHS, CAMHS, addiction treatment services, intellectual disability services, and community social services). Flexible ACT aims to enhance continuity of care by delivering and coordinating psychiatric treatment and practical support by the same team of professionals, as well as adjusting care to each client’s needs (through individual case management and intensive team care). Unlike Adult Flexible ACT, Youth Flexible ACT includes a systemic family therapist, an employment and education specialist, and parent and family counselors. The teams stimulate children, adolescents, and young adults in the aspects of personal identity, social contacts, school, work, and leisure. They boost their resilience by developing life skills appropriate to their life stages and transitions. Youth Flexible ACT provides recovery and development-oriented interventions addressing three recovery domains [24]: symptomatic (minimize clinical symptoms), social (i.e., regaining everyday functioning in education, work and leisure, social relationships, and self-care and living) and personal (i.e., regaining a grip on their life, establishing a positive identity, living a meaningful life) [25]. Furthermore, the teams are expected to provide high-quality care according to the Flexible ACT model guidelines, including clinical practice guidelines and evidence-based practices. The Centre for Certification and Flexible ACT (CCAF) determines model adherence via audits [17]. Our case study provides a detailed description of Youth Flexible ACT [26].

Data collection procedure

Youth Flexible ACT team members asked adolescents to participate in the study during their regular care intake process. After signing informed consent, participants and the mental health workers were asked to complete a baseline measurement. The participants completed the questionnaires during their regular appointment with a familiar mental health worker or independently in their own time. Researchers were in close contact with mental health workers and informed them timely on (1) completing the follow-up questionnaires (T1, T2, T3) about their enrolled clients and (2) notifying their enrolled clients to complete their follow-up questionnaires. Both paper and online questionnaires were available, although online versions were preferred to minimize the chance of missing data. An online data system was used to collect the data. Confidentiality of the data was guaranteed through a two-factor authentication login procedure. Adolescent participants received a remuneration of €10,- per assessment. Trimbos Ethics Committee approved this study and its procedures (201607_75-FACT2).

Study sample

During the enrolment period (October 2016–June 2018), 199 eligible clients signed the informed consent to participate and completed baseline questionnaires. This client group (69% was 18 years of age or older) showed a high diversity of severe psychiatric and social problems associated with significant trauma and developmental, mood, and anxiety disorders (Table 1) [8]. Their development in multiple life domains was hindered, especially since one-third did not attend a school or have a job, and almost all adolescents showed problems with family life and peer relationships. About half reported experiencing poor quality of life. Other frequently reported difficulties were substance misuse problems, the involvement of the police/legal system, problems with intellectual functioning, and personal finance. Before the Youth Flexible ACT referral, most adolescents had been involved with office-based (specialized) mental healthcare (see our baseline paper for a detailed description of the Youth Flexible ACT client group) [8].

Table 1 Sociodemographic and clinical characteristics of study sample
Full size table

Instruments

Table 2 presents an overview of outcomes and respective measures. Additional information about the questionnaires is described in our study protocol [21].

Table 2 Overview of outcomes and respective measures
Full size table

Statistical analyses

Originally, we opted for an intention-to-treat analysis and created a link to the remaining questionnaires we sent via email to participants who exited Youth Flexible ACT within the data collection period. However, following up with clients after exiting Youth Flexible ACT was unsuccessful. Only a very small group (n = 12) did respond and had incomplete data. The analysis was, therefore, restricted to a per-protocol analysis of data from the period in which clients were enrolled in Youth Flexible ACT.

To test the hypothesis that treatment with Youth Flexible ACT is associated with improved symptomatic and functional outcomes, we applied latent growth curve analyses (LGCA) [42] using the statistical package Mplus, version 7.2 [43]. A linear growth curve between time (in months) and outcomes was assumed for each individual with intercept i (starting value) and slope s (increase or decrease per month) as parameter estimates. For the continuous outcomes, normal linear regression was used to estimate the means of the growth parameters i and s. A one-unit change in time (1 month) means that the outcome variable changes with the value of s. For binary and ordinal outcomes, logistic regression was used. The relationship between time and the outcome variable’s logit (log odds) was linear. The intercept i was fixed at zero (for model identification purposes) and the slope s was estimated. A one-unit change in time (1 month) means that the outcome variable’s logit (or log odds) changes with the value of s. Because clients were nested within 16 teams, COMPLEX was used to correct for possible non-independence of the data to obtain unbiased standard errors of the growth curve parameter estimates. To handle missingness in the data, we assumed that the missing data mechanism was Missing At Random (MAR); see also the Statistical Appendix (Online Resource, S1 Statistical Appendix). The Full Information Maximum Likelihood estimator (FIML; using all available information in the data) was used to handle missing values [44].

In our dataset, the actual time points for completing questionnaires varied. Consequently, the growth models could not be estimated with equidistant time points but with individually varying time points. Model fit indices for this kind of random growth models could not be calculated. Moreover, in longitudinal analysis, the effect of missingness on the results is related to study dropout, not intermittent missing values [45]. For intermittent missing values (e.g., respondents did not drop out of the study but did not fill out the questionnaire at one or more time points), random missingness was assumed, and it had no effect on the point estimates of i and s, but may have affected the standard errors of these point estimates (leading to higher standard errors and therefore lowering the probability of significant results). To account for study dropout, we performed sensitivity analyses (Online Resource, S1 Statistical Appendix) in which we compared the results of LGCA assuming MAR with LGCA under several conditions of Missing Not At Random (MNAR).

Effect sizes were estimated as Cohen’s d [46]: d = (s * duration)/SD with s as the slope (change per month) and duration as the treatment period. For SD, the estimated standard deviation at baseline was used (the most reliable estimate due to the absence of dropouts) for continuous outcomes. For binary or ordinal outcomes, the SD of the standard logistic distribution (π/√3=1.814) was used [47, 48]. Effect sizes of non-significant slopes were also non-significant.

Results

Study dropout and response rates

Research participants

Regarding study dropout, 109 (54.8%) of the 199 participants were still enrolled in the study at T1, 83 (41.7%) at T2, and 54 (27.1%) at T3 (Fig. 1). The most frequent reasons for study dropout were: (1) research participants exited Youth Flexible ACT and subsequently discontinued participation in the study (n = 79) and (2) failure to engage research participants in the study (n = 52). In these cases, teams indicated that they were too busy to administer the questionnaires (due to high-pressure situations, such as clients in crisis or changes in staff). Completing a research questionnaire (and to continue with the study) was not a priority if a client was in the ‘meddling’ care phase.

Fig. 1
figure 1

Flow diagram of study participants

Full size image

Not all enrolled clients completed the questionnaires at each measurement (Fig. 1). At T0, all 199 clients filled out the questionnaires. At T1, 82 of the 109 clients; at T2, 64 of the 83 clients; and at T3, all 54 clients completed the survey. The missed measurements of the enrolled clients are denoted as ‘intermittently missing’. As described above, the measurements were missing mainly at staff and organizational levels, and in other cases, clients indicated no motivation to complete questionnaires at that moment.

To get insights into the effect of study dropout, we tested whether the 145 study dropouts differed from the 54 study completers in age, gender, and symptomatic and functional outcomes completed at baseline. For the continuous variables, we used t-tests for independent samples; for the binary variables, we applied Chi-square tests. We found four significant results: study dropouts scored significantly lower on total difficulties (SDQ; t = − 2.06, p = 0.033), lower on SDQ impact (SDQ; t = − 2.71, p = 0.007), lower on depressive symptoms (CDI-2; t = − 2.31, p = 0.022), and lower on treatment satisfaction (t = − 2.01, p = 0.046).

Mental health workers

Mental health workers completed the HoNOSCA for all 199 clients at T0, 110 out of 171 clients at T1, 101 out of 140 clients at T2, and all 94 clients at T3. We compared the total number of study dropouts (n = 105) with the study completers (n = 94) (Fig. 1). For age and the HoNOSCA total score, the t-test for independent samples was used, and for the 13 HoNOSCA-items (ordinal variables), the Mann–Whitney test for independent samples. No significant differences were found for these 15 variables.

Time of response

Most adolescents completed T0 within 3 months (74.9%; mean 2.7 months; SD 2.28; range 1–14) after the start of the Flexible ACT care. T1 was completed after an average of 7.6 months after T0 (SD 1.53; range 5–13 months). T2 was completed after an average of 7.8 months after T1 (SD 2.18; range 5–16 months), and T3 was completed after an average of 6.7 months (SD 2.53; range 3–12 months) after T2.

Most mental health workers (70.4%; mean 3.0 months; SD 2.38; range 1–11) completed T0 within 3.0 months after the Flexible ACT care. T1 was completed after an average of 7.1 months (SD 1.92; range 2–13 months) after T0, T2 was completed after an average of 8.4 months after T1 (SD 2.97; range 4–17 months), and T3 was completed after an average of 7.6 months (SD 3.35; range 3–17 months) after T2.

Latent growth curve analyses

LGCA for the client-reported outcomes under the assumption of MAR with varying time points and correction for team effects

Table 3 displays means and standard deviations and the results of LGCA for the client-reported outcomes. The continuous outcome variables showed significant reductions in problems over time, all with small effect sizes. This included significant decreases (negative slopes) for the SDQ total score (psychosocial difficulties), SDQ impact score (impact of difficulties on daily life), CDI-2 total score (depressive symptoms), PQ-16 total score, and distress score (subclinical psychosis symptoms). Significant increases (positive slopes) were found for Kidscreen-10 (quality of life), EMPO 3.1 (empowerment), and Social Support and Peers (subscale Kidscreen-27). The binary outcome variables showed a negative slope for contact with the ‘Police/Legal System’, with a very large effect size. Reductions in problems were not significant for ‘School and/or work situation’, ‘Financial problems’, and ‘Satisfaction with Youth Flexible ACT’.

Table 3 LGCA results of client-reported outcomes
Full size table

LGCA for the clinician-reported outcomes under the assumption of MAR with varying time points and correction for team effects

Table 4 displays means and standard deviations and the results of LGCA for the clinician-reported outcomes. Of the 13 ordinal outcome variables, we found negative slopes with small to medium effect sizes for 5 HoNOSCA variables related to ‘overactivity and attentional problems’, ‘emotional problems’, ‘peer relationship problems’, ‘problems with family life’, and ‘poor school/work attendance’. The total score of the HoNOSCA, a continuous variable, showed a negative slope, meaning that the total score decreased over time. The effect size was medium. Reductions in problems were not significant for ‘disruptive and aggressive behavior’, ‘self-injury’, ‘substance misuse’, ‘problems with scholastic or language skills’, ‘physical illness or disability problems’, ‘hallucinations and delusions’, ‘non-organic somatic problems’ and ‘self-care and independence’. Yet, it should be noted that roughly 30% of the clients who scored problematic (severity score 2–4) on these HoNOSCA domains at baseline returned to a non-problematic score (severity score 0–1) at T2 or T3 (see Online Resource, Table S1). The number of psychiatric hospital admissions did not change significantly throughout the treatment period.

Table 4 LGCA results of clinician-reported outcomes
Full size table

Discussion

Main findings

This study investigated changes in symptomatic, social, and personal recovery outcomes over 18 months of Youth Flexible ACT. In line with previous Dutch pilot studies on the effects of the Youth Flexible ACT service model [6, 18], adolescents improved on both the symptomatic, social, and personal recovery domains during the treatment. Regarding (1) symptomatic recovery, client-reported outcomes included a reduction in overall psychosocial difficulties, depressive symptoms, and subclinical psychosis symptoms. In addition, clinician-reported outcomes involved a reduction in emotional and attentional problems. Concerning (2) social recovery, client-reported social outcomes included improved social interaction with peers and fewer contact moments with the police/legal system. In addition, clinician-reported outcomes showed fewer problems with school and/or work attendance, family life, and peer relationships. In terms of (3) personal recovery, client-reported outcomes included increased feelings of empowerment and health-related quality of life. Moreover, clients indicated that their psychosocial difficulties interfered less with everyday life.

Vocational and educational recovery

As outlined above, mental health workers reported decreased problems with school and/or work attendance throughout Flexible ACT. This means that adolescents who attend their classes and work more responsibly, and those who stay home, feel less resistance to finding appropriate education or work. Yet, client reports showed that education and/or employment status did not significantly change. Although the overall attitude towards education and work improved, this improvement did not result in more adolescents finding appropriate education or work. A possible reason could be that the study period of 18 months was too short to detect changes in educational and employment status. The desire to seek education or work at the onset of Youth Flexible ACT is not always immediate, and there can be a considerable delay between program enrollment and its start. Another explanation could be the type of care employed to address educational and vocational recovery varied between teams. Individual Placement and Support (IPS) is the gold standard for vocational recovery and has shown positive results for young people in a similar youth target group [49, 50]. At the time of data collection, IPS was being implemented in some of the teams.

Clinician-reported outcomes

While the LGCA of clinician reports showed a significant reduction in emotional problems, attentional problems, problems with school and/or work attendance, and problems with family life and peer relationships, it did not show reduced problems related to the other problem areas, such as substance misuse, self-injury, hallucinations and delusions, disruptive and aggressive behavior, and self-care and independence. It is interesting to note that for the outcomes that showed a significant reduction, a much larger percentage of adolescents reported ‘problematic’ at baseline (71.1% average over 5 scores; see Online Resource, Table S1) compared to outcomes that did not show a significant reduction (29.9% average over 8 scores). This suggests that only those symptoms that were severe at baseline were reduced. Focusing on adolescents who had a ‘problematic’ score on substance misuse, self-injury, hallucinations and delusions, disruptive and aggressive behavior, and self-care and independence at baseline, roughly half (48.2–67.7% see Online Resource, Table S1) of those who participated at T2 or T3 experienced a drop in score to ‘non-problematic’. In sum, we only found improvements over time for those outcomes that were severe at baseline.

It is important to note that not all interventions commenced at the start of the study period (T0). This can be because clients do not always have the immediate desire to commit to intervention at the onset of Youth Flexible ACT. The study period might not have been long enough to measure effects if treatment had started later during the study period, especially when there was no fourth measurement.

Strengths and limitations

This observational cohort study had several strengths: the naturalistic character of the study, the use of both clinician and client informants, the monitoring recovery on three domains simultaneously, and the participation of multiple teams across the Netherlands. Some limitations should be considered while interpreting our results.

First, we had no control or comparison group; therefore, outcomes could have also improved if adolescents had received another type of treatment or no treatment at all. We attempted to follow-up with adolescents who exited care to allow for some comparison between adolescents in and outside Youth Flexible ACT. However, these adolescents were difficult to reach and did not respond to our queries, which means that we only have data for adolescents who received Youth Flexible ACT at the time of completing the questionnaires. Our conclusions are restricted to adolescents who received Youth Flexible ACT.

Second, we had a high attrition rate in the follow-up. Although this is not surprising for a client group with an intensive intervention and four assessment points over 18 months, it could result in attrition bias in which participants who dropped out of the study differed from those who remained. In our study, dropouts were less satisfied with Flexible ACT than those who remained. However, dropouts still reported positive satisfaction with Flexible ACT (7.49 on a scale of 10, SD 1.78). In addition, the client-reported data showed that adolescents with less severe problems on SDQ (psychosocial difficulties) and CDI-2 (depressive symptoms) were more likely to drop out of the study. In addition, mental health workers did not report any differences in the HoNOSCA domains between dropouts and those who remained. As dropout data showed that most study dropouts also exited Youth Flexible ACT, it is possible that adolescents with less severe problems were less motivated to engage in treatment and participate in our study.

Moreover, it could be that these adolescents were no longer in need of Flexible ACT and a lighter form of care was more appropriate. Consequently, the follow-up measurements included more clients with severe problems on some self-reported scales. This could be an indication that the external validity of our results is limited to those adolescents who had severe problems at baseline. Additionally, the previous discussion section suggested that Youth Flexible ACT is more likely to reduce symptoms for individuals scoring higher at baseline. In sum, our results suggest that Youth Flexible ACT can be more useful for those adolescents and those domains that show severe problems at the onset.

Furthermore, the dropout data also showed that some adolescents had already exited Youth Flexible ACT between T0 and T1 (Fig. 1). Team members indicated that another form of care was more appropriate. On one hand, the referral to the Flexible ACT team might not have been right in the first place. This suggests that triage conducted at the point of entry could have been more accurate. On the other hand, these young people might have had difficulty accessing and engaging in regular mental healthcare services and possibly required a Flexible ACT approach (team’s easy accessibility, flexibility and outreaching character) to find the most appropriate form of care. Youth Flexible ACT could then act as a consultation hub for this group of clients.

Also, we performed a multitude of statistical tests, which could indicate the need for multiple test corrections. While such tests indeed decrease the probability of a type-I error (false-positive: the chance to label non-existing effects as significant), they also increase the probability of a type-II error (false-negative: the chance to miss real effects). The use of multiple test adjustments is, therefore, advised in experimental designs, especially in the setting of confirmatory clinical trials [51]. However, as the purpose of our study is more exploratory, we did not opt to perform multiple test corrections. It is, therefore, important to note that all of our reported effects should be considered as exploratory findings, and warrant confirmation in future research.

Finally, effect sizes from non-experimental studies tend to be larger than those from experimental studies with a control group [52]. The magnitude of the effect size should, therefore, be considered in relation to our observational research design.

Clinical implications and future research

Our study findings indicated that the Youth Flexible ACT service delivery model is promising for adolescents unable to engage successfully in regular (office-based) mental health support services. These adolescents with persistent psychiatric and social care needs are at risk of long-term problems. Even though the period of this study is relatively short (i.e., 18 months), the adolescents included already showed signs of improvement on important outcomes. Future research should include long-term follow-up (for instance, 3 years) to understand the recovery process at the symptomatic, social, and personal levels (for example, to examine how the school and/or work situation is improving) and to evaluate whether clients maintain a positive development after the treatment. Moreover, we need to explore the types of health care services provided during Flexible ACT and specific elements of the Youth Flexible ACT model that contribute to change over time to make the intervention more precise and personalized.

In summary, the Youth Flexible ACT model is an integrated and recovery- and developmental-oriented service delivery model for transitional age youth with persistent and multifaceted mental health and social care needs who find engagement in mainstream mental health services difficult. This study provides promising initial evidence of Youth Flexible ACT being associated with improved symptomatic, social, and personal recovery. Future research is needed to further corroborate the effects of this service delivery model.