Introduction

Health care service networks are an organizational solution to strengthening system integration across different health, mental health, and social services, in order to improve outcomes for patients with multiple, complex, and long-term needs. However, the literature on the association between service network effectiveness and patient-level outcomes remains inconclusive. This paper aims to assess the effects of system integration on patient outcomes, using the current context of the Belgian mental health care reform, which established networks of health, mental health, and social care services across the country. In this introduction, we first review the literature on service network effectiveness. We then describe the Belgian reform setting and finally we present the research questions.

Literature Review

For some time now, mental health care services have sought to organize care in the community, with the aim of providing continuity of care (Burns et al. 2009) and supporting recovery (Slade et al. 2014). However, since the shift in mental health care towards deinstitutionalization, continuity of care for severely mentally ill (SMI) patients living in the community has been suboptimal. In a Taiwanese study (Chi et al. 2016), a significant proportion of patients with schizophrenia was shown to be readmitted to hospitals within the first few months following discharge. Furthermore, a US study (Fontanella et al. 2014) showed that only a minority of patients with schizophrenia received sufficient follow-up visits. The adherence to drug treatment of such patients quickly decreases, as evidenced in a Finnish prospective study (Tiihonen et al. 2011), and, even in OECD countries, the overall quality of mental health care for patients with psychiatric disorders is considered low [Organisation for Economic Cooperation and Development (OECD) 2014]. One of the major consequences of this situation is the high rate of suicide in the weeks following hospital discharge (Olfson et al. 2016; Qin and Nordentoft 2005).

The low quality of continuity of care negatively affects patient-level outcomes such as quality of life (Priebe et al. 2010) and social integration (Corrigan and Phelan 2004; Fenton et al. 1997). In most countries, the employability and earnings of SMI patients are low compared to the general population (Kessler et al. 2008; Marwaha et al. 2007). Most of these patients are also isolated or have few social contacts (Palumbo et al. 2015).

Several interventions and initiatives have been implemented in an effort to improve continuity of care using system integration mechanisms (Crawford et al. 2004; Durbin et al. 2006). Health care service networks have been promoted as an organizational solution that helps to improve coordination across different services. Service networks are long-term agreements between organizations or services with the aim of providing patients with a comprehensive and coordinated range of interventions (Mitchell and Shortell 2000; Shortell et al. 2014).

However, evidence for the association between system-level arrangements and patient-level outcomes remains inconclusive (Bickman et al. 1999; Bickman 1996; Lehman et al. 1994; Morrissey et al. 1994, 2002; Provan and Milward 1994, 1995, 2001; Rosenheck et al. 2002). Several large-scale studies, e.g. the Fort Bragg experiment (Bickman et al. 1999), the ACCESS program for mentally ill homeless people (Rosenheck et al. 1998), and the Robert Wood Johnson Foundation Program on Chronic Mental Illness (Lehman et al. 1994; Morrissey et al. 1994), have failed to evidence clear effects of system interventions on clinical and social outcomes for patients. One review found that system integration had a more consistent and positive effect on continuity of care than on clinical and social integration outcomes (Durbin et al. 2006).

However, these initiatives were all carried out in North America and focused on specific subgroups of patients, such as homeless people and veterans (Bauer et al. 2006). Also, more research is needed on how network integration can be measured. Indeed, network integration involves several components, i.e. the network structure, the collaborative processes by which the network operates, and network governance. The structural dimension relates to the density of relations between services and the structural position of services within the network, e.g. one central service that brokers relations with other services (Provan and Milward 2001). Our previous work suggested that continuity of care and social integration require different network structures (Lorant et al. 2017). The processual dimension relates to the capacity of services and health professionals to collaborate. Systematic reviews of collaborative care approaches for people with mental illnesses have indicated that collaboration between primary and specialized care may have reduced hospitalizations (Bauer et al. 2006; Reilly et al. 2013) or improved medication adherence and quality of life (Archer et al. 2012). Finally, the network governance dimension relates to how the network is managed. Some studies suggest that a dedicated network governance organization is more effective (Milward et al. 2010; Provan and Kenis 2008). To assess the effectiveness of network integration, therefore, a multi-component approach is required (Fisher and Elnitsky 2012).

There is a need, therefore, to broaden the empirical basis of system integration studies and, in particular, a need for studies on how to assess the effects of system integration on outcomes for SMI patients in European healthcare systems.

Setting: The Belgian Mental Health Care System

Health care governance in Belgium is shared between the federal state and the three regional authorities. As far as mental health is concerned, most residential services, including general and psychiatric hospitals, are funded and regulated by the federal authority, while community and social care services are mainly funded and regulated by regional and local authorities. Belgium has comprehensive and uniform social health care insurance, which covers the entire population and is mostly financed by social contributions. Providers are predominantly non-profit organizations and self-employed medical doctors, who are mainly paid using a fee-for-service model. Most expenditures are paid by sickness funds and by patients (out-of-pocket payments, 25%). This implies a high level of decision-making autonomy for users, clinicians, and providers, as well as a high level of fragmentation in the system (Gerkens and Merkur 2010). Finally, Belgium has kept a high number of psychiatric beds available for long-term care and, despite the development of community care services, large psychiatric hospitals remained the basic care supply in adult psychiatric care until recently (Nicaise et al. 2014; Conférence Interministérielle Santé Publique 2010).

Goals of the Reform

However, since 2011, Belgium has been reforming the organization of mental health care delivery by commissioning networks of health, mental health, and social care services for adults with mental health needs across the country (Nicaise et al. 2014). The reform has four overarching aims: (i) to strengthen care supply in the community, (ii) to improve continuity of care, (iii) to reduce and intensify use of hospitals, and, (iv) to support the social integration of patients with mental health needs. The newly established networks were expected to provide five basic care functionalities: (1) prevention and early detection of mental illness; (2) crisis management and outreach; (3) recovery and social rehabilitation; (4) intensive, acute residential interventions; and (5) long-term care and accommodation.

Network Design

The networks were intended to target all adults with mental health problems within a catchment area. In practice, however, most clinicians involved in networks considered SMI patients to be the actual target group of the reform policy (Lorant et al. 2016). All networks were chaired by a coordinator and managed by a steering committee that was composed of representatives of each of the five functions mentioned above. Funding was provided by reallocating funds from the long-term psychiatric beds that were closed and using these funds to set up new outreach, assertive and crisis resolution teams. The policy blueprint of the reform also suggested several collaborative tools for achieving the goals of the reform (Conférence Interministérielle Santé Publique 2010), e.g. the use of individualized care plans and the designation of case managers. A detailed analysis of the program theory underlying the reform was presented elsewhere (Nicaise et al. 2014).

Reform Implementation

The reform process has, however, been implemented from the bottom up, with each network developing its own membership and coordination mechanisms, resulting in a great diversity of network projects, as explained elsewhere (Lorant et al. 2017). The commissioned networks were composed of 50 services on average, the smallest network including only 11 services and the largest including up to 115 services. Networks were composed of a well-balanced distribution of social services, psychiatric wards, community mental health teams, and primary care services, the latter two groups being somewhat less numerous.

Aims

This paper aims to assess the effectiveness of the reform policy for patient outcomes, in relation to the reform’s main objectives:

  • Was the establishment of service networks effective in improving patients’ continuity of care?

  • Was the establishment of service networks effective in reducing the resort to hospital use?

  • Was the establishment of service networks effective in supporting patients’ social integration?

  • Was the establishment of service networks effective in improving patients’ quality of life?

Firstly, effective system integration should strengthen SMI patients’ continuity of care by improving the exchange of information across the range of service types (i.e. social, medical, and psychological) and facilitating referrals between them. In addition, system integration should facilitate the identification and use of alternatives to hospitals whenever possible and become a vehicle for optimal discharge after hospitalization. Secondly, effective service networks should contribute to the social integration of patients by helping them to access care while remaining in the community, maintain social activity, and find resources to assist with access to the (competitive) labor market, independent housing, social interaction, and participation in society. Thirdly, overall, service networks might improve patients’ global, subjective quality of life.

Methods

This paper is part of a broader evaluation of the Belgian mental health care delivery reform that took place in 2014 and 2015. At the time, 19 networks had been established, covering approximately two-thirds of the country: ten networks were approved to start in 2012 and nine additional networks were approved to start in 2014. As we explained above, the hospitals commissioned to promote these service networks participated voluntarily. Some rehabilitation services also received additional funds as part of the reform policy. All the networks implemented new outreach services. Several collaboration tools were also suggested for the networks to implement.

Data Collection

We carried out a repeated cross-sectional survey on patients with severe mental illness in 2014 and 2015. Severe mental illness (SMI) was defined as affecting patients with a psychiatric diagnosis, who had been in contact with psychiatric services for at least 2 years, and who presented serious psychological or social limitations (De Rick et al. 2002). The data were collected from the 19 commissioned networks and from three control sites in areas not covered by the networks. The areas covered by the networks and control sites comprised most metropolitan areas across the country as well as some rural areas. In each site, 80 SMI patients were recruited, ten patients in each of eight service types: primary care, community mental health, crisis resolution, assertive outreach, psychiatric wards, long-term residential care, social care, and rehabilitation services (cluster sampling).

Patients had to provide informed consent in order to participate. Those who were unable to consent or to complete a questionnaire in Dutch or French, as well as patients whose participation might have affected their mental health, were not deemed eligible. The eligibility of patients was assessed at the service level by clinical staff, who made a list of all eligible patients, based on the admissions or resident directory. Ten patients were randomly selected from the list. The questionnaire included one self-administered section for the patient, and one section to be filled out by a reference clinician. Ethical approval was obtained from the Ethical Committee of KU Leuven Medical Centre under the reference no. B322201215190—study no. S54355.

Each service that participated was also asked to complete a questionnaire that included questions about the use of the collaborative tools suggested in the reform policy: individualized care plans, case management, multidisciplinary team case review, and managers involved in clinical activity. The sample selection flowchart is provided in Fig. 1. We expected to recruit up to 3040 patients in the experimental areas (80 patients × 19 areas × two waves) and 480 patients in the control areas (80 patients × three areas × two waves). We ended up with, respectively, 1831 and 333 patients. Because the selection was performed by clinicians, it was not possible to distinguish non-eligible patients from participation refusals. After discarding patient data that had missing information, we were left with 1407 valid records.

Fig. 1
figure 1

Sample selection flowchart

Full size image

Measurements: Outcome

In this study, we assessed the effect of the reform on four outcomes that correspond to the main goals of the reform policy: continuity of care, quality of life, social integration, and hospitalization. The first three outcomes were self-reported by patients, while the last outcome was reported by reference clinicians.

Continuity of care was measured using the Alberta Continuity of Service Scale for Mental Health (ACSS-MH) (Adair et al. 2003, 2005; Joyce et al. 2010). ACSS-MH is a 31-item scale that captures how the patient perceives continuity of care across three dimensions: individualized care, system responsiveness, and clinician responsiveness. Each item ranges from 1 (completely disagree) to 5 (completely agree) and the scale total score has a maximum value of 155. Quality of life was measured with the Manchester Short Assessment of Quality of Life (MANSA) (Priebe et al. 1999). The MANSA includes 16 items, four of which target objective domains (i.e. contacts with friends, other social contacts, being victimized, and being accused of crime); and 12 target subjective domains (i.e. satisfaction with work, finances, family, sexual life, and mental and physical health). Social integration was measured with the SIX Index, a measure of social integration suited to long-term psychiatric adult patients that summarizes indicators of social outcomes across four main dimensions: employment, accommodation, family relationships, and friendship (Priebe et al. 2008). The SIX returns a score ranging from 0 (no social integration) to 6 (high social integration). Finally, the reference clinician was asked whether the patient had been admitted to an inpatient unit in the 6 months preceding the questionnaire being filled out.

Additional socio-demographic and clinical data were collected, including the Health of the Nation Outcome Scale (HoNOS) score (Wing et al. 1998). HoNOS is a scale that measures how patients are functioning psychosocially. It was completed by the reference clinician. The HoNOS score ranges from 0 (no impairment) to 48 (extreme impairment).

Measurements: Exposure

It was not possible to implement an experimental design in such a far-reaching, nationwide reform. Here, for each patient, a composite index of intensity of exposure to the reform was computed based on the reform model presented above. Each patient was assigned a total score composed of five components that indicate the degree of exposure to the reform process: (i) recruitment in one of the newly established networks or in control-group areas (yes = 1, 0 otherwise), (ii) recruitment in the first wave of commissioned networks (starting in 2012, scored 1) or in the second wave of commissioned networks (starting in 2014, scored 0), (iii) recruitment in services directly commissioned or funded by the government (scored 1) or in other services (scored 0), (iv) recruitment in the newly established outreach teams (scored 1) or in other services (scored 0), and (v) recruitment in services implementing the suggested collaboration tools (scored 1) or in other services (scored 0). The index score was increased by one point for each of these components and thus ranged from 0 (lowest exposure) to 5 (highest exposure).

Statistical Analysis

We used multilevel regressions with random intercepts at the service level nested by network. The inclusion of a random component at the network level takes into account the heterogeneity of the population covered and of the composition and functioning of the network. Multilevel linear regression was used for the ACSS-MH and for the MANSA. Multinomial regression was used for the SIX Index and logistic regression was used for re-hospitalization. The effect of exposure to the reform policy was adjusted for the patients’ level of education, sex, age, and HoNOS score.

Results

Sample socio-demographic status, clinical status, and outcomes were tabulated by exposure group and are presented in Table 1. Patients were, on average, 46 years old, living in the community, and had at least an intermediate level of education. Most patients had a diagnosis of schizophrenia or mood disorder. Patients had rather poor outcomes: one-third had been hospitalized within the previous 6 months and their quality of life and social integration were low (mean MANSA score = 39.3/72; STD = 11.6; mean SIX score = 2.4/6; STD = 1.1). Perceived continuity of care, however, was assessed as good (mean ACSS-MH score = 114.8/155; STD = 13.9).

Table 1 Socio-demographics, clinical status, and patient outcomes according to the intensity of exposure to the reform policy: descriptive statistics and tests
Full size table

Across the exposure groups, patients were quite similar in terms of age, sex, and nationality. However, patients with no or low exposure scores (0, 1) had a lower level of education and a slightly higher HoNOS score than patients with higher exposure scores (F-test = 5.5).

The effects of exposure to the reform policy on each outcome are displayed in Table 2. The scores were controlled for socio-demographics, HoNOS score, and random intercepts at service-network level. Overall, patients with low reform exposure had a higher rate of hospitalization, a lower quality of life, lower perceived continuity of care, and slightly lower social integration. A higher reform exposure score, however, was associated with few positive effects at the patient level. Exposure to the reform had no effect on quality of life, social integration, or rate of hospitalization. However, a higher reform exposure score was associated with higher perceived continuity of care (β = 1.46, p < 0.001), but to a very limited extent (standard deviation of the ACSS-MH score = 13.9).

Table 2 Effect of exposure to the reform policy: beta from the multilevel regression models
Full size table

As explained above, exposure to the reform policy was composed of several elements that could affect outcomes unevenly. The overall effect measure may mask the effect of some specific, effective component. We therefore decomposed the overall exposure score into its several components. The results are presented in Table 3. Patients recruited from the first-wave networks had higher continuity of care than patients recruited from second-wave networks. The other components were not associated with continuity of care. Patients recruited from the newly established outreach teams had lower hospitalization rates than patients recruited from elsewhere. Hospitalization rates were also lower for patients recruited from the commissioned networks than from the control areas. Patients recruited from the commissioned services were, however, more likely to be hospitalized than patients recruited from other services. Social integration was also higher for the patients recruited from outreach services, as well as for the patients recruited from commissioned services and commissioned networks (but with borderline statistical significance). No component was significantly associated with quality of life.

Table 3 Effect of each reform component on patient outcomes, beta from the multilevel regression
Full size table

Discussion

Findings

We found that, over a period of 2 years, patients who had been more exposed to the reform policy had a slightly better perception of continuity of care. This perception was associated with earlier exposure to the reform. The rate of hospitalization was not affected by overall exposure to the reform, though it was lower for patients cared for by the newly established assertive or crisis management outreach services and for patients recruited from the commissioned networks than for those from the control areas. Similarly, social integration was not affected by the reform process but was higher for patients cared for by outreach teams. Finally, the reform had no effect on quality of life.

Despite the rush to collaborative community psychiatric care, there is little empirical evidence that networks of health, mental health, and social care services are effective in improving patient outcomes. This paper contributes to the body of knowledge about mental health networks by describing the effect of a nationwide reform, involving more than a thousand mental health, health, and social services, on key outcomes at the patient level, including hospitalization, continuity of care, social integration, and quality of life.

Interpretation

Our findings are consistent with the Fort Bragg experiment, which found that the experimental, integrated sites performed better than control sites in terms of access to and continuity of care, but did not display better clinical outcomes at the patient level (Bickman 1996; Bickman et al. 1999). Similarly, the ACCESS study found that system integration was associated with improvements in all outcome domains, although no difference was found between the experimental and comparison sites (Morrissey et al. 2002; Rosenheck et al. 1998, 2002). In other words, explicit strategies and funding for system integration did not succeed in improving patient outcomes. Similarly the Robert Wood Johnson Program on chronic mental illness found that continuity of case management was improved, but that patients’ symptoms were not (Lehman et al. 1994). As in the case of the ACCESS study, the role of system integration may owe much to the role of assertive community services in driving the effect of integration on continuity of care. Our study also suggests that, in terms of continuity of care, networks that were commissioned earlier on achieved better outcomes than networks that were commissioned later. This may suggest that it takes time for system integration to deliver its outcomes, as suggested by Leutz in his integration laws (Leutz 1999).

The effect of the reform on hospitalization was modest and ambiguous. One possible explanation has to do with the reform implementation process. As explained elsewhere (Nicaise et al. 2014), hospitals are still driving the implementation of the reform: hospitals are steering the service networks, selecting their partners and collaboration procedures, paying for the network coordinators, and financing the new outreach teams and the whole networking process. Interestingly, patients cared for by the new outreach services had better results, which is also consistent with the literature (Mueser et al. 1998) and with the reform objectives. Our results support this interpretation as we found that patients from reform-funded services (some of which are psychiatric wards) had a higher hospitalization rate than patients from other services. However, the higher intra-class correlation coefficient for hospitalization suggests that the local context matters: psychiatric bed density and the structural position of the hospitals within the network may dampen the effect of the reform on hospitalization (Lorant et al. 2017).

The different components of exposure to the reform had different effects on the outcomes. Two components, being recruited from the new outreach services and being recruited from the commissioned networks, were associated with a lower rate of hospitalization and with higher social integration. Time of exposure influenced continuity of care slightly. Implementing collaboration tools had no effect on any outcome. We should be cautious not to over-interpret these associations. It is worth noting that the new outreach services were more effective than the other tools promoted by the reform policy. However, we must consider how the collaboration tools were actually implemented. Results show that the higher the level of exposure to the reform, the greater the number of tools implemented by services (increasing from 1.5 for patients with no exposure to 4.7 for the most exposed patients). Hence, exposure to the reform is associated with more implementation of collaboration tools. Yet, we did not assess whether the tools were actually implemented with fidelity to guidelines, and it is unclear whether, for example, the case management or individual care plan interventions were implemented in a uniform way similarly across sites. A lack of fidelity in their implementation might explain possible mixed effects on outcomes (McHugo et al. 1999). Another possible explanation would be the time needed to produce effects at the patient level. Indeed, there was a borderline association between the number of tools implemented and the continuity of care score (β = 0.57, p = 0.06).

The overall, modest (at best) impact of the reform on patient-level outcomes can be explained by three shortcomings of the reform. First, no target group was explicitly defined, and this led to very vulnerable groups of patients being overlooked and to leaving considerable autonomy to different services and networks in selecting their patients (Lorant et al. 2016; Nicaise et al. 2014). Second, the design of these networks was very much bottom-up and thus largely influenced by the promoting hospitals, which were able to retain their control of resources (Lorant et al. 2017). Third, one key network leverage relating to the pooling and joint allocation of existing of resources was left unchanged by policymakers, possibly because financial issues would have led to more institutional squabbles.

Limitations

There are three main limitations to this study: the observational design, the sampling procedures, and the heterogeneity of exposure. The lack of a randomized trial is due to the type of intervention assessed and to the policymaking process. Although a classical evaluation of effectiveness usually requires a randomized trial, this is not the most appropriate method where organizational interventions are involved, for at least two reasons. Firstly, the number of variables to control for a random exposure allocation at the level of the organizations was too high for the intervention being evaluated. Secondly, there may be a “contamination effect”, i.e. the diffusion of information and practices and the circulation of patients between sites, some of which were exposed to the reform process and some of which were not. The reform implementation rapidly covered most of the country’s main metropolitan areas, leaving a thin margin for control group sites. Furthermore, as the reform targeted all patients and most services in each area, it was impossible to design a randomized approach within areas. Differences across exposure groups may thus ignore unobserved heterogeneity, even if we took a rather conservative analytical approach by controlling for potential confounders such as the HoNOS score. For example, due to the nature of the reform process, the control sites were not funded to implement outreach teams. This may have affected the comparability of the control and experimental sites.

The sampling process was handled by clinicians at the service level. Thus, although we provided clinicians with clear guidelines on how to recruit SMI patients, we cannot rule out a potential selection bias. In addition, this selection process prevented us from providing the total number of eligible patients and of patients who turned down the invitation to participate. It is worth noting that patients were recruited from different services, including from social and primary care services. This may explain our low percentage of diagnosed psychotic patients compared to other studies. Finally, exposure to the reform was limited to a maximum period of 2 years. When the evaluation was commissioned, baseline data collection was not considered.

We evidenced a dose–response association between exposure to the reform process and the number of collaborative tools implemented, even within the commissioned networks. This suggests that implementation procedures may mediate the effectiveness of the reform, as evidenced in the evaluation literature (Rossi et al. 2004), and confirm the importance of the processual dimension in service networks.

Conclusions

This paper underlines the pivotal role of outreach teams in the integration process but also the need to consider integration over time. As far as patient-level outcomes are concerned, setting up service networks, in itself, does not make much difference at the patient level. In the case of the Belgian reform, the main benefit for patients was associated with a small improvement in continuity of care. If other outcomes, such as social integration or quality of life, are targeted, shifting more resources from inpatient services to services that are targeting social integration more specifically, such as Individual Placement and Support (IPS) or Housing First programs, should be prioritized. Finally, this study shows that the bottom-up approach to setting up networks of services comes at the price of implementation across settings being heterogeneous.