Introduction

In the Netherlands, 1811 people died by suicide in 2019 [1], more than two and a half times the number of fatal road casualties of that same year (661) [2]. Dutch suicide rates appear to have stabilised since 2013 with around 11 suicides per 100,000 inhabitants, of whom more than two-thirds (68%) are men [3]. Almost 40% of Dutch citizens who died by suicide were in specialised mental health care treatment at the time of their death, meaning that the majority did not find their way to specialised care [4].

The World Health Organisation (WHO) recognises suicide as preventable and argues that it should be high on the global public health agenda. It proposes that it is essential to use comprehensive multisectoral strategies with community-level approaches that take social, psychological and cultural impact into account. As suicide is surrounded by stigma, shame and misunderstanding, community engagement is vital. Communities can reduce risk and strengthen protective factors by providing social support and follow-up care, raising awareness, fighting stigma and supporting those bereaved by suicide [5]. Raising awareness and fighting stigmas on help-seeking behaviour is crucial as research shows that stigmas related to mental health disorders and -services are an important reason for insufficient help-seeking [6,7,8]. For this reason, focussing on attitudinal factors is a key aspect of suicide prevention policies.

In 2001, a four-level intervention programme was implemented in the German city of Nuremberg, called The Nuremberg Alliance against Depression (NAD). The NAD was a high-intensity, 2-year action programme against depression and suicidality. Evaluation of the programme showed a significant reduction in the number of suicidal acts [9, 10]. Since then, this four-level intervention concept has been adopted by more than 70 regions in Germany within the German Alliance against Depression and has led in 2004 to a European-wide collaboration funded by the European Commission; the European Alliance against Depression (EAAD) [11]. The research project OSPI-Europe (Optimising Suicide Prevention Programmes and their Implementation in Europe) has evaluated the intervention in four European cities in Germany, Hungary, Portugal and Ireland. Aggregated data demonstrated no significant effect of the intervention on suicidal behaviour. The previously observed preventive effects on suicidal behaviour in the Nuremberg programme were only replicated in Portugal [12]. In contrast, Hofstra and colleagues conducted a systematic review and meta-analysis and found that suicide prevention interventions are effective in preventing both completed and attempted suicides, with a significant effect of the number of levels in the intervention on effect size [13].

This paper provides a report of a test of the effects of an intervention based on the shared multilevel approach of the EAAD model, namely: the Suicide Prevention Action Network (SUPRANET) Community programme [14]. This programme was introduced in six intervention regions in the Netherlands in January 2017, as part of the Dutch National Agenda for Suicide Prevention. Although both interventions consider suicidality to be an important outcome measure, there is a significant difference in communication strategy. While EAAD's public awareness campaign focuses mainly on depression, the SUPRANET campaign addresses suicide directly [14, 15]. The SUPRANET programme is coordinated by 113 Suicide Prevention, the suicide prevention helpline and expertise centre in the Netherlands.

A companion paper on the SUPRANET public awareness campaign, showed that the campaign was predominantly visible among the younger generation (< 25 years). The respondents who indicated having seen the public awareness campaign showed more openness towards seeking professional help and were considerably more likely to be familiar with the Dutch suicide prevention helpline. In contrast to expectations, campaign awareness also seemed to relate to a higher perceived taboo on suicide and a lower estimation of the value of professional help [16].

Because of the interactions between components and the synergistic effects that might arise within a multilevel prevention programme, it is not possible to determine the reach and contribution of each component in the totality of this intervention [17, 18]. Given this, the aim of our quasi-experimental study was to gain insight into the impact of the Suicide Prevention Action Networks on attitudinal changes in the general public by studying the exposure–response relationship. This is possible because the level of implementation of the intervention varied across the six intervention regions. We hypothesise that the attitudinal change and brand awareness of the Dutch helpline among the general public was greater in the intervention regions as compared to the control region and the highest in the region with the most exposure to the SUPRANET Community intervention. The findings of this study will be used to facilitate the nationwide implementation of this programme and will add to a growing body of research on the effectiveness of multilevel prevention programmes.

Methods

The SUPRANET community programme

In 2017, the SUPRANET Community programme was introduced in the Netherlands. SUPRANET Community is a 2-year multicomponent and multisetting suicide prevention programme. It was modelled on EAAD and consists of four levels: (1) increasing the awareness of suicide by local public awareness campaigns; (2) training local gatekeepers; (3) targeting high-risk persons in the community; and (4) the training and support of professionals in primary care settings. The implementation of the programme started in six intervention regions (see Fig. 1). In such a community, a network of local multidisciplinary teams and organisations shared ownership and responsibility for the prevention of suicide within a geographical region. More information on the selection process of the intervention regions can be found in the study protocol by Gilissen et al. [14].

Fig. 1
figure 1

The six SUPRANET communities. Colouring represents the population density of the region

Full size image

Level 1: local public awareness campaigns

Local public awareness campaigns were based on a national campaign launched by 113 Suicide Prevention. The campaign called De vraag van je leven, which can be translated as “The question of your life”, was aimed at the general public, as well as healthcare providers, general practitioners and other professionals. The campaign carried the message that asking one question can make a difference and intended to contribute to breaking the taboo on talking about suicide. The campaign aimed to educate the public in recognising the signs of suicidal thoughts, giving practical tips and guidelines on how to manage suicidality and bring to the attention that one can turn to 113 Suicide Prevention for help. The national public awareness campaign had five different “waves” of two or three weeks and started in February 2017 and ended in November 2018. More detailed information about the national and regional public awareness campaigns can be found in Van der Burgt et al. [16].

Level 2: training local gatekeepers

The fact that the majority of people who died by suicide did not find their way to specialised care, illustrates the importance of recognising and assisting people with suicidal ideation within community settings. In 2008, the gatekeeper training was tailored for implementation in the Netherlands [19]. Since then, many professionals in various employment sectors, such as teachers, police officers, railway employees and healthcare professionals, have completed the training. The Dutch gatekeeper training has shown to be effective in improving participant’s knowledge on suicide and addressing suicidality and in their self-confidence to conduct a dialogue on suicide and suicidal thoughts [20].

Level 3: targeting high-risk persons in the community

The SUPRANET regions were encouraged to actively approach specific risk groups, who are known to be especially vulnerable for developing suicidal thoughts. In some regions this included an intervention for people within the LGBT + community, other regions preferred to target people on disability or welfare, youth, students, farmers or middle-aged people (especially men). Examples of interventions include providing information about suicide prevention at schools and establishing contacts with organizations involved or working with high-risk groups.

Level 4: the support of professionals in primary care settings

The final component of the SUPRANET Community intervention includes the support of primary care professionals (PCPs) to stimulate the signalling and exploration of suicidal ideation and concrete suicide plans among their patients. In the SUPRANET regions, a suicide prevention training was offered to primary care professionals. This training was based upon the multidisciplinary guideline for diagnosis and treatment of suicidal behaviour [21]. In addition to the training, materials were offered to the PCPs. These materials included a checklist to use during consultations, flyers and posters aimed at patients to encourage talking about suicidal thoughts and modules on suicide and medication to inform about which medication might induce or reduce suicidality and which medication is often used for suicide attempts [22].

The exposure–response measure

The implementation of the SUPRANET Community programme varied between the six intervention regions, with most of them missing components of the EAAD model. There was only one region, in the north of the Netherlands, where the intervention was implemented as intended in the studied timeframe. In this region, general practitioners and gatekeepers were trained, the public awareness campaign was widely spread, with posters placed in various public places (like pharmacies, supermarkets, churches, sports clubs and inside buses). Advertorials were used, information meetings were organised and there was a collaboration with local broadcasters for radio- and TV spots and interviews in the local language. High-risk groups were, amongst others, targeted by a photo exposition aimed at LGBT+ youth, participation in a local canal pride and collaborations with the educational and agricultural sector. The national public awareness campaign was visible across the country but only the intervention regions were part of the multilevel and multicomponent intervention. For this reason, the Netherlands as a whole was categorised as “low exposure”. The one region that had the ultimate implementation of all four levels was categorised as “high exposure”. The remaining five intervention regions, that managed to implement two or three levels, were labelled as “medium exposure”.

Study design and procedure

This study has a repeated cross-sectional design and uses general population surveys to measure key variables over time. The surveys were conducted in the six intervention regions and in the Netherlands as a whole as a control region. Respondents were recruited through Survey Sampling International (now known as Dynata), which maintains a demographically diverse online panel of people who have registered to participate in selected surveys. When members of the panel log onto Dynata’s website, they are randomly directed (with the use of a survey router system) to available surveys based on their demographic characteristics. The panel members also receive a modest reward [23]. Due to this router system, it was not possible to collect response rates. Although Dynata made several efforts to ensure that the samples in this study represent different age groups, educational levels, genders, household situations and occupational statuses, it should be considered as a convenience sample. The data consists of four measurements, timed to be able to capture the effect of the intervention as best as possible. The baseline measurement was conducted before SUPRANET Community’s start in January 2017 (T0, N = 1670). The second measurement took place just after the first public awareness campaign wave in May 2017 (T1, N = 1662). The third and fourth measurements took place one year (T2, N = 1722) and  two years after the start of SUPRANET Community (T3, N = 1719), respectively. Respondents in longitudinal studies may become conditioned to the study in which they are participating; they may remember and repeat their previous answers, or become sensitised to the research topic [24]. This is why each measurement contains a new representative sample of adults (aged 18 +) from the Dutch general population; 1000 for the control region and approximately 100 per intervention region. Figure 2 illustrates a timeline of those four measurements with the number of respondents for each measurement.

Fig. 2
figure 2

Timeline of the four measurements

Full size image

Survey instrument

The same digital survey was used for each measurement. This survey gathered information on sociodemographic information (including gender, age, region, educational level and household situation), brand awareness of the Dutch helpline and perceived taboo on suicide. Moreover, the Dutch Depression Stigma Scale (DSS) and the Attitude towards Seeking Professional Psychological Help Short Form (ATSPPH-SF) were included. The Dutch DSS measures the stigma associated with depression and consists of two subscales: the Personal Stigma Scale and the Perceived Stigma Scale. Both subscales consist of nine items about depression. Responses to the items are measured on a five-point Likert scale ranging from “strongly disagree” to “strongly agree” [25]. ATSPPH-SF also contains two subscales: “Openness to Seek Treatment for Emotional Problems” and “Value and Need in Seeking Treatment”. Both subscales consist of five items assessed on a four-point Likert scale ranging from “disagree” to “agree” [26]. More detailed information about the survey instrument can be found in Van der Burgt et al.[16].

Statistical analysis

All statistical analyses were conducted using IBM SPSS Statistics version 25.0 and the “stats” package in R version 3.5.1. To investigate the overall impact of the intervention, the six intervention regions have been as compared to the control region using multivariable linear regression models to predict respondents’ scores on the five continuous outcomes: perceived taboo, the two subscales of the DSS and the ATSPPH-SF. Multiple logistic regression analysis was used for the binary outcome; brand awareness. Time, age, gender and educational level were expected to have an effect on the key variables and were, therefore, included as confounders. In order to gain insight into the differences between the intervention regions and the control region over time, these models were extended with interaction variables between “time” and “intervention region”. Additionally, to investigate the exposure–response relationship, the effect of “high exposure” (i.e. living in a high exposure region) and “medium exposure” as compared to control category “low exposure”, these models were extended by adding “level of exposure region” by “time” interaction terms as predictors. To measure the internal consistency of the ATSPPH-SF and DSS subscales in this sample, the Cronbach’s alpha value of each subscale was calculated.

Results

Between January 2017 and January 2019, four surveys have been carried out. Table 1 displays the respondents’ characteristics of those four measurements. These characteristics were comparable regarding gender, educational level and household situation. Only age (χ2 (15) = 83.18, p < 0.05) and occupational status (χ2 (21) = 68.40, p < 0.05) of the respondents were significantly different between the four measurements. We could not come up with an explanation of why these differences occurred and attributed these differences to chance. In this sample, the Cronbach’s alpha value of the two ATSPPH-SF subscales ‘Openness to Seek Treatment for Emotional Problems’ and ‘Value and Need in Seeking Treatment’ were 0.78 and 0.67 respectively. The alpha values of the DSS subscales were 0.84 for personal stigma and 0.85 for perceived stigma.

Table 1 Respondents’ characteristics (in %)
Full size table

Notice that the data have a hierarchical structure, as the respondents were clustered within regions. We, therefore, initially tried fitting multilevel models to take this clustering into account. However, multilevel analyses in R indicated a very small intraclass correlation, most likely due to the homogeneity of the Netherlands as a small country. Likewise, adding the regions as a second level did not significantly improve the models.

Table 2 presents the multiple regression models predicting respondents’ scores on the six key variables; brand awareness of 113 Suicide Prevention, perceived taboo on suicide, the two DSS subscales (perceived and personal stigma towards depression) and the two ATSPPH-SF subscales (Openness to Seek Treatment for Emotional Problems and Value and Need in Seeking Treatment). Taking a focus on the effect of the intervention, the results show that respondents in the intervention regions scored significantly better on four out of six key variables. When controlled for time, age, educational level and gender, respondents in the intervention regions were considerably more likely to be familiar with the Dutch helpline (OR = 1.26, p < 0.001) and scored 0.38 point higher on the Value and Need in Seeking Treatment subscale, that ranges from 0 to 15 (β = 0.38, p < 0.001) as compared to the respondents in the control region. Moreover, respondents in the intervention regions showed less personal (β = − 1.01, p < 0.001) and perceived stigma (β = − 0.73, p < 0.001) towards depression than respondents in the control region. The results indicated no significant difference in the perceived taboo on suicide and the scores on the Openness to Seek Treatment for Emotional Problems subscale.

Table 2 Overview of regression models predicting the scores on six key variables
Full size table

A more in-depth analysis (see Table 3) on the differences between the intervention regions and the control region over time, shows an increasing trend in the proportion of respondents who are familiar with 113 Suicide Prevention in both the control region and the intervention regions, with a significantly larger proportion in the intervention regions at T3 (OR = 1.43, p < 0.050). The difference between the intervention regions and the control region on the Value and Need in Seeking Treatment subscale seems to be driven by a significantly higher score in the intervention regions at T2 (β = 0.43, p < 0.050). The initially found difference between the intervention regions and the control region on personal stigma towards depression can be explained by a difference at baseline; the intervention regions already scored significantly lower on personal stigma regarding depression (β = − 0.68, p < 0.050) before the start of the intervention at T0. The earlier found difference in perceived stigma towards depression also disappears when we take the different measurements into account. The interaction variables reveal fluctuations in the intervention regions regarding perceived stigma towards depression, with a lower perceived stigma in the intervention regions at T1 and T3 but higher at T2.

Table 3 Overview of regression models predicting the scores on six key variables including interaction variables
Full size table

The exposure–response relationship is analysed by comparing the high exposure region with the other five interventions regions (medium exposure) and the control region (low exposure). Table 4 presents the regression models for the six key variables. The main effect of the exposure variable displays the difference between the low, medium and high exposure regions at baseline. The interaction variables show the scores on the key variables in the medium and high exposure regions over time. For example, we see that the medium exposure regions scored significantly lower at baseline on perceived stigma than the low exposure regions (β = − 0.70, p < 0.050). When looking at the interaction variables for “perceived stigma”, we see a decreasing trend in the high exposure region over time but differences are too little to be statistically significant. Overall, the results of the exposure–response analyses show that although the high exposure region seems to score better on multiple key variables, differences were too little to show significant effects. We, therefore, have to conclude that we found no evidence for an exposure–response relationship.

Table 4 Overview of exposure–response regression models including interaction variables
Full size table

When looking at the different predictors of the key variables we see differences between genders, age groups and educational levels. On average, women considered professional help to be more valuable (β = 0.77, p < 0.001), reported more openness towards seeking professional help (β = 0.42, p < 0.001) and reported less personal stigma (β = − 2.88, p < 0.001) and perceived stigma (β = − 0.94, p < 0.001) towards depression than men. Women were also less likely to be familiar with the Dutch helpline (OR = 0.88, p < 0.050) and scored higher on the perceived taboo on suicide (β = 0.27, p < 0.001). High educated people considered professional help to be more valuable (β = 0.62, p < 0.001) and reported more openness towards seeking professional help (β = 0.49, p < 0.001) than low educated respondents. High educated respondents also reported less personal stigma towards depression (β = − 2.13, p < 0.001), had more chance to be familiar with the Dutch helpline (OR = 1.44, p < 0.001) and scored a little higher on the perceived taboo on suicide (β = 0.36, p < 0.001) than low educated respondents.

Discussion

The main goal of this study was to test the effectiveness of the SUPRANET Community programme in achieving attitudinal change in the general public. Given the systematic variation in the level of implementation and thereby exposure to the intervention in the regions of the Netherlands participating, we were able to test for the exposure–response relationships. We hypothesised that the attitudinal change and brand awareness of the Dutch helpline among the general public was greater in the intervention regions as compared to the control region and the highest in the high exposure region. Multiple (logistic) regression analyses revealed that respondents in the intervention regions were considerably more likely to be familiar with the Dutch helpline 113 Suicide Prevention, considered professional help to be more valuable and showed less personal and perceived stigma towards depression than respondents in the control region. Overall, four of the six outcomes that were used to test the effectiveness of the programme showed significant results.

A more in-depth analysis of the differences between the intervention regions and the control region over time shows an increasing trend in the proportion of respondents who are familiar with 113 Suicide Prevention in both the control region and the intervention regions, with a significantly larger proportion in the intervention regions at T3. The previously found difference between the intervention regions and the control region regarding the Value and Need in Seeking Treatment subscale was driven by a significantly higher score in the intervention regions at T2. The initially found difference in personal stigma towards depression can be partially explained by a difference at baseline; the intervention regions already scored significantly lower on personal stigma regarding depression before the start of the intervention. And also, the difference in perceived stigma disappears when looking at the different measurements, due to the fluctuations between T1 and T3. We could not come up with an explanation for the difference in personal stigma towards depression at baseline, or the fluctuations in perceived stigma, but the fact that these results came to light in our follow-up analysis, underlines the importance of longitudinal studies. At the higher end of the exposure, the exposure–response analyses showed that, although the high exposure region seems to score better on multiple key variables, differences were too little to yield statistically significant results.

Strengths of this study are the repeated cross-sectional design, with relatively large sample sizes, the use of internationally validated instruments and the exposure–response analyses. However, our study has some limitations that should be taken into consideration when interpreting its findings. First, a randomised controlled trial would have been a preferred design. However, for many reasons, this was not feasible. Testing for effects of systematic variation in the exposure to the intervention, utilizing a longitudinal design with multiple observations and controlling for confounders were three strategies we employed in order to test the effect of the intervention. Although this limits the causal inference, it is the best we can do given the context and the type of intervention under study. Second, self-report can lead to desirability bias, especially when measuring attitudes and stigmas. But the use of an anonymous online survey likely minimised this bias, as self-administration of questionnaires can increase the willingness to disclose sensitive information as compared to face-to-face or telephone interviews [27]. Furthermore, this study looked at differences in time using the same questionnaires, so this bias is likely negligible.

In conclusion, our study provides the first evidence for the effectiveness of the SUPRANET Community programme on creating attitudinal change in the general public. The results revealed that respondents in the intervention regions considered professional help to be more valuable and were more likely to be familiar with the Dutch helpline than respondents in the control region. In contrast to our hypothesis, we found no evidence for an exposure–response relationship.