Background

Prevention is key to public mental health just as it is key to public health generally [1]. Prevention for mental health focuses on two key issues: Reducing severe psychopathology prior to developing mental disorder, and preventing new incidence of psychopathology [2,3,4]. Identifying and reducing early risk markers would be a valuable route to implementing mental health prevention. One such risk factor are psychotic experiences (PEs). PEs are hallucinations/delusions which can occur outside of a psychotic disorder and in the general population [5]. PEs are generally transitory and remit [6], and are considered as an early indicator of developing mental ill health [7]. PEs are common within the general population (~ 5%)[8], particularly in youth (between 8–17%)[9]. PEs are associated with a 4-fold increased risk for psychotic disorder, and a 3-fold increased risk for any mental disorder [10]. PEs are also associated with suicidal thoughts and behaviours [11,12,13] poorer functioning [14,15,16], healthcare needs [17,18,19] and psychiatric multi-morbidity [9, 20]. Adverse outcomes have been found even in those who report transient PEs [10, 15, 16, 19, 21, 22]. PEs and psychopathology are significantly associated [23, 24] and show a bi-directional relationship [25]. This substantial association has been proposed as evidence that PEs represent a marker of severe psychopathology [26, 27].

In comparison to intervention research, prevention is likely to be a more effective approach for treatment, due to the difficulties in recovery from a disorder [28,29,30], and additional deficits in functioning following a disorder [31, 32]. Prevention of PEs could be a valuable avenue of inquiry, particularly in the context of PEs and psychopathology. Additionally, studies have found participants who report PEs in addition to mental disorders, show slower rates of recovery when in treatment, due to higher rates of symptomology at baseline [33, 34]. Current school based prevention interventions often don’t differentiate between those with/without ill-health at baseline [4], which can achieve prevention aim of reducing psychopathology [2, 3], but does not determine if interventions can stop new incidence of psychopathology [4].

To date few studies have focused specifically on interventions for subclinical psychotic symptoms [35], and a majority which do exist rely on a clinical high risk model [35,36,37], which may represent only a small proportion of psychotic disorder [38,39,40]. Interventions for PEs without these criteria are rare [41,42,43,44]. Only one study to date has had a large (n > 1000) sample size [43], and did find that digital cognitive behavioural therapy was effective. One study to date has focused on PEs prevention [45], finding resilience training was effective at reducing PEs in a college sample (n = 107). To date, one intervention [42], and no prevention studies, have examined these approaches in adolescence, despite the highest incidence of PEs being in adolescence [46]. Additionally, to our knowledge, no study has examined the effectiveness of school-based interventions for preventing PEs.

Given the lack of knowledge regarding the efficacy of preventing PEs using school based interventions, we opted to conduct a secondary data analysis on a pre-existing school based randomized control trial (RCT) examining a suicide ideation & behaviour intervention. The aim of our study is to investigate the potential effectiveness of three school-based interventions for preventing PEs over a one-year period. We examined prevention in two ways (1) Reduction, at a whole group level were there less PEs? & (2) Prevention, were there fewer incident PEs? Our secondary aim was to examine whether there was a change in depression and anxiety scores following these interventions in those who reported PEs at baseline.

Methods

Ethical considerations

Ethical approval for the Saving and Empowering Young Lives in Europe (SEYLE) study was sought at each site of the study. For the Irish site, this was granted by the Clinical Research Ethics Committee of the Cork Teaching Hospitals. Three arms of the study (Control, Question-Persuade-Refer & Youth Aware of Mental Health) had opt-out consent and the other (Professional Screening) had opt-in consent. For safety during the RCT, students across all treatment arms who indicated acute suicide risk during baseline or follow-up assessments were immediately contacted by the SEYLE team and invited for a subsequent interview and referral to care.

Trial design

Full details for the SEYLE study have been described elsewhere [47, 48]. Briefly, SEYLE study is a clustered-RCT, investigating the effectiveness of prevention strategies for suicidal behaviour in adolescents across 11 countries. The study was registered at the German Clinical Trials Registry (DRKS00000214). A full breakdown of the methodology, implementation, and cost effectiveness of the SEYLE study can be found in [47,48,49,50]. Full CONSORT checklist are available (eTable 1).

Study setting

The Irish site was the only site that incorporated questions on PEs into the study questionnaire [51], and so was the only included site for this study. The Irish site identified twenty-four schools in the south-west of Ireland who were approached for participation. Of these, 17 participated. Informed consent and assent was obtained from a parent/guardian and all participating students. For three arms (Control, Question-Persuade-Refer & Youth Aware of Mental Health) flyers were sent home and opt-out consent was used. One arm (Screening by Professional and Refer) had opt-in consent, and so required parental signature to participate.

Participants

Participants were school-students, aged 13–15 years old. Students in classes where the majority were 14 years old were invited to participate (n = 1602) and 69% (n = 1112) completed a baseline self-report questionnaires. Follow-up questionnaires were administered at 3-months and 12-months after baseline, with 89% (n = 993) and 86% (n = 959) taking part at 3-months and 12-months, respectively (Fig. 1). For additional participant information (see eMethods).

Fig. 1
figure 1

CONSORT diagram of the Irish site of the SEYLE study. (Note: Grey shading indicates the intervention period. Partial PEs data indicates the number of individuals who provided at least one wave of PEs data. These individuals were included in the main analysis. A supplementary analysis restricted to participants who provided data at all three waves of the study is presented in eAnalysis 1)

Full size image

Interventions

The SEYLE study consisted of three active intervention arms and one control arm: (1) A gatekeeper intervention: Question, Persuade and Refer (QPR), (2) A psychoeducational intervention: Youth Aware of Mental health programme (YAM), (3) A universal screener and selective intervention: Screening by Professional and Refer (ProfScreen) and (4) A Minimal Intervention arm (Control). The interventions were provided between baseline and the three-month follow-up. Two interventions (QPR, YAM) use a universal intervention approach, while ProfScreen utilizes a universal screening and selective-indicative intervention approach.

A full description of each arm can be found in [48].

  1. 1.

    The QPR is a standardised gate-keeper programme [52]. For the SEYLE study, QPR was used to train teachers and school staff to recognise the risk of suicidal behaviour in students and improve their communication skills with students to motivate the at-risk pupil to seek professional help.

  2. 2.

    The YAM is a standardised mental health awareness programme developed for the SEYLE study targeting pupils [53]. This intervention includes interactive lectures on mental health, a series of role-play sessions and workshops, a 32-page information booklet that pupils could take home and psycho-educational posters that were hung in the classrooms.

  3. 3.

    ProfScreen is a two-stage school-based screening approach, which was developed for the SEYLE study[54]. In the first step, students exceeding one or more predetermined cut-offs (eTable 2) on psychopathology, risk-behaviour, or both, on the baseline screening questionnaire were considered “at-risk”. Endorsement of PEs was not a criterion for ProfScreen referral. In cases where a participant was determined as “at-risk”, the child’s parents were contacted by phone and letter to arrange an interview with a mental health professional.

The ProfScreen clinical interview was a semi-structured clinical interview, which was based on the Schedule for Affective Disorders and Schizophrenia for School-Age Children [55], and carried out by a child and adolescent psychiatrist or registrar trainee. It was developed to assess the need for mental healthcare, rather than to determine a psychiatric diagnosis. The interview was used to distinguish between pupils with psychological problems that required referral to mental healthcare and those who did not. Wherever possible, cut-offs were defined according to DSM-IV diagnostic criteria. All those who were considered in need were referred for appropriate intervention. Interviews were arranged between the baseline assessment and 3-month follow-up.

4) The Control arm were, for ethical reasons, provided with minimal intervention. Psycho-educational posters were displayed in the pupils’ classrooms. Contact information for local health care providers was provided on these posters.

Outcomes

Measurement of Psychotic Experiences: PEs were measured at baseline, three-month follow-up and 12-month follow-up using a question on auditory hallucinations from the Adolescent Psychotic Symptom Screener (APSS)[51]. This question has been shown to have excellent psychometric properties [56] and a “definite” endorsement of this items has been validated against clinical interview with excellent sensitivity and specificity for all PEs [51]. This was the only question from the APSS that was included at all waves of the study.

Measurement of Depression: The Beck Depression Inventory II (BDI) was used as a self-reported measure of depression at all waves of the study[57]. The BDI is a 21-item clinically validated screening tool for depression with higher scoring indicating greater levels of depression.

Measurement of Anxiety: The Zung Self-Rating Anxiety Scale (SAS) was used as a self-reported measure of anxiety at all waves of the study[58]. The SAS is a 20-item clinically validated screening tool for anxiety with higher scoring indicating greater levels of anxiety.

Measurement of demographics: Information was obtained at baseline assessment on the participant’s age, gender, and nationality (Irish or non-Irish). In addition, participants were asked to respond “yes” or “no” to the question “Have you, during the past 12-months, been physically attacked?”. Physical victimization is used a measure of bullying, known to be a confounder to PE rates [59].

Sample size

The target sample for each intervention arm as well as for the control arm is 250 pupils, i.e. 1,000 subjects in each participating country [48]. For the present study, a total of 1096 were included from the Irish site, including all four arms; Controls (n = 353), QPR (n = 212), YAM (n = 256), ProfScreen (n = 169). Previous prevention studies for PE [45] had a sample of 107, and Cohen’s d of 0.57, utilizing this power, we estimate the need for a minimum of 102 per group to accurately calculate the research questions for the current study.

Randomization

Randomization was done at a school level, 17 schools which consented to participated were randomized into one of the arms of the study. Schools were stratified into large (> median school size in Ireland) or small (< median school size in Ireland) and randomized using a random number generator [48].

Statistical Methods

All analysis was completed using Stata 14 [60].

Aim 1

A mixed-effect logit model with random effect of school was used, due to intervention clustering, and accounting for the within-subject repeated-measures effect. The main effects of intervention arm, time and their interactions are reported. Interaction effects (examining the effectiveness of the intervention) at 3-months and 12-months are displayed in odds ratios at 3-months and 12-months are displayed in odds ratios before and after adjustment-1 (baseline psychotic experiences) and adjustment-2 (baseline psychotic experiences, age, gender, nationality and exposure to physical victimisation). A stratified analysis excluded those who reported PEs at baseline, to examine prevention of PEs. Secondary data analysis were conducted on only complete cases using the same statistical technique (eAnalysis 1).

Aim 2

Depression and anxiety symptom scores were grouped by intervention arm, and further divided by PE status at baseline. The main effects, the interaction between those with/without PEs at baseline, and time, on depression and anxiety scores, were examined. Depression scores formed a negative binomial distribution and were analysed using a mixed effect negative binomial model. The anxiety scores were normally distributed, thus we applied a linear mixed effects model.

Results

Baseline Demographics and Clinical Characteristics

Table 1 Baseline demographic and clinical profiles of study, based on intervention arm
Full size table

Participants in the ProfScreen group were slightly older than controls (β = 0.56, 95%CI: 0.45–0.68, p < .001)) (Table 1). There was a higher percentage of males in the YAM group relative to controls (χ2 = 6.47, p = .01). The QPR(χ2 = 15.12, p < .001) and YAM (χ2 = 13.22, p < .001) had slightly lower percentage of Irish born participants relative to controls (Table 1). A greater percentage of participants assigned to the ProfScreen arm reported PEs at baseline than the control arm (RR:2.07,CI:1.08–3.99,p = .029). They also had higher anxiety scores at baseline (β:1.95,CI:0.56–3.35,p = .006) but this effect size was small (η2 = 0.01,CI:0.001–0.041).

Participants with and without PEs

In the total sample and within each study arm, there were no significant differences in age, gender or nationality between participants with and without PEs (eTable 3). There were significant differences in physical victimisation, depression and/or anxiety scores between those with and without PEs in the overall sample and within the study arms (eTable 3).

Aim 1

Examining the effect of the interventions on the point prevalence of PEs at follow-up.

Over the course of the study 11% (n = 120) of study participants reported PEs on at least one occasion. PEs were reported by 7.1% (n = 76) of participants at baseline, 5.3% (n = 5) at 3-months and 4.5% (n = 42) at 12-months.

Table 2 Effect sizes for each intervention arm at 3 months and 12 months on point prevalence & incidence of PEs.
Full size table

There was no significant main effect of intervention arm, time or interaction between intervention arm and time at 3-months follow-up. At 12-months follow-up, there was a significant interaction indicating a reduction in the point prevalence of PEs in the ProfScreen arm when compared to the Control arm (Fig. 2). This effect was retained after adjustment (Table 2). A test for trend indicated a significant linear effect in the ProfScreen arm over the three time points (χ2 = 11.42,p < .001). There were no other significant interactions, or linear or quadratic effects in any group.

Excluding those who reported PEs at baseline, there was no significant main effect at 3-month for any intervention arm(Table 2). At 12-month, there was a significantly lower incidence of PE in the ProfScreen arm, compared to controls (Table 2). No other arm showed a significantly lower number of PEs, compared to controls (Table 2). Secondary data analysis (eAnalysis 1) of complete cases and excluding those who reported PEs at baseline showed a similar reduction in PEs only in the ProfScreen arm at 12-months.

Fig. 2
figure 2

The point prevalence of psychotic experiences at baseline, 3-months and 12-months follow-up in each arm of the study. (Note: Dotted line is the point prevalence in each of the participating schools. Grey shading indicates the intervention period)

Full size image

Aim 2

To examine whether there are changes in depression & anxiety scores in those with and without PEs at baseline.

Depression Scores

There were significant main effects of PE group and time in all arms of the study. This indicated that participants with baseline PEs had significantly higher depression scores than their peers and that depression scores were lower at follow-up relative to baseline (Fig. 3). We observed an interaction between PEs group and time in the ProfScreen arm (3-months:IRR:0.67,CI:0.57–0.79,p < .001; 12-months:IRR:0.63,CI:0.39–1.01,p = .057) and in the YAM arm at 12-months (3-months:IRR:1.05,CI:0.91–1.21,p = .477; 12-months:IRR:0.59,CI:0.40–0.85,p = .005). Both suggested that those with baseline PEs had a greater reduction in depression scores relative to those without baseline PEs. There was no interaction in either the control or the QPR arms (eFigure 3). Adjusting for baseline depression and other characteristics had very little effect on the interaction effect size in the ProfScreen arm (3-months:IRR:0.70,CI:0.69–0.73,p < .001; 12-months:IRR:0.71,CI:0.49–1.02,p = .06) (eFigure 1). However, there was some attenuation in the interaction in the YAM arm (3-months:IRR:0.97,CI:0.79–1.19,p = .80 12-months:IRR:0.62,CI:0.36–1.07,p = .08) (eFigure 2).

Fig. 3
figure 3

Depression scores at baseline, 3-months and 12-months follow-up in each arm of the study when stratified by the presence or absence of psychotic experiences at baseline. (Note: Beck depression Inventory. Grey shading indicates the intervention period. Dotted lines are linear trend lines. Error bars represent +/-1 standard error of the mean)

Full size image

Anxiety Scores

There were significant main effects of PEs group in all study arms, which indicated that participants with baseline PEs had significantly higher anxiety scores than their peers (Fig. 4). We observed an interaction between PEs group and time in the ProfScreen arm at 3-months (β:-4.76,CI:-5.61- -3.92,p < .001) and 12-months (β:-4.20,CI:-5.59- -2.81,p = .001), indicating a greater reduction in anxiety scores in those with PEs when compared with their peers (Fig. 4). There was also an interaction between PEs and time on anxiety scores in the control arm at 12-months (β:-4.63,CI:-8.52- -0.74,p = .02). There was no interaction in either of the other two intervention arms (eFigs. 2 and 3). All significant interactions were retained even after adjustment for baseline anxiety scores and other characteristics (eFigure 1).

Fig. 4
figure 4

Anxiety scores at baseline, 3-months and 12-months follow-up in each arm of the study when stratified by the presence or absence of psychotic experiences at baseline. (Note: Zung Anxiety Score. Grey shading indicates the intervention period. Dotted lines are linear trend lines. Error bars represent +/-1 standard error of the mean)

Full size image

Discussion

There is growing interest in a prevention based approach to mental disorders [61, 62]. This study aimed to examine for the first time the effectiveness of a school-based intervention to reduce and prevent PEs. We utilized a pre-existing RCT for suicide prevention to conduct a secondary data analysis, therefore results should be viewed as indicative of potential efficacy of school based interventions. Within this context, we found; The universal psychopathology screener, and integrated referral intervention (ProfScreen) showed a significant reduction and prevention in 12-month point prevalence of PEs, compared to controls. The universal & gate keeper intervention arms (YAM and QPR), did not show an effect on 12-month reduction or incidence of PEs. Those with PEs reported higher depression & anxiety scores than those without PEs at baseline. In the ProfScreen arm, those with PEs showed significantly greater reductions in both depression and anxiety scores, than their peers without baseline PEs.

Our results suggest that a two-stage screening and referral system can be effective at reducing the point prevalence and incidence of PEs at 12-months. Our results expand the findings from the university setting [43, 45] by highlighting that certain school-based interventions can reduce and prevent PEs. Potential explanations for this improvement may be due to the direct effect of the intervention with a health professional. Previous research on the SEYLE study found those who received the ProfScreen intervention did show a general (but non-significant) trend of higher help-seeking behaviour than controls [63]. However, previous research [64] examining those who were referred from the ProfScreen intervention, found overall only 38% of the participants in the SEYLE study attended the ProfScreen intervention. In the Irish site, 37% attended, and ranges across sites was substantial, from 5.7% in Italy, to 96.7% in France [64] .

The reason for the efficacy of the universal screener and selective intervention in this study then may be better explained by the act of being contacted by a health professional, rather than the meeting with the professional. Previous RCTs have observed an improved effect of intervention driven by more frequent follow-up contact [65]. For SEYLE the phone calls, letter, and/or referral, may have resulted in greater family awareness of potential problems. Previous research has shown parental support and lower child-parent conflict mediate the relationship between childhood adversity and PEs [66], and PEs and subsequent psychopathology [67]. It is also possible that the monitoring of psychiatric symptoms and the awareness that professional help was actively available assisted in reducing psychiatric morbidity, as has been observed in clinical studies [68]. Finally, it is possible that families sought supports/services outside of the ProfScreen intervention, although data is not available to examine service use of non-attendant referral participants for the SEYLE study.

An alternative explanation may be a down-stream effect of interventions. The primary goal of the SEYLE study was to examine the effectiveness of interventions for suicidal behaviour, rather than PEs. PEs are associated with suicidal thoughts and behaviour [12]. Another study has shown that CBT for improving sleeping patterns has been shown to reduce PEs in university students [43]. Evidence appears to show that interventions designed for different primary outcomes can reduce rates of PEs. Therefore, interventions which target symptoms known to be associated with PEs, can also improve PEs, without being a primary target i.e. a down-stream effect. The mechanisms for the ProfScreen intervention effect require further investigation.

Neither the teacher training (QPR) or universal psychoeducational (YAM) intervention showed significant evidence of preventing PEs. The universal psychoeducational intervention did reduce depression scores in young people with PEs at 12-months. In the full SEYLE sample, this intervention also significantly reduced the incidence of suicidal ideation & attempts at 12-month follow-up [47]. The differing outcomes of the current study may be as the interventions were insufficiently specialized for PEs i.e. the YAM arm was universal but with a focus on depression and suicidality. This interpretation is supported by the effectiveness of the universal screener and referral intervention arm (ProfScreen), which screened for psychopathology, highly associated with PEs [27, 69], and was not only targeting suicidality, but general poor mental health. However, specifically designed universal school-based interventions for PEs would need to be examined before such a result could be concluded.

Our secondary aim found those who reported PEs at baseline showed a greater improvement in anxiety and depression scores, relative to those without PEs, following the ProfScreen intervention. Previous research has observed differences in outcomes for PEs with and without co-occurring mental disorders [19, 69], and a recent intervention study found those with PEs showed slower rates of recovery from anxiety and depressive disorders [33]. In line with this research [33], those who reported baseline PEs showed substantially higher depression and anxiety symptoms, relative to those without PEs. This may therefore support literature of PEs being a marker of more severe psychopathology [27, 38, 69]. Within this context, studies which aim at prevention of PEs may be valuable to improving mental wellness.

Limitations

The main limitation of the study was that the primary goal of the SEYLE study was to examine the effectiveness of interventions for suicidal behaviour, rather than PEs. Only the Irish sites used PEs as a measure, limiting the numbers involved. Additionally, due to the random sampling design, more individuals with PEs at baseline happened to be in the schools which were assigned to the ProfScreen arm. This was controlled for in analysis, but with a larger sample of schools i.e. the whole SEYLE sample, this random imbalance was unlikely to have occurred. Finally, PEs were measured using a single item (auditory hallucinations). Auditory hallucinations been shown have good clinical and construct validity for measuring PEs [51]. However, future studies should broaden their PEs definitions to include items on other psychotic phenomena.

Another consideration is a selective intervention approach can lead to stigma [70]. Selectively targeting students may result in peer scrutiny, which could increase isolation. We propose that perhaps the most effective preventative treatment may be a selective interventions, embedded within a universal intervention. It is plausible that this combination would provide psycho-educational information for all young people while simultaneously identifying those most at-risk. Such an intervention requires investigation of the efficacy and cost.

Implication and future directions

Prevention and reduction is an important area of PE research; Firstly, PEs, even when transient, show long-term poorer functioning, psychopathology, and elevated healthcare needs [10, 15, 16, 19, 21, 22]. Therefore PEs prevention could be a valuable in improving general well-being. School based prevention interventions efficacy has been questioned, a recent systematic review of school-based prevention studies concluded that there was little evidence of their effectiveness for reducing symptoms of anxiety and depression [71]. The conclusions of Caldwell et al., [71] have been challenged, with [4] identifying, among other limitations, that examining reduction, but not incidence, is not truly reflective of prevention. Building from this proposal, our study examined all intervention arms for reduction and prevention. In line with Fazel et al., [4], and subsequent work [72, 73], our study supports the role of school-based preventions, finding positive improvements in PEs.

This study utilized pre-existing data to examine preliminary utility of school-based interventions in PEs prevention. There is now a need to replicate and expand these findings using a specifically designed RCT. However, based on these results, a priority should be given to approaches incorporating a universal screening tool, and referral interventions.