This study evaluates the effects and reception of a pilot of the Universal Curriculum Against Radicalization in Europe (UCARE). UCARE is a curricular intervention that aims to prevent the processes of radicalization through fostering social and civic competences. Radicalization and terrorism affect societies on a global level. Between 1970 and 2017, 170,000 terrorist incidents were reported (Institute for Economics and Peace 2019). In 2018, 103 countries experienced at least one terrorist incident and a total of 15,952 people were killed in attacks. The economic impact of terrorism worldwide in 2018 was estimated to be 33 billion US dollars (Institute for Economics and Peace 2019). There is increasing concern regarding the phenomenon of radicalization, and the need for preventative measures is paramount (European Commission 2016; Fataliyeva 2016; United Nations Educational, Scientific and Cultural Organization 2015).

Radicalization has many definitions and may relate to any process by which individuals increasingly reject the status quo accepted by mainstream society. In our work, we refer specifically to the process of radicalization into violent extremism (Schmid 2013). Hence, the “radical” aspect is not determined by how remote an individual’s ideology is from the current status quo, but by acceptance of violence as a legitimate means of promoting one’s ideology (Sklad and Park 2017). Almost any ideology may serve as a justification for violence and, therefore, form a basis for radicalization. For instance, the ideologies inspiring terrorist acts committed in Europe between 2015 and 2019 included separatist and ethno-nationalist ideology (383 attacks), jihadismFootnote 1 (87), left-wing extremism (83), and right-wing extremism (16) according to Europol (2020).

Not only is there a large spectrum of ideologies that can attract different societal groups to justify violent means, but the characteristics that lead an individual to engage in violent extremism also remain undefined (McGilloway et al. 2015). Furthermore, many individuals who never engage in violence often share the characteristics of individuals who have committed terrorist acts (Bartlett and Miller 2012). Therefore, it is difficult to identify radicalizing individuals, and efforts to de-radicalize individuals are very challenging (Schmid 2013). Consequently, universal primary prevention that aims to prevent radicalization by fostering resilience against radicalization processes among youth in general, rather than targeting a particular ideology or group at risk, should be incorporated in the daily environment of young individuals (Fataliyeva 2016). By definition, primary prevention must occur before radicalization. Therefore, adolescence appears to be the most suitable period for these preventative programs (Aly and Striegher 2012), particularly because during this period, values and beliefs are shaped (Aly et al. 2014; Fataliyeva 2016; Feddes et al. 2015).

The majority of adolescents spend a substantial part of their time in school; thus, educational institutions provide a convenient opportunity to deploy social and emotional interventions (Davies 2009; Durlak et al. 2015; Shriver and Buffett 2015; Torney-Purta 2002; Torney-Purta et al. 2001). Furthermore, schools may be considered an ideal context in which these interventions may be delivered because these are—per definition—environments that provide ample opportunities to learn. The European Recommendation of 2006 (European Commission 2006) declared that social and civic skills were among key competences, and that most European school systems are obliged to prepare young individuals for participation in society through civic education, taught as either a stand-alone subject or a topic integrated within another subject (Eurydice 2012). Fostering soft skills such as civic competences may be crucial in creating resilience to radicalization (Sklad and Park 2017).

To facilitate the inclusion of primary prevention of radicalization in the citizenship education agenda of schools, the authors developed the curricular intervention Universal Curriculum Against Radicalization in Europe (UCARE). The intervention was developed using a modified version of the intervention mapping (IM) framework (Bartholomew et al. 2011), which was originally designed to develop systematic evidence-based interventions in the field of health promotion. To establish the goals of the UCARE intervention, the theoretical background on the processes of radicalization was examined by reviewing the literature (Sklad and Park 2017). The authors identified six groups of psychological mechanisms involved in radicalization based on a review of existing research (e.g., Schmid 2013; Young et al. 2015) and theories, including Wiktorowicz’s (2004) al-Muhajiroun model, the staircase to terrorism theory (Moghaddam 2005), and the 12 mechanisms of terrorism involvement theory (McCauley and Moskalenko 2012). The psychological mechanisms involved in radicalization were identified as follows: (a) a need to belong and be accepted in a group (Sageman 2008; Wiktorowicz 2004), (b) trying to establish a positive social identity and meaning (De Wolf and Doosje 2010; Kruglanski 2013; Meertens 2007; Moghaddam 2005; Van den Boset al., 2009; Wiktorowicz 2004), (c) conforming to group norms (Aly and Striegher 2012; Sageman 2008), (d) frustration caused by relative deprivation or a sense of discrimination (Doosje et al. 2012; Sageman 2008), (e) a belief that one is lacking legitimate or effective means (Moghaddam 2005), and (f) perceiving opposition between the in-group and the out-group—“Us vs. Them” (Moghaddam 2005; Stern 2004) and subsequently dehumanizing out-group members (McCauley and Moskalenko 2012).

In an effort to prevent radicalization, it is imperative to utilize strategies to intervene effectively with the previously outlined processes (Sklad and Park 2017). Therefore, the main purpose of the UCARE curriculum is to confront the six identified processes in a way that they no longer facilitate violent extremism. The intervention aims to achieve this goal by increasing the social and civic competences of students, while also providing alternatives to extremism to satisfy the psychological needs and address the social grievances of young individuals. Consequently, UCARE aims to halt the development of radicalization at its source. Additionally, it provides students with tools and skills to live a meaningful life and contribute to society (Sklad and Park 2017). Based on the review of current research, the specific goals of the curricular intervention are (1) to facilitate a sense of acceptance and belonging and to strengthen the capacity to form positive identities, without the necessity to rely on the offer of cult-like extremist groups; (2) to create awareness of, and resilience to, group pressure and detrimental group norms; (3) to reduce the vulnerability of individuals to hold “Us vs. Them” thinking and in-group biases, as well as a propensity to stereotypical perception, discrimination, and dehumanization of out-group members; and (4) to overcome the sense that legitimate democratic means of addressing social issues are ineffective, by empowering participants politically. Several theory-informed intervention methods fitting one or more of these goals were subsequently identified.

One approach identified to potentially create resilience to radicalization is mindfulness meditation (Sklad and Park 2017). Mindfulness has been shown to have a diverse range of psychological benefits, which fall in line with the goal of creating resilience to radicalization by addressing underlying psychological processes. For example, a meta-analysis (McConville et al. 2017) indicated that post-training, students participating in mindfulness practice reported higher levels of general self-efficacy, defined as an increased belief in their own abilities (Bandura 1997; Grossman et al. 2003). Research has indicated that increased self-efficacy may lead to improved attitudes to learning and problem-solving, educational achievements, increased self-determination, and increased volitional control of behavior and decision-making (Richardson et al. 2012; Schunk 1991). The results of a meta-analysis (Roeser and Pinela 2014) show that practicing mindfulness improves the construct accessibility of oneself and other individuals; thus, participants are more prone to exhibit prosocial behaviors and empathy (Leiberg et al. 2011; Shapiro et al. 1998).

Another promising set of approaches to prevent radicalization targets the group processes involved. Empirical studies have demonstrated that exercises aimed at building resilience towards in-group conformity pressures, indirectly as well as directly, improve the opinions and perceptions of out-groups (Gaertner et al. 1996; Lemmer and Wagner 2015; Sessa 1996). A meta-analysis demonstrated that curricular and co-curricular interventions to reduce racial bias are successful (Denson 2009). These interventions include discovering commonalities or shared identities with out-groups through de-categorization, re-categorization, learning and understanding multiple perspectives (including the out-groups’ points of view), intergroup contact, raising awareness of interconnectedness, learning and practicing empathy towards out-groups, learning how false stereotypes may be applied to individuals, and promoting respect and appreciation of differences (Sklad and Park 2017). A large meta-analysis indicated that interventions based on direct contact experiences, along with social-cognitive training programs designed to promote empathy and perspective taking, exhibited the strongest effects (Beelmann and Heinemann 2014). Thus, training in taking multiple perspectives may also be instrumental in reducing the propensity for the derogation of out-groups in adolescents.

One key method to reduce out-group hostility may be through building resilience to in-group pressure. Raising awareness of conformity pressure (Asch 1955; Harakeh and Vollebergh 2012) and in-group bias (Billig and Tajfel 1973; Maas and Acuri 1996) may represent the first step in acknowledging in-group pressures. Scholars propose that once students have learned about these phenomena, it is important to begin practicing emotional and social skills in resisting group pressure (Steinberg and Monahan 2007; Sumter et al. 2009). Practicing critical social thinking, problem-solving, assertiveness, and mindfulness may theoretically strengthen the ability of students to resist group conformity and foster prosocial behaviors (Kuiper 2002; Lipman 2003; Ten Dam and Volman 2004). Activities to reduce discrimination and prejudice have been identified as practicing empathy towards out-groups, discovering shared identities through de-categorization, learning about false stereotypes, promoting respect for differences between individuals, and creating more intergroup contact (Beelmann and Heinemann 2014; Dutton et al. 2010; Fett et al. 2014; Gaertner et al. 1996; Goldstein and Winner 2012; Lemmer and Wagner 2015; Robbins and Krueger 2005; Sessa 1996).

Furthermore, there are indications that interventions that address identity issues may be effective at preventing radicalization. According to Rydell et al. (2010), students may come to reduce their discrimination and prejudice against out-groups through the adoption and exploration of multiple identities different from their identity. Positive youth development programs may enable young individuals to identify their individual strengths, which subsequently allows them to improve their social and emotional skills and create a positive self-identity (American Camp Association 2007; Caulfield and Wilson 2012; Cohen 2005). By holding a positive self-identity and a favorable image of one’s own social group, individuals may be able to address discrimination in a non-aggressive and resilient manner (Rueben 2015; Zirkel 2002).

The elements identified of effective interventions have the potential to strengthen competences underlying resilience to radicalization and as such are incorporated into the UCARE intervention.

Intervention

The UCARE intervention aims to prevent radicalization by strengthening social and civic competences of adolescents, as well as improving their sense of belonging and citizenship. Interventions that specifically target high-risk individuals carry a threat of labeling and stigmatization (Jones et al. 2014; Offord et al. 1998). In addition, interventions targeting specific groups may miss unidentified at-risk individuals (Offord et al. 1998). Therefore, this curriculum is designed as primary prevention intended to be delivered to all youth indiscriminately. The curriculum is universal in two ways. Firstly, competences relevant to all young individuals, independent of whether they are at risk, were elected to be taught and attention was devoted to making the personal relevance of these competences apparent to students. Secondly, the curriculum is universal in the sense that it is intended to help all young individuals potentially at risk, regardless of the violent ideology they may feel attracted to.

Throughout interactive classes, the curriculum utilizes a democratic teaching style (Dewey 1916), which is particularly relevant for this curriculum as it involves teaching “the skills and virtues of citizenship” (Gutmann and Ben-Porath 2014, p. 2). The curriculum adopts an active learning approach, more specifically, a guided discovery approach (Bruner 1961; Mayer 2004), in which students are allowed to construct their own knowledge rather than memorizing what the trainer says. A meta-analysis (Alfieri et al. 2011) showed that when trainers provide assistance in the form of feedback, examples, explanations, and scaffolding, better learning outcomes are achieved compared to other instructional methods, such as explicit instruction and unassisted discovery. In addition, the curriculum uses problem-based learning, in which example problems are studied to provide a broader context, with the aim of increasing student understanding (Barrows 1985; Barrows and Kelson 1995). Meta-analytic evidence indicates that problem-based learning, compared to traditional learning approaches, is associated with increased long-term knowledge retention and examination scores, improved self-directed and collaborative learning, skill development and application, better learning attitudes, higher levels of student and staff satisfaction, and improvements in problem-solving skills (Ding et al. 2014; Strobel and van Barneveld 2009).

The curriculum is disseminated through a manual (Sklad and Park 2016), and was developed so that it may be executed by regular teachers or trainers, without prior knowledge of the processes that underlie radicalization or psychology in general. These trainers may include youth workers or members of (non-governmental) organizations working with the schools. In seven consecutive workshops, the competences related to processes involved in radicalization are targeted. All workshops are designed to be conducted with a group of students the size of a regular school class (20–32 students) and take approximately 45–60 min (regular class time). However, the workshops may easily be extended to 90 min. The workshops are suitable for 12‑21-year-old students, at all levels of education. While the exercises and core objectives of the curriculum should be adhered to for maximum efficacy, trainers may adapt the language and discussion topics to suit the age and level of the students. The manual provides guidance on the execution of the workshops, as well as how the material may be adapted to different groups in line with the curriculum’s overall aims and objectives.

Workshop 1 covers the topic “Identity, Our Roles” and is titled “Who are we?”. This workshop aims to increase class cohesion, as well as the perception of similarities between classmates by allowing students to identify their strengths and roles. The workshop targets the underlying radicalization process of the fulfillment of the need to be accepted and belong (Sageman 2008; Wiktorowicz 2004) and the search for a positive identity (De Wolf and Doosje 2010; Meertens 2007; Moghaddam 2005; Van den Bos et al. 2009; Wiktorowicz 2004).

Workshop 2, titled “You’re in Control,” covers the topics of mindfulness, self-awareness, and self-control, and aims to make students more resilient to group pressure via increasing emotional control through teaching self-regulation and self-reflection skills.

Workshop 3, “How to Resist Pressure,” aims to increase the awareness of group pressure and promote resilience to group pressure through assertiveness. Workshops 2 and 3 together target the radicalization process of conformity to group norms (Aly and Striegher 2012; Sageman 2008).

Workshop 4, “Standing in Someone Else’s Shoes,” covers perspective taking and conflict resolution skills, and aims to reduce “Us vs. Them” perceptions (Moghaddam 2005; Stern 2004) through teaching perspective-taking skills. Moreover, the workshop aims to tackle the radicalization processes originated by feelings of frustration due to relative deprivation, discrimination, and resentment towards other individuals (Doosje et al. 2012; Sageman 2008), experiencing opposition between groups, and “Us vs. Them” thinking and in-group bias (Moghaddam 2005; Stern 2004).

Workshop 5, titled “Thinking Things Through,” focuses on critical open-mindedness (Lambie 2014) and focuses on stimulating open-mindedness and critical thinking skills. More specifically, this workshop aims to tackle intergroup conflict and the “Us vs. Them” constructions (Moghaddam 2005; Stern 2004).

Workshop 6, “Supporting the Cause,” and workshop 7, “Time to Make a Change,” seek to build political self-efficacy. The workshops are designed to stimulate empowerment by allowing students to practice advocacy skills on a self-selected issue. The goal of the final two workshops is to demonstrate viable alternatives to violence, thereby targeting the radicalization process of believing that legitimate means are ineffective (Moghaddam 2005).

Methods

Participants

The pilot intervention was delivered to 191 Dutch students. Schools from all levels of Dutch secondary education were deliberately selected to ensure the generalizability of the findings. Students from seven classes, from five different schools in the southern part of the Netherlands, participated in the intervention. The majority of the participants were between 15 and 17 years old (see Table 1) and participated in bilingual educationFootnote 2 (Dutch/English). The percentage of females varied between 44.8 and 60.6% between the classes. The percentage of students with a migration background varied between 4.0 and 21.4%Footnote 3 between the classes (Table 1).

Table 1 Demographic description of the participating classes
Full size table

The ten teachers, who were the respective students’ permanent teacher or subject teachers within which the workshops were embedded, observed and evaluated the workshops. However, only three of the class teachers evaluated all seven workshops taught in their classes, and one teacher evaluated all seven workshops of two different classes in his/her school (14 workshops in total). The workshops in one class were evaluated by four different teachers, and the workshops in another class were evaluated by two different teachers.

Procedure

The curriculum was piloted by seven pairs of trainers in seven Dutch high school classes. Each pair delivered the seven workshops to the same class of high school students. The workshops were intended to be delivered weekly and, when possible, on consecutive weeks, as part of the tutoring hour or civic education class. However, as a result of holidays and school events, the delivery period varied between 7 and 9 weeks. Due to organizational issues in several cases, some workshops were shorter than the full-allocated 45 min and in one class, the last two workshops were combined into one longer class. Prior to delivering the workshops, the trainers (second- or third-year university college students) received approximately 4 h of general instruction on guiding group discussions and educational methods. The trainers obtained the curriculum manual and were instructed to adapt the implementation of the exercises as described in the manual to their assigned class level and age of the students. Before the lessons were taught in the schools, the trainers piloted and discussed each workshop among themselves under the supervision of the manual authors. In the final version of the manual, minor adjustments were implemented according to the feedback obtained during these pilot sessions.

Before starting the intervention, the trainers visited their assigned class to become acquainted with the students and to talk to the teacher responsible for the class. During this time, the participating students completed a pre-test questionnaire (see the “Measures” section for more details), which contained measures of the intended direct outcomes of the intervention.

Regular resident class teachers were present during the workshops; however, they only observed and evaluated the workshops and did not actively participate. At the end of the intervention, the participating students completed the evaluation questionnaire and a post-test questionnaire (identical to the pre-test questionnaire) to measure intended direct outcomes of the intervention. Pre- and post-test questionnaires were also collected in the following semesters in a comparison group, consisting of students of analogous classes in the same or similar schools who did not receive the intervention. Instead, the control group received their usual classes on various topics, including citizenship, as part of the Dutch secondary school curriculum.

Measures

Data regarding three aspects of the intervention were collected: information on the implementation fidelity, information on the reception of the curriculum by the students and teachers, and information on the effects of the curriculum.

To assess the implementation fidelity, the following measures were collected and analyzed: (a) attendance and attrition rates, (b) the fidelity of execution as measured by the proportion of prescribed activities conducted, and (c) the proportion of prescribed activities conducted completely in line with the manual. All workshops were recorded, transcribed, and input into the open qualitative analysis software AQUAD (Huber 1989). Based on the curriculum goals and objectives, lists of keywords were developed inductively for each workshop. Researchers who were familiar with the curriculum, but not responsible for the intervention, independently coded the transcriptions. First, the transcripts were coded for the presence of activities described in the manual. Then, within every transcribed activity, the questions and objectives outlined in the manual were given corresponding codes to determine whether they were adequately included in each workshop. Using qualitative content analysis, the researchers independently determined whether they believed that all of the manual objectives had been met or not, before checking for convergence.

The second aspect, the reception of the curriculum, was independently assessed by the resident school teachers who observed the workshops and the participating students, using two respective evaluation questionnaires. The teacher of the class evaluated the workshops using a short questionnaire after each workshop he/she observed. This questionnaire included statements, with a 5-point Likert scale, that covered the evaluation of the suitability of the workshop level for the participants, the level of trainer preparation, the level of workshop execution, and the capacity to motivate students and stimulate prosocial attitudes.

The participating students also evaluated the workshops using a questionnaire that was completed within 1 week after the final workshop. The questionnaire contained two types of evaluations. Firstly, the students expressed agreement or disagreement with eleven positive statements regarding the workshops. Secondly, the students provided a general evaluation of the curricular intervention by grading it on a scale ranging from one to ten (this grading range corresponds to the grade scale used in Dutch schools).

The third and final aspect, the curriculum’s effect, was assessed in multiple ways. The first indicator of the potential impact was the self-perceived effectiveness, which was assessed via two measures. Firstly, an adaptation of the most significant change technique (Davies and Dart 2005) was used, which instructed the participants to answer an open question regarding the most important aspects the students felt they had learned. The answers of the participants were first coded deductively, derived from the workshop objectives, and then inductively using open-coding for the remaining answers. Subsequently, categories were formed which covered all codes in a process of axial coding. All answers were then categorized by the third, fourth, and fifth authors. Each author’s categorizations were checked for convergence and the few divergent categorizations were discussed until a consensus was reached.

The participants were also explicitly asked whether they thought that they improved on each of the competences addressed in the workshops as a result of their participation. The evaluation form included eight statements, to which the respondents could agree or disagree, indicating whether they felt the workshops had helped them to improve each competency. The eight statements covered improvement in understanding of one’s roles and identities, self-control, resilience against peer pressure, ability to assume different perspectives, metacognitive awareness, the ability to judge the validity of information provided by (social) media, political efficacy, and knowledge of classmates.

Finally, the potential effectiveness of the curriculum was assessed by comparing the pre-test and post-test scores on measures of the intended outcomes. Changes in the intervention group were compared to changes in the comparison group to increase internal validity. The outcomes were measured using a set of scales developed for this purpose and included self-assessed mindfulness (9 items), conflict resolution (4 items), assertiveness (6 items), (multiple) identity awareness (6 items), political self-efficacy (6 items), critical open-mindedness and perspective taking (9 items), and awareness of group pressure (1 item). All scales had satisfactory internal consistency, with a Cronbach’s alpha ≥ .7 (Table 8). Awareness of group pressure comprised a one-item measure and as such, the internal consistency could not be assessed.

Analysis

The results were initially analyzed by describing summary statistics of the fidelity indicators, evaluations, and self-perceived effects. The pre-test and post-test scores on indicators of the intended outcomes for both the intervention participants and the comparison group were compared to indicate the effects of the intervention. Finally, the variability of the results across the schools/classes/trainers and outcomes was assessed. As the data demonstrated a substantial dependence of cases within classes, rather than traditional tests, a multilevel model was fitted via means of a linear mixed model implemented in the SPSS version 21 software. Hypotheses were tested at a significance level of α = .05.

Results

Fidelity

Of the 49 planned workshops, 48 workshops were conducted. In one school, the last workshop was canceled because of an illness of a school staff member. In another school, the last two workshops were delivered to 46.88% of the students because half of the class switched to a schedule that coincided with the intervention. The general attendance rate, excluding these three workshops, was 90.87% (S = 5.57% across the workshops and schools). Including the three workshops affected by organizational issues, the total attendance rate was 87.40% (S = 16.16% across the workshops and schools).

With respect to the delivery, the trainers delivered on average 91.93% (S = 4.31% across the classes/trainers) of the 55 mandatory activities prescribed by the curriculum manual and 89.02% (S = 4.65%) of all activities, including optional activities (57; Table 2). Activities not being delivered were predominately a result of the trainers running out of time and skipping the wrap-up activity at the end of the workshop. On average, 68.37% (S = 18.38% across the classes/trainers) of the wrap-up activities occurred as planned. The discussion activities following demonstrations and interactions were conducted in 87.00% of the cases (S = 7.09% across the classes/trainers).

Table 2 Fidelity: proportion of activities implemented
Full size table

Regarding the internal fidelity of the activities, 72.82% of the conducted activities were coded as having been executed in complete compliance with the manual (Table 3). The fidelity varied between activity types; for example, 85.71% of the conducted closing activities reached complete compliance compared with 38.70% of the discussion activities. Discussions coded as not in full compliance often omitted or modified one or more questions from the manual.

Table 3 Fidelity of execution of implemented activities
Full size table

Evaluation by Observing Teachers

The results of the evaluation of the intervention by the observing teachers demonstrated an overall positive reception. The mean scores of all evaluated aspects of the intervention ranged between four and five, which corresponds to the responses well done and exceptionally well done, respectively. Taking all seven workshops together, the highest evaluation was awarded to the level of preparation of the trainers and workshops, followed by the suitability of the material level for the recipients, the quality of execution, and the workshops’ capacity to motivate students and stimulate prosocial attitudes (Table 4).

Table 4 Evaluation of the workshops by observing teachers
Full size table

When the results of each separate workshop were considered, the vast majority of the scores received by the workshops ranged between four and five, which corresponded to the answers well done and exceptionally well done, respectively. Four exceptions (out of 35 evaluations) concerned the fifth workshop’s (critical open-mindedness) execution and capacity to motivate student, and the third and seventh workshops’ (which targeted resilience to group pressure and political self-efficacy, respectively) capacity to stimulate prosocial attitudes. Nevertheless, it is important to note that these aspects scored above satisfactory.

A linear mixed model repeated measures analysis tested the statistical significance of the differences between the workshops. A compound symmetry variance structure was selected because of its best model fit (AIC = 362.07). The results indicated significant differences between the evaluations of the different aspects of the workshops (F(4, 222.12) = 8.81, p < .001), no significant difference between the workshops in the overall evaluation (F(6, 222.07) = 0.95, p = .464), and no significant interaction between the workshop topic and aspect evaluated (F(24, 222.01) = 0.61, p = .927). Post hoc pairwise comparisons of the aspects indicated that the preparation level was evaluated significantly higher than every other aspect (p < .001) and no other difference between the five evaluated aspects was statistically significant.

Evaluation by Participating Youth

In terms of the general evaluation, the students awarded the workshops with an average grade of 7.10 (S = 1.43) on a scale of 1 to 10, which corresponds to the Dutch grade amply sufficient or a B in the American grade system. There was a small, statistically insignificant, variation across the trainers/classes for the average evaluation Z = 1.17, p = .241. The intra-class correlation coefficient was ICC = .10 (Table 5).

Table 5 General workshop evaluation by participants, per class
Full size table

Concerning the evaluation of specific aspects of the program, the vast majority of the participating youth positively evaluated the workshops and agreed with 11 positive statements regarding the workshops (Table 6). Nearly all participants agreed that the workshops were well prepared and the trainers were helpful. More than 80% indicated that the workshops were interesting, clear, understandable, and had an appropriate level and tempo. More than three-quarters agreed that they learned new things, goals were reached, and the workshops met the expectations. More than 70% stated that they found the material personally relevant.

Table 6 Recipient evaluations of workshops
Full size table

The evaluations of the workshops across the classes/trainers were subsequently compared (Table 7). To account for a potential dependence of the observations because of students originating from the same classes, a linear mixed model analysis was used to test the statistical significance of the differences in the evaluations between the classes and among the evaluated aspects. A compound symmetry variance structure was selected because of its best model fit (AIC = 7269.43). There was no significant main effect of class (F(6, 180.91) = 1.23, p = .29), but there were significant differences across the evaluated aspects, (F(10, 1802.10) = 49.91, p < .001), and a statistically significant interaction of the evaluated aspects and class (F(10, 1802.11) = 3.97, p < .001), which indicates that the pattern of evaluations differed across the classes. To examine the differences, a cluster analysis was performed on the aggregated evaluation scores of each class. The classes were grouped into three clusters based on their evaluation of all aspects. The first cluster contained the two classes with the youngest students of bilingual preparatory education for academia (VWO) and the older group of professional lower level education (VMBO). The second cluster also contained three classes: All classes had students of nominal age 15–16 years old and of both higher levels of education: VWO level bilingual education and professional education (HAVO). The final cluster contained one class of higher-level bilingual high school education (VWO) with relatively older students (age 16–17). This class was also the class that did not receive the final workshop as a result of organizational issues at the school.

Table 7 Recipient evaluations of workshop aspects. Proportion of students who agreed with the statement in percentage
Full size table

The clusters significantly differed on the following three items: evaluation of the comprehensibility of the material (F(2,4) = 23.27, p = .006), appropriateness of the level of the workshops (F(2,4) = 8.89, p = .034), and whether the learning goals were reached (F(2,4) = 18.19, p = .010). The F tests should be interpreted only for descriptive purposes as the clusters were selected to maximize the differences among cases in different clusters. The observed significance levels are not corrected for this factor and thus cannot be interpreted as tests of the hypothesis that the cluster means are equal (SPSS, 2016). The analysis of the cluster centroids (refer to profiles in Fig. 1) indicates that cluster 1 (younger students, lower education level) and cluster 3 (incomplete intervention) differed from cluster 2 (median age, higher education level). Classes in cluster 1 (younger students, lower education level) evaluated the appropriateness of the level and the comprehensibility of the workshops relatively lower than cluster 2. Finally, cluster 3 (incomplete intervention) evaluated reaching the learning goals relatively lower than classes in cluster 2.

Fig. 1
figure 1

Student evaluations, class cluster comparisons

Full size image

Indicators of Effectiveness of the Intervention

The effects of the intervention were assessed by means of mixed-design ANOVA, comparing the pre-test to the post-test scores within the participants, as well as assessing between-group differences for the comparison and intervention group on the measures of the intended outcomes (self-assessed mindfulness, conflict resolution, assertiveness, (multiple) identity awareness, political self-efficacy, critical open-mindedness and perspective taking). A multilevel analysis was considered to account for clustering of the sample. However, an analysis of the intra-class correlations indicated that for all but one indicator (awareness of group pressure, ICC = .07), the intra-class correlations were negligible (ICC < .01). Therefore, a multilevel model was deemed unnecessary.

The results indicated that during the intervention, the participants significantly improved on nearly all measures (mindfulness: t(171) = 1.66, p < .05, d = 0.12, conflict resolution: t(171) = 1.64, p < .05, d = 0.13, political self-efficacy: t(169) = 4.18, p < .001, d = 0.32, critical open-mindedness and perspective taking: t(171) = 2.26, p < .01, d = 0.16, awareness of group pressure: t(171) = 3.25, p < .001, d = 0.26), with the exception of the marginally significant, identity: t(170) = 1.49, p = .07, d = 0.11 and self-assessment of assertiveness, t(171) = 0.33, p = .37, d = 0.03.

The intervention group increased significantly more than the comparison group on measures of conflict resolution, critical open-mindedness and perspective taking, political self-efficacy, and awareness of group pressure, bearing substantial effect sizes for all four outcomes (see Table 8). The effects of the intervention were strongest for improvements in political self-efficacy and critical open-mindedness and perspective-taking outcomes.

Table 8 Intervention vs. comparison group results on the intended outcome measures
Full size table

In addition to the more objective comparison presented above, the participants’ subjective perceptions of the effects of the intervention were assessed. Firstly, the most significant change analysis indicated that the participants perceived perspective taking and mindfulness competences as the most prominent areas in which the intervention had had an impact on them, followed by political self-efficacy, assertiveness, and awareness of automatic thinking processes. The complete list of the most significant outcomes spontaneously noted by the participants contained six more categories, listed in Table 9.

Table 9 Most important/interesting thing learned as spontaneously noted by participants
Full size table

The analyses of the eight closed questions pertaining to self-perceived effects of the intervention showed that the participants perceived that the intervention improved their capacity to consider issues from more than one perspective (81.77%), followed by understanding one’s roles and identities and being able to critically examine one’s views (both indicated by more than 70% of the respondents). The recipients were least convinced that they knew their classmates better as a result of the intervention, with approximately only half of the students agreeing with this statement.

The mixed linear model analysis indicated a significant main fixed effect of the type of the outcome (F(7, 41.19) = 9.78, p < .001), and a significant random interaction effect between trainer/class and subject of the question (Z = 2.06, p < .05). There were substantial differences between the trainers/classes in terms of the effects on particular outcomes of the training as perceived by the participants, ranging from 5.26 to 92.31% of the participants believing that they improved. However, a cluster analysis indicated that the differences were predominately located between class 3, who missed the last workshop, and the other classes. When class 3 was omitted, there were no significant random effects of class or the perceived effect among the other six classes, Z = 1.48, p = .14 (Tables 10 and 11).

Table 10 Self-perceived effects
Full size table
Table 11 Self-perceived effects by class/trainer
Full size table

Conclusion

The purpose of the piloted intervention was to address the six previously identified psychological needs and factors that contribute to radicalization (Sklad and Park 2017): (a) frustration for the need to be accepted and belong to a group (Sageman 2008; Wiktorowicz 2004), (b) a need for a positive social identity and meaning (De Wolf and Doosje 2010; Kruglanski 2013; Meertens 2007; Moghaddam 2005; Van den Bos et al. 2009; Wiktorowicz 2004), (c) conforming to group norms (Aly and Striegher 2012; Sageman 2008), (d) frustration caused by relative deprivation or a sense of discrimination (Doosje et al. 2012; Sageman 2008), (e) a sense that legitimate means are ineffective (Moghaddam 2005), and (f) perceiving opposition between the in- and out-groups (Moghaddam 2005; Stern 2004) followed by dehumanizing out-group members (McCauley and Moskalenko 2012). The piloted intervention consists of seven workshops, which each foster competences that address the aforementioned factors contributing to radicalization.

The analysis of the piloted intervention results indicates that the Universal Curriculum Against Radicalization in Europe (UCARE) has the potential to foster specific skills and competences, which may have preventative effects on the processes that contribute to radicalization. The participants significantly improved on the majority of the intended outcomes, including mindfulness, positive identity (on the trend level), conflict resolution, political self-efficacy, critical open-mindedness and perspective taking, and awareness of group pressure. Moreover, the evaluations demonstrate that the workshops were very well received by observing teachers and favorably by participating students. These findings confirm that schools may effectively engage in fostering civic and social competences with the intention to prevent processes that contribute to radicalization.

Despite the positive results and reception, several limitations must be considered before the intervention may be proclaimed effective in preventing radicalization. As previously discussed, although radicalization may have a substantial impact, prevalence of violent behavioral expressions of radicalization is relatively low and they may occur years after the processes contributing to radicalization have started. Therefore, large-scale longitudinal studies following participants after the intervention are required to ascertain the effects of primary interventions in the domain of radicalization. In a relatively short-term and small-scale study, such as the research presented here, only the immediate direct effects of the curriculum could be assessed. The connection between the prosocial competences stimulated and radicalization is only assumed based on the theory and existing empirical data.

The general reception of the curriculum was positive, but there was considerable variability across the participating classes. The number of participating classes was limited, and each interventional team only worked with one class. Therefore, it is unclear to what degree the variability was determined by the trainers, recipients, and context, and to what degree by the implementation fidelity. All materials were developed to be as context-independent as possible, to make the curriculum applicable in different countries, but most participants of the pilot were residents in the South West of the Netherlands. In addition, a substantial proportion of the sample was enrolled in a more demanding mode of secondary education and only one of the seven classes originated from the most basic level of secondary education. Therefore, the curriculum may present different levels of success among different populations and across different contexts. To reach a more conclusive insight regarding the determinants of the effectiveness and reception of the intervention, a larger study including different populations, classes, and trainers in every level of education is necessary. However, the results show that the implementation of the curriculum had been successful even without it being delivered by the authors or trained teachers, which suggests that the curriculum may be adequately implemented by individuals other than teachers. Consequently, educational institutes, as well as other relevant programs, could easily use the curriculum with adolescents.

Finally, for methodological reasons, the curriculum was implemented as a stand-alone module separate from the remaining school activities. Nevertheless, it may be expected that its effects may be stronger if embedded in a whole school approach (Watkins 1994; Wyn et al. 2000), thereby creating a climate and culture that supports the development of citizenship, social, and emotional skills. After the competences are introduced by the curriculum, students should be encouraged to practice, use, and further develop them within a framework of regular school activities.