Introduction

A lack of social skills has been associated with various behavioral and developmental problems in children and adolescents, including delinquency (van der Laan et al. 2009). Specifically, social skill deficits have been related to a higher risk for both offending and criminal offense recidivism (Loeber et al. 2008; van der Put et al. 2012). Social skills training (SST) for juvenile delinquents and juveniles at risk for offending aims to improve social skills as a means to reduce the risk for (re)offending. However, the effectiveness of many of these programs is yet to be shown. The present study aimed to examine the effectiveness of an SST that is imposed by judges as a sentence for juvenile delinquents.

SST is considered one of the widely recognized generic program types (Lipsey et al. 2010). Although SSTs come in a variety of types, they generally address social skills through addressing social interaction, pro-social behavior, and social cognitive skills (Gresham 2002; Gresham et al. 2004; Merrell and Gimpel 1998). SST treatment techniques are based on different theories: social learning theory (Bandura 1977), operant learning theory (Skinner 1953), social information processing theory (Ladd and Mize 1983), structured learning theory (Goldstein 1981), and multiple cognitive approaches (Cook et al. 2008; Kazdin et al. 1992), which are incorporated both independently and jointly in training (Maag 2006). Based on these theories, treatment techniques such as modeling, positive reinforcement, coaching, and role-playing are frequently used. SST has been implemented in different shapes and forms and with different target populations, resulting in a number of meta- and even mega-analyses on the subject.

Meta-analytic studies have found small to moderate effects of SST on secondary-aged students with emotional and behavioral disorders (Cook et al. 2008; Maag 2006) and antisocial youths (Ang and Hughes 2002). Notably, the majority of existing studies compared SST treatment to a waiting list control group or a placebo control group receiving minimal or no treatment. The added value of SST over other treatments (or treatments as usual, TAU) has yet to be established. In juvenile offender treatment effectiveness meta-analyses, SST has been included under different treatment types, but is generally found to have positive treatment effects (Lipsey 2009; Lipsey et al. 2010). Arguably, the distinction between SST and (cognitive) behavioral programs is arbitrary: meta-analyses have included SST as a type of cognitive-behavioral therapy (e.g., Lipsey et al. 2007), while others have made a distinction between (cognitive and/or behavioral) approaches within SST (e.g., Cook et al. 2008).

The present research base has led to multiple recommendations to improve SST effectiveness. First, treatment implementation should be monitored and improved in order to be able to attribute treatment effects to SST and specific SST techniques (Cook et al. 2008; Gresham et al. 2004; Lipsey 2009; Maag 2006). Second, SST should be implemented with adolescent “late starters” (Moffitt 1993), because their problems are not too severe to be treatable and they have developed the (cognitive) abilities to profit from the training (Cook et al. 2008). In addition, SST has been found most effective for older, less aggressive juvenile offenders, who were treated on diversion (not probation or incarceration; Lipsey 2009). Third, follow-up treatment effects are generally found to be small(er), indicating a lack of generalizability of the targeted skills to outside the training. This generalizability needs to be improved to enhance long-term SST treatment effects (Cook et al. 2008; Maag 2006). Finally, including the peer group is thought to enhance treatment effects (Maag 2006), especially because the treatment effects for deviant-only SST groups have been found to be smaller (Ang and Hughes 2002).

The present study compared Tools4U, an outpatient individual SST applied as a (penal) sanction for juvenile delinquents (Albrecht and Spanjaard 2011), with an alternative treatment (TAU) for juvenile delinquents. While most studies do not pay enough attention to treatment integrity (Perepletchikova et al. 2007), treatment integrity was measured to assure that possible treatment effects could be attributed to the intervention. The training is intended for delinquent juveniles (aged 12 to 18 years) for whom the lack of cognitive and social skills is related to delinquent behavior. With 8 (to 12) weekly training meetings of 1.5 h, the intervention is relatively short and “light”. The training is specifically intended for adolescent onset delinquents with moderate delinquency trajectory severity (see Loeber et al. 1998).

Tools4U generally uses cognitive-behavioral techniques to train juveniles to better deal with at-risk situations (e.g., offending, heightened emotions, peer pressure, and conflicts) and to develop pro-social behavioral alternatives. The intervention targets four skill deficits related to delinquency. Impulsivity is targeted because it is a risk factor for delinquency (Cruise et al. 2008; Veltri et al. 2014), which can moderate the effects of other risk factors (Lösel and Farrington 2012) by preventing the juvenile from overseeing the consequences of his or her delinquent behavior (Gottfredson and Hirschi 1990). Furthermore, social problem-solving skills are targeted to replace antisocial (delinquent) ways of coping with life stressors (Agnew 1992; Brezina 1996). Therefore, pro-social coping skills (such as confrontation and seeking social support) are taught to replace delinquency-related avoidant coping (Aebi et al. 2014). Moreover, Tools4U aims to improve social perspective-taking skills, because delinquent juveniles inadequately recognize and interpret the intentions and motivations of others (Dodge et al. 1990) through attributing more hostile intent to others (Dodge et al. 1990; Nas et al. 2005) and showing a lack of cognitive empathy (Sierksma et al. 2014; van Langen et al. 2014). Finally, the intervention targets critical reasoning to aid juveniles in the process of pro-social decision-making (Robinson and Porporino 2004) by challenging rigid and dichotomous thinking, including cognitive distortions (Helmond et al. 2015; Paternoster et al. 2011).

In addition to reducing risk factors for delinquency, the intervention aims at improving skills regarding the juvenile’s developmental tasks to increase protective factors against future delinquency. These include making choices and planning, having non-delinquent friends/partner, having meaningful daytime and appropriate leisure activities, and having a good relationship with parents/family (Lösel and Farrington 2012; Stouthamer-Loeber et al. 2002).

To improve the generalizability of Tools4U (i.e., to ensure that juveniles are able to apply the trained skills outside the training), two measures have been taken. First, parents are involved in the Tools4U training: parents are included for those juveniles where parental supervision and problem-solving abilities are lacking. Furthermore, in the regular (individual) version, parents are at least expected to attend the first and final meetings, and the trainer and juvenile will keep parents updated on the training process. As positive parental involvement is associated with abstinence of or desistance from offending (Hoeve et al. 2009; Lösel and Farrington 2012), and treatment effects can be greater with additional parenting components (Dowell and Ogles 2010; Webster-Stratton et al. 2001), parental involvement in the training is believed to enhance the (long-term) outcome effects. Second, positive feedback, behavioral exercises, homework assignments, and positive (parental) reinforcement are included in the training. These techniques are believed to increase the juvenile’s self-esteem, so the juvenile has enough confidence to apply the trained tools and techniques in daily life (Albrecht and Spanjaard 2011), thus enhancing generalizability.

Although Tools4U should be seen as a potentially effective intervention because it targets delinquency-related social skills individually and provides an optional parental component, the effectiveness of this specific training has yet to be shown. Therefore, the first aim of the present study was to examine the effectiveness of SST Tools4U by assessing impulsivity, coping skills, hostile intent attribution, lack of cognitive empathy, and cognitive distortions. Furthermore, because the developmental task-related target areas are treated secondarily and differently for every juvenile, we hypothesized that this would lead to more general improvement in social acceptance, behavioral adjustment, and self-worth. These factors are particularly relevant because they have been shown to be related to delinquency through their influence on other life outcomes (Trzesniewski et al. 2006; Wild et al. 2004). Finally, the effects of involving parents were examined through measures of positive parenting and parental rewards.

Despite the large number of studies conducted to examine the effectiveness of SST, it is still unknown for whom and under what circumstances treatments for juvenile delinquents are likely to be most effective (see e.g., Kazdin 2007, 2008; Kraemer et al. 2002). Therefore, moderator effects were tested to examine differential effects of Tools4U for different subgroups: age, gender, ethnicity, and risk of reoffending.

First, age was examined as a possible moderator of Tools4U effectiveness. As the nature and influence of risk factors change during adolescence (van der Put et al. 2012), specifically regarding family factors, the effects of (parental involvement in) Tools4U were expected to be larger for younger juveniles than for older juveniles. Second, gender was examined as a moderator. In line with the delinquency trajectory model by Loeber and colleagues (1998), the moderately severe delinquent population of Tools4U consists of a relatively large proportion of girls (i.e., 29 % in the current sample). The current study, therefore, provided an excellent opportunity to investigate gender differences in SST treatment effects, given the limited knowledge about gender differences in risk factors for delinquency and juvenile justice program effects (Steketee et al. 2013; Zahn et al. 2009). Based on a previous review (Zahn et al. 2009) suggesting that effective non-gender-specific interventions work for both boys and girls, we expected no sex difference in the treatment of Tools4U. Third, although international studies have found no differences in intervention effects between indigenous and ethnic minority youths (Huey and Polo 2008), offending behavior and risk factors for delinquency have been found to vary by ethnicity (van der Put et al. 2013). Therefore, we examined whether treatment effects of Tools4U differed between juveniles from various ethnic backgrounds. In line with previous research (Huey and Polo 2008), we expected no differences in treatment effects for indigenous versus non-indigenous juveniles. Finally, we examined whether treatment effects differed between groups with high and low risks of reoffending. According to the risk–need–responsivity (RNR) principles, treatment is most effective when it is tailored to the reoffending risk of the offender (Andrews and Bonta 2010; Andrews and Dowden 2007; Andrews et al. 1990). Therefore, we expected Tools4U, which is a relatively short and “light” intervention, to be most effective for juveniles with a relatively low risk of reoffending.

Methods

Participants

The majority were boys (n = 159, 71 %) and almost a third consisted of girls (n = 64 girls, 29 %), with an average age of M = 15.71 (standard deviation, SD = 1.53) years (Table 1). Half of the juveniles had a Dutch ethnicity (n = 111, 50 %). Of the ethnic minority group, most had a non-Western background (total: 87 %; Turkey: n = 10, 9 %; Dutch Antilles: n = 17, 15 %; Morocco: n = 22, 20 %; Surinam: n = 22, 20 %; other: n = 24, 21 %). The majority lived with one or both biological parents. Over half received their sentence for a property offense (n = 66, 30 %) or person offense (n = 63, 28 %), with an average of M = 31.74 (SD = 25.82) sentenced hours. Half of the juveniles (n = 111, 50 %) was at low risk for reoffending.

Table 1 Characteristics of the Tools4U and treatment as usual (TAU) groups after propensity score matching
Full size table

Setting and inclusion criteria

The present sample consisted of delinquent juveniles who received a behavioral training sentence or community service order by disposal by the district attorney or by the juvenile court. In the Netherlands, these are imposed independently, without an accompanying probation order (although some juveniles receive an additional specific probation, both with and without probation guidance). The treatment group consisted of juveniles who attended a Tools4U training program in the Netherlands starting between May and August 2012. The control group was recruited among juveniles with a community service order or another behavioral training order similar to Tools4U in duration and intensity (treatment as usual, TAU), starting between June 2013 and February 2014. Post-treatment assessment continued until January 2013 for the treatment group and until June 2014 for the comparison group. Additionally, case file analysis was conducted for both the Tools4U and control group juveniles. For the current effectiveness analyses, n = 108 control group juveniles (of a total N = 354) were matched to the Tools4U juveniles (N = 115) by means of a propensity score.

Questionnaires about social skills and treatment integrity were administered to juveniles, parents, and trainers at the beginning and end of the training. Immediately after the first meeting, a research assistant explained research procedures to the juveniles and their parents to obtain their informed consent, and to collect the first (pre-test, T1) questionnaires for the juvenile, parent, and trainer or community service coordination officer. After the final meeting, a research assistant obtained the second (post-test, T2) questionnaire for the juvenile, parent, and trainer or community service coordination officer. Routine Outcome Monitoring questionnaires administered by trainers at the start and end of Tools4U were collected for the treatment group and administered to the control group juveniles. If face-to-face questionnaire administration was not possible, questionnaires were administered by phone. Face-to-face assessment was conducted with 94 % of participants at pre-test (Tools4U n = 115, 100 %; TAU n = 94, 87 %) and with 65 % of participants at post-test (Tools4U n = 101, 94 %; TAU n = 17, 23 %). Analysis of covariance (ANCOVA) with treatment condition as a covariate showed no significant differences in any of the outcome measures (pre- and post-test) between face-to-face and telephonic assessment, either at pre- or post-test. Juveniles received a 15 Euro gift certificate for completing two assessments, while the parent received a 7.50 Euro gift certificate for the assessments.

Matching procedure

A comparison group was abstracted from the N = 354 control group juveniles by means of propensity score matching. Following this procedure, the probability of (treatment) group membership is calculated based on a set of relevant participant characteristics (Beal and Kupzyk 2014). Juveniles in the treatment group were then matched to juveniles in the control group with a similar propensity to receive Tools4U. To ensure that every juvenile could be included in the matching procedure, missing values at pre-test were imputed using the expectation maximization algorithm (Graham 2009). The propensity score was calculated for every juvenile, based on gender, age, ethnicity, and all (non-overlapping) pre-test outcome scales. Self-perception and parenting were not included in the propensity score calculation, because these measures were only available for a limited number of Tools4U juveniles (self-perception: pre-test n = 88, 77 %; post-test n = 77, 67 %; parenting: pre-test n = 69, 60 %; post-test n = 53, 46 %). Using a conventional caliper of 0.025 (i.e., the difference in propensity score between Tools4U and matched TAU juveniles could be 0.025 at most; Beal and Kupzyk 2014), 108 control group juveniles could be matched to the 115 Tools4U juveniles. After matching, the missing values at post-test were imputed as well using the same approach.

After the matching procedure, differences between the Tools4U and control groups were found in degree of urbanization (χ2 = 20.52, p = 0.000), which was coded as more or less urban, based on the degree of urbanization of the juveniles’ residential town. The Netherlands is nationwide divided into several judicial districts, including juvenile justice regions, which were unevenly divided between the treatment and control groups (χ2 = 29.48, p = 0.001). These differences are most likely the result of the less standardized referral procedures for Tools4U: the number of Tools4U referrals per region did not correspond to the number that would be expected based on the total number of juvenile criminal investigations (per region). Furthermore, there was a difference in the number of weeks between pre- and post-test (Tools4U M = 11.20, SD = 4.80; TAU M = 13.80, SD = 8.30; t = −2.53, p = 0.012). This can be attributed to greater differences in duration for community service orders than for Tools4U. Finally, there was a difference in self-perception of behavioral adjustment, with control group juveniles being more positive than Tools4U juveniles (t = −1.99, p = 0.048). There were no differences in any of the other characteristics and measures. All analyses were controlled for pre-test differences between the Tools4U and comparison groups.

Treatment conditions

SST Tools4U

Juveniles in the treatment group received SST Tools4U, an outpatient behavioral intervention consisting of individual training in cognitive and social skills. The intervention uses techniques based on operant, cognitive, and social learning theories, the self-management model, and the social interaction model (Albrecht and Spanjaard 2011). Tools4U has a number of compulsory techniques that are used in every training course, which can be supplemented with optional techniques, depending on the juveniles’ needs. The intervention can be imposed by disposal by the district attorney or juvenile court independently, or with an additional community service order. Execution of Tools4U is supervised by a community service coordination officer with the Child Protection Council.

Tools4U training meetings occur weekly for 1.5 h per meeting. There are different Tools4U versions: the regular (individual) version (eight meetings), an extended version (12 meetings), and the plus version, which trains parents in parental monitoring and problem-solving (extra: two parent-only meetings and two combined parent–juvenile meetings). In the current study, a quarter (n = 29, 25 %) received the plus version and slightly more juveniles (n = 30, 26 %) received the long Tools4U version. For all versions, parents were expected to attend at least the first and final meetings of Tools4U, and were informed by the juvenile and trainer on the training process.

Tools4U trainers are (clinical) social workers, specifically trained in Tools4U techniques. They have to follow at least three trainings per year, and attend regular supervision and training meetings. Furthermore, video recordings of three trainings are assessed for license extension every year. Elaborate training and management manuals are available from the developer (Albrecht and Spanjaard 2011).

Inclusion criteria and treatment integrity were investigated in the current study by case file analyses and administering questionnaires about treatment techniques to trainers and juveniles. This showed that Tools4U juveniles mostly met the inclusion criteria for the intervention, although 17 % of Tools4U juveniles were referred for truancy instead of for having committed an offense. Furthermore, Tools4U “Top 10” intervention techniques (most important, see e.g., Spanjaard et al. 2012) and the compulsory intervention components were applied sufficiently (i.e., in 80–100 % of all cases) according to the 60 % standard of Durlak and DuPre (2008) for the regular version and plus version. Involving parents in the training was difficult: parents were only present in 60 % of the final training meetings and the compulsory plus version intervention techniques were applied less than the regular techniques, although still sufficiently so (i.e., in 74–79 % of all cases, >60 %; Durlak and DuPre 2008). Conclusively, treatment integrity was sufficient to be able to attribute treatment effects to Tools4U.

Treatment as usual (TAU)

Control group juveniles received any suitable treatment other than Tools4U. The vast majority (n = 102, 94 %) received a community service order, meaning that a juvenile is assigned to a workplace to do voluntary work for the sentenced amount of hours, under the supervision of a community service coordination officer. Alternatively, juveniles received another behavioral training sentence (n = 6, 6 %) with duration, training hours, and training intensity similar to Tools4U; that is, an individual aggression regulation training (n = 5, 5 %) or individual substance abuse training (n = 1, 1 %).

Dropout and attrition

The flow of participants for the Tools4U and control groups is depicted in Fig. 1. For the Tools4U group, differences between participants and non-participants could not be investigated, because no information was available on these juveniles. For the control group, differences between participants and non-participants were investigated based on the majority of non-participants (declining: n = 70, 73 %; already started: n = 80, 79 %).

Fig. 1
figure 1

Flow diagram of the participants

Full size image

Non-participants did not significantly differ from participants based on age, gender, ethnicity, control treatment, being a first offender, recidivism risk, and juvenile justice region. Only three differences were found: juveniles declining participation were more often from more urban cities (χ2 = 8.83, p = 0.003), and sentences for juveniles who had already started them were more often imposed by a juvenile judge than by disposal by the district attorney (χ2 = 4.21, p = 0.040), and they committed more property and drug offenses (χ2 = 14.85, p = 0.038).

Despite extensive tracing efforts, some juveniles were lost to post-intervention assessment. These juveniles did not differ from juveniles who did complete the second assessment based on gender, age, education level, living situation, modality, offense, and being a first offender. There were, however, more treatment non-completers among the juveniles without post-test (Tools4U: χ2 = 27.85, p = 0.000; TAU: χ2 = 16.27, p = 0.043). Additionally, control group juveniles were more often ethnic minority youths (χ2 = 9.37, p = 0.002), more often from urban cities (χ2 = 4.00, p = 0.045), had more sentenced hours (t = −2.15, p = 0.033), and differed according to juvenile justice region (χ2 = 27.66, p = 0.001).

It proved to be harder to reach the control group juveniles for face-to-face assessment (T1 n = 314; T2 n = 51) than those in the Tools4U group (T1 n = 115; T2 n = 108). Furthermore, questionnaires about parenting skills were available for almost half of the juveniles (Tools4U n = 45; TAU n = 59). Moreover, due to the Routine Outcome Monitoring collection, the self-perception questionnaire was available for a majority of the Tools4U juveniles (Tools4U n = 88), while it was collected for all control group juveniles (TAU n = 108).

Outcomes and measures

Consistent with the aims of Tools4U, we measured skill deficits related to delinquency (impulsivity, lack of social problem-solving skills, lack of social perspective-taking, and a lack of critical reasoning), developmental task-related skills, and positive parenting behavior. The measures used to investigate these outcomes are described below.

Impulsivity

Impulsivity was measured using the impulsivity subscale of the Antisocial Process Screening Device (APSD; Frick and Hare 2001; van Vugt et al. 2012). The five four-point items of the impulsivity subscale range from 0 = not at all to 3 = definitely true and Cronbach’s alphas were α T1 = 0.70 and α T2 = 0.66.

Social problem-solving

Social problem-solving skills were measured through assessment of pro-social coping styles. The subscales seeking social support (α T1 = 0.83; α T2 = 0.82) and confrontation (α T1 = 0.77; α T2 = 0.76) of the shortened version of the Utrechtse Coping Lijst (UCL; Schreurs et al. 1993; short version, van den Akker et al. 2000) were used. For these scales, eight items had to be answered on a four-point Likert scale ranging from 0 = rarely or never applies to 3 = very often applies.

Social perspective-taking

Information on social perspective-taking was collected by measuring hostile intent attribution and cognitive empathy. To measure hostile intent attribution, the hostile intent subscale of the Social Information Processing and Emotional Response Questionnaire Short Version (SIP-AEQ; Coccaro et al. 2009) was used. The SIP-AEQ consists of vignettes about social situations with direct or relational aggression. Three vignettes were used in the present study. For every situation, juveniles could indicate on a four-point scale how likely or unlikely (1 = very unlikely, 4 = very likely) they thought different motivations or the situation behaviors were. Because the two scales of hostile intent, direct and indirect hostile intent, separately proved to be unreliable, it was decided to group them under one overarching scale of hostile intent (α T1 = 0.77; α T2 = 0.81).

To measure empathy, juveniles were asked to fill out the Basic Empathy Scale (BES; Jolliffe and Farrington 2006; van Langen et al. 2012). This 20-item questionnaire with a five-point scale ranging from 1 = strongly disagree to 5 = strongly agree measures both cognitive and affective empathy, but only cognitive empathy (i.e., understanding how the other feels; α T1 = 0.73; α T2 = 0.65) was used.

Critical reasoning

Various cognitive distortions were measured using the How I Think Questionnaire (HIT; Barriga et al. 2001; Nas et al. 2008). The HIT consists of 54 items that can be answered on a six-point scale ranging from 1 = strongly disagree to 6 = strongly agree. The following cognitive distortions were assessed: self-centering (α T1 = 0.79; α T2 = 0.83), blaming others (α T1 = 0.75; α T2 = 0.78), minimizing/mislabeling (α T1 = 0.80; α T2 = 0.83), and assuming the worst (α T1 = 0.79; α T2 = 0.80).

Developmental task-related skills

Developmental task-related skills were measured using the self-perception profile for adolescents (Competentie Belevingsschaal voor Adolescenten, CBSA; Treffers et al. 2002). CBSA items consist of two sentences: juveniles first have to determine which sentence is most applicable to them and then choose whether this sentence is a little or completely true for them. The 20 items assessing social acceptance (α T1 = 0.68; α T2 = 0.64), behavioral adjustment (α T1 = 0.73; α T2 = 0.76), and self-worth (α T1 = 0.76; α T2 = 0.74) were used.

Parenting skills

The Abbreviated Scale for Parenting Behavior (Verkorte Schaal Opvoedersgedrag, VSOG; Vermulst et al. 2011) consists of 25 items with a five-point scale ranging from 0 = almost never to 4 = almost always measuring positive parenting, rule setting, punishing, harsh punishment, and rewards. As Tools4U aims at improving positive parenting behavior, only the subscales for positive parenting and rewards were used. Cronbach’s alphas were α T1 = 0.64 and α T2 = 0.84 for positive parenting, and α T1 = 0.79 and α T2 = 0.74 for rewards.

Analytic strategy

Overall effects were examined for all outcome measures by conducting an ANCOVA, with the outcome measures at post-test as dependent variables, treatment condition as factor, and pre-intervention scores of the outcome variables as covariates. To control for the significant differences in the degree of urbanization and time between pre- and post-test after matching, these variables were also entered as covariates. Effect sizes were computed as d, based on the F-value, with a positive sign indicating improvement in the Tools4U group relative to the control group. To conduct sensitivity analyses for including juveniles who could not be reached for post-treatment and truants, we conducted split ANCOVAs for juveniles who did or did not complete post-treatment measures and for truants versus offenders.

For the moderator analyses, the same ANCOVAs were conducted, with the moderators as factors. Post hoc analyses for moderator effects were conducted by splitting the file according to the moderator and again conducting an ANCOVA.

Results

To determine the influence of Tools4U variants, post-treatment differences between the regular, long, and plus version were investigated. Because there were no significant post-treatment differences between variants on any of the outcome measures, analyses were conducted for the complete Tools4U group. Furthermore, the influence of the control group treatment (community service order versus behavioral training sentence) on post-treatment measures was tested, but no significant differences were found.

Intervention effects

Results of the analyses of the intervention effects are presented in Table 2. Tools4U was the most effective in improving impulsivity (d = 0.31), social perspective-taking (mean d = 0.15, range = −0.15–0.42), and critical reasoning (mean d = 0.27, range = 0.12–0.41). Specifically, Tools4U juveniles showed significantly less impulsivity, less hostile intent attribution, and less cognitive distortions (self-centering and assuming the worst) than TAU juveniles. No significant treatment effects were found for social problem-solving (mean d = 0.05, range = 0.01–0.09) and parenting skills (mean d = 0.09, range = 0.01–0.27). Finally, treatment effects on developmental task-related skills were not in the expected direction. Compared to TAU, Tools4U juveniles reported less skills (mean d = −0.21, range = −0.04–0.30), specifically about social acceptance and self-worth.

Table 2 Means (M) and standard deviations (SD) for the Tools4U and TAU groups pre- and post-intervention
Full size table

Sensitivity analyses

Despite extensive tracing efforts, some juveniles were lost to post-treatment assessment. To examine the influence of including these juveniles in the analyses, the same analyses were conducted without juveniles that did not complete post-treatment assessment. Assessment completers showed similar effects compared to the complete sample. However, for this group, there was no significant treatment effect on social acceptance (although the effect was still marginally significant, F(1, 177) = 3.23, p = 0.074, d = −0.27). In addition, assessment completers showed significant treatment effects on cognitive empathy (F(1, 177) = 3.96, p = 0.048, d = −0.30), with Tools4U juveniles showing less cognitive empathy after treatment than TAU juveniles. Furthermore, Tools4U juveniles showed significantly less minimizing/mislabeling than TAU juveniles after treatment (F(1, 177) = 6.36, p = 0.013, d = 0.38).

To increase the generalizability of the present study results. we conducted intention to treat analyses. We, therefore, included both assessment and treatment dropouts, as well as incorrectly indicated juveniles, such as the 17 % (non-offending) truants. To examine the effects of including these truants, the same analyses were conducted without juveniles who received their penal sanction for truancy. These analyses showed effects that were similar to the effects based on the complete sample, with two exceptions. First, the effects of Tools4U on hostile intent attribution was only marginally significant when truants were excluded (F(1, 168) = 3.71, p = 0.056, d = 0.29). Second, the offenders-only analyses yielded a significant treatment effect on behavioral adjustment (F(1, 168) = 5.84, p = 0.016, d = 0.37): Tools4U offenders showed less behavioral adjustment after treatment than control group offenders.

Moderators of effectiveness

To examine how participants’ characteristics affected the effectiveness of Tools4U, moderator analyses were conducted for age, gender, and ethnicity.

Age

To investigate the influence of age on the effectiveness of Tools4U, the participants were divided into groups of juveniles younger than 16 years of age (n = 95) and those aged 16 years and older (n = 128). Moderator analyses for age revealed only one significant interaction effect. There was a significant interaction between age and behavioral adjustment (F(1, 216) = 4.33, p = 0.039). Post hoc analyses revealed that juveniles aged 12 to 15 years improved in behavioral adjustment after Tools4U (F(1, 90) = 1.83, p = 0.179, d = 0.28), whereas juveniles of the same age in the TAU group did not show this improvement. Juveniles aged 16 to 18 years, on the other hand, showed the largest improvement in behavioral adjustment after TAU (F(1, 123) = 3.41, p = 0.067, d = −0.33), whereas juveniles of this age did not show this improvement after Tools4U.

Gender

Moderator analyses for gender revealed significant interaction effects for both parenting measures: positive parenting (F(1, 97) = 4.28, p = 0.041) and rewards (F(1, 97) = 4.33, p = 0.040). Post hoc analyses showed that parents of girls reported more positive parenting after Tools4U (F(1, 23) = 3.16, p = 0.089, d = 0.62), whereas parents of boys reported more positive parenting after TAU (F(1, 71) = 1.03, p = 0.315, d = −0.23), although neither of them showed significant treatment effects. Furthermore, parents of girls reported significantly more rewards after Tools4U than after TAU (F(1, 23) = 4.37, p = 0.048, d = 0.73), while parents of boys showed no significant treatment effects (F(1, 71) = 0.04, p = 0.835, d = 0.05).

Ethnicity

To investigate the influence of ethnicity on treatment effects, participants were divided into two ethnic groups: Dutch natives (both parents born in the Netherlands, n = 111) and ethnic minority (one or both parents not born in the Netherlands, n = 112). No significant interaction effects were found for ethnicity.

Reoffending risk

A composite measure was constructed to determine risk of reoffending, based on available characteristics that are the most predictive of reoffending (according to van der Put et al. 2014). The measure included the number of previous offenses (0 = none, 1 = one, or 2 = more than one), violent index offense (0 = truancy, 1 = non-violent, 2 = violent), and gender (0 = girl, 1 = boy). Based on this measure, juveniles were divided into low- versus high-risk groups, with the low-risk group having less than three risk points and the high-risk group having three or more risk points. No significant interaction effects were found for risk of reoffending.

Discussion

The current study examined the effects of the SST Tools4U for juvenile offenders on impulsivity, social problem-solving skills, social perspective-taking, critical reasoning, developmental task-related skills. and parenting skills. We found that Tools4U was more effective than TAU in reducing impulsivity, social perspective-taking deficits, and promoting critical reasoning. However, no treatment effects were found on social problem-solving and caretakers’ parenting skills. Furthermore, for the social perspective-taking domain, positive effects were only found on hostile intent attribution, but not on cognitive empathy. Moreover, juveniles showed more critical reasoning (i.e., less self-centering and assuming the worst). Finally, effects on developmental task-related skills were not in the expected direction: after Tools4U, juveniles reported significantly less social acceptance and self-worth than juveniles receiving TAU.

Overall, we found small effects of Tools4U. On the one hand, as high treatment implementation quality leads to larger treatment effects (Lipsey 2009), we expected larger treatment effects because implementation proved to be sufficient. We did, however, find that (optional) techniques regarding the social network and social report were implemented the least, providing an explanation for the lack of effects on the social problem-solving outcome measures. Furthermore, we also expected larger effects given that the present intervention meets several recommendations for (improved) SST effectiveness: the target population consists of “late starters” (Moffitt 1993; Cook et al. 2008), the training is outpatient (Lipsey 2009), and it is applied individually (as opposed to in a deviant-only group format; Ang and Hughes 2002). On the other hand, the current intervention targets juveniles who had committed less severe offenses, and who were undergoing a treatment of short duration and low intensity. As studies on reoffending generally have found larger treatment effects with more severely troubled juveniles (Lipsey 2009), no major treatment effects could be expected. Additionally, the current relatively small sample would have made it hard to detect small treatment effects. It is, however, unclear how (long-term) reoffending outcomes relate to post-treatment changes on “softer” (dynamic risk factor) outcome measures (Andrews and Bonta 2010; Douglas and Skeem 2005). Arguably, the generalizability efforts of Tools4U (such as including parents in treatment) should lead to improved long-term treatment effects (Cook et al. 2008; Maag 2006).

When comparing the results of the present study to previous research on SST, the overall effects for Tools4U are smaller than those found in previous meta-analyses on SST (as delineated by Cook et al. 2008), which may indicate that Tools4U is less effective than other SST programs. However, this difference could likely be attributed to differences in comparison treatments. That is, the majority of studies in the meta-analyses used a control group receiving no or minimal treatment, whereas the present study used TAU as a comparison group. One could argue that a control treatment (TAU) would have produced more effects than no treatment, making it harder to demonstrate SST treatment effects, although research shows that TAU generally does not have any effects either (Weisz et al. 2013). In addition, the target population for Tools4U (i.e., delinquent juveniles) might be more severely troubled than the target population included in the meta-analyses on SST (i.e., juveniles with behavioral problems) and, therefore, more difficult to treat.

Generally, positive treatment effects were only found on outcomes that could be considered risk factors for delinquency (i.e., impulsivity, social perspective-taking: hostile intent attribution, and critical reasoning: cognitive distortions). For the outcomes that may be considered as protective factors, no effects (i.e., social problem-solving, social perspective-taking: cognitive empathy, developmental task-related skills: behavioral adjustment, and parenting skills) or even negative effects (i.e., developmental task-related skills: social acceptance, self-worth) were found. This could indicate that Tools4U mostly succeeds in reducing risk factors, but is unsuccessful in enhancing protective (buffering) factors. Especially with the emerging call for combining risk and protective factors in the treatment of juvenile delinquency as well as the double-sided (i.e., promotive and risk-reducing) potential effects of protective factors (Loeber et al. 2008; Lösel and Farrington 2012), this could limit the long-term treatment effects of Tools4U on delinquency. Alternatively, several studies have shown that targeting criminogenic needs is more effective, and thus more important, than targeting non-criminogenic needs, such as protective factors (Andrews and Bonta 2010; Andrews and Dowden 2007). It is possible that improvements in protective factors (Farrington et al. 2006) occur on a longer term basis than could be measured in this study. Given that most interventions only target (and measure) risk and not protective factors in treatment, it is unclear if and to what extent other interventions do actually succeed in improving protective factors in addition to decreasing risk factors.

Juveniles referred for Tools4U reported less developmental task-related skills (i.e., social acceptance and self-worth) than juveniles receiving TAU. This could be attributed to the nature of the control group treatment. In most cases, control treatment was a community service order for which juveniles had to do volunteer work. This functioning in a real-world workplace may have contributed to more positive beliefs about social acceptance and self-worth. Furthermore, it could indicate that targeting empowerment of juveniles and teaching parents to empower their children (Albrecht and Spanjaard 2011) was unsuccessful.

Notably, even though parental involvement is an important aspect of Tools4U training, no treatment effects were found on parenting skills, nor were there any outcome differences between the Tools4U plus version and the regular version. This may be explained by the fact that the actual involvement of parents has proven to be difficult, because parents attended the first and final Tools4U meeting in only 60 % of the cases. Furthermore, as the parental component is only a small part of an already short and “light” intervention, small treatment effects might have been hard to detect in the current, relatively small sample (although the current effects did not even approach significance). Alternatively, it is possible that the parental component does not improve parental skills per se, but has had a moderating effect on juvenile skills intervention outcomes.

Finally, only few moderating effects were found, and these effects were not consistent over the various outcome variables. One notable moderating effect is that of gender on parenting skills. Only parents of girls reported more positive parenting and more parental rewards after Tools4U, indicating that the parental component of Tools4U is only effective for girls. This is promising for long-term treatment results, because parental factors have been found to have stronger influence on delinquency in girls than in boys (McCabe et al. 2002), although these findings have not been found in meta-analytic research (Hoeve et al. 2012). Moreover, it could be a result of gender differences in upbringing: girls need emotional support from parents to improve self-esteem, whereas boys need more independence from their parents (Leaper 2002). The Tools4U emphasis on positive parental involvement might, therefore, only be effective for girls.

The current study is, to our knowledge, one of the first to evaluate the effectiveness of SST for juvenile delinquents with known and sufficient treatment integrity. It is one of the few studies comparing SST to TAU and not to a no-treatment control group. Furthermore, as previous SST research mostly targeted juveniles with behavioral problems and not juvenile delinquents, this study expands current knowledge about SST effectiveness for juvenile delinquents specifically. In spite of these strengths, several limitations should be mentioned. A first limitation of this study is that it does not meet the “golden standard” of random assignment to a treatment and control condition (Farrington 2003). Because of practical considerations, random assignment was not possible. Furthermore, the differences in data collection procedure and period resulted in differences between the treatment and control groups (e.g., difference in the number of face-to-face assessments and post-treatment assessment). Additionally, sensitivity analysis showed that the present study might underestimate treatment effects on cognitive empathy and minimizing/mislabeling. Although extensive efforts were made to ensure equality of the treatment and control groups, it is still possible that there are some unmeasured characteristics of juveniles that are responsible for differences in treatment effects. While, to our knowledge, no major economic or social events occurred in between data collection periods that would have resulted in systematic differences between groups, these differences could not be ruled out. Alternative explanations for the positive treatment effects could, therefore, not be completely precluded, and the results should be interpreted with some caution. A second limitation is that the focus was on the short-term effectiveness of Tools4U, immediately established after treatment. A follow-up assessment is needed, which should include juvenile and parent reports and official judicial records to assess recidivism. As a decrease in recidivism is the ultimate goal of Tools4U, it is important to include this outcome measure in future studies. Third, some outcome measures had a reliability between 0.60 and 0.70, which is considered questionable by some scientists (Nunnally and Bernstein 1994), in particular because it reduces statistical power, but is still acceptable according to others (Bijleveld 2009). This means that the present outcomes on impulsivity, cognitive empathy, social acceptance, and positive parenting should be considered with some caution. Finally, the statistical power for the parental skills outcome effects (e.g., the sample size for parenting skills was n = 104) and some of the subgroups that were examined in the moderator analyses may have been too low to detect subgroup effects, resulting in an underestimation of parenting, gender, age, and ethnicity effects.

In sum, the present study is the first effectiveness study of SST Tools4U for juvenile delinquents in a Dutch sample. Treatment integrity for Tools4U was found to be sufficient, so that, from this perspective, treatment effects could be attributed to the Tools4U training techniques. Overall, small treatment effects were found. Tools4U was effective in reducing impulsivity and social perspective-taking deficits and improving critical reasoning. There were, however, no treatment effects on social problem-solving, indicating that effectiveness can be improved by implementing additional techniques, specifically for seeking social support and using the social network. Finally, treatment effects on parenting skills were only found for girls, so the parental component can be improved for boys.

As Tools4U aims to reduce cognitive and social skills deficits related to delinquent behavior, thereby preventing subsequent offenses (recidivism), the current effects on risk factors for delinquency should result in reduced recidivism. It is, however, unclear how the lack of improvement in protective factors influences recidivism. Studies including recidivism data are, therefore, needed in order to make a definite statement on the effectiveness of Tools4U.