In the present quasi-experimental study, we found no indications that 4-day MST (strongly) improved the psychological resilience or mental health of the participating Dutch police officers during the 9 months post-training, compared to police officers that did not participate in MST. Psychological resilience during this period appeared to be relatively stable among both groups, although psychological resilience was higher among the comparison group. Among the total study group (N = 305), a significant training effect was found for Interpersonal Confidence only. However, this finding is probably best explained as a statistical artifact. Since scores on Interpersonal Confidence were distinctly higher in the comparison group as compared to the experimental group, the observed increase of Interpersonal Confidence in the experimental group and decrease in the comparison group seem like scores regressing to the mean over time. Analyses among respondents exposed to potentially traumatic events (N = 170) showed some additional significant training effects with similar patterns: Acceptance of Self and Life, Interpersonal Confidence, and Total Score on the MTQ-48 showed significant change. However, changes in levels of psychological resilience according to the MTQ-48 and RS-nl scores were very small. Therefore these findings seem of little practical relevance. Furthermore, in the case of mental health disturbances, no group differences nor change over time was found, in both the total and PTE groups. However, whether the current training lacks usefulness is subject to other considerations of which the following are the most important: (1) little evidence of low levels of psychological resilience or mental health, (2) the usefulness of psychological resilience as a concept for stress management, and (3) training enrollment regardless of the potential problems of individual officers. We will discuss these matters further below.
To the best of our knowledge, there are no validated cutoff norms for the MTQ-48 and RS-nl that help to identify police officers with (too) low levels of psychological resilience. However, scores on both questionnaires in the current study question findings from earlier reports (Andersson Elffers Felix 2011) regarding the (lack of) psychological resilience of Dutch officers that initiated the development of the MST. It must be noted that the white paper report by Anderson Elffers Felix (2011) does not present any empirical evidence to corroborate the lack of psychological resilience, but rather makes inferences based on earlier research. When psychological resilience is actually measured among Dutch police officers, as in the current study, these inferences clearly do not hold. For example, the lowest MTQ-48 total mean measured at all time points among the experimental and comparison groups was 177.6 (range 48–240); for RS-nl this was 4.1 (range 1–5). At baseline, 98.3 and 86% had MTQ-48 scores of 144 or higher (cutoff when scores across items are neutral = 3) and 168 or higher (cutoff when scores across items are above neutral = 3.5), respectively. Furthermore, the prevalence levels of very high and probable clinical levels of anxiety, depression, hostility (all below 3%), and PTSD-symptomatology (all below 2%) were very low, which is in line with our findings on levels of psychological resilience and corroborated in recent research among police officers in the Netherlands (van Beek et al. 2013; van der Velden et al. 2013). One explanation for these findings on psychological resilience and mental health could be the rigorous police academy selection processes: about 90% of those applying to the officer training program is rejected (van der Velden et al. 2013). This may suggest that there was little room for improvement of resilience. However, the differences between trained and comparison group at the start of the training suggest that there was room for improvement among the trained respondents. Furthermore, the additional analyses among those with relatively low scores (scores in 30th percentile) did not change main outcomes. Therefore, the lack of increase in psychological resilience post-training in the experimental group is presumable due to lack of training efficacy. This is further corroborated by the training evaluation provided by respondents in the experimental group. At both follow-ups, a large majority of respondents felt that the training did little or nothing in adding to their ability to deal with the impact of police work (86.9 and 88.4%, respectively) and/or enhancing their resilience (84.0 and 84.7%). Furthermore, only a minority of participants felt they were taught new things (37.3 and 32.9%, respectively). These findings suggest that the training did not meet individual needs or problems.
It might appear sensible to find similarities between sports and policing, as they both require the ability to perform well under stressful conditions. However, it is also apparent that the main concern of athletes in stressful circumstances is not to safeguard the safety of themselves or others, as is the case for police officers. The absence of effects might be induced by the incorrectness of translating the athletes’ competitive circumstances, to the police circumstances of handling critical incidents. Furthermore, although psychological resilience is often at the conceptual core of stress management, it remains questionable whether this is an effective choice when aiming to improve the ability of the police work force, or similar professions like the military to deal with stressful situations. A recent study by Hystad et al. (2015) stressed group level stability of hardiness, a concept akin to psychological resilience, after 3 years of military training and thereby disputing the changeability of psychological resilience within the military population. In contrast, the study by McCraty et al. (2009) did find positive training effects, but focused more on other aspects such as coping, motivation, and/or positive outlook. Also, when comparing MST-like concepts in a broader field of occupations, such as teachers, managers, military, and the police, it shows that resilience enhancement can be achieved, but effects are small, and in half the cases too small to reach statistical significance according to Robertson et al. (2015). This is in line with our findings; some effects were found, but all effects sizes were small. In contrast to the current study, however, larger effects were found in a review (Robertson et al. 2015) which focused on well-being or mental health outcome measures, rather than psychosocial or performance outcome measures. Absence of substantial MHD prevalence in the current study might be the cause of not replicating this finding.
Another reason as to why the training yielded little to no effects could be found in the form of its delivery. The training was provided to police officers regardless of their individual needs or subjectively experienced problems. Furthermore, it was given to groups of police officers. A recent meta-analysis by Vanhove et al. (2015) examining the effectiveness of resilience building programs has shown that the individual approach yielded more long-term effects, when compared, among others, to training provided to groups. They also showed that programs targeted at problems experienced by the individual work better than so-called universal programs; training given to the entire population without consideration of individual needs or problems (Vanhove et al. 2015). A targeted training given to police officers that experience problems could also enhance efficiency of workforce enhancement efforts. For example, police leadership identifying cases of inadequate stress coping of police officers that require subsequent attention (Chapin et al. 2008). In the current sample and in the sample of van Beek et al. (2013), prevalence levels of serious MHDs were 10% at the highest. It would be more feasible and efficient to provide support to this smaller subset of officers.
However, additional analyses among the subgroup with relatively low levels of resilience showed similar outcomes as among the total group of respondents. Although we did not ask respondents the question if they thought that they needed this training (it was mandatory), relatively low levels may serve as an indication that their need was higher than among those with high levels of resilience. This finding and other results also raises questions about the theoretical background of the training. Besides the aspect of stability of resilience and possibilities to change resilience (see above), in this perspective attention must be paid to the following. The MST training tried to improve challenge, control, and confidence skills and used components developed in other trainings. Although our findings are in line with for example the recent review of Robertson et al. (2015), perhaps other (theoretical) models on learning and training are needed to be able to develop a resilience training that can improve these skills, especially among those who need it or exhibit low levels of resilience. One perhaps simple but important element might be the duration or intensity of the training. The MST consisted of three subsequent days and one meeting months later, (implicitly) assuming that the skills could best be learned or improved by a training of three subsequent days. The outcomes of other studies question this assumption. For instance, Arnetz et al. (2013) described a 10-week training (weekly 2 h sessions) aimed at enhancing control over stressful situations, which was expected to improve coping abilities. Hence, these abilities should support mental well-being of officers in the context of operational stressors. In contrast to MST, this intervention does show some small to moderate effects on for example sleep quality and mental health (Arnetz et al. 2013). The US military, to some extent a comparable population, has implemented the Comprehensive Soldier Fitness program (Cornum et al. 2011). This continues resilience enhancement and maintenance program consists of tracking, supporting, and training resilience to enhance the mental and social well-being of soldiers and their spouses/family (Cornum et al. 2011; Vie et al. 2016). These two programs share the monitoring of the participants and prolonged efforts in enhancing resilience, and or not limited to a few days. As said, behavioral change will most likely occur when the change-effort is sensitive to the needs of the affected and anchorage of such behavior needs prolonged and intensive effort (Vanhove et al. 2015). Adopting this insight, i.e., another model of learning, might improve the effectiveness of a MST training among police officers with low levels of resilience.
Limitations of the Study
The current study does have certain limitations. We were unable to conduct a randomized controlled trial which is considered the “gold standard,” but a quasi-experimental design can be considered the second best and feasible design for examining the effects of such a training program according to Patterson et al. (2012). Additionally, since the training was provided to teams, conducting a randomized controlled trial, in which individual officers are randomly assigned to either an experimental or comparison group, might compromise ecological validity of the study. Team level randomization is hardly different from the current study, since team enrollment into MST is not based on either levels of psychological resilience nor MHDs, but rather on logistical issues. Therefore, selection bias based on psychological resilience or MHD’s is highly unlikely.
The training exclusion criteria based on seriously impeded functioning due to MHD could be a source of differences between the experimental and control groups. However, MHD was comparable between experimental and comparison group. The largest Cohen’s d for baseline MHD differences between the experimental and comparison group was .16 and thereby too small to suggest any substantial differences between the groups.
The comparison group in the current study did not undergo any intervention; placebo or “training as usual.” The results based on the comparison between the experimental and comparison group, therefore, was not attributable to training content, but was found in the distinction between being trained or not. However, when the conclusion is reached whether that delivering a training yields none too little effect, comparing an untrained group of police officers to a group of trained police officers should be considered sufficient proof. The trained officers remained unaffected by the training in their development of psychological resilience and/or mental health disturbances, since the development was comparable to their untrained colleagues.
Previous systematic reviews on resilience training stress that none of the individual studies on the topic were able to determine effectiveness of individual components, but provide an overall effect of the entire training. The current study is not different. However, for most types of training, including the one analyzed in the current study, training components are complementary to each other and therefore effectiveness of individual components is dependent on the availability of its complementary components. For example, the current training advances preparing, handling, and dealing with the aftermath of stressful events. Potential effects of solely focusing on better preparation for stressful events in a particular training might be offset by the absence of advancing in actual handling stressful events by officers.
Attrition occurred during follow-up measurements in both the experimental and the comparison group, especially among the experimental group after baseline. Higher attrition rates among experimental groups are not uncommon in health behavior change (HBC) trials (Crutzen et al. 2015). However, non-response analyses showed that attrition in the experimental and comparison group was barely associated with outcome measures of psychological resilience and MHDs.
Psychological resilience and mental health were assessed using self-report questionnaires. We did not conduct personal (clinical) interviews or use biological measures as for instance McCraty et al. (2009) did. We have no other sources of information, such as the perceptions of colleagues, superiors, or occupational physicians on the mental strength and/or mental health of the respondents in our study. Furthermore, we have no data on individuals’ potential needs or problems. It is possible that the MST is (more) effective for officers who experience needs or problems with respect to psychological resilience. Future studies should include questions assessing these needs or problems regardless of the inclusion criteria of the intervention.
Despite these characteristics and possible limitations, some strength should be stressed as well, such as the relatively large study sample, inclusion of several officer ranks/functions, using two different measures on psychological resilience, and the three-wave study design. As stressed in this review (Robertson et al. 2015), it is remarkable that most effect studies on resilience enhancement did not include resilience measures. To the best of our knowledge, there are no such studies thus far within law enforcement.
The 4-day training yielded small effects in enhancing a very limited number of domains that are considered to be part of psychological resilience. But to conclude low efficacy of the training is subject to other considerations. The main premise of the course, the supposed limited psychological resilience and high prevalence of MHD’s among Dutch law enforcement, is very unlikely to hold when one considers the measured levels among officers in our study. Furthermore, it is unlikely that psychological resilience is useful as a central concept for enhancing the workforce, as, for example, rigorous selection methods make it unlikely that psychological resilience deficits are prevalent among officers. Lastly, the training was not sensitive to any problems individual officers might have. The current study underpins findings on effectiveness of resilience enhancement training for police officers. It showed the relative ineffectiveness of a group-based training not specifically targeted at individual problems or needs.