Towards virtual training of emotion regulation
For professionals in military and law enforcement domains, learning to regulate one’s emotions under threatening circumstances is crucial. The STRESS project envisions a virtual reality-based system to enable such professionals to train their emotion regulation skills. To explore the possibilities for such a system, this article describes an experiment performed to investigate the impact of virtual training on participants’ experienced emotional responses in threatening situations. A set of 15 participants were asked to rate the subjective emotional intensity of a set of affective pictures at two different time points, separated by 6 h. The participants were divided into three groups: the first group performed a session of virtual training in between, in which they received a choice-reaction task; the second group performed a session of virtual training, in which they had to apply reappraisal strategies; and a control group, that did not have any training session. The results indicate that the reappraisal-based training caused the participants in that group to give significantly lower ratings for the emotional intensity of the negative pictures, whereas the content-based training resulted in significantly higher ratings compared to the group without training. Moreover, a second experiment, performed with the same participants 6 months later, indicated that these effects are fairly persistent over time, and that they transfer to different pictures with similar characteristics.
KeywordsVirtual training Emotion regulation Emotional response
The ability to cope with negative stimuli from the environment is a useful characteristic of human beings. Almost on a daily basis, we are confronted with situations that in one way or the other invoke negative emotions. A particular type of negative emotion, which is typically induced by perceived threats, is fear . Depending on the person, different types of stimuli that may trigger fear vary from horror movies and scary animals to enclosed spaces and public speaking. The probability of being confronted with such stimuli depends, among others, on the person’s profession. On average, professionals in domains such as the police, military and public transport are more likely to be confronted with fear-inducing stimuli than people with an office job. It is therefore not surprising that these types of job are usually more appropriate for people who are strong at regulating their levels of fear.
Nevertheless, even the ‘coolest’ of individuals may have difficulties to function adequately in case the stimuli are extreme, such as in cases of military missions or terrorist attacks. First, the extreme emotions experienced in these situations may impair their cognitive processes like attention and decision making [4, 5]. And second, even if they make optimal decisions from an external perspective, they have an increased risk of developing anxiety related disorders such as post-traumatic stress disorder (PTSD) . For these reasons, much time and money is spent on developing appropriate training in these domains. Increasingly, often, virtual environments are successfully used to train performance and decision making of professionals under more realistic and stressful situations (see for example  or ). Furthermore, methods to prevent or treat PTSD after a traumatic event are costly and may even have negative effects . Primary prevention, before any traumatic event has occurred, has been proposed as a promising alternative [10, 11]. A promising technique for primary prevention, which has recently received much attention, is ‘stress inoculation training’ based on virtual reality (VR). The assumption behind this approach is that, by gradually exposing a trainee to fear-provoking stimuli, a VR system is able to increase her ‘mental readiness’ [12, 13]. In that sense, this approach has similarities with exposure therapy [14, 15]. VR-based stress training has proved to be successful, among others, for bank employees  and airline crew , to increase preparation for hostage situations.
The research presented in this article is part of a large project called STRESS, which stands for simulation-based training of resilience in emergencies and stressful situations (http://stress.few.vu.nl). The main aim of the project as a whole is to develop an intelligent system that is able to analyse human emotion regulation and decision making processes in threatening circumstances, and analyse the causes of incorrect decisions and inadequate stress regulation. The system will be incorporated in an electronic training environment for employees in the public domain, based on VR, cf. . In this environment, trainees will be placed in a virtual scenario, in which they have to make difficult decisions, while negative emotions are induced. During the scenario, modern human computer interaction (HCI) techniques will be applied to measure their emotional state. This information will then be used as input for the intelligent system, to determine why the trainee made certain less optimal decisions and to teach her how to improve this.
What type of training should be provided in order to maximise training effectiveness in reducing negative emotional effects?
What are the long-term effects of such types of training?
To what extent is there transfer of training to different, but comparable stimuli?
The article makes some steps towards the investigation of these research questions by means of an experiment where participants were exposed to negative stimuli via a computer screen. The article is organised as follows. In Sect. 2, some theoretical background of the research is reviewed. Next, in Sect. 3, an experiment is introduced that was used to assess the impact of different types of virtual training on the experienced emotional intensity towards the stimuli presented. The experiment involves a first part that was mainly designed to investigate research question (1), and a second part (performed after 6 months) to address research question (2) and (3). In Sect. 4, the results of the experiment are presented, and Sect. 5 concludes the article with a discussion.
2 Theoretical background
Driven by the goal to develop a virtual environment to train mental readiness, it should be possible to obtain a learned effect of successfully lowering subjects’ stress responses for future situations in such an environment. Type of instructions given to the trainees, in order to obtain a successful learning process, are of great importance. Previous research outside the VR domain (e.g., ) suggests that the effectiveness of exposure therapy is partly determined by the specific type of therapy and the task instructions (e.g., related to emotion regulation) that are prescribed to the participant. To investigate the possibilities, we conducted an experiment in which participants’ reactions to viewing negative pictures from the International Affective Picture System (IAPS) picture set  were assessed, and the impact of performing reappraisal-based training in a VR environment was studied.
34 healthy adults (age: 18–30 years) were randomly assigned to one of two groups. Each performed two repeat fMRI tests (test 1, test 2), separated by 12 h containing a night of EEG-recorded sleep (sleep group, n = 18, ten females) or a waking day (wake group, n = 16, nine females). During each test, participants viewed and rated the subjective emotional intensity of 150 standardised affective pictures on a 1–5 scale, corresponding to increasing intensity. Importantly, participants viewed the same stimuli at both test sessions, affording a measure of change in emotional reactivity to previously experienced affective stimuli (test2–test1), following wake or sleep. Participants additionally performed a circadian control test at the second fMRI session, involving presentation of a novel set of affective stimuli. This control test allowed confirmation that behavioural and fMRI differences in reactivity identified following wake and sleep were independent of time of day. 
In the next section, our variant of this experiment is described, in which the difference between sleep and wake is replaced by the difference between virtual training and no virtual training.
3 Experimental design
The first research question addressed is what type of VR-based training is appropriate, in order to obtain a successful decrease of emotional responses towards negative stimuli. As mentioned above, the effectiveness of exposure therapy is partly determined by the way the person deals with the negative stimuli. Within the context of virtual training, a number of strategies can be used, varying from just looking at the stimuli to performing different emotion regulation strategies such as ‘attentional deployment’ (i.e., directing one’s attention away from the emotional stimulus), ‘cognitive change’ (i.e., changing how one appraises a stimulus so as to alter its emotional meaning), and ‘suppression’ (i.e., inhibiting one’s emotional expression) . As a first step to investigate and compare the effects of different strategies, an experiment was performed in which participants’ reactions to viewing negative pictures from the IAPS picture set, ‘developed to provide a set of normative emotional stimuli’ , were assessed for two different types of training. The setup of this experiment is described in the current section.
Fifteen healthy adults (of age between 26 and 32 years, with a mean of 28.2) participated in the experiment, and were randomly assigned to one of three groups (to which we will refer as the ‘training 1’ group, the ‘training 2’ group and the ‘no training’ group), in such a way that each group consisted of 5 participants. Six of the participants were female and nine were male.
The participants in both training groups also participated in these rounds, just like the control group. However, in between these two rounds they performed a virtual training session. This training occurred 3 h after the first round and 3 h before the second round (see Fig. 1, upper and middle line). The virtual training made use of the same pictures used in the other rounds. Within the training 1 group, the participants were given a choice-reaction task in which they had to assess the valence of the picture as quickly as possible while the image increased in size up to double the size of the original picture (i.e., whether it gave them a positive or a negative emotion). They could make this distinction by either clicking the mouse or pressing the spacebar. Within the training 2 group, the participants were asked to view them while actively reducing their emotional response until they felt comfortable looking at the picture (e.g., by assuring themselves that the pictures were not real). The motivation for using these two types of training is that we wanted to investigate the impact of type of training and task instructions (i.e., learning to work with potentially negative stimuli in a dynamic context versus learning to cope with negative stimuli directly by performing reappraisal) on subjective emotional response. Although a large number of emotion regulation strategies exist, we selected reappraisal because this strategy has proven successful in a number of related domains (e.g. ).
Both the test environment and the two training environments were implemented using the PsychoPy software (http://www.psychopy.org/). This package provides an API for creating psychological experiments using the programming language Python. In combination with the Python API provided by PLUX, all ingredients for implementing both environments were available. The implementation itself is relatively straightforward, looping through the different images in fixed intervals and recording both the physiological measurements from the PLUX device as well as the manual responses from the participants.
Control group: no training
Training 1 group: choice-reaction task
Training 2 group: reappraisal task
After this comparison, we investigate whether these types of training have lasting effects, based on the second part of the experiment. Finally, data gathered with a new set of images is compared to the original set to find out if transfer of learning takes place between similar images.
4.1 Type of training
If we focus on only those images that are negatively valenced (left part of the graph in Fig. 5), the results are different. The mean change for the control group is slightly lower at −0.11. Training 1 resulted in an increase of emotional ratings with a mean change of 0.11, while training 2 still has a mean change of −0.39. Both these changes were confirmed to be significant with t(56) = 3.32, p = 0.0016 for training 1 and t(56) = −4,65, p < 0.001 for training 2. From this we conclude that, again for this set of participants, training 1 resulted in a significant increase of emotional ratings for negative images, while training 2 significantly decreased emotional ratings for those images.
Looking at the left graph, showing the results for all images, it can be seen that there are small differences between the control group and training 1, in line with the findings above. For training 2, it is clear that there is a decrease in the pictures with a high rating and a corresponding increase of pictures with a low rating.
Focussing on only those images with a negative valence as shown in Fig. 6 on the right, the results for both the control group and training 2 show a similar pattern compared with all images. However, for training 1, a trend can be seen towards the higher emotional ratings. This is in compliance with the results above, where we found a significant increase of the mean ratings for this group.
4.2 Six months later
For training 1 (top graph) the difference is harder to see. However, statistically, the measurement in the afternoon gave slightly lower ratings overall [t(149) = 3.0478, p = 0.0027], while 6 months later the ratings had increased compared to the first measurement [t(149) = −8.1722, p < 0.0001]. Taking into account only the negative images, the mean rating increased from 3.29 in the morning, via 3.40 in the afternoon, to 3.56 after 6 months, with both the difference between the morning and the 6 months as well as that between the afternoon and 6 months measurement being significant [t(56) = −4.4742, p < 0.0001 and t(56) = −2.6364, p = 0.011 respectively].
The lower graph in Fig. 8 (training 2) shows that both the measurement taken in the afternoon and the one taken 6 months later resulted on average in lower ratings, whereby the ratings after 6 months roughly lie between the ratings of the other two time points measured. Furthermore, all these differences are statistically significant with p < 0.0001. Regarding the negative images, similar results are found with statistical significance of p < 0.001.
Thus, after 6 months, the control group still had a similar response towards the images as they had initially. The participants taking part in training 1 already showed an increased response towards the negative images in the afternoon, and after 6 months this had increased even more for the negative images as well as the complete set of images. The lowered emotional response caused by training 2 was still present (and significant) after 6 months, albeit less pronounced.
4.3 Transfer of training
Furthermore, Fig. 9 shows very few differences between the control group and training 1, indicating little transfer regarding this type of training. An unpaired t test confirms this: t(298) = -0.62703, p = 0.73 for all images and t(113) = −1.0934, p = 0.14 for the negative ones. For training 2, the mean rating has dropped, which can also been seen in Fig. 9. These differences were significant for all images [t(298) = 4.342, p < 0.0001] as well as for only the negative pictures [t(113) = 1.7808, p = 0.039]. Thus, although no transfer can be shown for training 1, transfer does seem to take place for training 2.
In this paper an experiment is reported addressing the impact of virtual training on participants’ experienced emotional responses towards negative stimuli. Participants were asked to rate the subjective emotional intensity of a set of affective pictures at two different points time. The participants were divided into a first group performing a session of virtual training in between these time points, a second group performing virtual training thereby applying reappraisal strategies, and a control group without any training session. The results are that the reappraisal-based training caused the participants in that group to give significantly lower ratings for the emotional intensity of the negative pictures, whereas the content-based training resulted in significantly higher ratings compared to the group without training. Moreover, a second experiment, performed with the same participants 6 months later, indicated that these effects are fairly persistent over time, and that transfer to pictures with similar characteristics takes place.
The outcomes of this experiment indicate that, depending on its setup, virtual training may either strengthen the emotional responses to stimuli or weaken them. The fact that the first group shows enhanced responses might be explained by a form of fear conditioning (e.g., ) taking place in this setup. By presenting the stimuli in an active, dynamic manner, and by asking the subjects to perform an action as a response, a state of enhanced attention on the stimuli was induced, which may have an opposite effect compared to, for example, the emotion regulation strategy called attention deployment (cf. ), and by a process of fear conditioning this may lead to a form of up-regulation as opposed to down-regulation. In contrast to this, in the second group it was explicitly asked to apply an emotion regulation strategy based on reappraisal (cf. ). The outcomes indicate that indeed such a setup can strengthen the emotion regulation, which can be explained as inducing a form of fear extinction learning [26, 27].
For further research, it is planned to perform more experiments like this, with more participants and a greater focus on interpersonal differences. At the moment, similar experiments are being conducted with different types of stimuli (such as sounds and games ) to elicit emotional responses. An interesting additional element here is a personality questionnaire to consider individual differences in relation to for example specific personality traits. This opens up the possibility to investigate whether particular personality traits indicate what type of training would be most beneficial for that particular person.
Finally, the aim of the project is to build a VR training environment in which the knowledge acquired is incorporated. In this respect, care should be taken not to over-generalise the results of the current study. It is obvious that there is still a large gap in realism between scary images or video clips on the one hand, and confrontations with real-world threatening stimuli on the other hand. As this gap cannot be closed by means of one single study, the STRESS project takes an incremental approach, where we gradually try to increase realism of the presented stimuli. For instance, in a study that is currently in progress, an experiment is performed in which participants are actually confronted with virtual reality-based stimuli. More specifically, they are interacting with a virtual character, which at some point starts behaving aggressively towards the participant. In addition, another group of participants is being confronted with a real human (an actress), who starts behaving aggressively as well, in the exact same manner as the virtual agent does. Initial results of this experiment are promising: they indicate that in both conditions, the threatening event invokes subjective and physiological responses in the participants, although there are still some subtle differences between the virtual and the real stimuli. In follow-up research, the nature of these differences, as well as their implications for training purposes, will be studied in more detail.
Note that this intensity was independent of the ‘valence’ of the picture (i.e., pos. vs. neg).
The results of these measurements are not further discussed in this paper. They were collected to gain more insight in the relation between the presented stimuli and physiological states, which will be further explored in a follow-up experiment.
In both the control and training 2 group one participant dropped out.
This test took, for each of the 150 pictures, the average change in rating given by the participants in the training group, and compared this with the average change in rating given to the same picture by the participants in the control group. Since this way of testing takes the pictures (instead of the participants) as a basis, the results cannot be generalised for the population as a whole. However, an additional (unpaired two sample) t-test has been performed in which for each participant the average change in rating over all 150 pictures was calculated, and these averages were used to compare the training 2 with the control group. Due to the low number of participants, these results were not statistically significant on the p < 0.05 level, but a clear trend was found in the results (with t(8) = 2.05, p = 0.07).
This research was supported by funding from the National Initiative Brain and Cognition, coordinated by the Netherlands Organisation for Scientific Research (NWO), under grant agreement No. 056-25-013. The authors are grateful to Els van der Helm and colleagues for sharing the details of their research with us, to Rianne van Lambalgen for her help regarding the statistical analysis, and all participants of the experiments.
- 1.Bosse T, Gerritsen C, Man J. de, Treur J. (2012). Measuring stress-reducing effects of virtual training based on subjective response. In: Huang T et al. (eds) Proceedings of the 19th international conference on neural information processing, ICONIP’12, part I. Lecture notes in computer science, vol 7663. Springer-Verlag, Berlin Heidelberg, pp 322–330Google Scholar
- 2.Bosse T, Gerritsen C, Man J. de, Treur J (2013) Effects of virtual training on emotional response: a comparison between different emotion regulation strategies. In: Proceedings of the 7th international conference on brain and health informatics, BHI’13. Lecture notes in artificial intelligence. Springer-Verlag, Berlin Heidelberg, pp 21–31Google Scholar
- 3.Öhman A (2000) Fear and anxiety: evolutionary, cognitive, and clinical perspectives. In: Lewis M, Haviland-Jones JM (eds) Handbook of emotions. The Guilford Press, New York, pp 573–593Google Scholar
- 4.Loewenstein GF, Lerner JS (2002) The role of affect in decision making. In: Davidson R, Scherer K, Goldsmith H (eds) Handbook of affective science. Oxford University Press, New York, pp 619–642Google Scholar
- 7.Bouchard S, Guitard T, Bernier F, Robillard G (2011) Virtual reality and the training of military personnel to cope with acute stressors. In: Advanced computational intelligence paradigms in healthcare 6. Virtual reality in psychotherapy, rehabilitation, and assessment. Springer Berlin Heidelberg, pp 109–128Google Scholar
- 10.Deahl M (1998) Traumatic stress-is prevention better than cure? J R Soc Med 91(1998):531–533Google Scholar
- 12.Popović S, Horvat M, Kukolja D, Dropuljić B, Ćosić K (2009) Stress inoculation training supported by physiology-driven adaptive virtual reality stimulation. Stud Health Technol Inform 144(2009):50–54Google Scholar
- 13.Rizzo AA, Reger G, Gahm G, Difede J, Rothbaum BO (2008) Virtual reality exposure therapy for combat related PTSD. In: Shiromani P, Keane T, LeDoux J (eds) Post-traumatic stress disorder: basic science and clinical practice, Springer Verlag, New YorkGoogle Scholar
- 16.Markus E (2000) Antwoord op agressie, het effect van training op het omgaan met agressief klantgedrag en bankovervallen. Ph.D. Thesis (in Dutch), Erasmus Universiteit, RotterdamGoogle Scholar
- 18.Marsella S, Gratch J (2002) Modeling the influence of emotion on belief for virtual training simulations. In: Proceedings of the 11th conference on computer-generated forces and behavior representation, Orlando, FL, May 2002Google Scholar
- 20.Lang PJ, Bradley MM, Cuthberth BN (1999) International affective picture system (IAPS): technical manual and affective ratings. The Center for Research in Psychophysiology, University of Florida, Gainesville, FlGoogle Scholar
- 28.Bosse T, Gerritsen C, Man J de, Stam M (2014) Inducing anxiety through video material. In: HCI international 2014—posters’ extended abstracts. Springer Verlag, pp 301–306Google Scholar
- 29.Cusveller JJ, Gerritsen C, Man J de (2014) Evoking and measuring arousal in game setting. In: Göbel S, Wiedemeyer J (eds) Proceedings of the 3rd international conference on serious games, GameDays 2014, vol 8395. Springer LNCS, pp 165–174Google Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.