Combination of a Serious Game Application and Direct Contact with Mental Health Patients

Currently, one of the difficulties associated with patient recovery from severe mental disorders is stigma. Some authors have even identified its effects as a “second illness.” In order to combat these difficulties, various stigma awareness programs have been launched. Furthermore, in recent years, new technologies have also been incorporated, such as in the cases of serious games (educational video games) designed for this purpose. The present study examines the combined effect of a serious game called Stigma-Stop with a stigma awareness program based on direct contact between students and mental health patients. A total of 313 students participated in the study. The individuals were divided into two experimental groups (one which utilized Stigma-Stop and another which did not) and a control group. The results demonstrated that the two interventions were effective in reducing stigma, but the group which featured the serious game obtained a greater improvement on the fear factor. The discussion section addresses the relevance of these results.

its adverse impacts can also be observed in decreased adherence to treatment, low employment rates, and poor physical health, along with other issues (Fresán et al., 2010;Michaels et al., 2012;Schomerus et al., 2019). Such aspects have a profound effect on the recovery of people with these conditions (National Academy of Science, Engineering, and Medicine, 2016), leading several authors to consider stigma to be a "second illness" .
Stigma is a priority of the WHO Europen Mental Health Action Plan (WHO, 2013). Similarly, many international and national organizations have designed a variety of campaigns to reduce the impact of this phenomenon (Hansson, 2017).
Anti-stigma programs have generally relied on three strategies. The first consists of providing information which explains mental disorders. In this case, it has been demonstrated that it is particularly important to highlight biographical aspects and the suffering these individuals endure as it elicits greater empathy towards people with mental health problems (Angermeyer & Matschinger, 1996;Longdon & Read, 2017). Similarly, recent studies have found that it is also very important to show the existence of a continuum between schizophrenia and less intense symptoms in the general population, which helps reduce public stigma and improve self-stigma (Violeau et al., 2020). The second strategy, which has traditionally proven to be more effective in reducing stigma, is direct contact with mental health patients (Corrigan & Nieweglowski, 2019;Lee & Seo, 2018). In this case, as described by Allport in his classic work on prejudice (Allport, 1954), it is important that direct contact features several characteristics, specifically equal group status, common goals, intergroup cooperation, and the support of authorities, law, or custom. Finally, the third strategy is protest activities, which are particularly effective when combined with the previous two strategies described (Corrigan, et al., 2001).
Regarding more recent studies, the use of new technologies is increasingly more common when working with teenagers, such as the application of serious games, apps, and virtual reality programs, among others. These tools offer advantages in that they reach wider populations, are highly appealing and dynamic, and can be combined with other, more traditional methods. What is more, the costs of developing such technologies are continually decreasing (Carmona et al., 2012). Thus, electronic resources are becoming more common in a range of applications in the fields of Psychiatry, Psychology, and Education (Kim & Kim, 2020;Shen et al., 2019).
In the case of stigma, studies have quite often resorted to using videos or vignettes, or drawings, but not so much to electronic resources (Stubbs, 2014). The first serious game designed specifically for this purpose was Stigma-Stop . This game features four individuals with different mental health disorders. While interacting with these characters, players must choose among different ways of behaving around them. The game provides players with information about the featured disorders and asks them questions about whether they have ever suffered from similar problems or if they believe they could help the characters in the video game.
Stigma-Stop has been successfully used with high school and university students . Furthermore, its effectiveness has been compared with other traditional methods, such as direct contact with mental health patients and a talk by a mental health professional. It was found that the serious game obtained similar results to those of direct contact and slightly better than those of a talk with a professional (Mullor et al., 2019).
However, what has not yet been tested is the combination of this serious game with other strategies, for example direct contact with mental health patients. This combined approach could foster additional benefits. The serious game would first provide students with information explaining the mental disorders. Then, they would have the opportunity to meet mental health patients who give first-hand accounts of their experiences and answer any questions students have about their conditions. Furthermore, the two groups would have the opportunity to participate in group activities together. It is quite possible that this combined strategy (contact plus Stigma-Stop) could have a superior effect over not applying the serious game prior to only direct contact.
Indeed, the objective of the present study is precisely to determine to what extent Stigma-Stop is able to provide additional value to an intervention based fundamentally on contact with mental health patients, both by means of direct talks with students and by carrying out group activities between students and mental health patients.

Participants
Three hundred thirteen students from both 4th year secondary school and 1st year baccalaureate studies participated in the study. These participants came from four Secondary Education and Baccalaureate schools in the province of Almería (Spain). The final sample was selected using intentional non-probability sampling, i.e., convenience sampling, which included a total of 313 students. The ages of the participants ranged between 15 and 17 (M = 15.83; SD = 1.24). As for the distribution according to gender, there were 180 (57%) women and 133 men (43%).
The distribution of the cluster was conducted randomly according to the classrooms where they were located. All classes belonged to the same school. Thus, four classes (125 students) comprised Experimental Group 1, another four classes (119 students) formed Experimental Group 2, and the last two classes were the control group (69 students).
Approval to conduct this study was obtained from the Ethics Committee of the University of Almeria. Students were only excluded from the sample if they refused to give their informed consent to participate. The participants received neither economic nor academic incentives for taking part in the study.

Questionnaire on Students Attitudes towards Schizophrenia (Schulze et al., 2003)
This instrument was initially developed in Germany and applied to secondary school students as part of the Global Program of the World Psychiatry Association against stigma and discrimination. It assesses students' stigma towards people with schizophrenia, which is one of the most stigmatized disorders . The questionnaire consists of 19 items with three alternative answers (I agree, I disagree, I am not sure). The present study utilizes the Spanish version of this instrument (validated by Navarro et al., 2017) which obtained an overall Cronbach's alpha of 0.95. The said version is comprised of two factors, i.e., uno es stereotypes (e.g., "I would be embarrassed if my friends knew that someone in my family had schizophrenia" or "I would not invite someone with schizophrenia to my birthday party") and fear/dangerousness (e.g., "Someone with schizophrenia might get angry over something silly" or "I would be afraid to talk to a person with schizophrenia").

Stigma Stop
A serious game which introduces four characters who suffer from various mental disorders (schizophrenia, depression, agoraphobia, and bipolar disorder). The player must interact with each character to gain their collaboration and together carry out a group task, which, in this case, is to participate in a video game design contest which is presented in the serious game itself. During the different meetings, the characters begin to display characteristics common to their disorders. The game provides the player with information and offers various strategies on how to respond in the situations depicted. The player is initially asked to choose what they believe to be the most appropriate response among several possible options (feedback is offered for each of the alternatives presented). Additionally, the game interacts with participants by asking them if they think the characters feel well emotionally, if they themselves have ever felt like the characters, and if they think they could help them. The objective is to foster empathy among participants towards individuals with these problems. Once interaction with each character has concluded, a corresponding mini-game is presented (i.e., Memory, Trivia, Race, and Shooting) whose content is related to mental health concepts: -Evaluation of Stigma Stop. A questionnaire was developed ad hoc for the present study so participants could score, with the following questions: 1. On a scale of 1-10, how would you rate this practice in terms of interest? (0 would be not interesting at all to 10 very interesting): 2. How would you rate it in terms of usefulness? (0 not at all useful to 10 very useful) 3. Of the three parts of the practice, how would you rate each of them, on a scale of 1-10, in terms of interest and usefulness (write it down in the corresponding box): Interest(0-10) Usefulness (0-10)

Procedure
The procedure was conducted in group format using the classes themselves as individual units. Firstly, the participants completed the Questionnaire on Students Attitudes towards Schizophrenia (QSAS; Schultze et al., 2003) in a classroom isolated from noise and outside interference while the researchers remained with the students to administer the survey and to answer any questions they may have had. The researchers emphasized that all answers were anonymous and completely confidential. The students completed the questionnaires in a period of 10 min on average. Individual class units were randomly assigned to one of three groups featuring the following conditions: -Experimental Group 1. The following protocol was applied: a) Firstly, the serious game Stigma-Stop was applied. During this stage, four volunteers came forward to play the game while the rest of the students followed the game's progress on a projector screen in the classroom. The same procedure was used in other previous studies which also applied the computer program . The duration of this phase was 1 h. b) Talk with patients. Subsequently, three mental health patients described first-hand their experiences of suffering from their mental health problems, such as how their difficulties began and what helped them most to overcome those difficulties. In addition, the speakers also responded to all the questions the students had regarding these topics. The duration of this part of the intervention was 1 h. c) Group activities. Finally, with the participation of 20 mental health patients from different mental health associations in the province of Almeria, students participated in a series of group activities that their school had prepared in advance. The activities included sports (tug of war), music and gastronomy (follow different beats, cook crepes) and culture and botanical workshops.
Once the entire program had finished, the participants responded again to the Questionnaire on Students Attitudes towards Schizophrenia , as they also did with the evaluation questionnaire on the overall experience.
-Experimental Group 2. The same procedure was applied as in the case of group 1, except for the fact that the serious game was not utilized. Before and after applying the program, the students filled out the Questionnaire on Students Attitudes towards Schizophrenia. -Control Group. These students participated in the everyday school activities of their respective schools, without being subjected to the application of any intervention. They were given the evaluation questionnaire on stigma before and after the same time interval as Group 1 (4 h of activities).

Data Analysis
Cases that did not fully answer the questionnaires used were discarded. Subsequently, to determine whether or not there were differences between the groups in relation to both the pretest and the posttest, different ANOVAs were carried out, completed with the Bonferroni post hoc test and quantification of the effect size using η2. Subsequently, an analysis of variance with repeated measures was used to determine whether there were differences in the dependent variables according to the different measurements, pretest, and posttest in relation to each of the groups. Finally, a multivariate analysis "MANOVA" was performed to assess the influence of age and gender on the benefits of the intervention. All analyses were completed by calculating the effect size using η2.

Results
As can be observed in Table 1, the average difference test between the pretest measurements of the experimental groups and the control group did not reveal the existence of statistically significant differences among the three with the variables analyzed. However, there were statistically significant differences between the three groups for all the variables evaluated following the intervention. By using Eta squared (η2), it was confirmed that the differences between the groups after the activity were moderate for fear/ dangerousness and the total score and low for stereotypes. Also, there were statistically significant differences between the pretest and posttest for the experimental groups and the control group in relation to all the variables and between Experimental Groups 1 and 2 in the fear variable. It is verified that there are no statistically significant differences in fear, dangerousness, nor in the total score of the questionnaire between the 3 groups, and this is confirmed by the effect size, which in all cases has been low.
The analysis of the posttest-pretest scores of the control group revealed no statistically significant differences with respect to any of the variables evaluated, as can be observed in Table 2. However, significant differences were found in the same analysis for the scores of Experimental Groups 1 and 2, both in the total of the questionnaire score in relation to stigma and its two factors, namely, fear/dangerousness and stereotypes. As regards the scope of the effect, it was observed that Experimental Group 1 had a strong impact on reducing stigma in dangerousness but was weak in terms of affecting stereotypes. The intervention had a moderate effect on Experimental Group 2.
As for the students' evaluation of the different phases of the program, the students gave the serious game a score of 8.85 points for usefulness (SD = 1,40) and 7.62 points for interest (SD = 1.73). The talk with mental health patients received 7.25 points for usefulness (SD = 1.85) and 8.41 for interest (SD = 1.47). Finally, students gave the group activities 8.50 points for usefulness (SD = 1.50) and 9.13 for interest (SD = 1.16).

Discussion
The use of electronic resources is increasingly more common in Psychiatry, Psychology, and Education (Kim & Kim, 2020;Shen et al., 2019). The first serious game designed specifically to reduce stigma among students was released in 2017 , obtaining positive results with both high school and university students . Furthermore, its effect was compared with other methods, namely direct contact with mental health patients and talks given by professionals. Research found that the game's results were similar to those of direct contact (Muyor et al., 2019). In this sense, resorting to an electronic format can be an effective way of bringing young people closer to these problems, using a medium that they use a lot, such as video games (Snodgrass, et al., 2018). However, what has yet to be tested is whether this program could be also be specifically combined with traditional methods, such as contact with mental health patients. In other words, this type of combination test would determine if providing information to students in an entertaining way would enhance the effects of the intervention with patients, thereby achieving better results. The said experiment was precisely the objective of the present study.
The results showed that the two interventions utilized (contact with patients, both through talks and group activities, combined with the serious game Stigma-Stop and without) obtained positive results in reducing stigma when compared to the results of the control group. However, the experiment group consisting of the complete procedure (Stigma-Stop combined with contact) had a greater effect on the fear/dangerousness factor when compared to the results of the group which was not exposed to the serious game. Therefore, incorporating the serious game specifically contributed to reducing one of the most common characteristics related to stigma, namely fear (Link et al., 1999).
This effect may have been the result of the information about mental health problems that the serious game provides, which offers a more personal view of the disorders and presents them as being common in society . These aspects are not perceived in the same way if information is not provided to participants beforehand. Without information, it is possible to reduce social distance but difficult to do so with the fear factor.
On the other hand, contact is a sufficient variable to reduce stereotypes, since in the interaction it is easy to change common beliefs about this population that are not as deeply rooted as the idea of fear/dangerousness (West, et al., 2014). The literature also shows that in order to maximize the effectiveness of this type of intervention, it is important that the contact is prolonged, so it can be expected that if the intervention lasts longer the effect could be greater (Pettigrew & Tropp, 2008).
As for the evaluation of the usefulness of their experience, the students gave an average score which was higher than eight points to the three phrases of the intervention. More specifically, Stigma-Stop was given 8.80 points, the group activities were given 8.5 points, and the talk with mental health patients was given 8.4 points. Furthermore, albeit the differences were negligible, the participants gave a higher score for usefulness to the serious game.
As for the level of interest of the experience, the highest-scoring phase was the group activities with students and mental health patients (9.1 points), followed by the face-to-face talk about life experiences (8.4 points) and, finally, Stigma-Stop (7.6 points). In this regard, the most participatory intervention, namely the group activities carried out, was what most interested the students, perhaps for the novelty of the activities and the opportunity for greater involvement. In this regard, the serious game had a lower score, perhaps owing to the fact that the use of video games is highly common nowadays among young people and not so appealing or novel (González & Aguilar, 2019).
In conclusion, the results showed that the complete program achieved greater effectiveness than the others. They also demonstrate that not only can the serious game be utilized alone or as an instrument in conjunction with other methods, but it may simply be more beneficial when used in interventions that include face-to-face activities with mental health patients, as was the case of this experiment.
Among the limitations of the study, the number of participants in the sample is relatively low and there were no follow-up measures taken to determine whether these results lasted over time. Furthermore, the assessment instrument focused on the assessment of stigma towards people with schizophrenia, and more instruments need to be added in future studies as well. Possible future lines could delve deeper into students' qualitative evaluation of the various phases of the intervention, along with investigating whether the order in which the program is applied may also be a variable that influences results.
Funding Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.

Declarations
We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property. We further confirm that any aspect of the work covered in this manuscript that has involved either experimental animals or human patients has been conducted with the ethical approval of all relevant bodies and that such approvals are acknowledged within the manuscript. This study has been approved by the bioethics committee of the University of Almeria in order to start the present study (Ref. UALBIO 2019/213). We understand that the corresponding author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). He/she is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible by the Corresponding Author.

Conflict of Interest
The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.