Introduction

At the start of 2022, the Federal Bureau of Investigation (FBI) (FBI, 2022) released a statement warning companies that a hacker group was pretending to be Amazon and sending out packages that contained BadUSB devices (Cimpanu, 2022), trying to trick companies into using them. This event is in line with the increase in the number of USB-based attacks over the past years (Proofpoint, 2022). To map out the attacks, researchers have identified around 29 different USB-based attack types (Nissim et al., 2017). These attacks make use of all kinds of different malware, but the one thing they have in common is that they rely on human error (Talamantes). Next to identifying different versions of attacks, researchers have investigated how effective these attacks are. For example, Matthew Tischer et al. emulated a USB-drop attack on the University of Illinois, Urbana-Champaign campus (Tischer et al., 2016) and found that out of all the USBs dropped 45% of them were plugged into a computer. Because USB-based attacks can effectively trick people, solutions have been developed that prevent malicious code from executing (Mueller et al., 2019) and that use AI to try to recognize malware on them (Singh et al., 2022). However, these solutions are not waterproof (Tischer et al., 2016). Because of this, some people have put a focus on training and prevention of human error (Zwilling et al., 2020). For example, Inspired eLearning (eLearning, 2019) and UTwente Security Education Platform (UTwente, 2022) produced videos on cyber-security topics. However, there the stream of information is one way, unlike in gamification.

To make training more engaging and interactive, researchers have done more and more research on the use of gamification (Bai et al., 2020; Manzano-León et al., 2021; Sailer & Homner, 2019). Gamification, or serious gaming, is applying game elements to education (Kalogiannakis et al., 2021). In their meta-analysis, Bai et al. (2020) found that gamification overall has a positive effect on learning and education. A similar trend can be seen in cyber-security (Chaskos et al., 2022). To make gamification easier to apply in different disciplines, researchers have identified key concepts for making gamification effective (Bai et al., 2020; Kalogiannakis et al., 2021; Saleem et al., 2022). One concept that was identified is the use of narrative. According to other research, storytelling has been popular in traditional education for a long time, but there is already a lack of systematic review of the effects of it outside of gamification (Wu & Chen, 2020). There are examples of gamification approaches that use perspective to engage the user (Santos et al., 2021). In the past, Zichermann and Cunningham (2011) explicitly excluded narrative and storytelling when discussing important gamification concepts. They argued that the focus of gamification should be on non-fiction, so a narrative has no place in this. Yet others kept on using it. For example, 4 years later, Nicholson proposed “meaningful gamification”, in which narrative plays a key role (Bai et al., 2020).

Coenraad et al. (2020), did a systematic review of digital games related to cyber-security. They looked at 181 different games related to this topic and found that 74% of those deploy story-telling. Yet, on databases such as Scopus, there seems to be a lack of research on the effect of story-telling in gamification. For example, a search which is done on 16/05/22 in the Scopus Database (Elsevier, 2022) for papers related to story-driven gamification for cyber-security which included resulted in 22 results being returned. But, a search in Scopus on 16/05/22 using a query which excludes story-telling, returns 297 results. The exact queries used can be found in Appendix 1 (Fig 9). A similar trend has been observed in other databases, such as Web of Science Core Collections. So, while research shows that most cyber-security-related gamification deploy story-telling, less than 10% of published papers seem to discuss how effective it is.

Knowing that there is a knowledge gap on the effectiveness of stories in gamification research is important, however it is also important to close this gap for cyber-security education. One example of this can be found in the review by Wu and Chen (2020) on Education Digital Stories (EDS). This work reviews the research done in EDS and looks at the direction it has been heading in. What is interesting is that this work goes into how the usage of EDS and the focus it has influenced its effectiveness in higher education. It shows that in education, the way a story is designed and the goal it tries to achieve is directly influencing how effective it is, as well as an overall positive influence of stories in education. Seeing that this can be observed in a field such as EDS might also have implications in other fields which use stories such as gamification.

Because of this, it is therefore important to test the effectiveness of story-elements and how they can be implemented. A combination of primary and secondary research will fit this purpose as it will be possible to review the role of story-elements in gamified education and how to form the way they apply to best complement what it is trying to teach. This will advance understanding of using story-elements and potential frameworks that could or should be built to guide implementation. It will also highlight old evidence and provide new evidence in the field for further research. Because of the lack of research on this as identified in the knowledge gap, it is important to provide further analysis of the effects that can be observed to close this gap, provide footing for future development and research, as well as possible reviews. At the same time, this research focuses on trying to convey information in a limited amount of play-throughs as possible. Instead of having people come back over and over again, the game is meant to be played once or twice and still keep the player engaged enough using the story to remember the information.

To do this, the goal of this study is to answer the following main question: What is the impact of story-driven gamification on USB-based cyber-attack prevention among students?

This is done by first answering the following sub-questions:

  1. 1.

    What story-driven gamification concepts were perceived as effective in previous research on gamification?

  2. 2.

    Which of these concepts can apply to gamified USB-based cyber-attack prevention?

  3. 3.

    What is the perceived effectiveness of an application created using the characteristics from Sub-Question 2 when applied to USB-based cyber-attack prevention?

The literature review included an analysis of five papers and articles, selected based on the following criteria. First off, all the articles have at least been cited by 50 other works. The second criterion was the quality of the journal in which it has been published. An example of such a conference is the ACM/SIGSAC (2021) Symposium on Computer and Communications Security, which gets an h5 index of 94 (Scholar, 2022). The final criterion was the author of the research and to what institutions are they connected. One issue to think about when deploying a game in research is that all participants have to be able to access and play it. To make sure that the game is playable by almost anyone, the game has been created as a web-based game that can be played in a browser. For this reason, the surveys have also been made accessible online, so any student in the target group with a browser can partake. To incentivize people to participate, they were offered the chance to get an overview of the results after participation. There are other methods out there for incentivizing people to participate, like offering monetary compensation. However, sending an overview is less likely to attract reward seekers or introduce bias (Neal et al., 2020).

The next sub-section of this paper discusses the papers that were reviewed in the literature review. It goes over papers that in some form studied the effectiveness of narrative and story in gamification and what made them (not) effective.

Related work

In the past, several studies have been done on the use of story-telling and gamification. These studies identified several things that are important to consider for story-driven gamification. Below, a selection of five papers (Armstrong & Landers, 2017; Chothia et al., 2017; Nicholson, 2014; O’Donovan et al., 2013; Prestopnik & Tang, 2015) and their findings are discussed.

First off, Nicholson (2014) created a framework for what they called “meaning full gamification” to create long-term behavior change in participants. It includes two concepts of interest for this research: exposition and information. Exposition comprises developing and presenting a meaningful narrative element. This can be done by balancing realism and analogies. If a game is realistic, it is easier for players to get the information and relate it to the real world. But using analogies might make it more intriguing. Next, they discuss the importance of selecting the correct type of narrative. This can influence the analogy-realism balance and how the user perceives information. This leads to the information element. According to Nicholson, it is important to present the “why” and “how” to a user and there are two relevant ways this can be done. First, they mention using a Non-Player-Character as a guide, which provides the user with information and help. The second method is to make sure the information supplements the exposition.

To further discuss the idea of balancing the story world and the real world, one can look at the work of Prestopnik and Tang (2015). They discuss the idea of Diegesis, which is the link between the real and the story world. Concepts from the story world can be revealed through the narrative, and as one plays. Further, it discusses the effect of exogenous versus endogenous fantasy. In exogenous fantasy, there is no direct link between the task someone does and the narrative. For example, calculating 2 + 2 = 4 might launch a rocket, while in an endogenous fantasy, the player would calculate the trajectory of the rocket. Why this link between task and story might be important comes up in research by O’Donovan et al. (2013). There, the researchers present the gamification of a university course on game development. The students that partook pointed out that the storyline was not sufficiently integrated into the course, causing them to not be invested in the tasks.

A major aspect that keeps on coming back in research is the idea of agency. In 2017, Chothia et al. (2017) gamified a cyber security course with a “choose your own adventure” type game. In this type of game, a lot of freedom is given to the students to shape the story and interact with it in their own game. Overall, students saw this as a positive element. They praised the story, however, were also concerned about the time commitment it might add to their already busy classes. Next to the agency, two things that were praised are the realism of the game and the writing level. There is also an important link between the type of knowledge that is presented in the story, and gamification effectiveness which came forward in research by Armstrong and Landers (2017). They found that story elements had a more positive effect on the declarative knowledge intake of students compared to procedural knowledge intake. So it is important to ensure that the type of knowledge presented in a training fits the story one is using. Finally, they demonstrated the usefulness of using a framework to design the story. Different frameworks have different goals and effects on students, and using the correct one can aid the effectiveness of the training (Ferguson et al., 2020; Thorndyke, 1977). Ferguson et al. (2020) Also found that in terms of different texts that can be presented to a person, texts that are based on stories are better understood than texts that do not hav e storylines in them.

What has been observed in this literature review is that other research has investigated using stories in gamification. However, story-elements were never the focus of the research, but part of a larger set of game mechanics. This makes it difficult to pin-point which effects they have when isolated. However the importance of story telling in regards to information recall by human-beings has already been acknowledged in traditional forms of education (Kartalkanat & Göksun, 2020; Thorndyke, 1977). Stories help improve the recall of information and help them retain more information over time from them than from a set of facts.The novelty of this research lies in the focus on story-elements in gamified training, and that it highlights if they have a positive or negative effect on gamification effectiveness. The rest of the paper has been structured as follows. First, the method of the research and how the research questions are answered are discussed. Next, the design of the game and survey are discussed, including how the results from the literature review were applied the the design. After this, the perceived effectiveness of the game is discussed and the research questions are answered. Finally, the conclusion and limitations of the research are considered.

Methodology

This section discusses the methodology behind the research, including the research model and procedure, research context and sample, as well as the instruments used and their validation.

Research model and procedure

This subsection gives a general overview of the research model, procedure, and how the research questions were answered. The research has been divided into three different stages which all answer one of the research sub-questions. Figure 1 depicts the stages and the flow of the research. Stages 2 and 3 were done in parallel, after designing the survey and while waiting for ethical approval the story and game were designed and only after both stage 2 and 3 were done was the game sent to participants.

Fig. 1
figure 1

Flowchart of the proposed method

Full size image

The study incorporates both qualitative and quantitative approaches (Lakshman et al., 2000). Qualitative aspects are encapsulated within the literature review and specific sections of the survey. The literature review delves into story-driven gamification concepts and theories established in previous research. Further elaboration on the literature review is available in the subsequent paragraph. Concurrently, a segment of the survey encompasses open-ended question designed to extract insights into participants’ previous experiences with, and knowledge of cyber-security. Hence both the literature review and survey yield qualitative findings. Simultaneously, the survey incorporates Likert-scale questions (mangiafico), a tool that facilitates the collection of quantitative data. The resulting quantitative dataset can be subjected to significance testing. A more comprehensive overview of the survey’s design can be found in Sect. 2.3.

Concerning sub-question 1, a literature review has been done on story-driven gamification concepts. The articles were analyzed, and the main story-driven concepts and their effectiveness were explored. This information was used to create an overview of what research already has been done for story-driven gamification. Using insights from the literature study, sub-question 2 has been answered. At this stage, an application had been designed that implements the concepts found in the literature study. This way, the most relevant concepts of story-driven gamification could be implemented into the game. This application has been used in an empirical study, which answered sub-question 3. The study starts with all participants participating in a pre-game survey asking about their background and how much they know about USB-based attacks. Following this, people played the game and experienced the story. Finally, all participants were asked to do a post-training survey that checked the perceived effectiveness of the gamification and compared it to other gamification.

Research context and sample

The research focused on master’s and bachelor’s students, ranging in age from 18 to 25 and based in European and North American countries, as past research has suggested that they are susceptible to cyber-attacks (Solutions, 2022). The students came from studies with an array of different backgrounds, such as engineering studies and studies related to economics. This allows the research to look at a wide array of students that might have different technical knowledge and knowledge about cyber security. Some students are considered to have a tech-background. These are students that do a technical study such as electrical engineering or computer science and will have learned about the workings of computers, software, and cyber-security concepts. This includes: (Technical) Computer Science, Electrical Engineering, Creative technology, and Advanced Technology.

In total, at least 28 students needed to participate as the results from the survey were tested on significance using both a single-sided t-test (LaMorte, 2021) and a one-sample Wilcoxon Signed-rank Test (LaMorte, 2017). The Wilcoxon test can be used for comparing non-continuous data to a base value, such as Likert-scale questions (mangiafico). The t-test is usually deployed for continuous data, but because of the Central Limit Theory, it can also analyze Liker

With the diverse background of students that are present in the research and the number of participants, it allows this research to make claims about the general student body. However, due to there being a limited amount of students in each group, no claims can be made about those. For example, some students partake in an engineering study with a technical focus, such as computer science and electrical engineering. However, there are not enough participants to verify the results with a significance test. Because of the nationalities of the participants, the analysis done on the results applies to students based in Western-Europe and Norther-America. To make it easily accessible for all participants in all the different countries, the research was hosted online. The game is a web-based game that has been played in the browser. The survey has been hosted online as well using the Qualtrics platform/ More information on the survey and game is given in Sect. 2.3. Because of this, participants could partake in the study in any place where they had access to a laptop and active internet connection, without the active supervision of the researchers. In total, two surveys were created which ask the games before the participants played the game, the pre-game survey, and after playing the game.

During the research, any invalid submissions were removed from the data sample. There were three criteria on which data could be removed to not influence the past results. First off, every person was given a random ID to keep the data anonymous. This ID had to be filled in at the start of every survey and was generated from the online consent form of the research, and was the only thing that could link a participant to data. Only the participant had access to this ID but they had some characteristics which could see if any invalid IDs had been filled in, which indicates someone did not fill in the consent form. The second criterion was completion, any incomplete survey was removed. And finally, any pre-game survey that did not have an associating post-game survey with a matching anonymous ID was removed and vice versa.

Instruments used and their validation

This section discusses the way the game and survey questions were designed and how their quality was assured.

Game and survey design

The game is an adventure text-based game that falls into the interactive novel genre (Camingue et al., 2020; Soboleva et al., 2022). It can be found at:https://research-12312.itch.io/usb-based-attack-prevention. The reason for picking this type of game is to limit any non-story-related aspects that might affect the effectiveness of the game. It was built in Twine (Foundation, 2022) where it comprises passages, and individual pages that players can navigate between. In the game, each passage has at least 3 elements: an image, some text, and one or more buttons. The photos and text explain what is going on in the frame and are used to display dialogue. And finally, the buttons allow the users to interact with the game, make decisions and solve problems. Examples of what frames look like in-game can be seen in Appendix 2.

In the game, the story follows a trial structure (Meer, 2019). In this structure, as seen in Fig. 2, all players start at the same point, go down one story-line where they decide and overcome tasks, and fail or successfully reach the end. However, this game slightly deviates from this structure, as it has multiple endings. In the game, the player plays as a cyber-security investigator who needs to discover how a company got hit by a ransomware attack.

Fig. 2
figure 2

The trial structure for stories

Full size image

The story was designed using Thorndyke’s framework for creating simple stories (Armstrong & Landers, 2017; Ferguson et al., 2020; Thorndyke, 1977). Thorndyke’s framework has been proven to successfully improve recollection of information over a long period of time. The structure it describes gives the user control over different parts of learning and what information will be absorbed by a student. It was deemed the best option for this research as the goal of this training is to provide information and patterns for students to recognize in the future, and recall is an important aspect of that. The focus is on declarative knowledge instead of procedural knowledge. Thorndyke’s framework is divided up into different sections, called episodes. Each episode teaches the player about, and tests them on one specific element of USB-based attack prevention. This allows the player to focus on learning in small chunks before progressing. For example, in one episode, the player discovers the common tactics are that attackers used to get people to use infected USBs. An overview of the design can be seen in Fig. 3. Next to structures for designing stories, the literature review found a set of concepts that should be considered when implementing story-elements in gamification. First off, it is important to make sure the information the training provides is fully integrated into the story (Nicholson, 2014; O’Donovan et al., 2013). For example, in the game, the knowledge that people need to know to prevent USB-based attacks is the same knowledge they need to discover how the ransomware got onto the company’s computers. This integration is helped by making the story simulation-like and aiming for realism (Chothia et al., 2017; O’Donovan et al., 2013). This also applies to the tasks, which are endogenous and are narratively linked to the game (Prestopnik & Tang, 2015).

Fig. 3
figure 3

The structure of the game design

Full size image

For communicating information, both a “guide” and an embedded narrative were used (Nicholson, 2014). The user interacts with the guide and other characters through dialogue where characters talk with each other or with the player, and where the player can sometimes, respond. In the game, a broad spectrum of information is provided to fit the varied target group (Nicholson, 2014). Throughout the story, the game combines both enacted and emergent narratives (Nicholson, 2014). At the start, there are a few set sequences of interactions for exposition, like the cutscenes in a game, which is an enacted narrative. This is used to make sure all the users get the same basic information, such as what the mystery is they need to solve. Later in the game, the user gets to make decisions that influence the story and the narrative is emergent, it is built by the choices made by the player. Inside the story, there is a focus on declarative knowledge instead of procedural knowledge (Armstrong & Landers, 2017). Examples of the way exposition and dialogue are done in the game can be found in Appendix 2. Figure 10 shows what a frame looks like that provides exposition and information to a player, including descriptive text and buttons that allow a player to interact with the game. An example of dialogue can be found in Fig. 11. In this figure there is only one possible answer the player can give but there are also frames where multiple dialogue options can be chosen.

There are also differences between this game and previous work. Where previous work had a story-elements as one feature, this game puts emphasis on story elements. There is a difference in the research’s goal compared to previous research, where most other researchers looked at the effect of gamification and if it’s effective, or created frameworks for how to design gamified training. Meanwhile, this research focused on the effects of one specific element. The other research also mentioned improvement points that they could have implemented for story-elements in their games. So these were collected and considered for this game. And finally, the game focuses on the interactive novel genre of games, as explained earlier in this section.

Before the results of the research could be analyzed, the game and surveys had to be spread to participants. To make this easier to do, the game was developed as a web-based game. In total, two online surveys have been created. The first had to be filled in before the game is played, called the pre-game survey (link: https://qfreeaccountssjc1.az1.qualtrics.com/jfe/form/SV_bgh9wuCLVPV9K7k, and the second one after the game is played, the post-game survey (link: https://qfreeaccountssjc1.az1.qualtrics.com/jfe/form/SV_39rz7RJECQRYs5g. These surveys were analyzed with the use of qualitative analysis and significance tests.

The pre-game survey focuses on getting an idea of the knowledge that the participant already has on the topic. So, for example, they are asked what study they do. The post-game survey focuses on the perceived effectiveness of gamification. It does this by asking the participant what their experience was while playing the game. So, for example, they are asked to what extent the story elements in the game helped them feel engaged. This is done using a Likert scale of 5 points ranging from “completely disagree” to “completely agree”. Both surveys and their questions have been designed according to already developed frameworks (Högberg et al., 2019; Luo, 2021; Ros et al., 2020) for testing the effectiveness of gamification. These frameworks assure that derived survey questions will effectively measure how successfully gamified training has educated players. From the frameworks, we derived the variables as presented in Table 1 that were used to measure the effectiveness of the game.

Table 1 Overview of measures
Full size table

Data analysis

After the data was gathered, it was analyzed. The data was gathered in terms of Likert Scales with scores assigned to them so to analyze the results averages were given. Other researchers have investigated the best way to measure gamification effectiveness (Högberg et al., 2019; Ros et al., 2020). The measures found in those researches were adapted for this game and can be seen in Table 1. The original elements that were identified were as broad as possible, so instead of saying general cyber-security knowledge increase it discusses general knowledge increase in the target field for gamification.

These measures were then adapted into questions that could be answered using a Likert scale and a score was calculated as an average from the participants’ answers. Each answer has a score, as seen below, where a more positive score meant a more positive effect, and a more negative score a more negative effect. The numbers are chosen to represent the indicated effect and to have a neutral point to compare them to.

  • Completely disagree = − 2

  • Partly disagree = −  1

  • Neither disagree nor agree = 0

  • Partly agree = 1

  • Completely agree = 2

Using this system mentioned above, a score of 0 can be used as a baseline, which allowed a single-sided t-test (LaMorte, 2021), and a Wilcoxon Signed Rank Test (LaMorte, 2017), to be performed. A positive score means that the user perceived the story-elements to have a positive effect on the effectiveness, while a negative score means that a user experienced it to have a negative effect. If the significance test is passed, it means that the difference is significant and it can be used to discard the null-hypothesis, which, for example, is that story-elements have no impact on how engaged students were with the training. If the test is not passed, the difference is not significant and cannot be used to discard the null-hypothesis. The mean tha

The next section shows an overview of the results that were gathered using the surveys.

Results

In this section, the results of the surveys are shown.The survey studied different aspects of gamification. First off, the participants were asked to rate their overall experience with the game, the results of which can be seen in Fig. 4. They were also asked about the overall effect of the story, as seen in Fig. 5.

Fig. 4
figure 4

Perceived increase in USB-based attack knowledge

Full size image
Fig. 5
figure 5

Perceived effect of the story

Full size image

Not only did this research look at the overall effectiveness of the game and story, but also checked the effect in more detail using the measures in Table 1. For the responses, the number of the measure, questions that were generated from the measures, the averages and the results of the significance tests can be found in Table 2. With further analysis, two observations were made. First off, in the survey, participants were requested to compare this game to other gamification and cyber-security training. The results are found in Fig. 6. A link between study backgrounds and how participants rated the game was also found. As seen in Fig. 7, they were a wide variety of study backgrounds. The results are shown in Fig. 8.

Table 2 Overview of the average score per question
Full size table
Fig. 6
figure 6

Comparison of this game and other (gamified) pieces of training

Full size image
Fig. 7
figure 7

Pie chart demonstrating the distribution of studies among participants

Full size image
Fig. 8
figure 8

Overview of the difference in average scores

Full size image

The next section discusses the answers to the research questions drawn from the literature study and the survey.

Discussion

This section discusses the results found and what can be learned from them.

Discussion on Sub-research question 1 First off, the results of the literature review are used to answer sub-research question one. In this review, the following concepts were found to influence story-based gamification effectiveness.

  1. 1.

    Appropriate story-design framework The choice of story-design framework should align with the desired learning outcomes, as different frameworks impact students’ experiences differently. This is emphasized by Thorndyke (1977).

  2. 2.

    Narrative-instruction connection A strong connection between the narrative and instructional content is important for effective learning. Thorndyke (1977) underscores this relationship, for example that exogenous and endogenous fantasy can play a significant role in this connection.

  3. 3.

    Influence of narrative type The chosen narrative type affects how students engage with the content. Nicholson (2014) highlights its impact on the balance between analogy and realism, which affects information absorption and a students’ perception.

  4. 4.

    Balancing analogy and realism Prestopnik and Tang (2015) emphasizes the need to get a good balance between analogy and realism within the story. This balance contributes to participant engagement and linking content to real-world contexts.

  5. 5.

    Tailoring information for the audience Aligning presented information with the audience is vital. Armstrong and Landers (2017) emphasizes this alignment’s positive influence on declarative knowledge intake.

  6. 6.

    Integration of information and actions Integration of information with in-game actions improves the learning experience. For example Prestopnik and Tang (2015) discusses the impact of choosing Endogenous fantasy versus choosing exogenous fantasy.

  7. 7.

    Agency vs. directed sequences Achieving a balance between user agency and directed sequences for conveying information is important. Research by Chothia et al. (2017) highlights the value of user autonomy while ensuring that essential information communicated.

One thing that has become clear from the literature review is that balancing the concepts in a way that works for all story-based gamification is impossible. For example, balancing analogy and realism is something that differs per game (Nicholson, 2014). This also means there are concepts that were not present in the literature review, but that still might influence effectiveness.

Discussion on sub-research question 2 This paragraph discusses how the concepts mentioned in the list above were implemented as follows to answer sub-research question 2. A more detailed explanation was given in Sect. 2.3. For the design of the story (concept 1), Thorndyke’s structure was chosen because it improves the recalling of story elements by readers (Thorndyke, 1977). The story tries to simulate what could happen in the real world and makes little use of analogies (concept 4). This is so the users can more easily recall and use the information in real-world situations (Nicholson, 2014; Prestopnik & Tang, 2015). The information provided to the participants (concept 5) is mostly declarative knowledge and is as broad as possible (Armstrong & Landers, 2017) so that all the students from different backgrounds can use it to protect themselves. It explains how students can avoid the attacks from being successful but does not go into how the attacks are built and work on a technical level. Because of the simulation-like nature, all the tasks in the game are directly related to the narrative (Prestopnik & Tang, 2015) and influence the story (concept 6). On top of this, it also helps make the training information more integrated in the story (concept 2) so that users learn while exploring the story. In the game, the user’s agency was limited on purpose (concept 7) because giving the user a sandbox experience might cause them to miss information (Nicholson, 2014). Hence, enacted and emergent narratives were used (Nicholson, 2014) (concept 3).

With an increase in the usage of computers and technology in education through, for example, gamification, there are also questions being raised surrounding how this affects the well-being of users and students (Melo et al., 2020). First off, it was observed that the gamification successfully helped to increase people’s knowledge of USB-based attacks. This means that they are more aware of the types of attacks and can pass this knowledge along. This suggests an increase in social well-being (Luhmann et al., 2021) as now people can help to prevent this type of attack and they can pass this knowledge along, making others more aware, making them less attractive to attackers. It was stated that the story-elements and gamification were engaging, making the game more attractive to play, however this can lean into aspects of addiction such as focused immersion and heightened enjoyment (Melo et al., 2020). However, one thing that this game does not do is make the game highly replayable through things like high-scores and leader-boards. The game does not keep on drawing people by asking them to improve or with notifications; it has low re-playability. Participants also indicated that it was fun to play the game and learn about the topic through it, meaning it adds to the positive state of mind. Because of the challenges in the game, it also offers growth to participants by allowing them to overcome challenges.

Discussion on sub-research question 3 Sub-research question 3 was answered using the results from the surveys. First off, the participants were asked to rate their overall experience with the game. As seen in Fig. 4, participants found the game was successful at increasing their knowledge of USB-based attacks. Next to this, they also indicated that the story elements had a positive effect on their experience and helped them with acquiring knowledge, as seen in Fig. 5. So, overall it can be said that the game and story are effective.

Not only did this research want to look at the overall effectiveness of the game and story, but also wanted to check the effect in more detail, which was done with the elements in Table 1. As seen in Table 2, in all the measures of effectiveness, a positive effect can be noticed as the averages are positive. The story successfully helped to enhance the game. To test significance of all measures of the game’s effectiveness, a W-value was calculated for the Wilcoxon test and a t-statistic was calculated for the t-test. For all the measures, the w-values calculated from the scores are below 120 and the t-score values calculated from the scores are above 1.75. The reference numbers were taken from the respective tables with degrees of freedom being 27, the number of participants − 1, and Alpha being 0.05 (Guthrie, 2020; LaMorte, 2017). So, according to the data above, the game is effective enough to teach participants about USB-based attacks and raise their level of cyber-security knowledge. Next to this, the story elements were perceived to be effective in both engaging the player and making the game more effective at teaching, to where it surpasses the commonly used videos. Hence, the answer to sub-research question 3 is that the game was perceived to be efficient enough to raise people’s knowledge and make them more resilient against these types of attacks.

On top of this, there were also 2 observations that could not be verified and used to answer the research question, however, the observations still seemed of interest to the authors. First off, in the survey, participants were also requested to compare this game to other gamification and cyber-security training. The results of this can be found in Fig. 6. Initially looking at the data, it can be said that this type of game is more effective than other training and gamification. However, this link could not be verified because of the limited amount of people that could make the comparison, causing a lack of power in the t-test and Wilcoxon. Further analysis of the results found a link between study backgrounds and how participants rated the game. As seen in Fig. 7, they were a wide variety of study backgrounds. The results are shown in Fig. 8. As seen, non-technical students overall rate the game higher on all elements except for engagement. However, this link could also not be verified with a significance test. At the end of the post-game survey, the participants were also asked if they believed any improvements could be made to the game. Most participants did not give feedback, but a few did and their suggestions were implemented. The game was extended with more choices and more information on USB-based attacks, and it was made easier to navigate, so that exploration of the different story paths was easier.

Discussion on the main-research question A set of concepts and elements were identified in previous research for sub-research question one. These concepts provide guidelines for what concepts influence the story and should be thought about when designing a story-based game. These concepts were then used to build a story-based adventure game that teaches players how to avoid and recognize USB-based attacks. Finally, it was found that the game is effective at teaching people about USB-based attacks and that the story elements help improve the player’s experience. So overall, for the main research question, it can be said that story-based gamification is perceived to be effective enough to help students protect themselves against USB-based attacks successfully and that the story-elements enhance the game itself and makes it more effective.

The next and final section discusses the conclusion, limitations, and future work gathered from this research.

Conclusion, limitations, and future research

As there is both an increase in USB-based attacks and the use of gamification, it is important to understand the perceived effectiveness of story-driven gamification in USB-based attack prevention. This section goes into the conclusions that can be drawn from this research and what future work still can and should be done.

By doing a literature review, concepts were found that can influence the effectiveness of story-driven gamification. These range from choosing the correct level of realism, to the tasks and narrative types that should be in the game. These were then implemented to empirically test the effectiveness of story-driven gamification. Designing this game gave the insight that implementing the concepts should be adjusted to the goals of the training one wants to gamify. The empirical tests done with the game and students found that the game was perceived to be effective at teaching students about USB-based attacks and that the story-elements improved the game. Finally, by combining the answers for all the sub-research questions, the overarching research question can be answered. Story-driven gamification was perceived to have a positive impact on USB-based cyber-attack prevention when tested among students.

It was observed that the results of the research add significantly to the field in the following ways. What could be seen in current research is that story-elements often take a backseat to more popular gamification elements. They are an element in the game that is just there to engage the player and to give a reason for events. But this research shows that story on its own can facilitate learning. It can go beyond just making the game more interesting and can help students to get a further understanding about, for example, a cyber-security attack and give context. Making a game with a story at its center was said to be effective at helping the user with understanding USB-based attacks and helped them understand how to protect themselves. Beyond this, it was said that the story helped to ground the scenario in the real world and confronted the user with a realistic scenario, helping them to go beyond just knowing how the attacks happen. It allowed users to understand how an attack might come to be successful and to help recognize potentially dangerous scenarios. Where it also goes beyond other modern research is that the others mostly state that the story was present or that a “good” story-script was written, but it lacks details on the decisions that were made and what details to seek to make the story as effective as possible. Discussions on this can be found in older papers, but most do not do primary research with it and focus on secondary research. It advances on the theory that stories are effective by offering a more firm grip on how they affect gamification and can be deployed in education. Story-driven games can greatly enhance gamification among students when they are one of the main foci of educational computer systems and this research offers insight into how to design them around the goal of a training or topic. The results from this study can be used to further the theory of using gamification in education. It allows designers of educational computer systems to more effectively implement story-elements and improve the effectiveness of their work. The work made suggestions and pointers on the way story-elements should be incorporated in the design of a gamified training or course and to make sure it aids in its goal. Depending on what the course is trying to teach and what type of information is being shared, the way story-elements should be designed and implemented also changes.

There were also limitations to this research. First off, there was a lack of recent research with sufficient focus on story-elements in gamification. So all the concepts provided by this review predate the year 2020. In the meantime, new concepts might have come forward and as well as new ways to conceive how to deploy gamification. All these things might influence how the concepts found might be implemented to be more effective. Second, though the study looks at a diverse group of students, a majority of students still had a technical background. Because of this, the claims made about the differences in results were not verifiable and the effect of story-elements could be different among different groups of students. The study also just focuses on students, so the results can be used to only apply to this group as the situation might be different for younger people still going to school or working adults who would do such training for work or personal reasons. However, the significance test, it can be said that the results fit this group and so can be used to make claims about students, but it cannot be verified if they might differ between groups. The final restriction can be found in how the study was performed. Participants took the surveys and played the game at home without supervision, this means that the results were limited to the answers in the survey. If the participants would have been observed while playing the game in a controlled environment, there might have been observations that could have been made such as how the player interacts with the game and how quickly they played through the different sections of the game. However, this could also influence students to behave differently and influence the results. Further, the results are based on a self-report survey. Participants report on the effectiveness of the game based on their own feelings and how much they believed they learned, instead of an achievement test. This can introduce some subjectivity into the results but this is partly removed with the use of significance testing. The final point that the authors wish to address is that story-elements can never be fully isolated by students from other gamification aspects. There can always be other factors in the game or learning experiences that influence the students’ experience.

Future research into story-driven gamification should focus on creating a more concise framework for implementing stories in gamification for cyber-security. The focus of this paper is limited to one cyber-security topic and how stories should be implemented for other topics might be different. The results for the comparison to other games and the difference between students from technical and non-technical studies could not be verified with significance testing, and so should be further researched. There are also limitations with how the measurements have been done. And in the end, the field of cyber-security is constantly changing and updating. As the field evolves, so should the research into how people are educated.