Guide to developing case-based attack scenarios and establishing defense strategies for cybersecurity exercise in ICS environment

Abstract

Critical infrastructure mainly performs its role through an industrial control system (ICS). Organizations conduct cyber exercises between red and blue teams, focusing on offense and defense. Practical exercises require explicit attack scenarios and corresponding defense strategies. However, systematic guides for deriving cyberattack scenarios or defense strategies still need to be improved. This paper proposes a guide for establishing realistic attack scenarios and defense strategies for cybersecurity exercises in ICS environments. Attack scenario generation is divided into four steps: generating attack references, deriving attack sequences, mapping threat information, and mapping vulnerable implementation patterns. Deriving a defensive strategy consists of two steps parallel to developing an attack scenario: deriving containment and eradication. The methodology we propose guides exercise planning based on a knowledge base, thereby assisting exercise planners in generating various scenarios and deriving clear defense strategies. We showed that a clear exercise plan could be established through a case study.

1 Introduction

Cyber incident response exercises allow organization members to practice responding to security threats and incidents to respond to actual incidents effectively. Such exercise is part of an organization’s risk management to minimize potential financial loss, reputational damage, and legal issues when an actual cyberattack occurs by detecting the attack early, preventing its propagation, and quickly eliminating the threat. In addition, when new security technology is introduced, organizational members can quickly adapt to the change and master the new technology. Above all, by periodically conducting cyber incident response exercises, we can check the organization’s current security policy and level.

Cyber incident response exercises are essential in organizations that require high levels of security, such as critical infrastructure. Because infrastructure provides essential functions to society, such as energy, transportation, and communications, disruption of operations due to a cyber incident can have significant economic and social impacts. Each infrastructure facility often adopts an industrial control system (ICS) to perform its role. Furthermore, organizations operating security-related facilities frequently conduct adversarial simulation exercises between the so-called red team, which carries out attacks, and the blue team, responsible for defense. For simulated exercise to be carried out effectively, explicit attack scenarios similar to actual events and an exercise field where scenarios can be realized are required. In addition, as the attack progresses, the defender’s strategy should be systematically derived. This study focuses on establishing concrete attack scenarios and corresponding defense strategies for cyber exercise.

As mentioned earlier, a cyber incident response exercise is essential in running an organization, but conducting it systematically and effectively is very difficult. To this end, several government agencies present guidelines for the exercise life cycle [1,2,3]. Researchers also develop scenarios or evaluation indicators to conduct practical exercises [4,5,6]. Additionally, these studies often introduce well-known threat modeling frameworks or knowledge bases to establish exercise objectives specifically and specify specific threats [7,8,9].

Although there is a shared awareness of the need for realistic attack scenarios and defense strategies comparable to actual cyber incidents, there are several challenges. First, attack scenarios used in exercise should describe adversarial activities in a flow similar to actual events. If there is no proper guide or method to derive attack sequences, unrealistic scenarios may be derived and ultimately lead to meaningless exercise; Second, even if the sequence that constitutes the attack scenario is realistically derived, specific threat information is needed for each stage for the red team to implement it. Threat information includes details such as attack targets, attack techniques, and vulnerabilities, but consideration is needed to derive related information consistently; Third, even if a precise attack scenario has been created, an exercise field for this scenario to be realized is required. In other words, it may be necessary to implement a system or network that intentionally exposes vulnerabilities or reduces security levels to facilitate exercise. Therefore, in addition to threat information, vulnerable source code patterns for exercise field implementation should be considered in planning; Lastly, explicit attack scenarios and corresponding defense strategies should be considered for adequate exercise. In particular, strategies to respond to threats posed at each stage of the attack sequence should be accurately linked.

Our insights to tackle the identified challenges point-by-point are as follows. First, we analyze existing ICS attack cases and derive realistic attack sequences; Second, a well-known threat knowledge bases are introduced to assign specific threat information to each attack stage; Third, we use secure coding standards to search for vulnerable source code patterns associated with the threat information assigned to each attack stage; Finally, an accurate response strategy corresponding to the attack stage is derived through the knowledge base from the defender’s perspective. Standardized attack methods or response strategies can be explored by employing known threat modeling frameworks or knowledge bases, but a method for accurately linking them is required.

This paper proposes a guide for establishing realistic attack scenarios and defense strategies for cybersecurity exercises in ICS environments. In particular, this study focuses on an adversarial simulation exercise in which the red and blue teams interact as a cybersecurity exercise. In other words, it is assumed that the red team follows the attack scenario and infiltrates the system implemented at the exercise field, and the blue team performs an appropriate response as the attack progresses. Therefore, the proposed guide is largely divided into attack scenario generation and defense strategy derivation. Attack scenario generation is further divided into four steps: generating attack references, deriving attack sequence, mapping threat information, and mapping vulnerable implementation patterns. The step generating attack reference analyzes cases reported in the existing ICS environment and creates an attack reference expressed based on MITRE adversarial tactics, techniques, and common knowledge (ATT &CK) [7]; The sequence derivation step extracts the attack flow to be used for exercise from the attack reference; The threat information mapping step connects common attack pattern enumeration and classification (CAPEC) [9], common weakness enumeration (CWE) [10], and common vulnerabilities and exposures (CVE) [11]; The step for mapping vulnerable implementation patterns explores vulnerable source code patterns from secure code standards for implementing an exercise environment or system in which attack scenarios can easily be realized at each attack state. Next, deriving a response strategy consists of two steps in parallel with developing an attack scenario: deriving containment and eradication. The deriving a response strategy aims to specify mitigations for each attack stage using information from the attack scenario development process. First, the containment derivation step utilizes the information generated in the attack sequence derivation step and the MITRE detection, denial, and disruption framework empowering network defense (D3FEND) [8] as base information. In this step, when a cyberattack occurs, it is immediately detected and a strategy is drawn to prevent the attack site from propagating or spreading; Second, the eradication derivation step uses information generated by previous threat information mapping and vulnerable implementation patterns and secure coding standards as materials. This step leads to a method to completely eliminate attacks whose spread has been stopped by the containment strategy. Following this procedure, we can create realistic attack scenarios with specific threat information and response strategies for each stage. We also provide directions for searching for accurate and consistent information from the knowledge base. We conduct a case study with three well-known cyberattack cases (Stuxnet, Maroochy, and Blackenergy3). In the case study, we show the outputs along with our guide, which is utilized for cybersecurity exercise. The contributions of this paper are three folds as follows:

• We present a guide for generating a concrete attack scenario for the red team and establishing a response strategy for the blue team in an adversarial simulation exercise.

• We leverage well-known knowledge bases in developing attack scenarios and response strategies, and present methods to extract accurate and consistent threat information and mitigations from these.

• We present a result of case study on three well-known attack cases and produce an attack scenario and corresponding defensive strategies.

The remainder of this paper is organized as follows. Section 2 introduces the knowledge bases needed to describe the proposal, guidelines, and existing studies related to cyber exercise; Sect. 3 describes a step-by-step guide for deriving the attack scenarios and response strategies proposed in this paper; Sect. 4 performs a case study to generate attack scenarios and response strategies following the proposed guide and confirms their effectiveness; Sect. 5 discusses possible uses for the proposal and some limitations; Finally, we summarize this paper with conclusions in Sect. 6.

2 Background and related work

This section provides background regarding cyber exercise (Sect. 2.1) and some existing works (Sect. 2.2). For background, we describe several guidelines for exercise and knowledge bases related to threat modeling. Also, we present several studies to derive cyberattack scenarios and defense strategies against them for related work.

2.1 Background

2.1.1 Cyber exercise guidelines

Cybersecurity exercise strengthens the security posture of the entire organization by ensuring that employees are aware of the latest cyber threats and learn how to respond to them. This reduces the risk of potential security incidents and fosters the ability to respond quickly and effectively in the event of a security incident. For this purpose, various cyber exercise guidelines have been proposed. These guidelines help organizations design and implement systematic and effective exercise programs.

“Homeland Security Exercise and Evaluation Program (HSEEP)" [1] is a comprehensive set of exercise guidelines consisting of a set of standardized policies, methodologies, and terminology developed by the U.S. Department of Homeland Security (DHS). HSEEP provides an approach to exercise in general, not just cybersecurity exercise, that encourages exercise to be flexible and accessible while maintaining consistency in exercise delivery and evaluation. HSEEP broadly divides the exercise cycle into three stages: exercise design and development, exercise conduct, and exercise evaluation. Exercise design and development stages are design (training purpose, scope, objectives, evaluation parameters, scenario, and documentation) and development (planning for discussion-based and operations-based logistics, facilitation, control, exercise evaluation, and exercise conduct) process. The exercise conduct stage includes a series of activities such as preparing for exercise play, managing exercise play, and conducting immediate exercise wrap-up; The exercise evaluation stage evaluates all activities (exercise planning, data collected during exercise, and reports) from the previous stage. Above all, HSEEP specifies the responsibilities of each participant’s role in the training at each stage.

HSEEP is a general guideline for exercise issued by a governmental agency in the United States. At the same time, the “Good Practice Guide on National Exercise" [2] is a guideline for European member countries. This document provides best practices to help Member State authorities better understand the complexities of exercise and prepare for regional and national exercise. This guide has been compiled through interviews with experts on exercise across the European Union (EU) and beyond to identify best practices that are already in place and have proven effective. This document also emphasizes planning and performance for effective exercise and presents a life cycle of four stages: identifying, planning, conducting, and evaluating. In the Identifying stage, the organizer identifies exercise needs and includes identifying procedures or actions that should be implemented. Depending on these needs, the organizer selects the type of exercise and the organization to be involved in; The planning stage includes recruiting participants and securing sufficient resources for exercise. Additionally, locations, scenarios, rules, and tools for training should be selected. The organizer also determines who will monitor the exercise, their roles, and evaluation methods; The Conducting stage includes activities for actually carrying out the exercise. In addition, throughout the exercise, it is checked that participants behave according to the planned process and that the scenarios are followed; The Evaluating stage reviews the exercise conducted to identify the organization’s weaknesses and derive measures to overcome them.

“Cyber Exercise Playbook" [3] published by MITRE corporation is another popular guideline for cyber exercise. This document defines terms used in cyber exercise and the life cycle from inception to reporting. Like HSEEP, the playbook divides the exercise life cycle into three stages and presents activities and products for each stage: exercise planning, execution, and post-exercise. The exercise planning stage is an important process that determines the direction of the entire exercise and includes activities that determine the objectives, outputs, and types of exercise. In addition, general exercise objectives and sample scenarios are provided to help you develop a plan; The exercise execution stage monitors audience response through designated observers and performs exercise according to the plan established in the previous stage; The post-exercise evaluates the effectiveness and collects feedback from participants after the exercise is completed. The participants share lessons, accumulate experiences, and reflect on them in the next exercise.

In addition, various guidelines [12, 13] for effective cyber exercise are provided. These documents commonly emphasize planning, conducting, and evaluating training. Scenario development is mostly an activity belonging to the planning stage. However, while these guidelines emphasize the importance of scenarios, they do not provide specific directions for writing quality scenarios.

2.1.2 Knowledge bases regarding threat modeling

Threat modeling frameworks or threat knowledge bases help organizations identify, understand, and effectively respond to potential cyber threats. These tools are essential to strengthening one’s security strategy and systematically managing cybersecurity risks. Although various threat knowledge bases have been proposed and have currently been studied, we enumerate only those required to describe the proposed guide in this section.

MITRE ATT &CK [7] is a knowledge-based framework encompassing attackers’ tactics, techniques, and procedures (TTP), helping security experts understand threats and develop response strategies. This framework is divided into enterprise (14 tactics, 201 techniques, and 424 sub-techniques), mobile (14 tactics, 72 techniques, and 42 sub-techniques), and ICS (12 tactics and 81 techniques) depending on the modeling environment. It is designed to specifically identify and respond to cyber threats suited to the characteristics of each environment. The Enterprise area provides information on attack tactics and techniques that can be carried out against a typical corporate IT environment. So, the enterprise assets we describe are commonly used assets like servers, desktops, and Windows OS. The ICS domain provides information on attack tactics and techniques specific to industrial environments such as energy and manufacturing. That is why they represent assets used in industrial environments, such as supervisory control and data acquisition (SCADA) systems, Programmable logic controller (PLCs), human-machine interfaces (HMIs), and remote terminal units (RTUs), as targets for attacks. The mobile area provides the same attack tactics as the enterprise area and provides information specialized for mobile devices such as smartphones in attack techniques. In the mobile area, assets described as attack targets target mobile operating systems such as Android and iOS. In addition, it provides various cyber threat intelligence on adversaries, malware, and campaigns that caused existing cyber incidents and is used as a standard dataset in many studies [14, 15]. Our goal is to establish attack scenarios for cyber training of critical infrastructure with ICS environments. To achieve this, we utilize attack tactics and techniques from the Enterprise and ICS domains of ATT &CK during the exercise planning process. The Purdue model, which represents a typical ICS environment, divides the ICS environment into IT and OT areas. Additionally, as IT technology has recently been incorporated into the ICS area, it can be used in combination with ATT &CK enterprise attack tactics and techniques in the IT and OT areas of ICS. That is why we try to use a mix of both areas of ATT &CK to achieve our goals.

MITRE D3FEND [8] is a framework that organizes and describes cyber defense technologies, strategies, and methodologies, helping security experts understand and apply defense mechanisms. D3FEND classifies each tactic into seven distinct categories and provides a comprehensive set of 679 defense techniques. This framework categorizes and describes defense technologies, providing essential guidance for organizations to design and strengthen their security architecture. Additionally, this framework is compatible with MITRE ATT &CK framework and presents defense strategies suitable for countering specific attack techniques expressed in the MITRE ATT &CK framework. By leveraging both frameworks, exercise planners can develop scenarios that provide insights into offensive techniques and tactics while also imparting an understanding of defensive strategies and techniques. Furthermore, our methodology outlines the application of distinct D3FEND tactics, explicitly focusing on detection and response.

CAPEC [9] is a knowledge-based framework that systematically enumerates and classifies cyberattack patterns. It provides detailed information to understand and respond to security threats, including the various TTPs used by attackers. CAPEC broadly divides cyberattacks according to mechanism and domain and supports a hierarchical view. Attacks are specified from high-level categories into meta attack patterns, standard attack patterns, and detailed attack patterns. The framework also suggests other attack patterns, attack flows, prerequisites, required skills, consequences, and mitigations associated with a specific attack pattern. In addition, MITRE ATT &CK’s taxonomy and CWE’s security weaknesses for specific attack patterns are mapped in conjunction with other knowledge bases.

CWE [10] is an international standard that systematically lists and classifies security weaknesses in software and hardware. This framework describes common types, structures, and causes of security weaknesses. A unique CWE ID identifies each security weakness and information such as a detailed description, source code-level examples, impact, and response strategy are provided. Similar to CAPEC, security weaknesses are classified into class, base, and variant, starting from high-level categories. CWE publishes the top 10 dominant security weaknesses each year, helping security experts assess system weaknesses and prioritize risks.

CVE [11] is a framework that assigns unique identification numbers to publicly known computer security vulnerabilities and provides information about those vulnerabilities. Each CVE item has a unique identification number, CVE ID, allowing one to find and reference information about a specific vulnerability easily. A CVE entry includes comprehensive details on an overview of the vulnerability, affected products, its impact, patches, and countermeasures. In the past, exploit code was often provided to prove vulnerabilities, so-called proof-of-concept (PoC). However, an overview of vulnerabilities is currently offered due to the potential for exploitation by adversaries.

Finally, we briefly mention the secure coding standards. Secure coding standards provide safe coding practices and principles to prevent vulnerabilities and enhance software security when implementing computer systems. Secure coding practices like input validation and application programming interface (API) utilization are critical to software development. These methodologies enhance developers’ comprehension of security and vulnerability management, ultimately contributing to the mitigation of cyberattacks. Adversaries often carry cyberattacks that exploit source code’s lack of secure coding practices. Defenders can effectively counteract such attacks by promptly identifying and patching insecure code segments. In our study, we implement software without secure coding for exercise and guide exercise participants to perform patches by implementing secure coding. Depending on the programming languages, secure coding standards are prepared in various ways [16,17,18].

The knowledge bases on the threats listed in this chapter have sufficient value independently and are useful to security experts. However, these knowledge bases currently support interoperability, providing related information together. For example, when CAPEC searches for a specific attack pattern, it provides the security weaknesses of MITRE ATT &CK’s TTPs and CWE related. Based on this information, we can search for related CVE vulnerabilities. Additionally, we can map out a defense strategy to counter this from MITRE D3FEND through MITRE ATT &CK’s TTP.

2.2 Related work

2.2.1 Cybersecurity training methods

Various studies have been presented for practical cyber exercise activities. Song et al. [4] proposed GENICS, a formal framework for generating cyberattack scenarios. GENICS creates attack scenarios in five major steps. This framework identifies all assets in the environment where the exercise is performed, maps threat information to the assets, models attack paths via an attack tree, quantifies them through a common vulnerability scoring system (CVSS), and creates cyberattack scenarios. GENICS not only expresses various possible attack paths through an attack tree but also allows the creation of high-impact scenarios through quantification. Yamin et al. [5] proposed an improved method for cybersecurity training using Cyber Ranges [19]. Cyber Ranges are a simulation environment designed to develop, test, and train cybersecurity skills. This study pointed out inefficiencies caused by the complexity of the training process and the problem that the training infrastructure needs to reflect the dynamic environment of the actual system. This study tried to automate many tasks in the training life cycle to solve this. As a result, they could quickly build a training environment with sufficient complexity. Nakata et al. [20, 21] proposed CyExec$*$, creating a virtual environment with various attack scenarios during cybersecurity training using Cyber Ranges. The study pointed out the problems with using a single scenario multiple times in training. CyExec* adopted scenario randomization based on the directed acyclic graph (DAG) to solve this. In addition, CyExec* was designed based on Docker [22], so it used resources more efficiently than other virtual machine (VM)-based systems and could create complex scenarios. Angafor et al. [6] emphasize the importance of tabletop exercises (TTX) for training cybersecurity incident response teams. This study reviewed TTX-related literature and analyzed strengths and weaknesses in training existing incident response teams. The results of this study showed that TTX can improve the level of awareness, understanding, and preparedness of incident response teams and enhance decision-making processes when an actual incident occurs. Mases et al. [23] derived key evaluation indicators to evaluate training organizations regardless of training type or goal through existing cyber incident response training examples. This study analyzed past literature to identify a 10-step training organization process and emphasized the roles of participants in each step. Additionally, the identified indicators were used as a checklist for organizational training. Andreolini et al. [24] proposed a framework for evaluating trainees’ activities in Cyber Ranges-based training. More specifically, this study includes a monitoring system to collect information about trainees’ activities, modeling trainees’ activities based on a directed graph, and a scoring system. As a result, the framework identified how quickly trainees achieved their objectives and what misbehavior they performed.

Research on existing cybersecurity movements aims to improve their effectiveness. Research related to exercise planning has been conducted to generate high-impact and diverse scenarios. Existing studies establishing exercise scenarios have yet to discuss implementation methods that can make exercise feasible Research related to conducting exercise proposed a realistic and efficient way to build an exercise environment, and research related to evaluating exercise proposed derived exercise evaluation indicators and an exercise scoring framework. Existing research on exercise environment implementation and exercise evaluation needs to discuss research that creates highly realistic scenarios. Our research guides how to utilize a knowledge base to derive scenarios for exercise systematically and how to support their implementation.

2.2.2 Strategic approaches to cyber defense

Various studies have been presented to define effective defense strategies. Zahra et al. [25] propose the ICS threat-hunting framework (ICS-THF), a threat-hunting solution that integrates ICS’s IT and OT networks. As IT technology continues to be incorporated into ICS, new types of attacks emerge, and attack mechanisms change. Previously, methods for threat-hunting targeting ICS have been proposed, but it is challenging to perform threat hunting in the new ICS environment. In ICS-THF, they solve the problem by combining the diamond model and the MITRE ATT &CK framework. They ultimately used a defense strategy of updating security devices by creating indicators of compromise (IoC) by combining the diamond model and MITRE ATT &CK framework. Additionally, this study validates the proposed solution ICS-THF over Black energy3, PLC-Blaster, and SWAT datasets. Serra et al. [26] developed a game theory model based on Pareto optimality to derive an effective defense strategy under constrained conditions. This study derived a defense strategy through vulnerability patching and product deactivation when the cost required for incident response is limited and minimum productivity must be guaranteed. Additionally, attack scenarios are generated through vulnerability dependency graph (VDG), and frameworks such as CVSS are used to derive vulnerabilities that maximize the impact of attacks. The findings of this study enable defenders to formulate a defense strategy by considering the company’s productivity, time constraints, and costs. Patzer et al. [27] proposed a solution aiming at automating incident response in an ICS environment utilizing software-defined networking (SDN). This study identified the problem that existing solutions targeting ICS environments need to adequately consider critical characteristics of ICS, such as time-sensitivity, redundancy, and availability. This study solved the problem by defining rules for incident response in advance and classifying ICS assets to automate incident response. As a result, the SDN-IR solution was able to perform incident response procedures such as containment, detection, and analysis without compromising the critical characteristics of the ICS. Rullo et al. [28] proposed Kalis 2.0 based on security-as-a-service (SECaas), which performs context-aware intrusion detection in Internet of Things (IoT) networks. This study pointed out the heterogeneity caused by the IoT network composed of various types of devices and the difficulty of intrusion detection due to continuous changes in the IoT network. This study tried to evaluate the risk by performing feature discovery when changes in the IoT network were detected or at regular intervals. In addition, to solve problems caused by heterogeneity, the threats of IoT devices are divided into four types through the open web application security project (OWASP). Information such as device name, protocol, and port is provided through open repositories such as CVE. As a result, this study was able to derive threat information that may occur in IoT networks and develop an appropriate detection strategy.

Research on deriving existing defense strategies has proposed methods for deriving defense strategies by considering specific conditions. These conditions included availability, minimizing incident response costs, ensuring productivity, and unique network environments such as ICS and IoT environments. However, existing studies on deriving defense strategies need to discuss deriving defense strategies according to explicit attack scenarios for exercise. Our study guides leverage a knowledge base and link it to exercise scenarios to derive systematic defense strategies.

3 Development case-based attack scenarios and defense strategies in ICS environment

This section proposes a method for developing case-based attack scenarios and deriving defense strategies for cybersecurity exercises in ICS environments. As previously mentioned, this study assumes an adversarial simulation exercise in which the red and blue teams interact. Therefore, this study aims to develop an attack scenario in which adversarial activities and threat intelligence are specified explicitly for the red team (Sect. 3.1) and to derive a defense strategy to accurately respond to adversarial activities for the blue team (Sect. 3.2). In addition, when implementing an exercise field to realize an attack scenario, the research scope includes the search for required vulnerable implementation patterns.

Fig. 1

Overview of the proposed method

Figure 1 is an overview of the framework proposed in this paper. This framework is largely divided into cyberattack scenario generation (upper side in Fig. 1) and defense strategy establishment (lower side in Fig. 1). Cyberattack scenario generation involves four major steps: attack reference generation (A1), attack sequence extraction (A2), threat intelligence mapping (A3), and vulnerable-implementation-pattern mapping (A4). Attack reference generation (Sect. 3.1.1) derives attack reference by analyzing existing cyber incident cases based on the MITRE ATT &CK framework. The attack reference means expressing the attack flow of several incident cases on a single diagram; Attack sequence extraction (Sect. 3.1.2) draws out a single attack flow to be used for exercise from the attack reference. We extract our attack flow based on the attack graph. The subsequent process is based on the attack sequence extracted in this step; Threat intelligence mapping (Sect. 3.1.3) connects specific threat information to each stage of the extracted attack sequence. In other words, a security weakness or vulnerability is assigned to each attack stage; Vulnerable-implementation-pattern mapping (Sect. 3.1.4) maps vulnerable implementation patterns so that each attack stage in the attack sequence can be easily performed when implementing an exercise environment. In other words, this is the stage of collecting the information required when implementing an environment with an intentionally low-security level for easy exercise.

Establishing a defense strategy proceeds with the cyberattack scenario generation and consists of two major steps: containment derivation (D1) and eradication derivation (D2). Containment is an initial action to prevent an identified attack or its effects from spreading. Containment derivation (Sect. 3.2.1) assigns means to prevent the attack from spreading to each stage of the attack sequence generated in step A2 through MITRE D3FEND; Eradication is to eliminate the cause of an attack or accident and take action to prevent it from recurring. In other words, eradication derivation (Sect. 3.2.2) derives an appropriate patching method for the security vulnerability used to realize the attack scenario and utilizes secure coding standards. The remainder of this section describes the detailed process of generating a cyberattack and establishing a defense strategy.

3.1 Cyberattack scenario generation

This section describes the cyberattack scenario generation process in detail. More specifically, it presents the tasks performed at each step from A1 to A4 and the outputs. The meaning of the symbols in the pictures that appear from here on are specified in the Appendix B.

3.1.1 A1: Attack reference generation

This step begins with analyzing past cyber incidents in the ICS environment. Stuxnet [29], which attacked Iran’s nuclear facility in 2010, and Blackenergy 3 [30], which attacked the Ukraine power grid in 2015, are representative incidents in the ICS environment. Stuxnet was the first cyberattack to prove the possibility of an attack by targeting the ICS environment. It was evaluated as one of the most sophisticated cyberattacks in the world at the time [29, 31,32,33]. Understanding attack techniques such as air-gap overcoming techniques and PLC malware injection used in the attack case can provide insight into cyberattacks targeting the ICS environment. Black Energy3 is a large-scale attack targeting the ICS environment, and an example of how ICS-targeted cyberattacks can impact people’s lives [30, 34]. In this case, an attack was carried out from the IT area to field devices in the OT area in the Purdue model, which represents the ICS environment’s structure. BlackEnergy3 provides insight into realistic attack sequences from cyberattacks targeting ICS environments. Most attacks on the ICS environment are characterized by extremely sophisticated attack flows. Therefore, it is very difficult to standardize multiple cases without an appropriate attack library or knowledge base.

We adopt the MITRE ATT &CK framework to tackle this issue. In summary, we analyze attack flows from arbitrary existing attack cases. Then, with MITRE ATT &CK framework, the tactic corresponding to each stage of this flow is identified, and an adversarial technique is further assigned. The attack progress can be expressed by connecting all sequences in the MITRE ATT &CK framework. Now, the above operations are performed for various ICS attack scenarios, and the results are expressed in a single drawing to create an attack reference.

Fig. 2

Example of attack reference

Figure 2 is an example of an attack reference.¹ The figure shows that the attack flow analyzed from multiple cases is expressed in a single diagram. In other words, the attack reference is an intensive expression of existing ICS attack cases. However, the exercise planner’s expertise relies on analyzing attack cases and creating attack references. Fortunately, the MITRE ATT &CK framework provides a variety of attack libraries, making it relatively easy to map TTPs used in cyberattacks. In addition, analysis reports on recent cyberattacks often analyze the adversary’s infiltration flow step-by-step in detail, so there is not much variation in deriving TTP [35]. The attack reference created in this way can be an index for generating realistic cyberattack scenarios because it includes attack paths that occurred.

3.1.2 A2: attack sequence extraction

The next step is an attack sequence extraction. This step aims to draw out a single sequence used in a cyberattack scenario from the previously created attack reference. Since we have already expressed the attack reference in the MITRE ATT &CK framework in the previous step, we can learn the beginnings of past cyberattack cases and the means of initial access. Therefore, the technique observed as the initial access on the attack reference is selected arbitrarily or intentionally. In cyberattack cases, attackers’ targets can also be informed of the ultimate impact of the cyberattack and its goals. The technique observed as impact in the attack reference is arbitrarily or intentionally selected as the final attack technique’s target. We can extract the attack sequence in this technique by repeatedly selecting lines extending to the next tactic. These methods follow the attack techniques of the attack examples, but because they mix multiple attack examples, they lead to extracting the attack sequences as intended by the exercise planner.

Exercise planners should consider the following when generating attack sequences:

• The attack sequence must start with initial access among the attack tactics.

• Exercise planners must consider whether the exercise scenario can achieve the intended impact. The impact can be verified by comparing the impact of attack tactics and the outcomes of attack cases.

• Exercise planners should consider which attack techniques they would like to improve their response to through exercise. For example, movement planners can support an understanding of attack techniques that propagate internally through lateral movement tactics or attack techniques that disrupt the regular operation of physical processes through impaired process control tactics.

Fig. 3

Example of attack sequence

Figure 3 is an example of extracting an attack sequence from the attack reference created in the previous step (blue line in Fig. 3). This example draws out an attack sequence starting with initial access to external remote services and ending with data destruction. Although the attack sequence generated through our proposed method may not perfectly match an actual cyberattack, the attack sequence generated in this step is very realistic because the attack reference already reflects cyberattack cases.

3.1.3 A3: threat intelligence mapping

This step links specific threat information to each previously extracted attack sequence stage. Threat information considered here are attack patterns, security weaknesses, and vulnerabilities. We adopt CAPEC [9] to explore attack patterns, CWE [10] for security weaknesses, and CVE [11] for vulnerabilities. Doing this allows one to write a scenario that reveals the attack method more clearly. The process of mapping threat information to attack sequences is as follows. First, we find the ATT &CK ID² for each stage of the attack sequence extracted earlier from the MITRE ATT &CK framework. Then, we retrieve the corresponding ATT &CK ID from CAPEC and concatenate the CAPEC ID. Once we find the CAPEC ID, we select the CWE ID. A single attack pattern (CAPEC ID) typically contains multiple security weaknesses (CWE ID), so we select one appropriate CWE ID. Additionally, we recommend selecting relevant security weaknesses at each stage, considering the consistency of the attack sequence as much as possible. Finally, we retrieve the CVE ID from the CWE ID and assign it to each attack step. Exercise planners can subjectively determine the method of selecting security weaknesses and vulnerabilities. However, to map appropriate threat information, specific criteria should be considered:

• Exercise planners must consider the resources used in the field, including devices, software, and protocols. They should avoid exercises that target resources that are not actually in use.

• Exercise planners must select threat information appropriate to the level of exercise participants to enable them to participate in the exercise.

Because threat information (CAPEC, CWE, CVE) may not be mapped to all ATT &CK IDs, attack steps lacking threat information still need to be completed. Moreover, even with available threat information, it may not be applicable if no security weaknesses exist in the exercise environment. When deriving threat information for ATT &CK proves difficult following the sequence outlined above, exercise planners can map the threat information(CVE) using appropriate data. Criteria for selecting appropriate data include:

• Exercise planners map ATT &CK ID and threat information with the threat information used in the attack case. This is the easiest method for exercise planners to use when it is challenging to map threat information in the above order.

• Exercise planners leverage threat intelligence that maps to resources used in exercising. In CVE, vulnerabilities are mapped to resources such as the common product list (CPE) so exercise planners can easily use them. This can support exercise planners’ decision-making even when multiple CVE IDs exist.

• Exercise planners utilizes threat information directly mapped to ATT &CK IDs. ATT &CK’s attack techniques are mapped to threat information found in attack cases. This allows the exercise planner to check threat information from multiple attack cases at once, allowing the exercise planner to check various threat information. However, threat information is not mapped to all ATT &CK IDs, and since threat information only provides CVE IDs, it is challenging to utilize universally.

While mapping threat information with our method, exercise planners can consider choices as multiple threat information is provided. In cases like CVE, information such as CPE may be a selection criterion, as described above. However, because CWE is general-purpose, this information is difficult to obtain. Therefore, exercise planners can refer to the following criteria when selecting multiple CWE IDs.

• Among the listed CWE IDs, exercise planners may consider the highest priority those CWE IDs from which training participants can patch or derive response strategies.

• Exercise planners may consider the CWE IDs with the highest occurrence frequency the highest priority.

• Exercise planners may consider the CWE ID with the highest severity among the listed CWE IDs as the highest priority.

Fig. 4

Example of mapping threat intelligence with attack sequence

If our methodology makes linking ATT &CK IDs and CAPEC IDs and mapping CWE IDs difficult, exercise planners can skip that process and link them to CWEs. Exercise planners can also utilize CWE’s "CWE VIEW: Weaknesses in SEI ETF Categories of Security Vulnerabilities in ICS" article. This item allows the exercise planner to link the CWE ID and ATT &CK ID based on subjective judgment. However, the exercise planner must provide the basis for the exercise subjective judgment of the plan and obtain the consent of the exercise plan team working with the exercise planner.

Figure 4 shows an example of mapping threat information to the attack sequence extracted in Fig. 3. In this example, CAPEC IDs were not assigned because no attack patterns correspond to the last two steps of the attack sequence (exfiltration over physical medium and data destruction). In other attack stages, CAPEC ID, CWE ID, and CVE ID were mapped according to the method described above. By doing this, the red team can better understand the attack flow and activities to be performed during the exercise. Additionally, the attack scenarios used in the exercise are derived from here.

3.1.4 Vulnerable implementation pattern mapping

Cybersecurity exercise is often conducted in intentionally reduced security environments. If an organization performs a cybersecurity exercise in a real business environment, unintended damage may occur to real assets, and attack scenarios may not proceed smoothly due to pre-installed security equipment. Therefore, organizations with an ICS environment generally build a virtual environment for cybersecurity exercises or implement a vulnerable system that mimics the real environment. In particular, in the latter case, information about vulnerable implementation patterns that make each attack stage that constitutes the attack scenario feasible is required. To do this, we obtain references to vulnerable implementation patterns using the CWE ID assigned to each stage of the previously created attack scenario. There are two main ways to do this. First, searching the CWE ID provides typical vulnerable source code patterns for each programming language for the corresponding security weakness; Second, secure coding standards that provide pairs of vulnerable code patterns with their correct patches often provide associated CWE IDs. Through this method, we can explore vulnerable implementation patterns for attack scenarios.

Fig. 5

Example of mapping a vulnerable implementation pattern with CWE ID [36]

Figure 5 shows an example of a weak implementation pattern for CWE-522. CWE-522 suggests insufficiently protected credentials. To put it more succinctly, this CWE ID is a security weakness, meaning a weak authentication process. CWE-522 provides vulnerable code patterns for PHP, JAVA, C, and ASP.NET. Figure 5 is an example of code written in C language. This example code implements a simple authentication mechanism that compares the password entered through the strcmp function and compress_password. Generally, password-based authentication uses a one-way function, such as the hash function, but a compression function was used in this code. Therefore, an attacker can easily obtain the correct password through decompression. Since this code is only an example, it is difficult to use it to build an actual vulnerable exercise environment, but it can be used as a hint for building a exercise environment.

3.2 Defense strategy establishment

This section describes deriving a defense strategy for the previously created cyberattack scenario to respond to adversarial activities. More specifically, it presents the tasks performed at each stage from D1 to D2 and their outputs.

3.2.1 D1: containment derivation

Defense strategies are derived at each stage of the attack scenario. The process of deriving a defense strategy begins with deriving a containment method. At this step, we take the information from the step A2 in cyberattack scenario generation. Note that the attack sequence extracted in step A2 is expressed on the MITRE ATT &CK framework. We have an ATT &CK ID corresponding to each stage of the attack sequence. From this, we derive a containment strategy in connection with the MITRE D3FEND framework. The MTIRE D3FEND framework is similar to the ATT &CK framework. It lists tactics and techniques that defenders can employ. However, since availability is paramount in an ICS environment, any compromise to availability due to deception tactics should be avoided and consider three tactics (detect, isolate, and deceive) as containment strategies. Among defense tactics, detect is a crucial technique for identifying attacks by adversarial actors. Defenders need to search for various artifacts indicating attacks within the movement. Exercise planners should include detection artifacts in documents such as exercise plans. Isolate, another defensive tactic, aims to prevent attacks by hostile actors in a movement from spreading to other systems or internal layers. Exercise participants should implement isolation while minimizing compromise to system availability. Exercise planners can monitor system availability and ensure that exercises are performed smoothly. Deceive, a defensive tactic, increases the attack cost and time for hostile actors by draining their resources. Movement planners need to strategize on implementing deception techniques. However, since availability is paramount in an ICS environment, any compromise to availability due to deception tactics should be avoided. The process of deriving an appropriate defense strategy from ATT &CK ID is simple. The related defense strategy is retrieved when we look up the ATT &CK ID in the MITRE D3FEND framework. We are only interested in detecting, isolating, deceiving, and choosing the appropriate techniques. The process of deriving an appropriate defense strategy from ATT &CK ID is simple. The related defense strategy is retrieved when we look up the ATT &CK ID in the MITRE D3FEND framework. We are only interested in detecting, isolating, deceiving, and choosing the appropriate techniques.

Fig. 6

Example of deriving containment strategies

Figure 6 shows an example of mapping containment strategies to the attack sequence extracted in Fig. 3. In general, various containment strategies corresponding to any ATT &CK ID are searched. Since some may not be suitable for the targeted attack scenario or environment, we recommend selecting one or more appropriate strategies considering the exercise context. The reason is that the defender’s actions to respond to a specific attack may vary. Therefore, the defender’s actions can be checked by excluding inappropriate strategies but considering various possible containment methods.

3.2.2 D2: eradication derivation

In the previous step, we explored containment strategies to prevent attacks from spreading. In general, the incident response process recommends blocking the spread of the attack and then eliminating its cause [37]. Therefore, we derive an eradication method as the final step in establishing a defense strategy. In this study, we consider patching as an eradication strategy. We explored vulnerable implementation patterns in step A4 of cyberattack scenario generation. We propose two methods to patch vulnerable code using a compliant solution to eliminate the cause of the accident. First, when a vulnerable code pattern is discovered from CWE ID, a patch method is derived from the code description; Second, if a vulnerable code pattern is derived from a secure coding standard, use the correct code provided by the standard.

Fig. 7

Example of deriving eradication strategy

Figure 7 shows how a patch method was derived from the vulnerable implementation pattern received in A4 of cyberattack scenario generation. In the example in Fig. 5, the security weakness was due to a weak authentication pattern. CWE-522 [36] describes this vulnerable code snippet. We can easily get hints about the patch from this description. Figure 7 patches the code by changing the compression function used to protect the password in the example in Fig. 5 to a hash function. Of course, code patching cannot be an eradication strategy for all cyberattacks. Therefore, we recommend using evict as an auxiliary tool among MITRE D3FEND’s tactics in these cases.

4 Evaluation

This section evaluates the guide for case-based attack scenarios and defense strategy development in ICS environment proposed in Sect. 3. This evaluation was conducted to objectively explain the effectiveness and use of the proposal. The evaluation is largely divided into qualitative comparison with existing related studies (Sect. 4.1) and case study (Sect. 4.2).

4.1 Comparative study

We compare our proposal with existing studies analyzed in Sect. 2.2.1. Existing studies have not explored exactly the same topic as our proposal. Nevertheless, we conduct a qualitative comparison to clarify the characteristics of the proposed guide and to easily derive ways to utilize it. The comparative study is expressed in the notation below.

• $\bigcirc$: Matches evaluation item.

• $\triangle$: Partially matches the evaluation item.

• : Inconsistency with evaluation items.

Table 1 Comparative results between existing works and our guide

Table 1 compares existing studies and our proposal from the following six perspectives: generating attack scenarios, deriving defense strategies, supporting training field, supporting evaluation criteria, considering environment, and supporting automation. The evaluation criteria were carefully selected to encompass the results of the literature analysis and the characteristics of the proposal. As a result of the comparative analysis, our proposal corresponded (or partially corresponded) to the remaining evaluation items except for automation support, but showed the following differences from existing studies. Our proposal systematically derives specific threat information about attack scenarios and response strategies to be used in training. On the other hand, Yamin et al. [5] and Nakata et al. [20, 21] provide a systematic method to derive each stage of an attack scenario, but lacks detailed threat information. Detailed threat information can be used to evaluate participants’ activities in training. Although we do not explicitly provide evaluation criteria for training, activities can be checked using the details of the threats derived. Mases et al. [23] and Andreolini et al. [24] explicitly presented evaluation criteria or framework. Additionally, while we generate realistic scenarios based on past cases, Song et al. [4] utilize the structure and attack tree of a given ICS facility. Our proposal provides clues for implementing a training field, but it does not build a complete training field and lacks automation features. On the other hand, Yamin et al. [5] and Nakata et al. [20, 21] provide a method to automatically create a virtual training site because the research purpose is focused on creating a training site.

To summarize the analysis results, the attack scenarios and response strategies derived through our proposal can be used for incident response training in an ICS environment. In addition, the detailed threat information contained in the scenario can be introduced to establish criteria for evaluating trainees’ activities. Additionally, our proposal provides direction for building a training ground with inherent vulnerabilities. However, creating a complete training ground requires further research and automation efforts.

4.2 Case study

In this section, we conduct a case study of the proposed methodology. To conduct a case study, we analyze cyberattacks targeting ICS facilities and derive an attack sequence. In the case study, we confirm that our proposed methodology supports developing and implementing cybersecurity training scenarios in a standardized manner. The case studies utilize three examples of cyberattacks targeting ICS facilities: the Ukraine electric power attack, the Maroochy water breach attack, and the Stuxnet attack. The methodology for generating attack scenarios utilizes MITRE ATT &CK, CAPEC, CWE, CVE, and secure coding standards. The methodology for deriving response strategies for attack scenarios utilizes MITRE D3FEND and secure coding standards.

Fig. 8

Result of attack reference

4.2.1 Cyberattack scenario generation

A1. Attack reference generation: The cases analyzed in step A1 were representative of attacks targeting ICS facilities and were selected as cases in which attacks were carried out on field devices of ICS facilities. We analyzed three cases and created attack references for the attack techniques used. Attack reference (see Fig. 8) describes attack techniques based on referenced analysis reports and unifies the method of expressing attack techniques through MITRE ATT &CK. The attack references generated based on the three cases can be found in Appendix A.

Fig. 9

Result of attack sequence

A2. Attack sequence extraction: Fig. 9 shows the complete attack sequence generated from three cases. Using the attack techniques standardized in step A1, we can connect the attack techniques in each case. An attack sequence can be derived by connecting the attack techniques for each case. Each case study connects attack techniques to derive attack sequences, with the sequence varying depending on the attack techniques chosen by the exercise planner. The attack sequence in this case study starts with the replication through removable media technique (T1091) used for initial access in the Stuxnet attack. This is followed by the application layer protocol technique (T1071.001), utilized in both Stuxnet and BlackEnergy3 attacks. Next, the attacker establishes remote connections through the remote service technique (T1105). The attacker then employs lateral tool transfer (T1570) to download additional malware. Subsequently, the malware initiates unauthorized command messages (T855) to execute malicious commands, a technique observed in the Maroochy Water Breach attack. Ultimately, this leads to causing damage to property (T0879) in the ICS facility. The key takeaway from this case study is to outline the fundamental attack techniques for use in exercises.

Fig. 10

Result of threat intelligence mapping

A3. Threat intelligence mapping: Fig. 10 shows the results of mapping threat intelligence to each stage of the cyberattack scenario derived through step A2. We can see that the CVE is not mapped using one attack technique (Remote Services) if we check the mapped results. Some threat information may not be expressed in this format if there are no appropriate cases for the information provided by the CWE and CVE frameworks. Additionally, among the attack techniques in the attack sequence, the CWE of the unauthorized command message technique results from a subjective judgment. Since it is not directly mapped to CAPEC, CWE-306 was connected using CWE’s "CWE VIEW: Weaknesses in SEI ETF Categories of Security Vulnerabilities in ICS" section. We envisioned sending inappropriate commands to devices in exercise scenarios as a possible attack due to a lack of authentication implementation. We map from CWE-306 to CVE-2022-30317, a vulnerability in ICS devices(distributed control system. DCS).

Fig. 11

Result of mapping a vulnerable implementation pattern with CWE-308 [38]

A4. Vulnerable implementation pattern mapping: Fig. 11 shows the results of mapping CWE-308 vulnerable implementation patterns through “Demonstrative Examples" provided by CWE. CWE-308 is a security weakness that occurs when using single-factor authentication. Checking the code presented in Fig. 10 shows the process of hashing the password through the SHA1 hash algorithm and comparing it with the input value. The weak implementation pattern presented in step A4 is characterized by verifying the input value only once, making it difficult to respond to threat actors who have collected passwords in advance. Additionally, security features need to be stronger. The hash algorithm SHA1 does not have a hash collision function, making it vulnerable to threat actors.

Fig. 12

Result of deriving containment strategies

4.2.2 Defense strategy establishment

D1. Containment derivation: Fig. 12 shows the results of the defensive strategies through D3FEND. This step utilizes the attack sequence derived from step A2 of the methodology. We can check the available defensive strategies by searching techniques corresponding to each step of the attack sequence through D3FEND lookup. The available defensive strategies we have found are detect, isolate, and deceive. The related defensive strategies are searched when we look up the ATT &CK ID of the attack sequence to D3FEND. We derived the defender’s behavior by considering defensive strategies appropriate for Containment strategies and exercise, which is the purpose of the D1 step.

Fig. 13

Result of deriving eradication strategy

D2. Eradication derivation: Fig. 13 shows the results from the eradication method for eliminating the cause of the attack from the attack sequence derived in step A2. Among the threat information mapped in step A3, we proceed through the process of patching by referring to the description of CWE-308, where a vulnerable code pattern was detected. In our case study, a routine of authentication with OTP was added to use multi-factor authentication to address the problem of relying on single-factor authentication in CWE-308. Additionally, we have patched to use SHA-256 instead of using the SHA1 algorithm, which has weak strength in security features.

4.3 Quantitative study

We compare our proposal with existing studies analyzed in Sect. 2.2.1. In this section, we conduct a quantitative assessment of the impact of attack scenarios. For quantitative evaluation, we perform a comparative analysis with our case study and the attack scenarios of studies that include attack scenario generation among the evaluation items of the qualitative evaluation performed in Sect. 4.1. We exploit the vulnerabilities of attack scenarios to analyze their impact. We integrate vulnerability information mapped to other knowledge bases from existing studies based on their CVE IDs. This allows you to take advantage of CVSS, which is assigned per CVE ID. CVSS is an indicator of the severity of a CVE vulnerability. CVSS receives a higher score, the more significant the impact a vulnerability will have on the target system if exploited. We use CVSS as a parameter to identify high-impact scenarios that can cause significant damage.

Our quantitative evaluation assesses the high impact of attack scenarios exploiting many CVE vulnerabilities. In actual cyberattacks, the attack’s success rate varies depending on the availability of vulnerabilities [39]. Based on this, we determine that the vulnerability contributes to the success of the attack technique in the attack scenario and achieves the attack goal. In particular, for ICS, there are cases where security patches are delayed or not provided due to availability issues. Based on this, the CVSS score is given weight (+1) as more CVE vulnerabilities are identified in the attack scenario. Finally, the impact is derived by dividing the final sum of CVSS by the number of vulnerabilities. Attack steps in which a CVE is not used will result in a CVSS score 0. The formula we used is below:

$$\begin{aligned} Impact = \sum _{count=1}^{CVE Count} \frac{(count + CVSS_{step})}{CVE count} \end{aligned}$$

Table 2 Comparative quantitative results between existing works and our guide

Table 2 shows the results of evaluating the impact of attack scenarios in existing research and our case study. In our quantitative evaluation, we only evaluate case studies presented among the studies [4, 5, 20, 21] that generate attack scenarios. This is because their research can be overinterpreted and lead to inaccurate quantitative evaluation results. Excluded through this method are [20] and [21]. The reasons for excluded studies are as follows.

[20, 21]: The vulnerabilities in the scenario are mosaic and not represented, making identification difficult.

In the study by Song et al. [4], three attack scenarios were derived, and a quantitative evaluation was performed on two attack scenarios performed in one flow. The two attack scenarios consisted of 11 steps, and three vulnerabilities were used. The vulnerabilities used in the attack scenarios are the same, except for the initial access vulnerability. The CVSS scores for each vulnerability are CVE-2020-13699 (8.8), CVE-2010-2568 (9.3), CVE-2018-8872 (8.1), and CVE-2018-7522 (6.7). The quantitative evaluation result is 9.87, 10.83, which has a higher impact score than our case study.

In the study by Yamin et al. [5], the attack goal and vulnerabilities according to the attack goal are defined in the scenario as a study on building a training environment, but the attack step is not defined. Since attack steps are not presented separately in the scenario of this study, a quantitative evaluation was performed through individual vulnerabilities assigned to the system. This study used several vulnerabilities to implement exercise scenarios, but the vulnerabilities mapped to the knowledge base were MS17-010 and MS14-068. When mapped to CVE and derived CVSS scores, they were CVE-2017-0144 (8.1) and CVE-2014-6324 (9.0). The quantitative evaluation result is 8.1, 9.0, which has a lower impact score than our case study.

5 Discussion

This section discusses the method for generating case-based attack scenarios and establishing defense strategies for cybersecurity exercises in ICS environments proposed in this paper. In this study, we derived concrete attack scenarios and response strategies using existing cyberattack cases in ICS environments and various threat knowledge bases. At each stage of the attack scenario, the adversary’s TTP, security weaknesses, vulnerabilities, and weak implementation patterns are identified. Additionally, defense strategies that respond to scenarios include containment strategies to prevent the spread of attacks and eradication strategies to remove the cause. In particular, we allowed to identify multiple response strategies that a defender could adopt. This can be used as checklists and directions for the blue team’s activities when conducting cybersecurity exercises.

This study uses attack graph theory to derive attack scenarios systematically. The attack graph represents all the paths an attacker can exploit to achieve a malicious goal [40], resulting in a set of sequences to achieve the goal. Attack sequences are the core element of our methodology. They are used to establish the flow of attacks and defense strategies systematically. We adopt attack graph theory to derive attack graphs systematically and indicate a derivation method. Our methodology generates attack references in the first step of deriving the attack graph. In attack reference materials, adversarial techniques from different cyberattack cases are linked and presented in a single diagram. This is equivalent to representing the path to achieve the malicious goal pursued by the attack graph. Movement planners can now establish malicious goals and causes of movement from attack references and connect adversarial techniques to derive attack sequences. The attack sequence derived in this way can be used in exercises such as tabletop, but our ultimate goal is an adversarial simulation exercise. We provide methods to map specific threat information to adversarial techniques in the attack sequence to achieve the goal, establish detailed attack scenarios, identify implementation patterns, and build a training environment.

This study showed that attack scenarios and response strategies were systematically derived. However, this study has two major limitations. First, the exercise life cycle largely consists of planning, conducting, and evaluating. This study mainly deals with tasks related to planning. Therefore, research is also needed on how to perform the exercise and its accurate evaluation effectively; Second, in generating attack scenarios, we matched vulnerable implementation patterns to build an exercise environment, but this is still insufficient. Building the exercise environment requires a lot of resources and is a highly complex task. Recently, virtual environments such as Cyber Ranges have been used to build flexible exercise environments. Therefore, an efficient exercise environment can be created if this research and the virtual environment are appropriately combined.

6 Conclusion

To conduct an effective cybersecurity exercise to increase the security awareness of organization members, it should be based on exercise goals and an understanding of cyberattacks assumed by the red team and blue team. In other words, the red team and blue team should clearly understand their respective roles and activities to be performed in the exercise. For this purpose, a clearly written attack scenario is required, and specific adversarial activities to be performed by the red team should be assigned to each stage of the attack scenario. At the same time, the blue team’s defensive activities to respond to these adversarial activities should also be accurately described. In this study, we propose a standardized guide to derive specific attack scenarios and response strategies. Attack scenarios are generated in four steps, starting with analyzing cyberattack cases in the existing ICS environment. The response strategy proceeds in two steps along with creating the attack scenario. We introduced well-known threat knowledge bases into this work, allowing us to drive objective and reliable exercise activities. A case study showed that effective exercise can be planned using the proposed guide. We also discussed how the proposed guide could be used and analyzed some of its limitations. From this, future research was naturally derived. Based on this study, we will study the remaining parts of the exercise life cycle: conducting and evaluating. In addition, research will be conducted on how to configure the exercise environment efficiently and flexibly.

Appendices

Analysis of existing cyberattack cases

see Tables 3, 4 and 5.

Table 3 Result of analysis on Stuxnet

Table 4 Result of analysis on Marrochy

Table 5 Result of analysis on Blackenergy3

Symbol description

Symbol	Description
$\bigcirc$	ATT &CK techniques used in attack cases
$\Box$	Attack tactics based on ATT &CK
−	The connection between attack techniques (does not indicate order)
$\rightarrow$	The selected connection by planner with direction
$\leadsto$	The threat information regarding the specific attach technique