CyberSAGE: The cyber security argument graph evaluation tool

Abstract

Cyber risk assessment is a critical step in securing the digital systems that support modern society. Typically this is a manual process carried out by consultants or working groups with little or no software support outside of spreadsheet tools. As cybersecurity threats and digital systems themselves become more complex and dynamic, there is a need for greater tool support in the risk assessment process to document and trace assumptions and facilitate the revision or extension of a threat and risk assessment throughout a system’s lifecycle. The Cyber Security Argument Graph Evaluation (CyberSAGE) tool provides a platform for model-based cybersecurity analysis of cyber failure and attack scenarios. It combines models of high-level workflow, system architecture, device properties, attacker capability and skill, to compute holistic, quantitative security metrics. In this paper we describe the models, algorithms, and software architecture of the CyberSAGE tool. To illustrate its application, we describe an assessment carried out on communication systems in two railway lines with the support of an industry partner. Finally, we summarize feedback on the CyberSAGE tool from the railway case study partner, as well as over 40 interviews with practitioners and domain experts and a multinational electronics company who carried out a one year independent evaluation.

Appendices

Appendix A: Extension templates for attack scenarios on rail radio systems

The logical relationship between the various input models namely workflow, system and attacker models are discovered via extension templates. We define extension templates as reusable modules that can be used repeatedly to combine heterogeneous pieces of information from our input models in a meaningful and concise manner. The argument graph generation process is the repeated application of a set of extension templates to produce the final security argument graph. In order to study the TETRA attack scenarios in our assessment, we define a new set of extension templates over and beyond what has been discussed in our previous work (Tippenhauer et al. 2014).

We define 3 graph expansion templates which help us automatically generate the security argument graph. Each of these templates (see templates T1, T2 & T3 below) grow a single vertex v based on the workflow, system and attack models present in Σ. The resulting star graph ω_r contains v and at least one additional vertex. Each of the newly added vertices will contain at least one outgoing edge towards v. The additional vertices from the star graph ω_r and the associated edges will be added to the original graph (not shown in pseudocode below). We provide a detailed explanation for template T3 while we present a pseudocode for all the other templates. Extension template T3 is used to decompose a mal-activity step v into all its previous activity steps γ obtained by calling the GetLastActivityList (Σ) function. This is because for an activity step W_f to have occurred its previous activity step W_f − 1 should also have occurred. We also map v to the component device v_d on which the operation takes place. Depending on the activity step an attacker v_a may arrive to compromise the component device and in process exploit the activity step towards fulfilling her malicious intentions. For example a hacker is more likely to attempt a network access exploit as compared to an inept installer who is more likely to use physical access as the means to enter into the system. The newly created vertices namely v_c, v_d, v_a and their associated edges (v_c,v), (v_d,v), (v_a,v) are added to the returned star graph ω_r.

While formalizing the set of TETRA attack scenarios into mal-activity diagrams, we see that mal-activity diagrams share common attack steps. For example, send command is an activity step that is shared across most scenarios as they have a pro-active step to issue a request to a system component. Similarly in certain scenarios the attacker enters the system either via a network access route or via a physical access route. The chance that each of these attack steps will succeed depend on the vulnerabilities, mitigations implemented in the device on which the operation is taking place and also on the threat agent’s skill to exploit these vulnerabilities. These common attack steps are made into extension templates enabling us to compute the success probability of the attacker to exploit these steps. We identify a total of 19 such evaluation extension templates and present 2 of them in the form of pseudo codes below. We believe that by distilling the common attack steps into reusable extension templates, the efforts to model attack scenarios for other relate components are considerably reduced as new mal-activity diagrams will share a subset of these extension templates.

• T4: Physical access template Certain threat agents (such as insiders) may have physical access to the TETRA equipment. The physical accessibility can be abused for tampering the equipment or changing settings, etc. and cause the TETRA system to enter into unintended failure state. The template T4 represents the chance that such an attack may be possible.

• T5: Network access template The attack vector via the network access route is a more likely scenario for threat agents as a) most TETRA installations are heavily guarded and b) it may be possible to leverage a public internet facing interface of the rail system such as an office network and then use it to penetrate deeper into the system eventually reaching the secure control system network. This template represents the chance that the attacker can successfully compromise a device by launching network based attacks.

Appendix B: Evaluation logic implemented in example templates

Calculating the success probability of a single attack step. We first present the quantitative relationship captured in our example extension templates, which describes how to calculate the success probability of an individual attack step based on various relevant factors, including the set of vulnerabilities that can be exploited for launching the attack step, and for each vulnerability, the set of relevant attacker properties and the set of relevant mitigation solutions. To simplify the definition of the quantitative relationship, we make a number of assumptions in our example extension templates, as detailed below:

• An attack step can be successfully launched by exploiting any one of the relevant vulnerabilities, and these vulnerabilities are independent from each other. Denote the relevant set of vulnerabilities as vul_list. With our assumption, we have:

  $$ p_{\text{attack step}} = 1 - {\prod}_{vul \in vul\_list} (1 - p_{vul}) $$

  where p_{attack step} is the probability for the attack step to succeed, and p_vul is the probability that the attacker exploits a specific vulnerability vul ∈ vul_list and successfully execute the attack step.

• The probability p_vul for each individual vulnerability vul depends on the relevant attacker properties (which we denote as A_vul) and the relevant mitigations (which we denote as M_vul).

• We first consider the case when there is no mitigation in place at all. Let us denote the success probability to exploit a particular vulnerability vul under this assumption as $p_{vul}^{\prime }$, which only depends on an attacker’s properties, such as its physical access privilege, IT skill, and domain knowledge. The exploitation of different vulnerability can depend on different attack properties. For example, the exploitation of the vulnerability ”Physical access may be obtained by unauthorized individuals”, depends on the attacker property of ”physical access privilege”, but not on other properties like ”IT skill”. We will then require the users to define $p_{vul}^{\prime }$, the success probability to exploit vulnerability i when there is no mitigations, based on all combinations of these relevant attacker properties.

  Specifically, if a vulnerability only depends on a single property, the user input is defined just as shown in Table 2 .

  While if a vulnerability depends on multiple relevant attacker properties, to reduce the number of user inputs required, we further assume these attacker properties affect the success probability in an independent manner, and we define q_j for each relevant attacker property j ∈ A_vul, such that $p_{vul}^{\prime } = {\prod }_{j} q_{j}$. With this assumption,

  the number of required user inputs is reduced.

  This savings can be significant when the number of relevant attacker properties is large.

• Now we consider the effectiveness of mitigations. Recall that each vulnerability vul has a set of relevant mitigations M_vul. We assume each mitigation takes effect in an independent way. We assume that user can provide good estimation of the effectiveness of each mitigation, as measured by the scale that the mitigation can reduce the probability of successful vulnerability exploitation. Hence, for each mitigation k ∈ M_vul, we will ask the user to define its probability-reduction scale, r_k ∈ [0, 1] (the lower the value of r_k, the more effective the mitigation k in reducing the security risk of vulnerability i). This will be defined as a property in the corresponding device. With a given set of mitigation inputs, the probability for the vulnerability to be exploited becomes $p_{vul} = p^{\prime }_{vul} \times {\prod }_{k \in M_{vul}} r_{k}$.

To summarize the above discussions (with all assumptions made earlier), the final equation the extension template will use to calculate p_attackstep is:

$$p_{\text{attack step}} = 1 - {\prod}_{vul \in vul\_list} (1 - {\prod}_{j \in A_{i} } q_{j} \times {\prod}_{k \in M_{i} } r_{k} ) $$

To calculate the success probability of a single attack step, we will need to solicit the following inputs from the user:

• The set of relevant vulnerabilities vul_list, which may consist of one or multiple vulnerabilities.

• For each vulnerability vul, the set of relevant attacker properties A_vul.

• For each attacker property j ∈ A_vul, its range of possible values, and for each value, the corresponding q_j.

• For each vulnerability vul, the set of relevant mitigations M_vul.

• For each mitigation k ∈ M_vul, its probability-reduction value r_k.

The pseudo code in Template 6 summarizes the main evaluation logic we implemented for example extension templates.

Calculating the success probability of the whole mal-activity diagram. The success probability of individual attack steps are then combined over our security argument graph according to the logical OR / AND relationship among them. We use LibDAI to deal with shared variables. See more details about how this combination is computed in our earlier work (Tippenhauer et al. 2014).

We further introduce here the parameters to characterize the malicious intent of the attacker, and gives three probabilities for an attacker to launch attacks that related to C / I / A respectively. With this, for a given mal-activity diagram, p_{failure scenario} calculated from the mal-activity diagram should be further multiplied by p_{intention to launch specific mal-activity}. This is intended to capture the case where an attacker may be able to launch an attack, but she doesn’t have the intention to do so.