# Differential Terror Queue Games

- First Online:

DOI: 10.1007/s13235-016-0195-1

- Cite this article as:
- Wrzaczek, S., Kaplan, E.H., Caulkins, J.P. et al. Dyn Games Appl (2016). doi:10.1007/s13235-016-0195-1

## Abstract

We present models of differential terror queue games, wherein terrorists seek to determine optimal attack rates over time, while simultaneously the government develops optimal counterterror staffing levels. The number of successful and interdicted terror attacks is determined via an underlying dynamic terror queue model. Different information structures and commitment abilities derive from different assumptions regarding what the players in the game can and cannot deduce about the underlying model. We compare and explain the impact of different information structures, i.e., open loop, closed loop, and asymmetric. We characterize the optimal controls for both the terrorists and the government in terms of the associated state and costate variables and deduce the costate equations that must be solved numerically to yield solutions to the game for the different cases. Using recently assembled data describing both terror attack and staffing levels, we compare the differential game models to each other as well as to the optimal control model of Seidl et al. (Eur J Oper Res 248:246–256, 2016). The paper concludes with a discussion of the lessons learned from the entire modeling exercise.

### Keywords

Counterterrorism Differential games Queues Intelligence## 1 Introduction

The threat of terrorism remains one of the world’s most pressing problems, and a growing academic literature seeking to understand how both terrorist organizations and governments make decisions has sprouted in response. Enders and Sandler [6] provide a very readable account of empirical and game-theoretic work in this area, while new articles and special journal issues continue to appear; for example, see the January 2015 *Oxford Economic Papers*, the December 2011 issue of *Public Choice*, the May 2011 issue of *Journal of Peace Research*, and the April 2010 issue of *Journal of Conflict Resolution*, among others. Among those articles focusing on the application of game theory to terrorism and counterterrorism, most focus on one- or two-period games at a high level of abstraction.

More recently, Kaplan et al. [10] initiated a series of studies that focus on basic operational questions such as estimating the number of undetected terror plots, determining optimal terrorist attack rates and government counterterror agent staffing levels in equilibrium [13, 14], optimal terrorist- and government-force allocation across different insurgent strongholds [11], and developing optimal time-dependent counterterror staffing plans [16]. The present paper builds on this recent work (and most directly upon [16]) to model differential terror queue games, wherein terrorists seek to determine optimal attack rates over time, while simultaneously the government develops optimal counterterror staffing levels. As is often the case with differential game models [5, 15], different assumptions governing information structure and commitment ability can lead to different results, and these differences are especially important to recognize in the terrorism setting.

The paper begins with a preliminary analysis of a terrorist organization operating in the absence of government counteractions. The resulting model applies to the decisions of a newly emerging group of terrorists whose existence is as of yet unknown, and provides the other side of the coin examined by Seidl et al. [16] who presumed that only the government acted strategically against a fixed terrorist threat. We will show it is optimal for terrorists to conduct attacks at a constant rate in this scenario, somewhat justifying what was taken as a simplifying assumption in Seidl et al.’s earlier work.

The paper then defines the differential game that is at the heart of this paper. Both the terrorists and the government are presumed rational in the economic sense. The terrorists seek to maximize the present value of the benefits of successful attacks minus the costs of all attacks initiated by selecting an optimal trajectory of attack rates over time, while the government seeks to minimize the present value of the cost of terror attacks plus the staffing and related costs from fielding counterterror agents by selecting an optimal time-dependent agent staffing plan. The number of successful and interdicted terror attacks is determined via the continuous version of the terror queue model first introduced in Kaplan [11] and most recently utilized in Seidl et al. [16].

We compare the impact of three different information structures which relate to the assumption that each player makes about his opponent’s decision rule. In all three cases, the players have full information about the form of the game and choose their strategy as the best response to their opponent’s strategy. In the open-loop structure, the players assume that their opponent is not able to observe the number of known and unknown terror plots, i.e., the state variables, and therefore see their opponent’s decision rule only as a function of time and as such assume a commitment. As such, the players neglect the impact of a change of the state variable on their opponent’s strategy in their determination of the optimal strategy. We contrast the results to closed-loop strategies, where the players are aware that their opponent actively responds to a change of the state variables, and to strategies based on asymmetric information, where players assume that their opponent’s strategy depends on only one of the state variables and time. The contribution of this analysis is to provide a deeper understanding about the impact of different information structures which are commonly used in differential games; see, e.g., [3] and [4].

From an analysis of the Hamiltonian and first-order conditions for these games, we are able to characterize the optimal controls for both the terrorists and the government in terms of the associated state and costate variables. For the different information structures, we deduce the costate equations which must be solved numerically to yield solutions to the game. Using recently assembled data describing both terror attack and staffing levels, we present numerical calculations to compare the differential game models to each other as well as to the optimal control model of Seidl et al. [16] in order to see the impact of the various assumptions involved. We include sensitivity analyses to illustrate how the model solutions depend upon the values of key model parameters. The paper concludes with a discussion of the lessons learned from the entire modeling exercise.

## 2 The Unobservable Terrorist

*X*(

*t*) as a state variable. The total number of terror plots increases by the number of newly initiated terror plots, i.e., the attack rate \(\alpha (t)\), which is the control variable of the terrorists and decreases by the number of successful attacks \(\mu X(t)\), where \(\mu \) is considered to be exogenous.

^{1}The dynamic state equation thus equals

*b*and

*k*, respectively. Therefore, the terrorists maximize the following objective function [subject to (1)] with respect to their optimal control variable \(\alpha (t)\),

From the above analysis, we can identify the behavior of the terrorists if they assume that they are not or cannot be observed by the government. Independent of the state and the time, they always initiate the same number of terror plots. In Seidl et al. [16], we used an optimal control model to derive the optimal strategy of the government given a constant arrival rate of new terror plots, equal to the above-derived rate. Having the above model of the unobservable terrorist in mind, the result of the optimal control model can be seen as the optimal behavior of the government if the terrorists are assumed to be unobservable (assumed by both the government and the terrorists). In Sect. 5, we compare the result of this optimal control model with the results of the differential game where both players are assumed to act strategically. This shows the effect of a wrong assumption concerning the terrorists observability.

## 3 A Differential Game Between the Government and Terrorists

### 3.1 The Basic Model

*X*denotes terror plots that are undetected by the government and

*Y*denotes terror plots that are detected by the government. The terrorists’ control variable \(\alpha \) (\(\alpha \ge 0\)) denotes the arrival rate of the undetected plots, i.e., the number of newly initiated terror plots by the terrorists. The control variable of the government

*f*denotes the number of (teams of) undercover agents deployed by the government to infiltrate terror plots. The agents can be divided into two groups: interdictors (one for each detected plot) and detectors. Analogously to a service point in a queuing model, there is a 1-to-1 correspondence between interdicting agents and detected terror plots. Thus,

*Y*agents (interdictors) are linked to already detected plots and \(f-Y\) (detectors) are seeking to detect further plots. The dynamical system reads

*X*and

*Y*are the states, \(\alpha \) and

*f*are the controls, \(\mu \) denotes the rate of successful undetected terror plots, and \(\delta \) is the rate that detecting agents reveal planned terror plots. For the additional time from detection to interdiction of a terror plot, we assume an exponential distribution with mean \(\frac{1}{\rho }\) implying the interdiction rate \(\rho \). The 1-to-1 correspondence between interdictors and detected terror plots implies the following constraint

*c*is the estimated cost by one successful terror attack.

^{2}

- open loop:
The information structure is symmetric. Neither player is able to observe the state variables instantaneously. Therefore, both the terrorists and the government commit to a strategy over time at the start of the game where they assume that their opponent does not adapt its optimal controls according to changes in the state variables.

^{3}- asymmetric:
The information structure is asymmetric; the terrorists assume that the government can observe the number of detected terror plots

*Y*but not the number of undetected terror plots*X*, while the government assumes that the terrorists are able to observe the total number of terror plots \(X+Y\), but do not know which have already been detected (so they do not know*X*or*Y*individually).- closed loop:
The information structure is symmetric, and both the government and the terrorists have full information regarding the state variables due to a combination of observable data and the ability to deduce the value of unobserved states from model logic and known data. In mathematical context, this information structure is called memoryless perfect state (see [3], page 214). Due to notational convenience, the term closed loop will be used within this paper.

The different information structures above reflect different possible assumptions governing the information players have at hand, or perhaps their flexibility, or even their sophistication. In the open-loop case, inflexibility in suddenly changing plans (e.g., the training required to dispatch counterterror agents, or the difficulty in suddenly increasing the planned rate of terror attacks absent advanced places) combined with literal belief in Eqs. (8–9) might lead both players to simply solve their associated optimal control problems as functions of time only. They do not resort to considerations of subgame perfection that might result from asking what to do should the other player deviate from the open-loop strategy (as such deviations would be viewed as exercising infeasible flexibility as just described).

The asymmetric information structure comes about from assuming that, realistically, the government can only observe terror plots it has already detected, while the terrorists do not know which plots of theirs are or are not known to the government. Also, the flexibility concerns associated with closed-loop play are not assumed to matter in this structure; both players are free to consider deviations from play that might come about for any number of reasons.

Finally, the closed-loop structure might make the most sense from a strictly mathematical point of view, for given that the underlying terror queue model is deterministic, it is possible to deduce the values of unobserved states from observable data. For example, consider the government. It observes the number of detected plots *Y*(*t*) and cannot observe undetected plots, but it can observe successful terror attacks, and the latter occur at rate \(\mu X(t)\). As the government knows the value of \(\mu \), it can easily deduce *X*(*t*). Similarly, while the terrorists do not observe the number of detected plots *Y*(*t*) directly, they do know the total number of plots in play, \(X(t)+Y(t)\), and like the government can observe successful attacks and hence deduce *X*(*t*) by subtraction. If both the government and the terrorists are sufficiently sophisticated to solve optimal control problems to produced equilibrium strategies in a differential game, then surely they are sophisticated enough to deduce full information even if only partial information is directly observable.

In the following section, we derive the optimality conditions for each of the three cases described above.

## 4 Analysis

*Y*,

*Z*)-space. Consequently, we can omit

*X*unless necessary otherwise. The resulting dynamics are

*Y*and

*Z*, respectively. The resulting first-order condition is

### 4.1 Open-Loop Case

### 4.2 Asymmetric Information Case

*Y*, but not according to the total number of terror plots

*Z*, which the government is not able to observe. On the other hand, the government takes into account that the terrorists are aware of the total number of terror plots and deduces how the terrorists adapt their strategy when

*Z*changes. The government does not, however, consider the impact of a change of the number of known terror plots on their opponents strategy as they assume that terrorists have no knowledge about

*X*. Thus, the government assumes that the terrorists control is a function of

*Z*(

*t*) and time, whereas the terrorists assume that the governments control is a function of

*Y*(

*t*) and time, i.e.,

### 4.3 Closed-Loop Case

When we look at the adjoint systems above [(20)–(23) for the open-loop case, (25)–(28) for the asymmetric information case, and (30)–(33) for the closed-loop case], the implications of the assumed information structure are straightforward. The open-loop case can be seen as the foundation of all three cases, since its terms also appear in the other two cases. In the asymmetric information case, the government’s and terrorists’ optimal strategies are assumed to depend on *Y* and *Z*, respectively. This implies that the marginal effect of these state variables on the decision rule weighted by the effect of the control variable on the Hamiltonian is added to the adjoint system. Thus, it incorporates the future effects (aggregated over time) related to the opponent’s adaptions of its decision rule corresponding to changes in *Y* and *Z*, respectively. Moving to the closed-loop case, we observe that additionally the marginal effect of the second state on the controls is added to the adjoint equations. The intuition behind it is analogous to that given before.

Taking now a closer look at the actual decision rules of the players, (17) and (19), it can be observed that *f* depends immediately on *Y* and *Z* (therefore, \(\frac{\partial \phi ^{G}}{\partial Y} \ne 0\) and \(\frac{\partial \phi ^{G}}{\partial Z} \ne 0\)), whereas there is no marginal effect on \(\alpha \) (therefore, \(\frac{\partial \phi ^{T}}{\partial Y} = \frac{\partial \phi ^{T}}{\partial Z} = 0\)). The reason for this lies in the way both controls occur in the dynamical system and the objective.

The terrorists’ control \(\alpha \) denotes the number of newly initiated terror plots and is an absolute rate. There is no interaction with the other states in the dynamical system nor in the objective, meaning that a change of *Y* or *Z* has no immediate effect on the efficiency of this control. This implies that the adjoint equations for \(\lambda _Y\) and \(\lambda _Z\) are the same no matter what information structure is used. The interpretation of this is that when the government takes into account that terrorists could adapt their strategy as direct response to a change in the state variable, it can deduce from the derivation of the actual decision rule of the terrorists that it actually is not optimal to do so.

It is different for the governments control *f* (the number of undercover agents). Unknown terror plots can be detected with rate \(\delta \), and (of course) the more the unknown plots, the easier they can be detected. Thus, in the dynamical system there is a multiplicative interaction term between the control *f* and the states. This means that the optimal *f* should directly be adapted corresponding to the value of the states and implies the additional term in the adjoint equations.

*Remark* Note that in general the optimal strategies (of all three above-derived information scenarios) of both players also depend on the initial states \((X(0),Y(0))=(X_{0},Y_{0})\). In contrast, the feedback Nash equilibrium does not depend on the initial states, but only on the current state and on the time. As a result, every feedback Nash equilibrium is strongly time-consistent. For a profound discussion on that issue, we refer to Section 6 of [3]. Due to the structure of our model, it is not possible to derive the feedback Nash equilibrium. The state–control interaction and the nonlinearity of the state in the dynamics imply that the standard way of deriving the feedback Nash equilibrium (HJB approach with polynomial guess of the value function) does not work, since the resulting exponents always exceed the exponent of the value function.

In all three cases, it is not possible to derive analytically a steady state. Thus, we have to resort on numerical calculations.

## 5 Numerical Results

In the following, we present numerical calculations of the above differential game. The model includes nine (positive) parameters: \(\mu \) (rate of successful undetected terror plots), \(\delta \) (rate of revealing undetected terror plots), \(\rho \) (interdiction rate), *c* (costs of one successful terror attack for the government), \(c_{f}\) (weight of quadratic costs of counterterror measures), *b* (damage of one successful terror attack for the terrorists), *k* (costs of planning new terror plots), \(r_{1}\) (discount rate of the government), and \(r_{2}\) (discount rate of the terrorists). To get a better intuition for the results of our model, the results are presented in the (*X*, *Y*)-space.

*b*,

*k*, and \(r_{2}\) only occur in the differential game. Inserting the given parameter values into (7) gives \(\alpha = \frac{b}{1.04 k}\), and using, like [16], \(\alpha =40\) as a reference point gives us parameters when assuming the utility of a terror plot to the terrorists is a tenth of the damage it causes to the government. For them, we assume

*b*and

*k*are difficult to estimate, and no data are available.

*b*can hardly be measured by pure monetary terms; here also some ideological factors have to be included.

^{4}

- open loop:
\(\hat{f} =1002.40, \,\,\, \hat{\alpha } =13.73, \,\,\, \hat{X} =4.57, \,\,\, \hat{Y} =2.29\),

- asymmetric:
\(\hat{f} =1097.40, \,\,\, \hat{\alpha } =16.97, \,\,\, \hat{X} =5.32, \,\,\, \hat{Y} =2.91\),

- closed loop:
\(\hat{f} =849.55, \,\,\, \hat{\alpha } =9.41, \,\,\, \hat{X} =3.49, \,\,\, \hat{Y} =1.48\).

- optimal control:
\(\hat{f} =1562.16, \,\,\, \hat{X} =9.73, \,\,\, \hat{Y} =7.56\),

^{5}

Furthermore, we observe crucial differences between the solutions due to the information structures. The asymmetric information case (which is a mixture of the open-loop and closed-loop case) has the greatest values for both controls and both states, followed by the open-loop case and the closed-loop case with the lowest values. The reason for the difference can be explained as follows:

The big difference between the open-loop and the asymmetric case is that, for the latter case, when calculating the shadow price of *Y*, \(\eta _y\), the terrorists take into account the extent to which the government adjusts its strategy when *Y* changes and its impact on the value of the Hamiltonian. The derivative of \(\phi ^G\) with respect to *Y* is strictly negative, and since \(\eta _y\) is positive,^{6}\(\dot{\eta }_y\) increases. In the asymmetric case, therefore, the terrorists conclude that when the number of known terror plots *Y* increases, *f* decreases (when *Z* remains constant). However, when determining the shadow price of *Z*, the terrorists do not directly take into account that an increase in *Z* leads to an increase in *f*. Thus, the higher evaluation of known terror plots leads to a higher evaluation of the total number of terror plots, and thus, the terrorists have a higher incentive in the asymmetric case to make more terror plots. Therefore, terrorists choose a higher \(\alpha \) than in the open-loop case, which results in a higher number of terror plots (known and unknown). This leads to a higher *f*, which in turn the terrorists need to compensate by a higher \(\alpha \).

Now in closed loop, terrorists do not only take into account that a change of *Y* affects *f* and consequentially their benefits from *Y*, but also how a change of *Z* affects *f* and therefore the shadow price of *Z*. The derivative of \(\phi _G\) with respect to *Z* is positive, i.e., when *Z* increases it is optimal to increase *f*. Thus, terrorists now take into account that it is not only beneficial to having a high number of terror plots *Z* (i.e., a higher number of successful terror plots), but also to a stronger extent that there is a negative impact due to a stronger response by the government. This negative impact dominates the attractiveness of known terror plots to occupy terror agents.

So information plays a crucial role in the optimal behavior and in the resulting optimal state values. Starting from the open-loop case, there are an increasing effect if both players can only observe one state (government: the known terror plots; terrorists: all terror plots) and an additional negative effect if both players are able to observe both states. The negative effect dominates the positive one.

### 5.1 Phase Portrait Analysis

We see that if the number of initial terror plots is low, the terrorists will initiate a comparatively high number of terror plots first. Since it is difficult for the government to detect terror plots when their number is low, they initially will not do much, because costs of terror detection would exceed the utility from it. Thus, the number of terror plots grows. When there are more terror plots, the government will increase its efforts to detect them, not only because it is easier to detect them, but also because more damage is caused by successful attacks. The terrorists will initiate less terror plots as due to the higher control efforts of the government planning new plots is less efficient. The number of terror plots grows until the number of newly initiated terror plots equals the number of successful attacks plus the number of detected plots.

Due to the high damage caused by terror attacks, and the high efficiency of detection, the government deploys a high number of agents for detection. As the number of unknown terror plots gets lower, the terrorists have to initiate more and more terror plots to be able to launch more successful attacks. Thus, the number of unknown terror plots *X* decreases, and the number of unknown terror plots *Y* increases. Therefore, the government can decrease its control efforts. Note, however, that after some time the number of unknown terror plots becomes so low that *Y* then decreases until its steady state value is reached. That is, *Y* decreases once the terrorists cannot draw any advantage from terror plots initiated before the government’s involvement into counterterror actions.

### 5.2 Sensitivity Analysis

Our results are mostly derived numerically; therefore, it is natural to look at the robustness of the results if some of the key parameters are changed. In Fig. 3, we plot the steady state values of three different solutions of the differential game and vary the interdiction rate \(\rho \) in the interval [0.5, 10]. The other parameters are the same as introduced above. The left panel in Fig. 3 shows the steady state values of *X*, and the right one that of *Y*.

The stability of the steady states remains the same. Similar to the optimal control model (see [16]), we find that there are more terror plots, both known and unknown, when the interdiction rate is low. This is reasonable since a high interdiction rate means that the known terror plots are handled quickly.

We can see that the smaller the \(\rho \), the greater is the difference between the information structures used. Interestingly, the value of *X* does not change crucially for the open- and the closed-loop case, but for the asymmetric case, the change is considerable.

The efficiency rate \(\delta \) is a parameter that is very hard to observe in reality. In Fig. 4, we plot the steady state values and vary \(\delta \) in the interval [0.02, 0.4] (again the rest of the parameters stay the same as above). The left panel shows the steady state values of *X*, and the right one that of *Y*.

## 6 Conclusions

The present paper provides two major contributions. First, we extend the terror queue model first introduced in Kaplan [11] to account for the fact that not only the government can choose its strategy optimal, but also the terrorists. We find that if the government is not able to observe terrorists by any means and therefore detect any terror plots, it is optimal for the terrorists to initiate terror plots at a constant rate, which provides a motivation for the assumption of a constant attack rate made in Seidl et al. [16].

To analyze the strategic interactions between the government and the terrorists, we set up a differential game. Comparing the results of the optimal control problem of the government and the differential game, we find that the government’s optimal strategy only differs quantitatively, but not qualitatively. We see how the optimal number of newly initiated terror plots, i.e., the strategy of the terrorists, is linked to the number of known and unknown terror plots. We find that it is optimal to initiate more terror plots if the number of existing terror plots is high than if it is low.

The second main contribution of the paper is to provide a better understanding about the impact of the choice of commonly used assumptions on the underlying information structure in differential games. In addition to the standard open-loop and closed-loop case, we also assume the case of asymmetric information. In this context, we provide an extensive discussion on different information structures, their meaning, and their implications. While the results of the present model do not differ qualitatively, the underlying information structure has a considerable impact on the extent to which the two players apply their control instruments and consequentially on the long-run development of the number of terror plots.

We see that if terrorists think that the government cannot know (or derive) the total number of terror plots (despite having full information) and only take the adaption of the government’s strategy into account when the number of known terror plots changes, terrorists will initiate more terror plots than when they see the government’s actions as just a function of time or when they incorporate the government’s reaction to the change of the total number of plots. We argue that terrorists then evaluate terror plots as more valuable and initiate more plots, which in turn leads to a stronger response of the government in terms of counterterror actions. When terrorists incorporate the government’s adaptions to a change of both, the number of known and the total number of terror plots, they will attack less as they know that by planning terror plots they strongly provoke counteractions.

The presented model is purely deterministic, and the players have in principle complete information about all the parameters. They know about the initial number of terror plots and can deduce their development by their knowledge about the state equations and about their opponents objective. While the solution of the present model is an important first step to understand the optimal strategies of the government and the terrorists, it provides many possibilities for extensions. An interesting task would be to study the impact of imperfect information. One could consider what happens if the government errs about the initial number of terror plots or if the terrorists wrongly estimate the efficiency of the government’s agents. Furthermore, one could include uncertainties with respect to the detection of terror plots.

For a more detailed discussion on the different information structures, we refer to [3, 5] and Long (2000).

Thus, players do not deduce how their opponent actually determines its decision rule despite having full information about the game. This is probably the main reason why in the literature closed-loop or feedback solutions are seen as more reasonable than open-loop strategies.

In general, nonzero-sum differential games which admit a unique open-loop Nash equilibrium solution have uncountably many Nash equilibrium solutions if the information structure is extended (see, e.g., [2]). This *informationally nonuniqueness* is based on the dynamic nature of the information. However, using the above method implies a canonical system of ordinary differential equations which has a unique steady state.

Note, however, that this is also strongly related to the used parameters. If the costs of terror plots are lower or the utility is higher, results are the opposite.

The reason for the sign of \(\eta _y\) is that for the terrorists a high *Y* is beneficial since then interdiction prevents agents from detecting new plots.

## Acknowledgments

This research was supported by the Austrian Science Fund (FWF) under Grants P25979-N25 and P25275-G11.