# Mean-Field-Game Model for Botnet Defense in Cyber-Security

- 1.1k Downloads
- 14 Citations

## Abstract

We initiate the analysis of the response of computer owners to various offers of defence systems against a cyber-hacker (for instance, a botnet attack), as a stochastic game of a large number of interacting agents. We introduce a simple mean-field game that models their behavior. It takes into account both the random process of the propagation of the infection (controlled by the botner herder) and the decision making process of customers. Its stationary version turns out to be exactly solvable (but not at all trivial) under an additional natural assumption that the execution time of the decisions of the customers (say, switch on or out the defence system) is much faster that the infection rates.

## Keywords

Botnet defence Mean-field game Stable equilibrium Phase transitions## 1 Introduction

A botnet, or zombie network, is a network of computers infected with a malicious program that allows cybercriminals to control the infected machine remotely without the user’s knowledge. Botnets have become a source of income for entire groups of cybercriminals since the cost of running botnets is cheap and the risk of getting caught is relatively small due to the fact that other people’s assets are used to launch attacks. The interactive process of the attackers and defenders can be modeled as a Game. The use of game theory in modeling attacker-defender has been extensively adopted in the computer security domain recently; see [5, 24, 26] and bibliography there for more details. Two aspects are important. The first one is the contamination effect. The second one is the large number of computers. So, in fact, one deals with a stochastic game of a large number of interacting agents. This is amenable to Mean Field theory. To investigate this approach represents the main objective of this paper. Our model takes into account both the random process of the propagation of the infection (controlled by the botnet herder) and the decision making process of customers. We develop a stationary version which turns out to be exactly solvable (but not at all trivial) under an additional natural assumption that the execution time of the decisions of the customers (say, switch on or out the defense system) is much faster that the infection rates.

Similar models can be applied to the analysis of defense against a biological weapon, for instance by adding the active agent (principal interested in spreading the disease), into the general mean-field epidemic model of [25] that extends the well established SIS (susceptible-infectious-susceptible) and SIR (susceptible-infectious-recovered) models.

Mean-field games present a quickly developing area of the game theory. It was initiated by Lasry-Lions [23] and Huang–Malhame–Caines [15, 16], see [1, 4, 6, 13, 14] for recent surveys, as well as [2, 3, 7, 8, 9, 22, 27] and references therein. The papers [11, 12] initiated the study of finite-state space mean-field games that are the objects of our analysis here.

The paper is organized as follows. In the next section we introduce our model, formulate the basic mean-field game (MFG) consistency problem in its dynamic and stationary versions leading to precise formulation of our main problem of characterizing the stable solutions (equilibria) of the stationary problem. This problem is a consistency problem between an HJB equation for a stochastic control of individual players and a fixed point problem for an evolutionary dynamics. These two preliminary problems are fully analyzed in Sects. 3 and 4 respectively. Section 5 is devoted to the final synthesis of the stationary MFG problem from the solutions to these two preliminary problems. In particular, the phase transitions and the bifurcation points changing the number of solutions are explicitly found. In the last section further perspectives are discussed.

## 2 The Model

Assume that any computer can be in four states: *DI*, *DS*, *UI*, *US*, where the first letter, *D* or *U*, refers to the state of a defended (by some system, which effectiveness we are trying to analyze) or an unprotected computer, and the second letter, *S* and *I*, to susceptible or infected state. The change between *D* and *U* is subject to the decisions of computer owners (though the precise time of the execution of her intent is noisy) and the changes between *S* and *I* are random with distributions depending on the level of efforts \(v_H\) of the Herder and the state *D* or *U* of the computer.

Let \(n_{DI}, n_{DS}, n_{UI}, n_{US}\) denote the numbers of computers in the corresponding states with \(N=n_{DS}+n_{DI}+n_{UI}+n_{US}\) the total number of computers. By a state of the system we shall mean either the 4-vector \(n=(n_{DI}, n_{DS}, n_{UI}, n_{US})\) or its normalized version \(x=(x_{DI}, x_{DS}, x_{UI}, x_{US})=n/N\). The fraction of defended computers \(x_{DI}+x_{DS}\) represents the analogue of the control parameter \(v_D\) from [5], the level of defense of the system, though here it results as a compound effect of individual decisions of all players.

The control parameter *u* of each player may have two values, 0 and 1, meaning that the player is happy with the level of defense (*D* or *I*) or she prefers to switch one to another. When the updating decision 1 is made, the updating effectively occurs after some exponential time with the parameter \(\lambda \) (measuring the speed of the response of the defense system). The limit \(\lambda \rightarrow \infty \) corresponds to the immediate execution.

The recovery rates (the rates of change from *I* to *S*) are given constants \(q^D_{rec}\) and \(q^U_{rec}\) for defended and unprotected computers respectively, and the rates of infection from the direct attacks are \(v_Hq^D_{inf}\) and \(v_Hq^U_{inf}\) respectively with constants \(q^D_{inf}\) and \(q^U_{inf}\). The rates of infection spreading from infected to susceptible computers are \(\beta _{UU}/N, \beta _{UD}/N, \beta _{DU}/N, \beta _{DD}/N\), with numbers \(\beta _{UU}, \beta _{UD}, \beta _{DU}, \beta _{DD}\), where the first (resp second) letter in the index refers to the state of the infected (resp. susceptible) computer (the scaling 1 / *N* is necessary to make the rates of unilateral changes and binary interactions comparable in the \(N\rightarrow \infty \) limit).

*x*in the limit \(N\rightarrow \infty \) can be described by the following system of ODE:

### Remark 1

*N*players. The generator of this Markov evolution on the states

*n*is (where the unchanged values in the arguments of

*F*on the r.h.s are omitted)

*x*as

*F*is a differentiable function, the generator \(L_N\) turns to the generator

We shall now use the Markov model above to assess the actions of individual players.

*x*(

*t*) and \(v_H(t)\) are given, the dynamics of each individual player is the Markov chain on 4 states with the generator

*T*, that she tries to minimize, is

*DI*,

*DS*(resp. of the states

*DI*,

*UI*). Assuming that the Herder has to pay \(k_H v_H\) per unit of time using efforts \(v_H\) and receive the income

*f*(

*x*) depending on the distribution

*x*of the states of the computers, her payoff, that she tries to maximize, is

*x*(

*t*). Once

*x*(

*t*) and \(v_H(t)\) are known, each individual should solve the Markov control problem (5) with costs (6) thus finding the individual optimal strategy

*MFG consistency equation*can now be explicitly written as

*t*(then \(\mu \) describing the optimal average payoff), so that

*g*satisfies the stationary HJB equation:

*stationary MFG consistency*problem is in finding \(x=(x_{DI},x_{DS}, x_{UI}, x_{US})\) and \(u=(u_{DI},u_{DS}, u_{UI}, u_{US})\), where

*x*is the stationary point of evolution (1), that is

*x*is a fixed point of the limiting dynamics of the distribution of large number of agents such that the corresponding stationary control is individually optimal subject to this distribution. Yet in other words, \(x=(x_{DI},x_{DS}, x_{UI}, x_{US})\) and \(u=(u_{DI},u_{DS}, u_{UI}, u_{US})\) solve (11), (12) simultaneously.

Fixed points can practically model a stationary behavior only if they are stable. Thus we are interested in *stable solutions* (*x*, *u*) to the stationary MFG consistency problem (12), (8), where a solution is stable if the corresponding stationary distribution *x* is a stable equilibrium to (1) (with *u* fixed by this solution).

Apart from stability, the fixed points can be classified via their efficiency. Namely, let us say that a solution to the stationary MFG is *efficient* (or globally optimal) if the corresponding average cost \(\mu \) is minimal among all other solutions.

*u*only. In principle, there are 16 possible pure stationary strategies (functions from the state space to \(\{0,1\}\)). But not all of them can be realized as solutions to (11). In fact if \(u_{DI}=1\), then \(g(UI)< g(DI)\) (can be equal in degenerate case) and thus \(u_{UI}=0\). This argument forbids all but four strategies as possible solutions to (11), namely

*U*or always choose

*D*, are acyclic, that is the corresponding Markov processes are acyclic in the sense that there does not exist a cycle in a motion subject to these strategies. Other two strategies choose between

*U*and

*D*differently if infected or not.

Of course, allowing degenerate strategies, more possibilities arise.

## 3 Analysis of the Stationary HJB Equation

Let us start by solving HJB Eq. (11).

*g*is defined up to an additive constant we can set \(g(US)=0\). Then (18) becomes

*g*(

*DI*) and then the other values of

*g*:

*g*(

*UI*) and then the other values of

*g*:

*x*. The first observation in this direction is that the interior of the domain defined by (22) (that is, with a solution of case (i)) and the interior of the domain defined by (30) (that is, with a solution of case (iii)) do not intersect, because the first inequality in (22) contradicts the second inequality in (30) (apart from the boundary). Similarly, the interior of the domain defined by (22) (that is with a solution of case (i)) and the interior of the domain defined by (33) (that is, with a solution of case (iv)) do not intersect, and the interior of the domain defined by (27) (that is, with a solution of case (ii)) does not intersect with the domains having solutions in cases (iii) or (iv).

*x*classifying the solutions to HJB Eq. (11):

*x*belong to \(D_1\) (or its boundary), so that \(D_2\) is empty.

*x*belongs simultaneously to the interiors of the domains specified by (22) and (27) (that is, with solutions in cases (i) and (ii) simultaneously), then necessarily \(x\in D_1\) (that is, for \(x\in D_2\) the conditions specifying cases (i) and (ii) are incompatible). On the other hand, if

*x*belongs simultaneously to the interiors of the domains specified by (30) and (33) (that is, with solutions in cases (iii) and (iv) simultaneously), then necessarily \(x\in D_2\) (that is, for \(x\in D_1\) the conditions specifying cases (iii) and (iv) are incompatible).

Denoting \(\varkappa =k_D/k_i\), we can summarize the properties of HJB equation (11) as follows (uniqueness is always understood up to the shifts in *g*).

### Proposition 3.1

- (1)Ifthen there exists a unique solution to (11) belonging to case (iii) and there are no other solutions to (11).$$\begin{aligned} \frac{\left( \beta +\lambda \right) q_{rec}^D -\left( \alpha +\lambda \right) q_{rec}^U}{\left( \beta +q^U_{rec}+\lambda \right) \left( \alpha +q^D_{rec}\right) }< \varkappa < \frac{\beta \left( \lambda + q_{rec}^D\right) -\alpha \left( \lambda +q_{rec}^U\right) }{\left( \beta +q^U_{rec}\right) \left( \alpha +q^D_{rec}+\lambda \right) }, \end{aligned}$$(36)
- (2)Ifthen there exists a unique solution to (11) belonging to case (iv) and there are no other solutions to (11).$$\begin{aligned} \frac{\beta \left( \lambda + q_{rec}^D\right) -\alpha \left( \lambda +q_{rec}^U\right) }{\left( \beta +q^U_{rec}+\lambda \right) \left( \alpha +q^D_{rec}\right) }< \varkappa < \frac{\left( \beta +\lambda \right) q_{rec}^D -\left( \alpha +\lambda \right) q_{rec}^U}{\left( \beta +q^U_{rec}\right) \left( \alpha +q^D_{rec}+\lambda \right) }, \end{aligned}$$(37)
- (3)A solution belonging to case (i) exists if and only ifand is unique if this holds. A solution belonging to case (ii) exists if and only if$$\begin{aligned} \varkappa \ge \frac{\beta \left( \lambda + q_{rec}^D\right) -\alpha \left( \lambda +q_{rec}^U\right) }{\left( \beta +q^U_{rec}\right) \left( \alpha +q^D_{rec}+\lambda \right) }, \end{aligned}$$(38)and is unique if this holds. Either of conditions (38) or (39) is incompatible with either (36) or (37). In particular, equation (11) may have at most two solutions (if both (38) and (39) hold).$$\begin{aligned} \varkappa \le \frac{\left( \beta +\lambda \right) q_{rec}^D -\left( \alpha +\lambda \right) q_{rec}^U}{\left( \beta +q^U_{rec}+\lambda \right) \left( \alpha +q^D_{rec}\right) }, \end{aligned}$$(39)
- (4)Under (16), one has alwaysand$$\begin{aligned} \frac{\beta \left( \lambda + q_{rec}^D\right) -\alpha \left( \lambda +q_{rec}^U\right) }{\left( \beta +q^U_{rec}+\lambda \right) \left( \alpha +q^D_{rec}\right) } \ge \frac{\left( \beta +\lambda \right) q_{rec}^D -\left( \alpha +\lambda \right) q_{rec}^U}{\left( \beta +q^U_{rec}\right) \left( \alpha +q^D_{rec}+\lambda \right) }, \end{aligned}$$(40)Hence (37) becomes impossible and conditions (38) and (39) become incompatible implying the uniqueness of the solution to (11) for any \(x\in D_1\). This unique solution belongs to cases (ii), (iii) and (i) respectively for \(\varkappa \) satisfying (39), (36), (38) (when equality holds in (38) or (39), two solutions from different cases become coinciding).$$\begin{aligned} \frac{\left( \beta +\lambda \right) q_{rec}^D -\left( \alpha +\lambda \right) q_{rec}^U}{\left( \beta +q^U_{rec}+\lambda \right) \left( \alpha +q^D_{rec}\right) } \le \frac{\beta \left( \lambda + q_{rec}^D\right) -\alpha \left( \lambda +q_{rec}^U\right) }{\left( \beta +q^U_{rec}\right) \left( \alpha +q^D_{rec}+\lambda \right) }. \end{aligned}$$(41)

### Proof

### Remark 2

(1) When (16) does not hold one can find situations when solutions from cases (i) and (ii) exist simultaneously. To get simple examples one can assume \(\varkappa =1\). (2) When two solutions exist simultaneously one can discriminate them by the values of the average payoff \(\mu \). One sees from (20) and (25), that \(\mu \) arising from cases (i) and (ii) are different (apart from a single value of \(\varkappa \)). (3) The uniqueness result under (16) is quite remarkable, as it does not seem to follow a priori from any intuitive arguments.

Again directly from the argument above one can conclude the following.

### Proposition 3.2

- (1)Ifthen there exists a unique solution to (11) belonging to case (i) and there are no other solutions to (11).$$\begin{aligned} \varkappa > \frac{\left( \beta +\lambda \right) q_{rec}^D -\left( \alpha +\lambda \right) q_{rec}^U}{\left( \beta +q^U_{rec}\right) \left( \alpha +q^D_{rec}+\lambda \right) }, \end{aligned}$$(42)
- (2)Ifthen there exists a unique solution to (11) belonging to case (ii) and there are no other solutions to (11).$$\begin{aligned} \varkappa < \frac{\beta \left( \lambda + q_{rec}^D\right) -\alpha \left( \lambda +q_{rec}^U\right) }{\left( \beta +q^U_{rec}+\lambda \right) \left( \alpha +q^D_{rec}\right) }, \end{aligned}$$(43)
- (3)A solution belonging to case (iii) exists if and only ifand is unique if this holds. A solution belonging to case (iv) exists if and only if$$\begin{aligned} \frac{\left( \beta +\lambda \right) q_{rec}^D -\left( \alpha +\lambda \right) q_{rec}^U}{\left( \beta +q^U_{rec}+\lambda \right) \left( \alpha +q^D_{rec}\right) } \le \varkappa \le \frac{\beta \left( \lambda + q_{rec}^D\right) -\alpha \left( \lambda +q_{rec}^U\right) }{\left( \beta +q^U_{rec}\right) \left( \alpha +q^D_{rec}+\lambda \right) }, \end{aligned}$$(44)and is unique if this holds. Either of conditions (44) or (45) is incompatible with either (42) or (43). In particular, equation (11) may have at most two solutions (if (44) and (45) hold simultaneously).$$\begin{aligned} \frac{\beta \left( \lambda + q_{rec}^D\right) -\alpha \left( \lambda +q_{rec}^U\right) }{\left( \beta +q^U_{rec}+\lambda \right) \left( \alpha +q^D_{rec}\right) } \le \varkappa \le \frac{\left( \beta +\lambda \right) q_{rec}^D -\left( \alpha +\lambda \right) q_{rec}^U}{\left( \beta +q^U_{rec}\right) \left( \alpha +q^D_{rec}+\lambda \right) }, \end{aligned}$$(45)

### Proposition 3.3

- (1)Under (16) conditions (39), (36), (38) classifying the solutions to the HJB equation rewrite asrespectively. In particular, solutions of case (ii) become impossible.$$\begin{aligned} \varkappa \le 0, \quad 0< \varkappa < \frac{(\beta -\alpha )}{\beta +q}, \quad \varkappa \ge \frac{(\beta -\alpha )}{\beta +q}, \end{aligned}$$(47)
- (2)Suppose \(x\in D_1\) and (46) holds. If \(x\in D_{11}\), there exists a unique solution to (11), which belongs to cases (ii), (iii), (i) forrespectively. If \(x\in D_{12}\), solutions from case (iii) do not exist and there exist two solutions to (11) for$$\begin{aligned} \varkappa< \frac{\delta }{\alpha +q^D_{rec}}, \quad \frac{\delta }{\alpha +q^D_{rec}}<\varkappa < \frac{\beta -\alpha }{\beta +q^U_{rec}}, \quad \varkappa > \frac{\beta -\alpha }{\beta +q^U_{rec}}, \end{aligned}$$(48)belonging to cases (i) and (ii), and only one solution otherwise.$$\begin{aligned} \frac{\beta -\alpha }{\beta +q^U_{rec}}< \varkappa < \frac{\delta }{\alpha +q^D_{rec}}, \end{aligned}$$(49)
- (3)Suppose \(x \in D_2\). If \(x\in D_{22}\), solutions from case (iii) do not exist and there is always a unique solution to (11) belonging to case (ii), (iv) or (i), forrespectively. If \(x\in D_{21}\), then there are two solutions to (11) for$$\begin{aligned} \varkappa< \frac{\beta -\alpha }{\alpha +q^D_{rec}}, \quad \frac{\beta -\alpha }{\alpha +q^D_{rec}}< \varkappa < \frac{\delta }{\beta +q^U_{rec}}, \quad \varkappa > \frac{\delta }{\beta +q^U_{rec}}, \end{aligned}$$(50)which belong to cases (iii) and (iv), and one solution otherwise. This unique solution belongs to case (ii) or (i) for$$\begin{aligned} \frac{\delta }{\alpha +q^D_{rec}}< \varkappa < \frac{\beta -\alpha }{\beta +q^U_{rec}}, \end{aligned}$$(51)respectively and to case (iv) otherwise.$$\begin{aligned} \varkappa < \frac{\beta -\alpha }{\alpha +q^D_{rec}}, \quad \varkappa > \frac{\delta }{\beta +q^U_{rec}} \end{aligned}$$

### Proof

## 4 Analysis of the Fixed Points

Next we are solving the fixed point system (12).

Thus we proved the first part of the following statement and the second is analogous.

### Proposition 4.1

- (1)
There exists a unique solution to system (12) with the strategy

*U*being individually optimal (that is, with the first acyclic stationary strategy \(u_{UI}=u_{US}=0, u_{DI}=u_{DS}=1\)) and it is stable. It equals \(x=(0,0, x^*_{UI}, 1-x^*_{UI})\) with \(x^*_{UI}\) given by (55). - (2)There exists a unique solution to system (12) with the strategy
*D*being individually optimal (that is, with the second acyclic stationary strategy) and it is stable. It equals \(x=(x^*_{DI}, 1-x^*_{DI},0,0)\) with \(x^*_{DI}\) being the unique solution of equationon the interval (0, 1), that is$$\begin{aligned} Q_D(y)= \beta _{DD}y^2+y\left( q^D_{rec} -\beta _{DD} +q^D_{inf} v_H\right) -q^D_{inf} v_H=0 \end{aligned}$$(59)$$\begin{aligned} x^*_{DI}&=\frac{1}{2\beta _{DD}}\left[ \beta _{DD}-q^D_{rec}-q^D_{inf} v_H\nonumber \right. \\&\quad \left. +\,\sqrt{\left( \beta _{DD} +q^D_{inf} v_H\right) ^2+\left( q^D_{rec}\right) ^2-2 q^D_{rec} \left( \beta _{DD} -q^D_{inf} v_H\right) }\right] . \end{aligned}$$

### Proposition 4.2

- (1)
For large \(\lambda \) there exists a unique solution to system (12) in case (iii), that is with \(u_{UI}=u_{DS}=0, u_{DI}=u_{US}=1\), and it is stable. It has the form \(x=(0, 1-\bar{x}^*_{UI}, \bar{x}^*_{UI},0)\) up to corrections of order \(\lambda ^{-1}\), with \(\bar{x}^*_{UI}\) being the unique solution of equation (65) on (0, 1) given by (66).

- (2)For large \(\lambda \) there exists a unique solution to system (12) in case (iv), that is with \(u_{UI}=u_{DS}=1, u_{DI}=u_{US}=0\), and it is stable. It has the form \(x=(\bar{x}^*_{DI},0,0,1- \bar{x}^*_{UI})\) up to corrections of order \(\lambda ^{-1}\), with \(\bar{x}^*_{DI}\) being the unique solution of equationon (0, 1).$$\begin{aligned} \beta _{DU}x_{DI}^2+x_{DI}(q^U_{rec}-\beta _{DU}+q^U_{inf}v_H)-q^U_{inf}v_H =O(\lambda ^{-1}), \end{aligned}$$(69)

## 5 Solutions to the Stationary MFG Problem

Combining Propositions 4.1, 4.2 and 3.3 allows one to fully characterize the solutions to our stationary MFG consistency problem for large \(\lambda \).

The most straightforward general conclusion is the following.

### Theorem 5.1

For large \(\lambda \) there may exist up to 4 solutions to the stationary MFG problem, with only one in each of the cases (i) -(iv). All these solutions are stable.

### Remark 3

Notice that already this statement is not at all obvious a priori, and may not be true for finite \(\lambda \), where solutions to case (iii) or (iv) are found from an equation of fourth order.

As an example of more precise classification, let us present it under assumption (17) that ensures that all solutions lie in the domain \(D_1\).

### Theorem 5.2

- (1)
If (70) holds and \(x^*_{UI} >\bar{x}^*_{UI}\), or the opposite to (70) holds and \(x^*_{UI} <\bar{x}^*_{UI}\), then \(\varkappa ^* >\bar{\varkappa }^*\). Consequently, for \(\varkappa <\bar{\varkappa }^*\) there exists a unique solution to the stationary MFG problem, which is stable and belongs to case (iii); for \(\varkappa \in (\bar{\varkappa }^*, \varkappa ^*)\) there are no solutions to the stationary MFG problem; for \(\varkappa > \varkappa ^*\) there exists a unique solution to the stationary MFG problem, which is stable and belongs to case (i).

- (2)
If (70) holds and \(x^*_{UI} <\bar{x}^*_{UI}\), or the opposite to (70) holds and \(x^*_{UI} >\bar{x}^*_{UI}\), then \(\varkappa ^* <\bar{\varkappa }^*\). Consequently, for \(\varkappa <\varkappa ^*\) there exists a unique solution to the stationary MFG problem, which is stable and belongs to case (iii); for \(\varkappa \in (\varkappa ^*, \bar{\varkappa }^*)\) there exist two (stable) solutions to the stationary MFG problem; for \(\varkappa > \bar{\varkappa }^*\) there exists a unique solution to the stationary MFG problem, which is stable and belongs to case (i).

Thus if one considers the system for all parameters fixed except for \(\varkappa \) (essentially specifying the price of the defence service), points \(\varkappa ^*\) and \(\bar{\varkappa }^*\) are the bifurcation points, where the phase transitions occur.

### Theorem 5.3

Depending on the order relation between \(x^*_{UI}, x^*_{DI}, \bar{x}^*_{UI}\), one can have up to 3 solutions to the stationary MFG problem for large \(\lambda \), the characterization in each case being fully explicit, since for \(\varkappa >\varkappa _1\), there exists a unique solution in case (i), for \(\varkappa <\varkappa _2\), there exists a unique solution in case (ii), for \(\varkappa _3< \varkappa < \varkappa _4\), there exists a unique solution in case (iii).

Thus in this case the points \(\varkappa _1, \varkappa _2, \varkappa _3, \varkappa _4\) are the bifurcation points, where the phase transitions occur.

The situation when (17) does not hold is analogous, though there appears an additional bifurcation relating to *x* crossing the border between \(D_1\) and \(D_2\), and the possibility of having four solutions arises.

## 6 Discussion

Our model of four basic states is of course the simplest one that takes effective account of both interaction (infection) and rational decision making. It suggests extensions in various directions. For instance, it is practically important to allow for the choice of various competing protection systems, leading to a model with 2*d* basic states: *iI* and *iS*, where \(i\in \{1, \cdots , d\}\) denotes the *i*th defense system available (which can be alternatively interpreted as the levels of protection provided by a single or different firms), while *S* and *I* denote again susceptible or infected state, with all other parameters depending on *i*. On the other hand, in the spirit of papers [18, 19] that concentrate on modeling myopic behavior (rather than rational optimization) of players one can consider the set of computer owners consisting of two groups, rational optimizers and those changing their strategies by copying their neighbors.

The main theoretical question arising from our results concerns the rigorous relation between stationary and dynamic MFG solutions, which in general is in front of research in the mean-field game literature. We hope that working with our simple model with fully solved stationary version can help to get new insights in this direction. In the present context the question can be formulated as follows. Suppose that, if at some moment of time *N* players are distributed according certain frequency vector *x* among the four basic state, each player chooses the optimal strategy *u* arising from the solution of the stationary problem for fixed *x* (fully described in Sect. 3), and the Markov evolution continues according to the generator *L*. When two solutions are available, players may be supposed to choose the one with the lowest \(\mu \), see Remark 2 (2) (notice however that there exist cases when the *N* player game has several equilibria, but the mean-field game selects those with greater cost, see [10]). The resulting changes in *x* induce the corresponding changes of *u* specifying a well-defined Markov process on the states of *N* agents. Intuitively, we would expect this evolution stay near our stationary MFG solutions for large *N* and *t*. Can one prove something like that? A partial answer to this question is provided in [21].

## References

- 1.Bardi, M., Caines, P., Dolcetta, I.C.: Preface: DGAA special issue on mean field games. Dyn. Games Appl.
**3**(4), 443–445 (2013)MathSciNetCrossRefzbMATHGoogle Scholar - 2.Basna, R., Hilbert, A., Kolokoltsov, V.: An epsilon-Nash equilibrium for non-linear Markov games of mean-field-type on finite spaces. Commun. Stoch. Anal.
**8**(4), 449–468 (2014)MathSciNetGoogle Scholar - 3.Bauso, D., Tembine, H., Basar, T.: Robust mean-field games. Dyn. Games Appl.
**6**, (2015). doi: 10.1007/s13235-015-0160-4 - 4.Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Type Control Theory. Springer, New York (2013)CrossRefzbMATHGoogle Scholar
- 5.Bensoussan, A., Hoe, S., Kantarcioglu, M.: A game-theoretical approach for finding optimal strategies in a botnet defense model. In: Alpcan T., Buttyan, L., Baras, J. (eds.)Proceeding of the Decision and Game Theory for Security First International Conference, GameSec: Berlin, Germany, November 22–23, 2010, vol. 6442, pp. 135–148 (2010)Google Scholar
- 6.Caines, P. E.: Mean Field Games, Encyclopedia of Systems and Control. In: Samad , T., Ballieul, J. (eds.) Springer Reference 364780. Springer, London (2014). . doi: 10.1007/978-1-4471-5102-930-1
- 7.Cardaliaguet, P., Lasry, J.-M., Lions, P.-L., Porretta, A.: Long time average of mean field games with a nonlocal coupling. SIAM J. Control Optim.
**51**(5), 3558–3591 (2013)MathSciNetCrossRefzbMATHGoogle Scholar - 8.Carmona, R., Lacker, D.: A probabilistic weak formulation of mean field games and applications. Ann. Appl. Probab.
**25**(3), 1189–1231 (2015)MathSciNetCrossRefzbMATHGoogle Scholar - 9.Carmona, R., Delarue, F.: Probabilistic analysis of mean-field games. SIAM J. Control Optim.
**514**, 2705–2734 (2013)MathSciNetCrossRefzbMATHGoogle Scholar - 10.Doncel, J., Gast, N., Gaujal, B.: Mean-Field Games with Explicit Interactions. Preprint. https://hal.inria.fr/hal-01277098/file/main
- 11.Gomes, D.A., Mohr, J., Souza, R.R.: Discrete time, finite state space mean feld games. J. Math. Pures Appl.
**93**(3), 308–328 (2010)MathSciNetCrossRefzbMATHGoogle Scholar - 12.Gomes, D.A., Mohr, J., Souza, R.R.: Continuous time finite state space mean feld games. Appl. Math. Optim.
**68**(1), 99–143 (2013)MathSciNetCrossRefzbMATHGoogle Scholar - 13.Gomes, D.A., Saude, J.: Mean field games models - a brief survey. Dyn. Games Appl.
**4**(2), 110–154 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 14.Guéant, O., Lasry, J.-M., Lions, P.-L.: Mean Field Games and Applications. Paris-Princeton Lectures on Mathematical Finance 2010. Lecture Notes in Mathematics, pp. 205–266. Springer, Berlin (2003)Google Scholar
- 15.Huang, M., Malhamé, R., Caines, P.: Large population stochastic dynamic games: closed-loop Mckean-Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst.
**6**, 221–252 (2006)MathSciNetzbMATHGoogle Scholar - 16.Huang, M., Caines, P., Malhamé, R.: Large-population cost-coupled LQG Problems with nonuniform agents: individual-mass behavior and decentralized \(\epsilon \)-Nash equilibria. IEEE Trans. Automat. Control
**52**(9), 1560–1571 (2007)MathSciNetCrossRefGoogle Scholar - 17.Kolokoltsov, V.N.: Nonlinear Markov processes and kinetic equations. Cambridge Tracks in Mathematics 182, Cambridge University. Press, 2010. See the review of Prof. D. Applebaum in. Bulletin of the London Mathematical Society vol. 43(6), pp. 1245–1247 (2011)Google Scholar
- 18.Kolokoltsov, V. N.: Nonlinear Markov games on a finite state space (mean-field and binary interactions). Int. J. Stat. Probab.
**1:1**(2012), 77-91. http://www.ccsenet.org/journal/index.php/ijsp/article/view/16682 - 19.Kolokoltsov, V. N.: The evolutionary game of pressure (or interference), resistance and collaboration (2014). arxiv:1412.1269 (to appear in Mathematics of Operatoion Research (MOR))
- 20.Kolokoltsov, V. N., Malafeyev, O. A.: Mean field game model of corruption. Dyn. Games Appl. (2015). doi: 10.1007/s13235-015-0175-x (published with Open Access)
- 21.Kolokoltsov, V. N., Malafeyev, O. A.: Corruption and botnet defense: a mean field game approach arXiv:1607.07350
- 22.Kolokoltsov, V., Troeva, M., Yang, W.: On the rate of convergence for the mean-field approximation of controlled diffusions with large number of players. Dyn. Games Appl.
**4**(2), 208–230 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 23.Lasry, J.-M., Lions, P.-L.: Jeux à champ moyen. I. Le cas stationnaire (French). C.R. Math. Acad. Sci. Paris
**343**(9), 619–625 (2006)MathSciNetCrossRefGoogle Scholar - 24.Li, Z., Liao, Q., Striegel, A.: Botnet Economics: Uncertainty Matters. http://weis2008.econinfosec.org/papers/Liao
- 25.Liu, J., Tang, Y., Yang, Z. R.: The spread of disease with birth and death on networks. J. Stat. Mech. (2004). arXiv:q-bio/0402042v3
- 26.Lye, K.-W., Wing, J.M.: Game strategies in network security. Int. J. Inf. Secur.
**4**, 71–86 (2005)CrossRefGoogle Scholar - 27.Tembine, H., Zhu, Q., Basar, T.: Risk-sensitive mean-field games. IEEE Trans. Automat. Control
**59**(4), 835–850 (2014)MathSciNetCrossRefGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.