The Continuous Time Infection–Immunization Dynamics

Abstract

Recently a new evolutionary game dynamics, the Infection–Immunization Dynamics,has been introduced for discrete time. In this paper a continuous time version of this model is derived and the existence and structure of solutions is analysed. This is a very challenging task, since standard technique existence theorems for Differential Inclusions do not hold in general. An extended solution concept, the notion of Krasovsky solutions, can be applied though. Some stability results are stated and discussed.

1 Introduction

Historically speaking, the main goal of evolutionary game theory has been predicting the behaviour of animals by the means of non-cooperative game theory. But over the last years this focus has shifted, and evolutionary game theory has applications especially in economics, but also in optimization, imaging and computer vision, machine learning, network design and security (see, for example, [15–17, 19]). In these applications often equilibria of games with thousands or millions of pure strategies have to be found. Standard approaches for equilibrium selection like the Replicator Dynamics (RD) are not suited and efficient for such problems.

A recently published paper [18] aimed to overcome these restrictions and introduced a new dynamics in discrete time, the Infection–Immunization Dynamics (IID). This dynamics presents a computationally fast way to find a Nash Equilibrium Strategy (NES) starting from an arbitrary point on the simplex.

The basic ideas behind the IID is the following: whenever a population state is not a NES, there exists a strategy which is “infective” for the population. When a small share of “mutants” (who play an infective strategy) is injected into the population, they will spread until the population becomes “immune” against them. At this point, another infective strategy is selected and this process is started repeatedly. It can be shown that in discrete time less and less infective strategies remain—the population gets “vaccinated”—and in the end a state which is immune against all mutants—a Nash equilibrium—is reached.

The IID resembles to a certain extent the Best Reponse Dynamics (BRD) (or Fictitious Play in discrete time), a dynamics where only “best” (mixed) strategies are selected. On the contrary, the IID takes only pure strategies into account, while it also selects “best” strategies—then the dynamics moves into its direction. But also “worst” strategies are considered, then the dynamics moves away from them, this becomes possible via the concept of a co-strategy.

In this paper we introduce and analyse a continuous time version of the IID; it is structured as follows.

Section 2.1 recapitulates, and investigates further, the discrete time Infection–Immunization Dynamics (abbreviated dIID). For the readers’ convenience, these ideas are summarized here. Section 2.2 extends the model by making responses multi-valued, introducing the notion of a strategy selection correspondence.

These subsections are complemented by new observations relating maximum infection level to invasion barriers (Proposition 4), as well as bounding the population share of losing strategies in any incumbent state (Proposition 12). While these results are of interest on their own, they will not be needed in later sections.

Section 2.3 develops a continuous time Infection–Immunization Dynamics (abbreviated cIID) out of the discrete time version culminating in Definition 14.

Section 3 deals with questions regarding the existence of solutions for the cIID. In particular, it is shown here that standard techniques to prove existence of solutions do not apply to cIID. Although solutions can be constructed piecewise similarly as for the BRD [13], we were unable to prove existence of a solution.

Section 3.2 shows possible solutions to this dilemma, one can construct solutions “by hand”, as well known for the BRD. Although this approach works heuristically, we were not able to prove it, so this remains an open problem. The second approach is to utilize a more general solution concept for differential inclusions, the notion of Krasovsky solutions. Every differential inclusion possesses a Krasovsky solution, so also for the continuous time version of the model.

Section 4 finally states stability results and properties we were able to derive for the cIID. Furthermore, we give an example for an interesting behaviour of the dynamics on the faces of the simplex.

1.1 Evolutionary Game Theory

An evolutionary game theory set-up (see e.g. [23]) with pairwise contests and finitely many pure strategies $N=\left\{ 1,2,\ldots ,n\right\}$ is fully described by means of a payoff matrix $A=[a_{ij}]_{(i,j)\in N\times N}$ where $a_{ij}$ is payoff to pure strategy i when playing against pure strategy j. A population state is then a point $\mathbf {x}\in \Delta ^n$ where

$$\begin{aligned} \Delta ^n=\left\{ \mathbf {x}\in \mathbb {R}^{n}:\sum _{i=1}^{n}x_{i}=1,\, x_{i}\ge 0\quad \hbox {for all }i\in N\right\} \end{aligned}$$

is the standard simplex, spanned by the vertices $\mathbf {e}^{i}$, $i\in N$, with $[\mathbf {e}^{1},\ldots ,\mathbf {e}^{n}]$ the $n\times n$ identity matrix. The payoff function describes the mean payoff for the pure strategies in state $\mathbf {x}\in \Delta ^n$ and is denoted by $F(\mathbf {x})=A\mathbf {x}$, with components

$$\begin{aligned} u(\mathbf {e}^{i}|\mathbf {x})=F_{i}(\mathbf {x})=(A\mathbf {x})_{i}\,. \end{aligned}$$

The expected payoff of a population state $\mathbf {y}$ against the population state $\mathbf {x}$ is denoted by

$$\begin{aligned} u(\mathbf {y}|\mathbf {x})=\sum _{i\in N}y_{i}F_{i}(\mathbf {x})=\mathbf {y}^{T}F(\mathbf {x})=\mathbf {y}^{T}A\mathbf {x}\,, \end{aligned}$$

and the average population payoff within the population in state $\mathbf {x}$ by $u(\mathbf {x})=u(\mathbf {x}|\mathbf {x})$.

Then we denote by

$$\begin{aligned}\begin{array}{rcll} \sigma (\mathbf {x}) &{} = &{} \{i\in N:x_{i}>0\}\quad &{} \hbox {the }support\hbox { of }\mathbf {x}\,,\\ \tau _{-}(\mathbf {x}) &{} = &{} \{i\in N:u(\mathbf {x}-\mathbf {e}^{i}|\mathbf {x})>0\}\quad &{} \hbox {the }losing\hbox { pure strategies at }\mathbf {x},\\ \tau _{+}(\mathbf {x}) &{} = &{} \{i\in N:u(\mathbf {x}-\mathbf {e}^{i}|\mathbf {x})<0\}\quad &{} \hbox {the }winning\hbox { pure strategies at }\mathbf {x}\,\hbox {and }\\ \tau _{0}(\mathbf {x}) &{} = &{} \{i\in N:u(\mathbf {x}-\mathbf {e}^{i}|\mathbf {x})=0\}\quad &{} \hbox {the }balancing\hbox { pure strategies at }\mathbf {x}\hbox {.}\end{array} \end{aligned}$$

The terms “losing” and “winning” above always refer to a performance below or above the average performance (average population payoff), not in an absolute sense. The support $\sigma (\mathbf {x})$ defines the set of pure strategies used with positive probability in state $\mathbf {x}$. The set $\tau _{0}(\mathbf {x})$ is also called the extended support of $\mathbf {x}$, since at a Nash equilibrium (see below) we always have $\sigma (\mathbf {x})\subseteq \tau _{0}(\mathbf {x})$. The set of best replies or best responses $\beta (\mathbf {x})$ against a strategy $\mathbf {x}$ is given by

$$\begin{aligned} \beta (\mathbf {x})=\arg \max _{\mathbf {y}\in \Delta ^n}u(\mathbf {y}\vert \mathbf {x})\hbox {.} \end{aligned}$$

So $(\mathbf {x},\mathbf {x})\in \Delta ^{2n}$ is a symmetric Nash equilibrium (NE) if $\mathbf {x}\in \beta (\mathbf {x})$. A population state $\mathbf {x}$ which constitutes an NE $(\mathbf {x},\mathbf {x})$ will be called the Nash equilibrium strategy (NES), and $\mathbf {x}$ is a NES if

$$\begin{aligned} u(\mathbf {y}-\mathbf {x}\vert \mathbf {x})\le 0\quad \hbox {for all }\mathbf {y}\in \Delta ^n\hbox {.} \end{aligned}$$

A NES is called strict, if $\beta (\mathbf {x})=\left\{ \mathbf {x}\right\}$, i.e. if $\beta (\mathbf {x})$ is a singleton. Hence, a strict NES can be defined as an $\mathbf {x}$ satisfying

$$\begin{aligned} u(\mathbf {y}-\mathbf {x}\vert \mathbf {x})<0\quad \hbox {for all }\mathbf {y}\in \Delta ^n\backslash \left\{ \mathbf {x}\right\} \hbox {.} \end{aligned}$$

Now let $\mathbf {y}$ be an alternative state and $0<\epsilon <1$, and consider the postentry mix population state $\epsilon \mathbf {y}+(1-\epsilon )\mathbf {x}$. Following [7], we consider the score function of $\mathbf {y}$ versus $\mathbf {x}$

$$\begin{aligned} h_{\mathbf {x}}:\Delta ^n\times (0,1)\rightarrow & {} \mathbb {R}\\ (\mathbf {y},\epsilon )\mapsto & {} u\left( \mathbf {y}-\mathbf {x}|\epsilon \mathbf {y}+(1-\epsilon )\mathbf {x}\right) , \end{aligned}$$

and define the invasion barrier of $\mathbf {x}$ against $\mathbf {y}$, denoted ${\overline{\epsilon _{\mathbf {x}}}}(\mathbf {y})$, as the largest value of $\epsilon$, so that for all smaller values of $\epsilon$, $\mathbf {x}$ earns a higher payoff than $\mathbf {y}$ against the postentry mix. Formally

$$\begin{aligned} {\overline{\epsilon _{\mathbf {x}}}}(\mathbf {y})=\inf \left\{ \left\{ \epsilon \in (0,1):h_{\mathbf {x}}(\mathbf {y},\epsilon )\ge 0\right\} \cup \left\{ 1\right\} \right\} \hbox {.} \end{aligned}$$

So ${\overline{\epsilon _{\mathbf {x}}}}(\mathbf {y})=0$ if $\mathbf {y}$ fares better than $\mathbf {x}$ against all the postentry populations, and ${\overline{\epsilon _{\mathbf {x}}}}(\mathbf {y})=1$ if $\mathbf {x}$ always fares better than $\mathbf {y}$. A strategy $\mathbf {x}\in \Delta ^n$ is called an evolutionarily stable strategy (ESS), if

$$\begin{aligned} {\overline{\epsilon _{\mathbf {x}}}}(\mathbf {y})>0\quad \hbox {for all }\mathbf {y}\in \Delta ^n\backslash \left\{ \mathbf {x}\right\} \hbox {.} \end{aligned}$$

A slight modification of the ESS was called the neutral stability [8]. The neutral stability replaces the invasion barrier by the neutral invasion barrier of $\mathbf {x}$ against $\mathbf {y}$, denoted ${\overline{\epsilon _{\mathbf {x}}^{N}}}(\mathbf {y})$, is the largest value of $\epsilon$, so that for all smaller values of $\epsilon$, $\mathbf {x}$ earns a payoff higher or equal than $\mathbf {y}$ against the postentry mix, i.e.

$$\begin{aligned} {\overline{\epsilon _{\mathbf {x}}^{N}}}(\mathbf {y})=\inf \left\{ \epsilon \in (0,1):h_{\mathbf {x}}(\mathbf {y},\epsilon )>0\right\} \cup \left\{ 1\right\} \hbox {.} \end{aligned}$$

A strategy $\mathbf {x}\in \Delta ^n$ is called a neutrally stable strategy (NSS) if

$$\begin{aligned} {\overline{\epsilon _{\mathbf {x}}^{N}}}(\mathbf {y})>0\quad \hbox {for all }\mathbf {y}\in \Delta ^n\hbox {.} \end{aligned}$$

The conditions for being a NSS are clearly weaker than the one for an ESS, but stronger than for a NES. This also reflects in the dynamic behaviour—a NSS is Lyapunov stable fixed point under the RD, see [6].

2 The Infection–Immunization Dynamics

2.1 The Idea of Infection and Immunization

The ideas in this subsection—the concepts of infection and immunization—have been developed in [18], for the readers convenience we want to summarize it briefly.

Definition 1

(Immunity) A strategy $\mathbf {x}\in \Delta ^n$ is said to be immune against $\mathbf {y}\in \Delta ^n$ if $\mathbf {x}$ has a neutral invasion barrier against $\mathbf {y}$, i.e. ${\overline{\epsilon _{\mathbf {x}}^{N}}}(\mathbf {y})>0$.

Note that immunity concerns neutral stability / neutral invasion barriers. An implication of this is that a strategy $\mathbf {x}\in \Delta ^n$ is neutrally stable if and only if it is immune against all $\mathbf {y}\in \Delta ^n$.

Definition 2

(Infectivity) A strategy $\mathbf {y}$ is said to be infective for $\mathbf {x}$ if $u(\mathbf {y}-\mathbf {x}|\mathbf {x})>0$.

If we associate the concept of infectivity with the concept of NE, one can say that $\mathbf {x}$ is a NES if and only if there exists no $\mathbf {y}$ which is infective for $\mathbf {x}$. Note that in general there can be strategies $\mathbf {y}$, which are neither infective for $\mathbf {x}$ nor is $\mathbf {x}$ immune against them. We can define the set of infective strategies for $\mathbf {x}$ as

$$\begin{aligned} \Upsilon (\mathbf {x})=\left\{ \mathbf {y}\in \Delta ^n:u(\mathbf {y}|\mathbf {x})>u(\mathbf {x})\right\} \hbox {.} \end{aligned}$$

Consider now a $\mathbf {y}\in \Upsilon (\mathbf {x})$. Then $\mathbf {x}$ gets invaded by $\mathbf {y}$ as long as the payoff of $\mathbf {y}$ against the postentry population is higher than the one of $\mathbf {x}$, and that leads to the following definition.

Definition 3

(Maximum Infection Level) We will call the maximum infection level for an infective strategy $\mathbf {y}$ against $\mathbf {x}$, denoted $\delta _{\mathbf {y}}(\mathbf {x)}$, the smallest population share $\epsilon$ such that for all smaller shares $\mathbf {y}$ fares better against the postentry population than $\mathbf {x}$ does; that is,

$$\begin{aligned} \delta _{\mathbf {y}}(\mathbf {x})=\inf \left\{ \epsilon \in (0,1):h_{\mathbf {x}}(\mathbf {y},\epsilon )\le 0\right\} \cup \left\{ 1\right\} \hbox {.} \end{aligned}$$

(1)

Note that this notation was introduced in [18] and that, contrary to the definition of the invasion barrier and the score function, the incumbent strategy $\mathbf {x}$ serves as the function argument and the invading strategy $\mathbf {y}$ is expressed by the subscript.

According to this definition, $\delta _{\mathbf {y}}(\mathbf {x})>0$ if $\mathbf {y}$ is infective for $\mathbf {x}$ and $\delta _{\mathbf {y}}(\mathbf {x})=0$ as long as $\mathbf {x}$ is immune against $\mathbf {y}$.

A natural question to ask is at which infection level the postentry population becomes immune against $\mathbf {y}$. It can be shown that for any $\mathbf {y}\in \Upsilon (\mathbf {x})$, this happens at $\epsilon = \delta _{\mathbf {y}}(\mathbf {x})$, i.e. that $\mathbf {z}=\left( 1- \delta _{\mathbf {y}}(\mathbf {x)}\right) \mathbf {x}+ \delta _{\mathbf {y}}(\mathbf {x)}\mathbf {y}$ is immune against $\mathbf {y}$ [18]. The definition for the maximum infection level is formulated above for a very general set-up (arbitrary sets of pure strategies, more general interaction than pairwise contest, i.e. playing the field). In our case of a linear payoff functions $F(\mathbf {x})=A\mathbf {x}$, the maximum infection level can be written down more explicitly:

$$\begin{aligned} \delta _{\mathbf {y}}(\mathbf {x})={\left\{ \begin{array}{ll} 0 &{} \quad \hbox {if }u(\mathbf {y}-\mathbf {x}|\mathbf {x})\le 0\\ \min \left\{ \frac{u(\mathbf {x}-\mathbf {y}|\mathbf {x})}{u(\mathbf {y}-\mathbf {x})},1\right\} &{}\quad \hbox {if }u(\mathbf {y}-\mathbf {x}|\mathbf {x})>0\hbox { and }u(\mathbf {y}-\mathbf {x})<0\\ 1 &{}\quad \hbox {if }u(\mathbf {y}-\mathbf {x}|\mathbf {x})>0\hbox { and }u(\mathbf {y}-\mathbf {x})\ge 0\end{array}\right. }\hbox {.} \end{aligned}$$

An interesting association between the maximum infection level and the invasion barrier is given by the following proposition. Note that the invading strategy for the invasion barrier is the incumbent strategy for the maximum infection level and vice versa.

Proposition 4

It holds that

$$\begin{aligned} {\overline{\epsilon _{\mathbf {y}}}}(\mathbf {x})\le 1-\delta _{\mathbf {y}}(\mathbf {x})\hbox { .} \end{aligned}$$

Proof

First consider the case of infective $\mathbf {y}$, i.e. $u(\mathbf {y}-\mathbf {x}|\mathbf {x})>0$. By using the identity

$$\begin{aligned} h_{\mathbf {y}}(\mathbf {x},\epsilon )=-h_{\mathbf {x}}(\mathbf {y},1-\epsilon ) \end{aligned}$$

and putting $\eta =1-\epsilon$, one can rewrite

$$\begin{aligned} {\overline{\epsilon _{\mathbf {y}}}}(\mathbf {x})=1-\sup \left\{ \eta \in (0,1):h_{\mathbf {x}}(\mathbf {y},\eta )\le 0\right\} \cup \left\{ 1\right\} \hbox { .} \end{aligned}$$

Now using $1-\sup A \le 1-\inf A$ it follows that

$$\begin{aligned} {\overline{\epsilon _{\mathbf {y}}}}(\mathbf {x})\le 1-\delta _{\mathbf {y}}(\mathbf {x})\hbox { .} \end{aligned}$$

Now consider not infective $\mathbf {y}$, i.e. $u(\mathbf {y}-\mathbf {x}|\mathbf {x})\le 0$, then by definition $\delta _{\mathbf {y}}(\mathbf {x})=0$. Then the claim reduces to ${\overline{\epsilon _{\mathbf {y}}}}(\mathbf {x})\le 1$, which obviously holds. $\square$

The principal idea behind the discrete time Infection–Immunization Dynamics is the following. Take an arbitrary state $\mathbf {x}^{t}\in \Delta ^n$ and take an infective strategy $\mathbf {y}\in \Upsilon (\mathbf {x}^{t})$. In essence a signpost is wanted which leads to a NES / fixed points of the dynamics. So it is useful to infect $\mathbf {x}^{t}$ with $\mathbf {y}$, and it will be infected until the postentry population, which is now called $\mathbf {x}^{t+1}$, is immune against $\mathbf {y}$, i.e. the maximum infection level $\delta _{\mathbf {y}}(\mathbf {x}^{t})$ is reached. If $\delta _{\mathbf {y}}(\mathbf {x}^{t})=1$ then population state $\mathbf {y}$ is directly reached. Now the same process starts again, this time with $\mathbf {x}^{t+1}$ as starting point and another infective strategy out of $\Upsilon (\mathbf {x}^{t+1})$. If this process is reiterated and the game behaves as expected, usually fewer and fewer possibilities for infection remain, and the dynamics arrives at a fixed point at some time.

Definition 5

(Strategy Selection Function) A function $S:\Delta ^n\rightarrow \Delta ^n$ is called a strategy selection function if

$$\begin{aligned} S(\mathbf {x})={\left\{ \begin{array}{ll} \mathbf {y}&{} \hbox {for some }\mathbf {y}\in \Upsilon (\mathbf {x}) \hbox { if } \Upsilon (\mathbf {x})\ne \emptyset \\ \mathbf {x}&{} \hbox {otherwise}\end{array}\right. }\hbox {.} \end{aligned}$$

Now we can define the resulting dynamics.

Definition 6

(Discrete Time Infection–Immunization Dynamics) The discrete time Infection–Immunization Dynamics, abbreviated as dIID in the following, is defined as

$$\begin{aligned} \mathbf {x}^{t+1}= \delta _{S(\mathbf {x}^{t})}(\mathbf {x}^{t})\left[ S(\mathbf {x}^{t})-\mathbf {x}^{t}\right] +\mathbf {x}^{t}\hbox {.} \end{aligned}$$

(2)

It was shown in [18] that for a given $\mathbf {x}\in \Delta ^n$, the following statements are equivalent:

• 1. $\Upsilon (\mathbf {x})=\emptyset$

• 2. $\mathbf {x}$ is a NES

• 3. $\mathbf {x}$ is a fixed point under dynamics (2).

The way the strategy selection function has been defined until now is not very practical in application, because any infective strategy can be selected by the strategy selection function. It can be made more convenient by constraining the strategy selection function in an apt way.

Definition 7

(Co-Strategy) Let ${\overline{\epsilon }}=\max \left\{ \epsilon \in \mathbb {R}:\mathbf {x}+\epsilon (\mathbf {x}-\mathbf {y})\in \Delta ^n\right\} \ge 0$. The co-strategy of $\mathbf {y}$ with respect to $\mathbf {x}$ is then given by

$$\begin{aligned} {\overline{\mathbf {y}_\mathbf {x}}}=\mathbf {x}+{\overline{\epsilon }}(\mathbf {x}-\mathbf {y})\hbox {.} \end{aligned}$$

Consider $\Delta ^n\subset \mathbb R^n$; then, ${\overline{\epsilon }}$ for a co-strategy of a pure strategy $\mathbf {e}^{i}$ with respect to $\mathbf {x}$ is given by ${\overline{\epsilon }}=\frac{x_{i}}{1-x_{i}}$ for all $\mathbf {x}\ne \mathbf {e}^{i}$.

The main idea is now to search for an infective strategy not on the whole simplex, but only within the pure strategies or their co-strategies. Take the strategy which has the highest absolute value of the payoff difference compared to $\mathbf {x}$, i.e. maximize $|u(\mathbf {\mathbf {e}}^{i}-\mathbf {x}|\mathbf {x})|$ over all $i\in N$. For the maximizing i, take the pure strategy $\mathbf {e}^{i}$ if $u({\mathbf {e}}^{i}-\mathbf {x}|\mathbf {x})>0$ and take the co-strategy ${\overline{\mathbf {e}^i_\mathbf {x}}}$ if $u({\mathbf {e}}^{i}-\mathbf {x}|\mathbf {x})<0$.

Definition 8

(Pure Strategy Selection Function) For any $\mathbf {x}\in \Delta ^n$, denote by

$$\begin{aligned}\begin{array}{rcl} {\mathcal {V}}_{+}(\mathbf {x}) &{} = &{} \left\{ u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {x}):i\in \tau _{+}(\mathbf {x})\right\} \quad \hbox {and}\\[1ex] {\mathcal {V}}_{-}(\mathbf {x}) &{} = &{} \left\{ u(\mathbf {x}-\mathbf {e}^{i}|\mathbf {x}):i\in \tau _{-}(\mathbf {x})\cap \sigma (\mathbf {x})\right\} \hbox {.}\end{array} \end{aligned}$$

If $\tau _{+}(\mathbf {x})=\tau _{-}(\mathbf {x})\cap \sigma (\mathbf {x})=\emptyset$, put $S_{p}(\mathbf {x})=\mathbf {x}$. Else, let

$$\begin{aligned} M(\mathbf {x})=\min \left[ \arg \max _{i}\left( {\mathcal {V}}_{+}(\mathbf {x})\cup {\mathcal {V}}_{-}(\mathbf {x})\right) \right] \end{aligned}$$

and define

$$\begin{aligned} S_{p}(\mathbf {x})={\left\{ \begin{array}{ll} \mathbf {e}^{i} &{} \hbox {if }i=M(\mathbf {x})\in \tau _{+}(\mathbf {x})\\ {\overline{\mathbf {e}^i_\mathbf {x}}} &{} \hbox {if }i=M(\mathbf {x})\in \tau _{-}(\mathbf {x})\cap \sigma (\mathbf {x})\hbox {.}\end{array}\right. } \end{aligned}$$

Since there exists an infective strategy for $\mathbf {x}$ if and only if $S_{p}(\mathbf {x})$ is infective for $\mathbf {x}$ (i.e. $S_p(\mathbf {x})\ne \mathbf {x}$) [18, Proposition 2], this pure strategy selection function $S_{p}(\mathbf {x})$ is indeed well defined.

Definition 9

(Pure dIID) The dIID which uses a pure strategy selection function as in Definition 8 is called pure dIID.

The pure dIID reduces the amount of possible values for the selection function from a possibly uncountable number, namely parts of the whole simplex, to a finite number, namely $2n+1$ strategies. These are the n pure strategies plus their n co-strategies and $\mathbf {x}$ itself. So also from a computational point of view the pure dIID is very interesting, for example see [19], where the pure dIID is used very successfully in various graph-based computer vision problems.

2.2 Infective Strategy Selection Correspondences

Using the dIID always one infective strategy gets selected, but the definition of $S(\mathbf {x})$ does not specify which one. However, the definition of $S_{p}(\mathbf {x})$ specifies it as the pure strategy or co-strategy with the smallest index. So the most general case is to admit all the strategies, which are infective for $\mathbf {x}$, as return values of the strategy selection, this leads to a strategy selection correspondence.

Definition 10

(Full Strategy Selection Correspondence) The correspondence $\mathcal {S}:\Delta ^n\rightarrow {\mathcal {P}}(\Delta ^n)$, where $\mathcal {P}( \cdot )$ denotes the power set, is called full strategy selection correspondence if the following holds

$$\begin{aligned} \mathcal {S}(\mathbf {x})={\left\{ \begin{array}{ll} \Upsilon (\mathbf {x}) &{} \hbox { if }\Upsilon (\mathbf {x})\ne \emptyset \\ \left\{ \mathbf {x}\right\} &{} \hbox {otherwise.}\end{array}\right. } \end{aligned}$$

(3)

So all the infective strategies for $\mathbf {x}$ are returned by the correspondence, if there are infective ones, otherwise $\mathbf {x}$. We also can introduce a pure strategy selection correspondence, again similar to the pure strategy selection function.

Definition 11

(Pure Strategy Selection Correspondence) With ${\mathcal {V}}_{\pm }$ as in Definition 8, let

$$\begin{aligned} \mathcal {M}(\mathbf {x})=\arg \max _{i}\left( {\mathcal {V}}_{+}(\mathbf {x})\cup {\mathcal {V}}_{-}(\mathbf {x})\right) \hbox {.} \end{aligned}$$

The correspondence $\mathcal {S}_{p}:\Delta ^n\rightarrow {\mathcal {P}}(\Delta ^n)$ with

$$\begin{aligned} \mathcal {S}_{p}(\mathbf {x})={\left\{ \begin{array}{ll} \left\{ \mathbf {e}^{i}:i\in \mathcal {M}(\mathbf {x})\cap \tau _{+}(\mathbf {x})\right\} \cup \left\{ {\overline{\mathbf {e}^{i}}}_{\mathbf {x}}:i\in \mathcal {M}(\mathbf {x})\cap \tau _{-}(\mathbf {x})\cap \sigma (\mathbf {x})\right\} &{} \hbox {if }\mathcal {M}(\mathbf {x})\ne \emptyset \\ \left\{ \mathbf {x}\right\} &{} \hbox {otherwise}\end{array}\right. } \end{aligned}$$

is called pure strategy selection correspondence. Note that $\mathcal {M}(\mathbf {x})$ returns a set of indices (or is empty) while $\mathcal {S}_{p}(\mathbf {x})$ returns the corresponding pure strategies and co-strategies (or $\mathbf {x}$ itself).

So $\mathcal {S}_{p}(\mathbf {x})$ returns the set of all pure strategies or co-strategies with the highest relative payoff difference (otherwise $\mathbf {x}$), and it selects a set of infective strategies if and only if the full strategy selection correspondence selects a set of infective strategies. This can be proved analogously to [18, Proposition 2].

The following properties limit the population share of losing strategies.

Proposition 12

Consider the pure strategy selection correspondence.

• (a) At a point $\mathbf {x}$ where (infective) pure strategies and co-strategies are returned, we have ${\overline{\epsilon }}\le 1$ for any returned co-strategy, or equivalently, $x_{j}\le \frac{1}{2}$ for co-strategy ${\overline{\mathbf {e}^j_\mathbf {x}}}$.

• (b) At a point $\mathbf {x}$ where only (infective) co-strategies are returned, we have ${\overline{\epsilon }}<1$ for any returned co-strategy, or equivalently, $x_{j}<\frac{1}{2}$ for co-strategy ${\overline{\mathbf {e}^j_\mathbf {x}}}$.

Proof

• (a) Consider any (returned) co-strategy ${\overline{\mathbf {e}^j_\mathbf {x}}}$. Then ${\overline{\mathbf {e}^j_\mathbf {x}}}$ lies on the face of the simplex which is given by

  $$\begin{aligned} conv\left\{ \mathbf {e}^{1},\ldots ,\mathbf {e}^{j-1},\mathbf {e}^{j+1},\ldots ,\mathbf {e}^{n}\right\} \end{aligned}$$

  and $u({\overline{\mathbf {e}^j_\mathbf {x}}}-\mathbf {x}|\mathbf {x})={\overline{\epsilon }}u(\mathbf {x}-\mathbf {e}^{j}|\mathbf {x})$ by definition of a co-strategy. By the linearity of the payoff function it follows that

  $$\begin{aligned} {\overline{\epsilon }}u(\mathbf {x}-\mathbf {e}^{j}|\mathbf {x})=\sum _{i\in N \backslash \{j\} } \lambda _{i}u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {x}) \end{aligned}$$

  where $\lambda \in \triangle ^{n-1}$. Hence,

  $$\begin{aligned} {\overline{\epsilon }}u(\mathbf {x}-\mathbf {e}^{j}|\mathbf {x})=\sum _{i\in N \backslash \{j\} } \lambda _{i}u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {x})\le \max _{i\in N \backslash \{j\} }u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {x})\hbox { .} \end{aligned}$$

  But by the assumption that that at least one pure strategy is returned by $\mathcal {S}_{p}(\mathbf {x})$ and that ${\overline{\mathbf {e}^j_\mathbf {x}}}$ is also returned, it follows that $u(\mathbf {x}-\mathbf {e}^{j}|\mathbf {x})=\max _{i\in N \backslash \{j\} }u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {x})\ne 0$ by definition of the pure selection correspondence. From that one can see that ${\overline{\epsilon }}\le 1$. That $x_{j}\le \frac{1}{2}$ can be seen by using ${\overline{\epsilon }}=\frac{x_{j}}{1-x_{j}}$ from the definition of the co-strategy.

• (b) In this case

  $$\begin{aligned} {\overline{\epsilon }}u(\mathbf {x}-\mathbf {e}^{j}|\mathbf {x}) \le \max _{i\in N \backslash \{j\} }u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {x}) < u(\mathbf {x}-\mathbf {e}^{j}|\mathbf {x})\hbox { ,} \end{aligned}$$

  because no pure strategy is returned by the pure strategy selection correspondence. The inequality above implies that ${\overline{\epsilon }}<1$.

$\square$

2.3 The Continuous Time Infection–Immunization Dynamics

We now derive a continuous time Infection–Immunization Dynamics from the dIID, first in the most general sense possible. If we apply the full strategy selection correspondence to the dIID, we get a difference inclusion of the form

$$\begin{aligned} \mathbf {x}^{t+1}\in \delta _{\mathcal {S}(\mathbf {x}^{t})}(\mathbf {x}^{t})\left[ \mathcal {S}(\mathbf {x}^{t})-\mathbf {x}^{t}\right] +\mathbf {x}^{t}\hbox {.} \end{aligned}$$

We can derive a differential inclusion from the approximation ${\dot{\mathbf {x}}}\approx \mathbf {x}^{t+1}-\mathbf {x}^{t}$, putting $\mathbf {x}^{t}=\mathbf {x}$, which yields

$$\begin{aligned} {\dot{\mathbf {x}}}\in \delta _{S(\mathbf {x})}(\mathbf {x})\left( \mathcal {S}(\mathbf {x})-\mathbf {x}\right) \hbox {.} \end{aligned}$$

The offset term $\delta _{\mathcal {S}(\mathbf {x})}(\mathbf {x})$ can be omitted for the following reasons: if $\delta _{\mathcal {S}(\mathbf {x})}(\mathbf {x})>0$, then it is just a factor which rescales time and it does not influence the qualitative behaviour of the differential inclusion. But $\delta _{\mathcal {S}(\mathbf {x})}(\mathbf {x})\in [0,1]$ by definition, so the critical case is $\delta _{\mathcal {S}(\mathbf {x})}(\mathbf {x})=0$, which must not influence the differential inclusion, i.e. $\mathcal {S}(\mathbf {x})-\mathbf {x}=\{\mathbf {o}\}$ must hold in that case.

Lemma 13

$\delta _{\mathcal {S}(\mathbf {x})}(\mathbf {x})=0$ if and only if $\mathcal {S}(\mathbf {x})=\left\{ \mathbf {x}\right\}$.

Proof

By definition of the maximum infection level for linear payoff functions $\delta _{\mathcal {S}(\mathbf {x})}(\mathbf {x})=0$ implies that $u(\mathbf {y}-\mathbf {x}|\mathbf {x})\le 0, \; \forall \mathbf {y}\in \mathcal {S}(\mathbf {x})$. $\mathcal {S}(\mathbf {x})$ returns $\Upsilon (\mathbf {x})$ if there exists at least one infective strategy for $\mathbf {x}$, otherwise $\left\{ \mathbf {x}\right\}$. Let us assume that the strategy selection correspondence returns $\Upsilon (\mathbf {x})$ and we select an arbitrary $\mathbf {y}\in \Upsilon (\mathbf {x})$. Then $\mathbf {y}$ is infective, i.e. $u(\mathbf {y}-\mathbf {x}|\mathbf {x})>0$, resulting in a contradiction to the assumption. So $\mathcal {S}(\mathbf {x})$ must return $\left\{ \mathbf {x}\right\}$ then, which was to be shown. On the other hand, if $\mathcal {S}(\mathbf {x})=\left\{ \mathbf {x}\right\}$ then $\delta _{\mathcal {S}(\mathbf {x})}(\mathbf {x})$ becomes $\delta _{\mathbf {x}}(\mathbf {x})$ which is per definition zero. A strategy can never be infective for itself.$\square$

So we can drop $\delta _{\mathcal {S}(\mathbf {x})}(\mathbf {x})$ and define:

Definition 14

(Continuous Time IID, Pure Continuous Time IID) The following differential inclusion is called the continuous time Infection–Immunization Dynamics, abbreviated cIID:

$$\begin{aligned} {\dot{\mathbf {x}}}\in \mathcal {S}(\mathbf {x})-\mathbf {x} \end{aligned}$$

(4)

with $\mathcal {S}(\mathbf {x})$ defined as in (3). When the pure strategy selection correspondence $\mathcal {S}_{p}(\mathbf {x})$ is used, the model is called pure cIID.

Analogous to the dIID we can show the following.

Theorem 15

A given $\mathbf {x}\in \Delta ^n$ is a NES if and only if $\mathbf {x}$ is a fixed point under dynamics (4), i.e. $\mathcal {S}(\mathbf {x})=\{\mathbf {x}\}$.

Proof

We know that $\mathbf {x}$ is a NES if and only if $\Upsilon (\mathbf {x})=\emptyset$ [18, Theorem 1]. The rest is analogous to the proof of Lemma 13.$\square$

One may wonder about the relevance of cIID. While dIID was constructed mostly for its computational power, a continuous time version does not have this advantage. Still, we are driven to analyse the properties of the continuous time version mainly for theoretic insight. And the outcome is more than interesting—a reasonable and uncomplicated dynamics, which exhibits very non-trivial and non-standard behaviour in many senses. This will be explained in the next section.

It is interesting to relate the pure cIID to the Best Response Dynamics (abbreviated as BRD), which is defined as

$$\begin{aligned} {\dot{\mathbf {x}}}(t)\in \beta (\mathbf {x}(t))-\mathbf {x}(t)\hbox {.} \end{aligned}$$

They behave similar to a certain extent, the pure cIID differs from the BRD in two points, though, see Sect. 1. A property of the cIID which is shared by the BRD is the fact that its faces are not forward invariant, i.e. strategies, which are currently not played by the population can emerge over time. This property is not shared by many other popular dynamics, among them also the well-studied Replicator Dynamics.

3 When Do Solutions for the cIID Exist?

3.1 Existence Theorems for Differential Inclusions: A Short Review

Since the cIID is constructed similar to the BRD, we want to analyse this dynamics first. One approach to prove the existence of solutions for the BRD is to invoke existence theorems for differential inclusions. Standard conditions for solution existence can be found e.g. in [4] or in [21]. Usually, the solutions are not unique, however. For the readers’ convenience, we shortly repeat some of the existence theorems here; to this end, we first have to recall some technical notions for a correspondence $\mathcal {F}:X\rightarrow {\mathcal {P}}(Y)$. $\mathcal {F}$ is said to be upper hemicontinuous or u.h.c. at $\mathbf {x}_{0}$ if for any neighbourhood V of $\mathcal {F}(\mathbf {x}_{0})$ there exists a neighbourhood U of $\mathbf {x}_{0}$ such that $\mathcal {F}(\mathbf {x})$ is a subset of V for all $\mathbf {x}$ in U and it is said to be lower hemicontinuous or l.h.c. at $\mathbf {x}_{0}$ if for any open set V intersecting $\mathcal {F}(\mathbf {x}_{0})$ there exists a neighbourhood U of $\mathbf {x}_{0}$ such that $\mathcal {F}(\mathbf {x})$ intersects V for all $\mathbf {x}$ in U. Finally, $\mathcal {F}$ is said to be Lipschitz continuous if there exists a $k>0$ such that $\mathcal {F}(\mathbf {x}_{1})\subset \mathcal {F}(\mathbf {x}_{2})+k\Vert \mathbf {x}_{1}-\mathbf {x}_{2}\Vert _{2}\mathcal {B}$ for all $\mathbf {x}_{1},\mathbf {x}_{2}\in X$ where $\mathcal {B}=\left\{ \mathbf {y}\in \mathbb R^n:\Vert \mathbf {y}\Vert _{2}\le 1\right\}$. Now, a differential inclusion with correspondence $\mathcal {F}:X\rightarrow {\mathcal {P}}(Y)$ has (non-unique) solutions guaranteed, if either

• $\mathcal {F}$ is u.h.c. or $\mathcal {F}$ is l.h.c.

  and

• $\mathcal {F}$ has non-empty, compact and convex values

  or

• ($\mathcal {F}$ is u.h.c. and l.h.c.) or ($\mathcal {F}$ is Lipschitz continuous) or ($\mathcal {F}$ is l.h.c. including some further restrictions) or ($\mathcal {F}$ is u.h.c including some further restrictions)

  and

• $\mathcal {F}$ has non-empty and compact values, but it need not necessarily have convex values.

Summarizing this, one can see that the condition of convex values can be dropped if stronger restrictions on $\mathcal {F}$ are imposed. But the requirement of compact values remains and this will be crucial for the cIID as we will see below. The BRD has non-empty, convex and compact values and is u.h.c.; hence, existence theorems can be applied. If we try the same for the cIID, it turns out that no standard technique existence theorem holds:

Proposition 16

The correspondence $\mathcal {F}(\mathbf {x})=\mathcal {S}(\mathbf {x})-\mathbf {x}$ from the differential inclusion (4) is non-empty and has convex and bounded values, but they need not be closed (and therefore not compact). The correspondence $\mathcal {F}$ is neither necessarily u.h.c. nor Lipschitz continuous, not even for symmetric payoff matrices.

Proof

It is obvious that $\mathcal {F}(\mathbf {x})$ has non-empty, convex and bounded values by definition of $\mathcal {S}(\mathbf {x})$. For the other properties consider the payoff matrix

$$\begin{aligned}A=\begin{pmatrix} 1 &{} 1 &{} 0\\ 1 &{} 1 &{} 0\\ 0 &{} 0 &{} -1\end{pmatrix}\,. \end{aligned}$$

The payoffs of this game take the form $u(\mathbf {y}|\mathbf {x})=(y_{1}+y_{2})(x_{1}+x_{2})-x_{3}y_{3}$, which shows that only states with no positive weight on $\mathbf {e}^{3}$ are NES, i.e. the face of the simplex connecting $\mathbf {e}^{1}$ and $\mathbf {e}^{2}$. Now set $\mathbf {y}=\mathbf {e}^{3}$, then $\mathcal {S}(\mathbf {y})=\Delta ^3\backslash \left\{ \mathbf {y}\right\}$, and consider a sequence in $\mathcal {F}(\mathbf {y})=\mathcal {S}(\mathbf {y})-\mathbf {y}$ which converges towards $\mathbf {o}$. Because $\mathbf {o}\notin \mathcal {F}(\mathbf {y})$, the set $\mathcal {F}(\mathbf {y})$ is not closed.

Next we show that $\mathcal {F}(\mathbf {x})$ is not u.h.c. at the point $\mathbf {x}_{0}=\frac{1}{2}\mathbf {e}^{1}+\frac{1}{2}\mathbf {e}^{2}$, which is a NES, so $\mathcal {S}(\mathbf {x}_{0})=\left\{ \mathbf {x}_{0}\right\}$. Take a sufficiently small neighbourhood V of $\mathcal {F}(\mathbf {x}_{0})=\left\{ \mathbf {o}\right\}$.

For an arbitrarily given neighbourhood U of $\mathbf {x}_{0}$, choose $\mathbf {x}=(1-\epsilon )\mathbf {x}_{0}+\epsilon \mathbf {e}^{3}$ with $\epsilon \in (0,1]$ small enough that $\mathbf {x}\in U(\mathbf {x}_{0})$. Pure strategy $\mathbf {e}^{1}$ is infective for $\mathbf {x}$ because $u(\mathbf {x}|\mathbf {x})=1-2\epsilon <1-\epsilon =u(\mathbf {e}^{1}|\mathbf {x})$. Therefore, $\mathbf {e}^{1}-\mathbf {x}\in \mathcal {F}(\mathbf {x})$, but for any $\epsilon \in (0,1]$ we have $\mathbf {e}^{1}-\mathbf {x}\notin V$, so $\mathcal {F}(\mathbf {x})$ cannot be u.h.c.

Finally, we show that $\mathcal {F}$ is not Lipschitz continuous at the point $\mathbf {x}_{0}$. Let $k>0$, choose $\epsilon$ small enough, e.g. set $\epsilon :=\frac{1}{100k}$, and consider again $\mathbf {x}=(1-\epsilon )\mathbf {x}_{0}+\epsilon \mathbf {e}^{3}$. Then $\mathcal {F}(\mathbf {x}_{0})+k\Vert \mathbf {x}-\mathbf {x}_{0}\Vert _{2}\mathcal {B}=\mathbf {o}+\frac{1}{100}\sqrt{\frac{3}{2}}\mathcal {B}$, which definitely does not include $\mathbf {e}^{1}-\mathbf {x}$. So $\mathbf {e}^{1}-\mathbf {x}\in \mathcal {F}(\mathbf {x}) \not \subset \frac{1}{100}\sqrt{\frac{3}{2}}\mathcal {B}$, which was to be shown. $\square$

The reason why none of the existence theorems for differential inclusions hold for the cIID lies in the non-compactness of its values. But even if $\mathcal {F}(\mathbf {x})=\mathcal {S}(\mathbf {x})-\mathbf {x}$ would have compact values, the correspondence still would not be u.h.c. or Lipschitz continuous. It remains yet unclear whether it would necessarily be l.h.c.

Proposition 17

The correspondence $\mathcal {F}(\mathbf {x})=\mathcal {S}_{p}(\mathbf {x})-\mathbf {x}$ for the pure cIID takes compact values but it is not necessarily convex valued. The correspondence is neither necessarily u.h.c. nor l.h.c. nor Lipschitz continuous, not even for symmetric payoff matrices.

Proof

Since $\mathcal {F}(\mathbf {x})\ne \emptyset$ is a finite set it is compact but not convex, if it contains more than one element. For the other properties consider the game with payoff matrix

$$\begin{aligned}B=\begin{pmatrix}1 &{} 1 &{} 0\\ 1 &{} 1 &{} 2\\ 0 &{} 2 &{} 1 \end{pmatrix}\quad \hbox {and the NES}\quad \mathbf {x}_{0}=\begin{pmatrix}\frac{1}{2}\\ \frac{1}{2}\\ 0\end{pmatrix} \end{aligned}$$

(see [18] for details and graphics). The game has infinitely many NES and the one relevant for this counterexample is given by $\mathbf {x}_{0}$.

Now we can show that $\mathcal {F}(\mathbf {x})$ is neither u.h.c. nor l.h.c. at $\mathbf {x}_{0}$. Take a sufficiently small neighbourhood V of $\mathcal {F}(\mathbf {x}_{0})=\left\{ \mathbf {o}\right\}$. Let U be an arbitrarily given neighbourhood of $\mathbf {x}_{0}$ , then $\mathbf {x}=(1-\epsilon )\mathbf {x}_{0}+\epsilon \mathbf {e}^{3}\in U$ if $\epsilon >0$ is small enough. We have $u(\mathbf {e}^{1}-\mathbf {x}|\mathbf {x})=-\epsilon$ while $u(\mathbf {e}^{2}-\mathbf {x}|\mathbf {x})=\epsilon$ and $u(\mathbf {e}^{3}-\mathbf {x}|\mathbf {x})=0$, showing that $\mathcal {S}_{p}(\mathbf {x})=\left\{ {\overline{\mathbf {e}^{1}}}_{\mathbf {x}},\mathbf {e}^{2}\right\}$. So

$$\begin{aligned} \mathcal {F}(\mathbf {x})=\left\{ {\overline{\mathbf {e}^{1}}}_{\mathbf {x}}-\mathbf {x},\mathbf {e}^{2}-\mathbf {x}\right\} =\left\{ \begin{pmatrix}\frac{-1+\epsilon }{2}\\ \frac{1+\epsilon }{2}\\ -\epsilon \end{pmatrix}, \begin{pmatrix}\frac{\epsilon -1}{2}\\ \frac{(1-\epsilon )^{2}}{2(1+\epsilon )}\\ \frac{\epsilon (1-\epsilon )}{1+\epsilon }\end{pmatrix}\right\} \,, \end{aligned}$$

and thus $\mathcal {F}(\mathbf {x})\cap V=\emptyset$, implying also $\mathcal {F}(\mathbf {x})\not \subseteq V$ for small enough $\epsilon >0$. Therefore, $\mathcal {F}$ is neither u.h.c. nor l.h.c. at $\mathbf {x}_{0}$.

Finally, the same argument as in the proof of Proposition 16 applied to the payoff matrix B at the NES $\mathbf {x}_{0}$ with $\mathbf {x}=(1-\epsilon )\mathbf {x}_{0}+\epsilon \mathbf {e}^{3}$ and $\mathbf {e}^{2}-\mathbf {x}\in \mathcal {F}(\mathbf {x})$ establishes failure of Lipschitz continuity. $\square$

One other known method to establish existence of solutions is the use of directionally continuous selections, see [9] and [10]. This concept uses a weaker condition than u.h.c. or l.h.c., but is still strong enough to guarantee existence. Unfortunately the correspondence $\mathcal {S}_{p}(\mathbf {x})-\mathbf {x}$ of Proposition 17 does not satisfy this: there exists no selection such that $\mathbf {x}_0$ is directionally continuous.

3.2 Existence and Construction of Solutions

One approach to the existence of solutions is to construct the trajectory piecewise, as done for the BRD, see [13]. Generically, for “most” times t the pure strategy selection correspondence will only select one infective strategy $\mathbf {b}$. During an open time interval I where this strategy does not change, the system can be described as ${\dot{\mathbf {x}}}=\mathbf {b}-\mathbf {x}$. The solution starting, say, at $t_{0}\in I$ with $\mathbf {x}(t_{0})=\mathbf {x}_{0}$, has the form

$$\begin{aligned} \mathbf {x}(t)=\mathbf {x}_{0}e^{-(t-t_{0})}+\left( 1-e^{-(t-t_{0})}\right) \mathbf {b}\,,\; t\in I\,. \end{aligned}$$

(5)

This straight line movement towards $\mathbf {b}$ continues until a time point is reached where more than one strategy is selected, which is the case at any $\mathbf {x}$ where $|u(\mathbf {e}^i-\mathbf {x}|\mathbf {x})|=|u(\mathbf {e}^j-\mathbf {x}|\mathbf {x})|$ for two different $i,j\in N$. Then we can restart the whole process with a “new” $\mathbf {b}$, and in the end we get a path which is piecewise composed of straight line segments. This leads to a Carathéodory solution of the system. Clearly, $\mathbf {b}\in \mathcal {S}_{p}\left( \mathbf {x}\right)$ must hold in order that $\mathbf {b}$ can be considered a “solution target” for a positive time interval. But $\mathbf {b}$ must also be selected if the system moves slightly into the direction of $\mathbf {b}$, i.e. $\mathbf {b}\in \mathcal {S}_{p}\left( (1-\epsilon )\mathbf {x}+\epsilon \mathbf {b}\right)$ for small $\epsilon$, otherwise the trajectory could not move along this straight line towards $\mathbf {b}$. The important question is: does such a strategy $\mathbf {b}$, which enables further motion, always exist? Although heuristics show that this is the case, this problem remains open.

A different possibility is to use a more general solution concept for differential equations/inclusions, namely the notion of a Krasovsky solution, for the definition see e.g. [12] or [5] for a good comparison to other solution concepts.

Definition 18

(Krasovsky set-valued map) The Krasovsky set-valued map $\mathcal {F}_K:\mathbb R^n \rightarrow \mathcal {P}(\mathbb R^n)$ for a correspondence $\mathcal {F}:\mathbb R^n \rightarrow \mathcal {P}(\mathbb R^n)$ is defined by

$$\begin{aligned} \mathcal {F}_{K}(\mathbf {x})=\cap _{\delta >0}{\overline{conv}}\{\mathcal {F}(\mathcal {B}(\mathbf {x},\delta ))\},\,\mathbf {x}\in \mathbb {R}^{n} \end{aligned}$$

where ${\overline{conv}}$ denotes the convex closure and $\mathcal {B}(\mathbf {x},\delta )$ denotes the open ball around $\mathbf {x}$ (constrained to the simplex) with radius $\delta$.

Definition 19

(Krasovsky solution) A Krasovsky solution of ${\dot{\mathbf {x}}}(t) \in \mathcal {F}(\mathbf {x}(t))$ defined on $[t_{0},t_{1}]\subset [0,\infty )$ is an absolutely continuous function $\mathbf {x}:[t_{0},t_{1}]\rightarrow \mathbb {R}^{n}$ such that ${\dot{\mathbf {x}}}(t)\in \mathcal {F}_{K}(\mathbf {x}(t))$ for almost every $t\in [t_{0},t_{1}]$. In other words, a Krasovsky solution of $\mathcal {F}$ is a Carathéodory solution of $\mathcal {F}_{K}$.

It is obvious that every cIID has a solution in the Krasovsky sense. For the pure cIID it is not so obvious, for this recall the Closed Graph Theorem first, see e.g. [3].

Theorem 20

(Closed Graph Theorem) A correspondence $\mathcal {F}:\mathbb R^{n}\rightarrow \mathcal {P}(\mathbb R^{n})$ with closed values is u.h.c. if and only if it has a closed graph, i.e. if $x_{n}$ and $v_{n}$ are sequences with $x_{n}\rightarrow x_{0}$ and $v_{n}\rightarrow v_{0}$, and $v_{n}\in \mathcal {F}(x_{n})$ then $v_{0}\in \mathcal {F}(x_{0})$.

Corollary 21

Consider a pure strategy selection correspondence $\mathcal {S}_{p}:\Delta ^n\rightarrow \mathcal {P}(\Delta ^n)$. Then the Krasovsky map of $\mathcal {F}(\mathbf {x})=\mathcal {S}_{p}(\mathbf {x})-\mathbf {x}$, denoted by $\mathcal {F}_{K}$, is u.h.c., bounded and has non-empty closed convex values. Hence, a solution to any initial value problem exists.

Proof

By the properties of the pure strategy selection correspondence $\mathcal {F}$ is bounded and non-empty; hence, $\mathcal {F}_{K}$ is bounded and non-empty. Using the closed convex hull in the Krasovsky map implies that $\mathcal {F}_{K}$ has closed and convex values. Furthermore, it implies that $\mathcal {F}_K$ has a closed graph; hence, the closed graph theorem holds and $\mathcal {F}_K$ is u.h.c. $\square$

4 Some Stability Results

The concept of adjustment dynamics [20], also called positive correlation [22], is usually used for evolutionary dynamics ${\dot{\mathbf {x}}}= F(\mathbf {x})$, we extend it to differential inclusions ${\dot{\mathbf {x}}}\in \mathcal {F}(\mathbf {x})$ – the following two conditions have to hold:

• 1. $\mathbf {y}^\top \mathcal {F}(\mathbf {x})\ge 0\quad \hbox {for all }\mathbf {y}\in \mathcal {F}(\mathbf {x})\hbox { and for all }\mathbf {x}\in \Delta ^n$ and

• 2. $\mathbf {y}^\top \mathcal {F}(\mathbf {x})>0\quad \hbox {for all }\mathbf {y}\in \mathcal {F}(\mathbf {x})\hbox { if }\mathbf {o}\notin \mathcal {F}(\mathbf {x})\,$.

Proposition 22

The cIID is an adjustment dynamics.

Proof

Here $\mathcal {F}(\mathbf {x})=\mathcal {S}(\mathbf {x})-\mathbf {x}$. Let $\mathbf {z}\in \mathcal {S}(\mathbf {x})$ be infective, hence $u(\mathbf {z}-\mathbf {x}|\mathbf {x})>0$, what was to be shown. If $\mathcal {S}(\mathbf {x})=\{\mathbf {x}\}$, then $\mathcal {F}(\mathbf {x})=\left\{ \mathbf {o}\right\}$ and $u(\mathbf {o}|\mathbf {x})=\mathbf {o}^{T}A\mathbf {x}=0$; hence, the cIID is an adjustment dynamics. $\square$

Now for an adjustment dynamics we know (see e.g. [14]):

• If A is symmetric, then the average population payoff is monotonically increasing, i.e. $t_{1}<t_{2}\Rightarrow u\left( \mathbf {x}(t_{1})\right) \le u\left( \mathbf {x}(t_{2})\right)$.

• Strict NES are asymptotically stable for games when $n\le 3$.

Remark:

[14] claims that strict NES are always asymptotically stable. But it has been shown in [2] that this is not the case.

We want to show a further stability result, previously established for BRD [14]. As usual, we denote by $int(\Delta ^n)=\left\{ \mathbf {x}\in \Delta ^n:x_{i}>0\hbox { for all }i\in N\right\}$ the relative interior of the simplex. Recall that if a point $\mathbf {p}\in int(\Delta ^n)$ is an ESS then there exists no other NES than $\mathbf {p}$ in the game [14]. Further [1, 11, 14] proved that if $\mathbf {p}\in int(\Delta ^n)$ is a NES, then $\mathbf {p}$ is an ESS if and only if

$$\begin{aligned} u(\mathbf {y}|\mathbf {y})<0\hbox { for all }\mathbf {y}\ne \mathbf {o}\hbox { with }\sum _{i=1}^{n}y_{i}=0\hbox {.} \end{aligned}$$

(6)

Theorem 23

Let $\mathbf {p}\in int(\Delta ^n)$ be an ESS, then $\mathbf {p}$ is globally asymptotically stable for the pure cIID, provided that solutions exist for the game.

Proof

Consider the function $V(\mathbf {x})=\max _{j}|u(\mathbf {e^{j}}-\mathbf {x}|\mathbf {x})|$, then $V(\mathbf {x})$ is a strict Lyapunov function, because $V(\mathbf {x})>0$ whenever $\mathbf {x}$ is not a NES and $V(\mathbf {x})=0$ if $\mathbf {x}$ is a NES. According to above, this can happen only at the ESS $\mathbf {p}$.

Along a linear path where the system can be described as ${\dot{\mathbf {x}}}=\mathbf {e^{i}}-\mathbf {x}$ the Lyapunov function can be rewritten as $V(\mathbf {x})=u(\mathbf {e^{i}}-\mathbf {x}|\mathbf {x})$. Taking the derivative with respect to time leads to

$$\begin{aligned} {\dot{V}}(\mathbf {x})=-u({\dot{\mathbf {x}}}|\mathbf {x})+u(\mathbf {e}^{i}-\mathbf {x}|{\dot{\mathbf {x}}}) \end{aligned}$$

and by using ${\dot{\mathbf {x}}}=\mathbf {e}^{i}-\mathbf {x}$ this can be rewritten as

$$\begin{aligned} {\dot{V}}(\mathbf {x})=-u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {x})+u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {e}^{i}-\mathbf {x})\hbox {.} \end{aligned}$$

We know that $u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {x})>0$ because of infectivity and furthermore $u(\mathbf {e}^{i}-\mathbf {x}|\mathbf {e}^{i}-\mathbf {x})<0$ because of (6); hence, ${\dot{V}}(\mathbf {x})<0$.

Along a linear path where the system can be described as ${\dot{\mathbf {x}}}={\overline{\mathbf {e}^i_\mathbf {x}}}-\mathbf {x}$ the Lyapunov function can be rewritten as $V(\mathbf {x})=\frac{1-x_{i}}{x_{i}}u({\overline{\mathbf {e}^i_\mathbf {x}}}-\mathbf {x}|\mathbf {x})$. Taking the derivative with respect to time leads to

$$\begin{aligned} {\dot{V}}(\mathbf {x})=\frac{1}{x_{i}^{2}}u({\overline{\mathbf {e}^i_\mathbf {x}}}-\mathbf {x}|\mathbf {x}){\dot{x_{i}}}+\frac{1-x_{i}}{x_{i}}\left( u({\overline{\mathbf {e}^i_\mathbf {x}}}-\mathbf {x}|{\dot{\mathbf {x}}})-u({\dot{\mathbf {x}}}|\mathbf {x})\right) \end{aligned}$$

and by using ${\dot{\mathbf {x}}}={\overline{\mathbf {e}^{i}}}_{\mathbf {x}}-\mathbf {x}$ as well as ${\dot{x_{i}}}=-x_{i}$ this can be rewritten as

$$\begin{aligned} {\dot{V}}(\mathbf {x})=-\frac{2-x_{i}}{x_{i}}u({\overline{\mathbf {e}^i_\mathbf {x}}}-\mathbf {x}|\mathbf {x})+\frac{1-x_{i}}{x_{i}}u({\overline{\mathbf {e}^i_\mathbf {x}}}-\mathbf {x})\hbox {.} \end{aligned}$$

By the same argument as before this expression is always negative. Note that $x_{i}=0$ and $x_{i}=1$ cannot occur because of the definition of the pure strategy selection correspondence. Therefore, $V(\mathbf {x})$ is a strict Lyapunov function (for all the possible Carathéodory solutions) and $\mathbf {p}$ is globally asymptotically stable. $\square$

Although the pure cIID behaves as expected for a game dynamics in the interior of the simplex, the behaviour is problematic at the faces of the simplex. The system can converge towards a point on a face, which is not a fixed point. Consider the following example.

Example 24

Consider the game with payoff matrix $A=\left( \begin{array}{ccc} -2 &{} -9 &{} 8\\ -9 &{} 4 &{} 0\\ 8 &{} 0 &{} -4 \end{array}\right)$ and an initial value $\mathbf {x}_{0}(0)=(\frac{9}{20},\frac{1}{20},\frac{1}{2})^\top$. For $\mathbf {x}_{0}$ the only selected strategy is ${\overline{\mathbf {e}^2_{\mathbf {x}_0}}}=(\frac{9}{19},0,\frac{10}{19}^\top )$ and it remains the same when the system moves according to ${\dot{\mathbf {x}}}=S(\mathbf {x})-\mathbf {x}={\overline{\mathbf {e}^2_{\mathbf {x}_0}}}-\mathbf {x}$. Solving this differential equation leads to $\mathbf {x}(t)=\left( \begin{array}{c} \frac{9}{19}-\frac{9}{380}e^{-t}\\ \frac{1}{20}e^{-t}\\ \frac{10}{19}-\frac{1}{38}e^{-t} \end{array}\right)$ and $\lim _{t\rightarrow \infty }\mathbf {x}(t)={\overline{\mathbf {e}^2_{\mathbf {x}_0}}}$. But ${\overline{\mathbf {e}^2_{\mathbf {x}_0}}}$ is not a fixed point, $S({\overline{\mathbf {e}^2_{\mathbf {x}_0}}})=\mathbf {e}^{1}$.

This behaviour is explainable by the construction of the strategy selection function/correspondence, it also appears in the discrete version. But since the pure dIID “jumps”, this behaviour does not matter. It only becomes critical in the continuous version.

5 Conclusion

Using the recently proposed Infection–Immunization Dynamics, which was introduced for discrete time (dIID and pure dIID), we adapt this evolutionary dynamics for continuous time (cIID and pure cIID). It turns out that standard techniques for differential inclusions do not work for the cIID and the pure cIID, lacking compactness in the case of the cIID and lacking hemicontinuity in the case of the pure cIID.

However, we can show that an extended solution concept works, namely the one by Krasovsky: every cIID and every pure cIID has a (non-unique) solution in the Krasovsky sense. Similar to the Best Response Dynamics it can be shown what solutions look like.

Furthermore, we show that the cIID is an adjustment dynamics and show stability result for the pure cIID regarding interior ESS. Unfortunately the pure cIID exhibits a problematic behaviour at the faces of the simplex—solutions can converge towards points which are not fixed points—again by discontinuity of the right-hand side of the dynamics’ equation.