Lattice approximations of the first-order mean field type differential games

Abstract

The theory of first-order mean field type differential games examines the systems of infinitely many identical agents interacting via some external media under assumption that each agent is controlled by two players. We study the approximations of the value function of the first-order mean field type differential game using solutions of model finite-dimensional differential games. The model game appears as a mean field type continuous-time Markov game, i.e., the game theoretical problem with the infinitely many agents and dynamics of each agent determined by a controlled finite state nonlinear Markov chain. Given a supersolution (resp. subsolution) of the Hamilton–Jacobi equation for the model game, we construct a suboptimal strategy of the first (resp. second) player and evaluate the approximation accuracy using the modulus of continuity of the reward function and the distance between the original and model games. This gives the approximations of the value function of the mean field type differential game by values of the finite-dimensional differential games. Furthermore, we present the way to build a finite-dimensional differential game that approximates the original game with a given accuracy.

Appendices

Appendix A: Existence and uniqueness result for the flow of probabilities generated by the distribution of players’ controls

Here we give the proof of Proposition 2.2 that is the existence and uniqueness theorem for the flow of probabilities defined by Definition 2.1.

Proof of Proposition 2.2

To derive existence of the flow of probabilities, let us introduce the set of trajectories on [0, T] those are Lipschitz continuous with the constant R that is the upper bound for the norm of f (see (2.4)):

$$\begin{aligned} {\text {Lip}}_R\triangleq \{x(\cdot )\in \mathcal {C}:\Vert x(t')-x(t'')\Vert \le R|t'-t''|,\ \ t',t''\in [0,T]\}. \end{aligned}$$

Obviously, $x(\cdot ,s,y,\xi ,\zeta )$ lies in ${\text {Lip}}_R$ for every $y\in \mathbb {T}^d$, $\xi \in \mathcal {U}$, $\zeta \in \mathcal {V}$. Moreover, ${\text {Lip}}_R$ is compact. Thanks to the Prokhorov’s theorem, $\mathcal {P}({\text {Lip}}_R)$ is also compact in the topology of narrow convergence. Consider the mapping $\Phi$ from $\{\chi \in \mathcal {P}({\text {Lip}}_R):e_s\sharp \chi = m_*\}$ into itself acting by the rule: if $\chi \in \mathcal {P}({\text {Lip}}_R)$, then set $m(t)\triangleq e_t\sharp \chi$ and put

$$\begin{aligned} \Phi [\chi ]\triangleq {\text {traj}}^{s}_{m(\cdot )}\sharp \varkappa . \end{aligned}$$

Since the narrow convergence on $\mathcal {P}({\text {Lip}}_R)$ is equivalent to the convergence in the Wasserstein metric $W_2$ [3, Proposition 7.1.5] and $W_2(e_t\sharp \chi ',e_t\sharp \chi '')\le W_2(\chi ',\chi '')$ for every $t\in [0,T]$ and $\chi ',\chi ''\in \mathcal {P}^2(\mathcal {C})$, we have that, if $\{\chi _n\}_{n=1}^\infty \subset \mathcal {P}({\text {Lip}}_R)$ converges to some $\chi \in \mathcal {P}({\text {Lip}}_R)$, then the sequence of probabilities $\{m_n(t)\}_{n=1}^\infty$ defined by the rule $m_n(t)\triangleq e_t\sharp \chi _n$ converges to $m(t)\triangleq e_t\sharp \chi$ uniformly in the Wasserstein metric $W_2$. Using the continuous dependence of the solution of the differential equation on parameter, we obtain that the limit of sequence of operators $\{{\text {traj}}_{m_n(\cdot )}^{s}\}$ is ${\text {traj}}_{m(\cdot )}^{s}$. Lemma 5.2.1 of [3] implies that the sequence of probabilities $\{{\text {traj}}_{m_n(\cdot )}^{s}\sharp \varkappa \}$ converges to the distribution ${\text {traj}}_{m(\cdot )}^{s}\sharp \varkappa$. This means that $\Phi$ is continuous operator acting on compact $\mathcal {P}({\text {Lip}}_R)$ and, thus, admits a fixed point. It determines $m(\cdot ,s,m_*,\varkappa )$ through the evaluation operator $e_t$.

Now let us prove the uniqueness. Let $m^1(\cdot )$, $m^2(\cdot )$ be flows of probabilities generated by s, $m_*$ and $\varkappa$. Thanks to the Lipschitz continuity of the function f (see (2.3)), we have that

$$\begin{aligned} \begin{aligned}&\Vert x(t,s,y,m^1(\cdot ),\xi ,\zeta )-x(t,s,y,m^2(\cdot ),\xi ,\zeta )\Vert ^2 \\&\quad \le 2L^2\int _s^t \Vert x(t',s,y,m^1(\cdot ),\xi ,\zeta )-x(t',s,y,m^2(\cdot ),\xi ,\zeta )\Vert ^2dt' \\ {}&\qquad +2L^2\int _s^tW_2^2(m^1(t'),m^2(t'))dt'. \end{aligned} \end{aligned}$$

The Gronwall’s inequality gives that

$$\begin{aligned}\begin{aligned}&\Vert x(t,s,y,m^1(\cdot ),\xi ,\zeta )-x(t,s,y,m^2(\cdot ),\xi ,\zeta )\Vert ^2\\ {}&\quad \le C_6\int _s^tW_2^2(m^1(t'),m^2(t'))dt',\end{aligned} \end{aligned}$$

where

$$\begin{aligned} C_6\triangleq 2L^2e^{2L^2T}. \end{aligned}$$

Further, choosing the transportation plan $\pi ^{1,2}$ between $m^1(t)$ and $m^2(t)$ by the rule

$$\begin{aligned} \pi ^{1,2}\triangleq (e_t\circ {\text {traj}}^{s}_{m^1(t)},e_t\circ {\text {traj}}^{s}_{m^2(t)})\sharp \varkappa , \end{aligned}$$

we deduce that

$$\begin{aligned} \begin{aligned}&W_2^2(m^1(t),m^2(t))\le \int _{\mathbb {T}^d\times \mathbb {T}^d}\Vert x^1-x^2\Vert ^2\pi ^{1,2}(d(x^1,x^2))\\&\quad = \int _{\mathbb {T}^d\times \mathcal {U}\times \mathcal {V}}\Vert x(t,s,y,m^1(\cdot ),\xi ,\zeta )-x(t,s,y,m^2(\cdot ),\xi ,\zeta )\Vert ^2\varkappa (d(y,\xi ,\zeta )) \\ {}&\quad \le C_6\int _s^tW_2^2(m^1(t'),m^2(t'))dt'. \end{aligned} \end{aligned}$$

Using the Gronwall’s inequality once more time, we conclude that $W_2^2(m^1(t), m^2(t))=0$ for every $t\in [0,T]$. This implies the uniqueness of the flow of probabilities generated by s, $m_*$ and $\varkappa$. $\square$

Appendix B: A property of Wasserstein metric on a simplex

The purpose of Appendix B is to prove the inequalities between the Wasserstein metric on $\mathcal {P}(\mathcal {S})$ and the p-th metric on $\Sigma$ formulated in Proposition 3.1.

Proof of Proposition 3.1

Since the set $\mathcal {S}$ is finite we have that, for $\pi \in \Pi (\widetilde{\mu ^1},\widetilde{\mu ^2})$, there exist nonnegative numbers $b_{{\bar{x}},{\bar{y}}}[\pi ]$, ${\bar{x}},{\bar{y}}\in \mathcal {S}$, such that

$$\begin{aligned} \pi =\sum _{{\bar{x}},{\bar{y}}\in \mathcal {S}}b_{{\bar{x}},{\bar{y}}}[\pi ]\delta _{{\bar{x}},{\bar{y}}}. \end{aligned}$$

(B.1)

Notice that each number $b_{{\bar{x}},{\bar{y}}}[\pi ]$ is equal to the mass transported from ${\bar{x}}$ to ${\bar{y}}$ according to the transportation plan $\pi$.

Now let us prove inequality (3.4). Let $\pi _0\in \Pi (\widetilde{\mu ^1},\widetilde{\mu ^2})$ be such that

$$\begin{aligned} W_p^p\Bigl (\widetilde{\mu ^1},\widetilde{\mu ^2}\Bigr )=\int _{\mathcal {S}\times \mathcal {S}}\Vert x-y\Vert ^p\pi _0(d(x,y)). \end{aligned}$$

From (B.1) it follows that

$$\begin{aligned} W_p^p\Bigl (\widetilde{\mu ^1},\widetilde{\mu ^2}\Bigr )=\sum _{{\bar{x}},{\bar{y}}\in \mathcal {S}}b_{{\bar{x}},{\bar{y}}}[\pi _0]\Vert {\bar{x}}-{\bar{y}}\Vert ^p. \end{aligned}$$

The fact that $b_{{\bar{x}},{\bar{y}}}[\pi _0]$ is the mass transported from ${\bar{x}}$ to ${\bar{y}}$ according to the plan $\pi _0$ yields the following inequality:

$$\begin{aligned} \sum _{{\bar{y}}\in \mathcal {S},{\bar{y}}\ne {\bar{x}}}b_{{\bar{x}},{\bar{y}}}[\pi _0]\ge (\mu _{{\bar{x}}}^1-\mu _{{\bar{x}}}^2)^+, \end{aligned}$$

where $a^+$ is equal to a provided that a is positive, and 0 otherwise. Using this estimate and the definition of $d(\mathcal {S})$ (see (3.1)), we conclude that

$$\begin{aligned} W_p^p\Bigl (\widetilde{\mu ^1},\widetilde{\mu ^2}\Bigr )\ge \sum _{{\bar{x}}\in \mathcal {S}}\sum _{{\bar{y}}\in \mathcal {S},{\bar{y}}\ne {\bar{x}}}b_{{\bar{x}},{\bar{y}}}[\pi _0](d(\mathcal {S}))^p\ge (d(\mathcal {S}))^p\sum _{{\bar{x}}\in \mathcal {S}}(\mu _{{\bar{x}}}^1-\mu _{{\bar{x}}}^2)^+. \end{aligned}$$

Since $\mu ^1,\mu ^2\in \Sigma$, we have that

$$\begin{aligned} \sum _{{\bar{x}}\in \mathcal {S}}(\mu _{{\bar{x}}}^1-\mu _{{\bar{x}}}^2)^+=\frac{1}{2}\sum _{{\bar{x}}\in \mathcal {S}}|\mu _{{\bar{x}}}^2-\mu _{{\bar{x}}}^1|. \end{aligned}$$

(B.2)

Furthermore, for every ${\bar{x}}\in \mathcal {S}$, $|\mu _{{\bar{x}}}^2-\mu _{{\bar{x}}}^1|\le 1$. Thus, we have that

$$\begin{aligned} W_p^p\Bigl (\widetilde{\mu ^1},\widetilde{\mu ^2}\Bigr )\ge \frac{(d(\mathcal {S}))^p}{2}\sum _{{\bar{x}}\in \mathcal {S}}|\mu _{{\bar{x}}}^2-\mu _{{\bar{x}}}^1|\ge \frac{(d(\mathcal {S}))^p}{2}\sum _{{\bar{x}}\in \mathcal {S}}|\mu _{{\bar{x}}}^2-\mu _{{\bar{x}}}^1|^p. \end{aligned}$$

This proves (3.4).

To prove inequality (3.5), choose a probability $\pi '\in \Pi (\widetilde{\mu ^1},\widetilde{\mu ^2})$ such that

$$\begin{aligned} b_{{\bar{x}},{\bar{x}}}[\pi ']\triangleq \mu ^1_{{\bar{x}}}\wedge \mu _{{\bar{x}}}^2. \end{aligned}$$

Here $b_{{\bar{x}},{\bar{y}}}[\pi ']$ are given by representation (B.1). Therefore, for every ${\bar{x}}\in \mathcal {S}$

$$\begin{aligned} \sum _{{\bar{y}}\in \mathcal {S},{\bar{y}}\ne {\bar{x}}}b_{{\bar{x}},{\bar{y}}}[\pi ']=(\mu _{{\bar{x}}}^1-\mu _{{\bar{x}}}^2)^+. \end{aligned}$$

(B.3)

The inclusion $\mathcal {S}\subset \mathbb {T}^d$ yields

$$\begin{aligned} W_p^p\Bigl (\widetilde{\mu ^1},\widetilde{\mu ^2}\Bigr )\le \sum _{{\bar{x}},{\bar{y}}\in \mathcal {S}} \Vert {\bar{x}}-{\bar{y}}\Vert ^pb_{{\bar{x}},{\bar{y}}}[\pi ']\le d^{p/2}\sum _{{\bar{x}}\in \mathcal {S}}\sum _{{\bar{y}}\in \mathcal {S},{\bar{y}}\ne {\bar{x}}}b_{{\bar{x}},{\bar{y}}}[\pi ']. \end{aligned}$$

This and equalities (B.2), (B.3) imply that

$$\begin{aligned} W_p^p\Bigl (\widetilde{\mu ^1},\widetilde{\mu ^2}\Bigr )\le \frac{d^{p/2}}{2}\sum _{{\bar{x}}\in \mathcal {S}} |\mu _{{\bar{x}}}^2-\mu _{{\bar{x}}}^1|=\frac{d^{p/2}}{2}\Vert \mu ^2-\mu ^1\Vert _1. \end{aligned}$$

The Holder’s inequality gives that

$$\begin{aligned} W_p^p\Bigl (\widetilde{\mu ^1},\widetilde{\mu ^2}\Bigr )\le \frac{d^{p/2+p'}}{2}\Vert \mu ^2-\mu ^1\Vert _p, \end{aligned}$$

where $p'$ is such that $1/p'+1/p=1$. This proves (3.5) $\square$