Discrete potential mean field games: duality and numerical resolution

Abstract

We propose and investigate a general class of discrete time and finite state space mean field game (MFG) problems with potential structure. Our model incorporates interactions through a congestion term and a price variable. It also allows hard constraints on the distribution of the agents. We analyze the connection between the MFG problem and two optimal control problems in duality. We present two families of numerical methods and detail their implementation: (i) primal-dual proximal methods (and their extension with nonlinear proximity operators), (ii) the alternating direction method of multipliers (ADMM) and a variant called ADM-G. We give some convergence results. Numerical results are provided for two examples with hard constraints.

A Appendix

We detail here the calculation of the projection on Q and the non-linear proximity operator in (24), for a running cost of the form

$$\begin{aligned} \ell (t,x,\rho ) = \sum _{y\in S} \rho (y) \beta (t,x,y) + \chi _{\varDelta (S)}(\rho ). \end{aligned}$$

The adaptation to the case where $\ell$ is defined by (33) is straightforward.

1.1 A.1 Projection on Q

We detail the computation of ${{\,\textrm{proj}\,}}_Q$, as it appears in (20) and (29). First notice that the projection is decoupled in space and time, then for any $(t,x) \in {\mathcal {T}} \times S$ and $({\bar{a}},{\bar{b}}) \in {\mathbb {R}} \times {\mathbb {R}}(S)$, we need to compute

$$\begin{aligned} {{\,\textrm{proj}\,}}_{Q_{t,x}}({\bar{a}},{\bar{b}}) = \mathop {\mathrm {arg\,min}}\limits _{(a,b) \in Q_{t,x}} \ (a-{\bar{a}})^{2}/2 + \sum _{y\in S} (b(y)-{\bar{b}}(y))^2/2, \end{aligned}$$

where $Q_{t,x} = \left\{ (a,b) \in {\mathbb {R}} \times {\mathbb {R}}(S), \, a + b(y) - \beta (y) \le 0 \right\}$. The corresponding problem is

$$\begin{aligned} \min _{a \in {\mathbb {R}}} \, \Bigg ( (a-{\bar{a}})^{2}/2 + \min _{{\begin{array}{c} b \in {\mathbb {R}}(S) \\ b(y) \le \beta (y) - a, \; \forall y\in S \end{array}}} \ \, \sum _{y\in S} (b(y)-{\bar{b}}(y))^2/2 \Bigg ). \end{aligned}$$

(35)

For any $a\in {\mathbb {R}}$, the solution of the inner minimization problem is given by

$$\begin{aligned} b^\star (a,y) := \min \{{\bar{b}}(y),\beta (y) - a\}, \quad \forall y \in S. \end{aligned}$$

Then replacing into (35), the minimization problem is now given by

$$\begin{aligned} \min _{a \in {\mathbb {R}}} g(a), \quad g(a) := (a-{\bar{a}})^{2}/2 + \sum _{y\in S} \max (0,a-{\tilde{\beta }}(y))^2/2, \end{aligned}$$

where ${\tilde{\beta }}(y) := \beta (y)-{\bar{b}}(y)$. It is now relatively easy to minimize g. Let us sort the sequence $({\tilde{\beta }}(y))_{y \in S}$, that is, let us consider $(y_i)_{i\in \{0,\ldots ,n-1\}}$ such that ${\tilde{\beta }}(y_0) \le \cdots \le {\tilde{\beta }}(y_{n-1})$. It is obvious that the function g is strictly convex and polynomial of degree 2 on each of the intervals $(-\infty , {\tilde{\beta }}(y_0))$, $({\tilde{\beta }}(y_0),{\tilde{\beta }}(y_1))$,..., and $({\tilde{\beta }}(y_{n-1}),+\infty )$. One can identify on which of these intervals a stationary point of g exists, by evaluating $\partial g({\tilde{\beta }}(y_i)$, for all $i=0,...,n-1$. Then one can obtain an analytic expresison of the (unique) stationary point $a^\star$, which minimizes g. Finally, we have ${{\,\textrm{proj}\,}}_{Q_{t,x}}({\bar{a}},{\bar{b}})= (a^\star ,b^\star (a^\star , \cdot ))$.

1.2 A.2 Entropic proximity operator

Here we detail the computation of the solution to (24). For notational purpose we set $c_1 = \tau (- u' + \gamma ')$ and $c_2 =\tau (\beta + \varvec{A}^\star P' + \varvec{S}^\star u')$. By definition of the running cost $\ell$, we have that

$$\begin{aligned} \sum _{(t,x) \in {\mathcal {T}} \times S} \tilde{\varvec{\ell }}[m_1,w](t,x) = \langle w, \beta \rangle + \chi _{{{\,\mathrm{{{\textbf {dom}}}}\,}}(\tilde{\varvec{\ell }})}(m_1,w). \end{aligned}$$

Problem (24) writes

$$\begin{aligned} \min _{(m_1,w) \in {\mathcal {R}}}&\langle m_1, c_1 \rangle + \langle w ,c_2 \rangle + \frac{1}{\tau } d_{KL}((m_1,w),(m_1',w')) \\ \text {subject to: }&{\left\{ \begin{array}{ll} \begin{array}{l} m_1(t,x) \le 1 \\ m_1(t,x) - \sum _{y \in S} w(t,x,y)= 0. \end{array} \end{array}\right. } \end{aligned}$$

To find the solution, we define the following Lagrangian with associated multipliers $(\lambda _1,\lambda _2) \in {\mathbb {R}}({\mathcal {T}} \times S) \times {\mathbb {R}}_{+}(\bar{{\mathcal {T}}} \times S)$:

$$\begin{aligned}&{\mathcal {L}}(m_1,w,\lambda _1,\lambda _2) = \langle m_1, c_1 \rangle + \langle w ,c_2 \rangle + d_{KL}((m_1,w),(m_1',w')) \\&\quad + \sum _{(t,x) \in {\mathcal {T}} \times S} \lambda _1(t,x)\Big ( m_1(t,x) - \sum _{y\in S} w(t,x,y) \Big )+ \sum _{(s,x) \in \bar{{\mathcal {T}}} \times S} \lambda _2(s,x)( m_1(s,x) - 1). \end{aligned}$$

For any $(t,s,x,y) \in {\mathcal {T}} \times \bar{{\mathcal {T}}} \times S \times S$, a saddle point of the Lagrangian is given by the following first order conditions,

$$\begin{aligned} {\left\{ \begin{array}{ll} {\hat{m}}_1(T,x) &{}= m_1'(T,x) \exp (- \lambda _2(T,x) - c_1(T,x)),\\ {\hat{m}}_1(t,x) &{}= m_1'(t,x) \exp (-\lambda _1(t,x) - \lambda _2(t,x)-c_1(t,x)),\\ {\hat{w}}(t,x,y) &{}= w'(t,x,y) \exp (\lambda _1(t,x) - c_2(t,x,y)), \\ {\hat{m}}_1(t,x) &{}= \sum _{y' \in S}{\hat{w}}(t,x,y'),\\ 0 &{} = \min \left\{ \lambda _2(s,x), {\hat{m}}_1(s,x) - 1\right\} . \end{array}\right. } \end{aligned}$$

(36)

Case 1: $\lambda _2(s,x) > 0$. At time $s=T$ we have that ${\hat{m}}_1(s,x) = 1$. For any $s<T$ we have that ${\hat{m}}_1(s,x) = 1$ and $\sum _{y \in S}{\hat{w}}(s,x,y) = 1$ and by a direct computation we have that

$$\begin{aligned} {\left\{ \begin{array}{ll} {\hat{m}}_1(s,x) &{} = 1,\\ {\hat{w}}(s,x,y) &{} = w'(s,x,y) \exp (-c_2(s,x,y)) C(s,x), \\ \lambda _1(s,x) &{} = \ln \left( C(s,x) \right) , \\ \lambda _2(s,x) &{} = \ln \left( m_1'(s,x)/C(s,x)) \right) - c_1(s,x), \end{array}\right. } \end{aligned}$$

(37)

where $C(s,x) = \sum _{y \in S } w'(s,x,y) \exp (-c_2(s,x,y))$.

Case 2: $\lambda _2(s,x) = 0$. At time $s=T$ we have that ${\hat{m}}_1(s,x) = m_1'(s,x) \exp (-c_1(s,x))$. For any $s<T$ we have by a direct computation

$$\begin{aligned} {\left\{ \begin{array}{ll} {\hat{m}}_1(s,x) &{}= m_1'(s,x) C(s,x)^{-1} \exp (-c_1(s,x)),\\ {\hat{w}}(s,x,y) &{}= w'(s,x,y) C(s,x) \exp (- c_2(s,x,y)), \\ \lambda _1(s,x)&{} = \ln \left( C(s,x) \right) ,\\ \lambda _2(s,x) &{}= 0, \end{array}\right. } \end{aligned}$$

(38)

where $C(s,x) = \Big (m_1'(s,x) \exp (-c_1(s,x)) / \sum _{y \in S } w'(s,x,y) \exp (- c_2(s,x,y)) \Big )^{1/2}.$

In order to identify which of the two cases arises, one can compute a solution with formula (37) and check a posteriori that $\lambda _2(s,x) > 0$. If this is not the case, we deduce that the solution to (36) is given by (38).