A Mean Field Game Inverse Problem

Abstract

Mean-field games arise in various fields, including economics, engineering, and machine learning. They study strategic decision-making in large populations where the individuals interact via specific mean-field quantities. The games’ ground metrics and running costs are of essential importance but are often unknown or only partially known. This paper proposes mean-field game inverse-problem models to reconstruct the ground metrics and interaction kernels in the running costs. The observations are the macro motions, to be specific, the density distribution and the velocity field of the agents. They can be corrupted by noise to some extent. Our models are PDE constrained optimization problems, solvable by first-order primal-dual methods. We apply the Bregman iteration method to improve the parameter reconstruction. We numerically demonstrate that our model is both efficient and robust to the noise.

Appendices

Appendix A. Table of Notations

Table 4 Table of notations

Appendix B. Proofs of the Theorems in Sect. 3

Proof of Theorem 3

For simplicity, we just prove the case ${\mathcal {G}}$ being an indicator function, i.e., $\rho _T=\rho _{term}$. For the case ${\mathcal {G}}$ being differentiable, the proof is similar. We start our proof with one direction, if $(\rho ,{\mathbf {v}})$ is the solution of (1.1). According to the assumption that $\rho$ is strictly positive, the KKT condition of (1.1) is (2.1). The minimizer $(\rho ,{\mathbf {v}})$ solves (2.1) with some Lagrangian multiplier $\varphi$. Then, by substituting ${\mathbf {m}}$ with $\rho ,\varphi ,G_M$, the KKT condition is equivalent to (2.2), $(\rho ,\varphi )$ solves (2.2), ${\mathbf {v}}$ can be represented as ${\mathbf {v}}=G_M^{-1}\nabla \varphi$.

Note ${\mathbf {w}}=\nabla \varphi$. The HJE (2.2b) can be written as:

$$\begin{aligned} \begin{aligned} \varphi _t=\frac{\delta }{\delta \rho }{\mathcal {F}}(\rho )-\frac{1}{2}{\mathbf {w}}^TG_M^{-1}{\mathbf {w}}. \end{aligned} \end{aligned}$$

Let

$$\begin{aligned} \xi :=\frac{1}{2}{\mathbf {w}}^TG_M^{-1}{\mathbf {w}}-\frac{\delta }{\delta \rho }{\mathcal {F}}(\rho ). \end{aligned}$$

Since $\varphi$ has a second-order mixed derivative by assumption, we can substitute HJE with

$$\begin{aligned} {\mathbf {w}}_t+\nabla \xi =0,\quad \frac{\partial w_i}{\partial x_j}=\frac{\partial w_j}{\partial x_i},\quad i\not =j, \end{aligned}$$

where $w_i,w_j$ are the ith and jth component of ${\mathbf {w}}$. The MFG system is transformed into:

$$\begin{aligned} \left\{ \begin{aligned}&\rho _t+\nabla \cdot (\rho G_M^{-1}{\mathbf {w}})=0\\&{\mathbf {w}}_t+\nabla \left( \frac{1}{2}{\mathbf {w}}^TG_M^{-1}{\mathbf {w}}-\frac{\delta }{\delta \rho }{\mathcal {F}}(\rho )\right) =0\\&\frac{\partial w_i}{\partial {x_j}}=\frac{\partial w_j}{\partial {x_i}}, \quad i\not =j \end{aligned} \right. \end{aligned}$$

(B.1)

Since the domain ${\mathbb {T}}^d$ is multiply connected, for compatibility, the integral of $w_i$ over the path $s_i({\hat{x}}^i,\cdot )$ equals to 0:

$$\begin{aligned} \int _{s_i({\hat{x}}^i,\cdot )} w_i dS_x=0,\quad {\hat{x}}^i\in {\mathbb {T}}^{d-1},i=1,2,\ldots ,d. \end{aligned}$$

(B.2)

Since

$$\begin{aligned} {\mathbf {v}}=\frac{{\mathbf {m}}}{\rho }=G_M^{-1}\nabla \varphi =G_M^{-1}{\mathbf {w}}, \end{aligned}$$

by substituting ${\mathbf {w}}$ with $G_M{\mathbf {v}}$, (B.1) (B.2) can be reformulated into (3.1). Hence the minimizer $(\rho ,{\mathbf {v}})$ of the optimization problem solves (3.1).

Conversely, we assume that $(\rho ,{\mathbf {v}})$ is a solution of (3.1). Since $G_M{\mathbf {v}}$ satisfies (3.1c), (3.1d), according to Helmholtz’s theorem, $G_M{\mathbf {v}}$ is the gradient of some function $\varphi$ on ${\mathbb {T}}$, i.e. $G_M{\mathbf {v}}=\nabla \varphi$. Inserting $G_M{\mathbf {v}}=\nabla \varphi$ into (3.1a), (3.1b), we can get

$$\begin{aligned} \rho _t+\nabla \cdot (\rho G_{M}^{-1}\nabla \varphi )&=0 \\ (\nabla \varphi )_t+\nabla \left( \frac{1}{2}\nabla \varphi ^TG_{M}^{-1}\nabla \varphi -\frac{\delta }{\delta \rho }{\mathcal {F}}(\rho )\right)&=0 \end{aligned}$$

Further, we have

$$\begin{aligned}&\rho _t+\nabla \cdot (\rho G_{M}^{-1}\nabla \varphi )=0 \end{aligned}$$

(B.4a)

$$\begin{aligned}&\varphi _t+\frac{1}{2}\nabla \varphi ^TG_{M}^{-1}\nabla \varphi -\frac{\delta }{\delta \rho }{\mathcal {F}}(\rho )=C(t) \end{aligned}$$

(B.4b)

With out losing generality, we can assume $C(t)=0$, else, we can substitute $\varphi$ with ${\tilde{\varphi }}$:

$$\begin{aligned} {\tilde{\varphi }}=\varphi -\int _0^tC(s)\,ds, \end{aligned}$$

here $\nabla {\tilde{\varphi }}=G_M{\mathbf {v}}$ still holds. Set ${\mathbf {m}}=\rho G_M^{-1}\nabla \varphi$, and insert it into (B.4). By variable substitution among ${\mathbf {m}},\rho ,\varphi$, we can reach (2.1). Due to the strict convexity of the optimization problem (1.1), the above pair $(\rho ,{\mathbf {m}})$ satisfying the KKT condition is a minimizer of (1.1). $\square$

Proof of Theorem 5

Take $\Phi ,\pmb {\psi },\pmb {\chi },\Theta$ as the dual variable. Especially,

$$\begin{aligned} \Theta (x,t)=\left( \Theta _1((x_2,x_3),t),\Theta _2((x_1,x_2),t),\Theta _3((x_2,x_3),t)\right) ^T, \end{aligned}$$

and $\Theta _i$ is the dual variable of the constrain:

$$\begin{aligned} \int _{s_i({\hat{x}}^i,\cdot )} (G_M{\mathbf {v}})_i\,dS_x=0,\qquad i=1,2,3. \end{aligned}$$

The Lagrangian of (4.1) can be written as:

$$\begin{aligned} \begin{aligned}&\mathop {{\text {min}}}\limits _{ \begin{array}{c} \rho ,{\mathbf {v}},g_0 \end{array} }\mathop {{\text {max}}}\limits _{\Phi ,\pmb {\psi },\pmb {\chi },\Theta } {\mathcal {J}}(\rho ,{\mathbf {v}},\rho _0,\rho _T,{\mathbf {v}}_T;{\hat{\rho }},{\hat{{\mathbf {v}}}},{\hat{\rho }}_0,{\hat{\rho }}_T) +\frac{\gamma }{2}\Vert \nabla g_0 \Vert _2^2\\&\quad + \int _0^T\int _{{\mathbb {T}}^3}\Phi \left( \rho _t+\nabla \cdot (\rho {\mathbf {v}}) \right) + \pmb {\chi }\cdot \left( \nabla \times (G_M{\mathbf {v}})\right) \\&\quad + \pmb {\psi }\cdot \left( (G_M {\mathbf {v}})_t+\nabla \left( \frac{1}{2} {\mathbf {v}}^T G_M {\mathbf {v}}-F'(\rho )\right) \right) +\Theta \cdot (G_M{\mathbf {v}}) \,dx\,dt . \end{aligned} \end{aligned}$$

Besides, by integration by parts, we have

$$\begin{aligned} \begin{aligned}&\frac{1}{2}\int _{{\mathbb {T}}^3} \nabla g_0\cdot \nabla g_0\,dx=\frac{1}{2}\int _{{\mathbb {T}}^3}\nabla \cdot (g_0\nabla g_0)-g_0\triangle g_0 dx=-\frac{1}{2}\int _{{\mathbb {T}}^3} g_0\triangle g_0\,dx, \\&\quad \int _0^T\int _{{\mathbb {T}}^3} \Phi \left( \rho _t+\nabla \cdot (\rho {\mathbf {v}}) \right) \,dx\,dt\\&\quad = \int _0^T\int _{{\mathbb {T}}^3} (\rho \Phi )_t-\rho \Phi _t+\nabla \cdot (\Phi \rho {\mathbf {v}}) -\rho \nabla \Phi \cdot {\mathbf {v}}\,dx\,dt\\&\quad =\left. \int _{{\mathbb {T}}^3} \rho \Phi \,dx\right| _{t=0}^T +\int _0^T\int _{{\mathbb {T}}^3}-\rho \Phi _t-\rho \nabla \Phi \cdot {\mathbf {v}}\,dx\,dt, \\&\quad \int _0^T\int _{{\mathbb {T}}^3} \pmb {\psi }\cdot \left( (G_M {\mathbf {v}})_t+\nabla \left( \frac{1}{2} {\mathbf {v}}^T G_M {\mathbf {v}}-F'(\rho )\right) \right) \,dx\,dt\\&\quad =\int _0^T\int _{{\mathbb {T}}^3} (\pmb {\psi }^T G_M {\mathbf {v}})_t-\pmb {\psi }_t^TG_M{\mathbf {v}}+ \nabla \cdot \left( \pmb {\psi }\left( \frac{1}{2} {\mathbf {v}}^T G_M {\mathbf {v}}-F'(\rho )\right) \right) \\&\qquad -\nabla \cdot \pmb {\psi }\left( \frac{1}{2} {\mathbf {v}}^T G_M {\mathbf {v}}-F'(\rho )\right) \,dx\,dt,\\&\quad =\left. \int _{{\mathbb {T}}^3}\pmb {\psi }^T G_M {\mathbf {v}}\,dx \right| _{t=0}^{T} +\int _0^T\int _{{\mathbb {T}}^3} -\pmb {\psi }_t^TG_M{\mathbf {v}}-\nabla \cdot \pmb {\psi }\left( \frac{1}{2} {\mathbf {v}}^T G_M {\mathbf {v}}-F'(\rho )\right) \,dx\,dt \\&\quad \int _0^T\int _{{\mathbb {T}}^3}\pmb {\chi }\cdot (\nabla \times (G_M{\mathbf {v}}))\,dx\,dt\\&\quad = \int _0^T\int _{{\mathbb {T}}^3}\nabla \cdot \left( (G_M{\mathbf {v}})\times \pmb {\chi }\right) +(G_M{\mathbf {v}})\cdot (\nabla \times \pmb {\chi })\,dx\,dt\\&\quad =\int _0^T\int _{{\mathbb {T}}^3}(G_M{\mathbf {v}})\cdot (\nabla \times \pmb {\chi })\,dx\,dt. \end{aligned} \end{aligned}$$

We insert these equalities into the Lagrangian. By making first variation with regard to $\rho ,{\mathbf {v}}$ at initial and terminal time, i.e., in the space ${\mathbb {T}}^3\times \{0\}, {\mathbb {T}}^3\times \{T\}$, we can deduce

$$\begin{aligned} \frac{\delta }{\delta \rho _0}{\mathcal {J}}\!-\!\Phi (x,0)\!=\!0,\quad ,\frac{\delta }{\delta \rho _T}{\mathcal {J}}\!+\!\Phi (x,T)=0,\quad \pmb {\psi }(x,0)\!=\!0,\quad G_M\pmb {\psi }(x,T)\!+\!\frac{\delta }{\delta {\mathbf {v}}_T}{\mathcal {J}}\!=\!0. \end{aligned}$$

Taking first variation with regard to $\rho ,{\mathbf {v}},g_0$ in the density space ${\mathbb {T}}^3\times [0,T]$, combining with the fact that $\pmb {\psi }$ vanishes at $t=0,T$, we can get the first three equations in (4.3). And finally, the variation with regard to dual variable leads back to constraints in (3.4). $\square$

The proof of Theorem 6 is quite analog to the proof of Theorem 5, except for some details in dealing with the convolution operator. The summation over $x^*\in {\mathcal {S}}(x)$ comes from the symmetry of the convolution kernel $K(\cdot ,\cdot )$. We omit the proof here.

Appendix C. Complementary Computational Results

Test 7

We apply our algorithm to solve Model 3 in 1D. No noise impacts on the observations, $G_M=g_0$ is taken. We discretize the problem on a $50\times 30$ grid. The information about $G_M$ at a single grid is gained. The scaling parameters $\alpha ,\alpha _0,\beta$ are taken as in (6.1), $\gamma$ varies in $\{10^{-8},10^{-7},\ldots ,10^{-3}\}$. The settings for the iteration step size and the iteration time follows Test 3. We test our algorithm on two ground metrics of different shapes. The results are depicted in Figs. 9 and 10. The error estimate is in Table 5 (Table 6).

Fig. 9

The result for $G_M(x)=1-0.6 \sin (\pi x)^2$. From left to right, $\gamma =10^{-8},10^{-7},\ldots ,10^{-3}$. The red curve presents the leaned ground metric, and the blue curve depicts the real metric

Fig. 10

The result for $G_M(x)=1-0.6 \sin (2\pi x)^2$. From left to right, $\gamma =10^{-8},10^{-7},\ldots ,10^{-3}$

From the numeric results, we conclude that the optimal parameter for the inverse model depends on the shape of the ground metric. When $g_0$ has more fluctuation, we should choose a smaller $\gamma$ to preserve the oscillation of $g_0$. This matches our intuition.

Table 5 The absolute (relative) error in Test 7 for different source data

Test 8

In this test, we use 2-dimensional noisy observations. We design our experiment based on the data in Test 1, and the data is corrupted by additive noise defined in (6.2), with level $\epsilon ^*=0.1$. The scaling parameters are selected as $\alpha =1/\Vert {\hat{\rho }}\Vert ^2,\alpha _0=0,\beta = 1/{\hat{{\mathbf {v}}}}^T{\hat{{\mathbf {v}}}},\gamma =0.01$. During the iteration, as prior information, $g_0$ on one row of grids is fixed to be the ground truth. Other settings all follow Test 1. The result is depicted in Fig. 11.

Test 9

In Test 8, the prior information about $g_0$ on a row of grids is assumed to be known. In this test, we inspect the sensitivity of the reconstruction of the ground metric about the prior information. We variate the value of $g_0$ on the row to be $1.05\times$ the ground truth. Moreover, this row of grids is set fixed during the iteration. All other settings are the same as they are in Test 8. The computational result is in Fig. 12.

Table 6 The error in selected Bregman iteration, the last column is intended to compare with the result by taking optimal $\gamma$ in non-Bregman algorithm

Fig. 11

The recreated metric kernel from noisy data, in 2 dimensional case, when $\epsilon ^*=0.1$. From left to right, the figures correspond to the recreated metric kernel $g_0$, the ground truth, and the residue $|g_0-\bar{g_0}|$. The absolute error is 0.0900, and the relative error is 0.1683

Fig. 12

The sensitive test for 2D ground metric reconstruction. The prior information of $g_0$ is biased by 5%. From left to right, the figures are the heat maps for the reconstruction ground metric, ground truth and the absolute difference between them. The absolute error is 0.0907, and the relative error is 0.1694

Test 10

In this test, we run Algorithm 2 and use the same data as Test 4, the noise level is $\epsilon ^*=1$. We set $\gamma =10^{-1}$, and take $3\times 10^6$ primal-dual sub-iterations in each Bregman iteration. For comparison, a non-Bregman result of Algorithm 1 is also given. The iteration time for the non-Bregman algorithm is $3\times 10^6$. The results of the first 13 Bregman iterations are depicted in Fig. 13.

Fig. 13

The recreated convolution kernels from noisy data from the first 13 Bregman iterations. $\gamma =10^{-1}$. (The last figure shows the recreated convolution kernel from Algorithm 1 with the nearly optimal $\gamma =10^{-3}$)