Regularity of the value function and quantitative propagation of chaos for mean field control problems

Abstract

We investigate a mean field optimal control problem obtained in the limit of the optimal control of large particle systems with forcing and terminal data which are not assumed to be convex. We prove that the value function, which is known to be Lipschitz continuous but not of class $C^1$, in general, without convexity, is actually smooth in an open and dense subset of the space of times and probability measures. As a consequence, we prove a new quantitative propagation of chaos-type result for the optimal solutions of the particle system starting from this open and dense set.

Appendix A: The proof of Lemma 1.6

Proof

A fact similar to Lemma 1.6 was given in [2] for the torus and for smooth initial data. Here, we extend the argument for the whole space and general initial conditions and slightly simplify it.

We begin with the existence of a solution to (1.14), the uniqueness being obvious in view of the regularity of $\alpha$.

Fix $\beta \in C^\infty _c((t_0,T]\times {\mathbb {R}}^d; {\mathbb {R}}^d)$ and note that the product $m\beta$ is smooth, because the only singularity of m is at time $t_0$. Thus, there exists a unique classical solution to (1.14).

In order to prove its regularity, fix $t_0<t_1<t_2$, $\xi \in C^{2+\delta }({\mathbb {R}}^d)$, let w be the solution to

$$\begin{aligned} -\partial _t w-\Delta w-\alpha (t,x)\cdot Dw=0\ \ \textrm{in}\ \ (t_0,T)\times {\mathbb {R}}^d \ \ \ w(t_2)=\xi \ \ \textrm{in}\ \ {\mathbb {R}}^d, \end{aligned}$$

and note that, for a constant C depending only on the data of the problem, since the regularity of $\alpha$ depends only on the data of the problem,

$$\begin{aligned} \Vert w\Vert _\infty +\Vert Dw\Vert _{\infty }\le C \Vert \xi \Vert _{W^{1,\infty }} \ \ \textrm{and}\ \ \Vert w\Vert _{C^{\delta /2,\delta }}+ \Vert Dw\Vert _{C^{\delta /2,\delta }}\le C\Vert \xi \Vert _{C^{2+\delta }}.\nonumber \\ \end{aligned}$$

(A.1)

Then,

$$\begin{aligned} \int _{{\mathbb {R}}^d} \rho (t_2,x)\xi (x)dx = \int _{{\mathbb {R}}^d} w(t_1,x)\rho (t_1,x)dx- \int _{t_1}^{t_2}\int _{{\mathbb {R}}^d}\beta (t,x)\cdot Dw(t,x) m(t,dx)dx, \end{aligned}$$

and, choosing $t_1=t_0$ and $t_2$ arbitrary in $[t_0,T]$, we get

$$\begin{aligned} \sup _{t\in [t_0,T]} \Vert \rho (t)\Vert _{(W^{1,\infty })'} \le \Vert \beta (t,\cdot )\Vert _{L^1_m([0,T]\times {\mathbb {R}}^d)}. \end{aligned}$$

(A.2)

In addition, since, thanks to (A.1),

$$\begin{aligned} \Vert w(t_1,\cdot )-w(t_2,\cdot )\Vert _{W^{1,\infty }}\le C(t_2-t_1)^{\delta /2}\Vert \xi \Vert _{C^{2+\delta }} \end{aligned}$$

using (A.2) we find

$$\begin{aligned}&\int _{{\mathbb {R}}^d} (\rho (t_2,x)-\rho (t_1,x))\xi (x)dx\\&\quad = \int _{{\mathbb {R}}^d} (w(t_1,x)-w(t_2,x))\rho (t_1,x)dx\\&\qquad - \int _{t_1}^{t_2}\int _{{\mathbb {R}}^d}\beta (t,x)\cdot Dw(t,x) m(t,dx)dx\\&\quad \le \Vert w(t_1,\cdot )-w(t_2,\cdot )\Vert _{W^{1,\infty }}\Vert \rho (t_1,\cdot )\Vert _{(W^{1,\infty })'}\\&\qquad +C(t_2-t_1)^{1/2}\Vert \beta \Vert _{L^2_m([0,T]\times {\mathbb {R}}^d)}\Vert Dw\Vert _\infty \\&\quad \le C(t_2-t_1)^{\delta /2}\Vert \beta (t,\cdot )\Vert _{L^1_m([0,T]\times {\mathbb {R}}^d)} \ \Vert \xi \Vert _{C^{2+\delta }}\\&\qquad +C(t_2-t_1)^{1/2}\Vert \beta \Vert _{L^2_m([0,T]\times {\mathbb {R}}^d)}\Vert \xi \Vert _{W^{1,\infty }}. \end{aligned}$$

The last estimates proves the existence of a solution $\rho$ for $\beta \in C^0([t_0,T]\times {\mathbb {R}}^d; {\mathbb {R}}^d)$ or for $\beta \in L^\infty$ vanishing near $t=t_0$ by approximation.

Next, let

$$\begin{aligned} J(m',\alpha ') = \int _{t_0}^T \left( \int _{{\mathbb {R}}^d} L(x, \alpha '(t,x))m'(t,dx)+{\mathcal {F}}(m'(t))\right) dt +{\mathcal {G}}(m'(T)). \end{aligned}$$

The quantity $J(m',\alpha ')$ is defined, for instance, for $m'\in C^0([t_0,T], {\mathcal {P}}_1({\mathbb {R}}^d))$ and $\alpha '\in C^0([t_0,T]\times {\mathbb {R}}^d; {\mathbb {R}}^d)$. Let $\beta \in C^\infty _c((t_0,T]\times {\mathbb {R}}^d)$ and $\rho$ be the classical solution to (1.14), and, for $h>0$ small, let $m_h\in C^0([t_0,T],{\mathcal {P}}_1({\mathbb {R}}^d))$ be the solution to

$$\begin{aligned} \partial _t m_h -\Delta m_h +\textrm{div}(m_h (\alpha +h\beta ))=0\qquad \textrm{in}\; (t_0,T)\times {\mathbb {R}}^d \ \ \text {and} \ \ m_h(t_0)= m_0 \ \ \textrm{in}\ \ {\mathbb {R}}^d. \end{aligned}$$

Then $m_h=m+h\rho +h^2\xi _h$, where $\xi _h$ solves in the sense of distribution

$$\begin{aligned} \partial _t \xi _h -\Delta \xi _h +\textrm{div} (\xi _h (\alpha +h\beta )) +\textrm{div}(\beta \rho )= 0\ \ \textrm{in}\ \ (t_0,T)\times {\mathbb {R}}^d \ \ \text {and}\ \ \xi _h(t_0)= 0\ \ \textrm{in}\ \ {\mathbb {R}}^d. \end{aligned}$$

The regularity of $\alpha$, $\beta$ and $\rho$ imply that $\Vert \xi _h\Vert _\infty \le C$, with C depending on $\beta$, and, as $h\rightarrow 0$, the $(\xi _h)$s converges weakly in $L^\infty$-weak-$*$ to the solution $\xi$ of the same equation with $h=0$.

Then

$$\begin{aligned}&J(m_h,\alpha +h\beta ) =\int _{t_0}^T\left( \int _{{\mathbb {R}}^d} L(x,\alpha +h\beta )m_h(t,dx)+{\mathcal {F}}(m_h(t))\right) dt + {\mathcal {G}}(m_h(T)) \\&\quad = J(m,\alpha ) + h\Bigl \{ \int _{t_0}^T\Bigl (\int _{{\mathbb {R}}^d} D_\alpha L(x,\alpha )\cdot \beta (t,x) m(t,dx)+\int _{{\mathbb {R}}^d} L(x,\alpha )\rho (t,x)dx\\&\qquad +\frac{\delta {\mathcal {F}}}{\delta m}(m(t))(\rho (t))\Bigr )dt + \frac{\delta {\mathcal {G}}}{\delta m}(m(T))(\rho (T)) \Bigr \} \\&\qquad + \frac{h^2}{2} \Bigl \{ \int _{t_0}^T\Bigl (\int _{{\mathbb {R}}^d} D_{\alpha \alpha } L(x,\alpha )\beta (t,x)\cdot \beta (t,x) m(t,dx)\\&\qquad +2\int _{{\mathbb {R}}^d} D_\alpha L(x,\alpha )\cdot \beta (t,x) \rho (t,x)dx\\&\qquad +\int _{{\mathbb {R}}^d} 2 L(x,\alpha )\xi _h(t,x)dx+2\frac{\delta {\mathcal {F}}}{\delta m}(m(t))(\xi _h(t))+ \frac{\delta ^2 {\mathcal {F}}^2}{\delta m}(m(t))(\rho (t),\rho (t))\Bigr )dt \\&\qquad + 2\frac{\delta {\mathcal {G}}}{\delta m}(m(T))(\xi _h(T))+ \frac{\delta ^2 {\mathcal {G}}}{\delta m^2}(m(T))(\rho (T),\rho (T)) \Bigr \} + o(h^2). \end{aligned}$$

The first-order necessary optimality condition implies that the factor of h above vanishes and, therefore, the limit as h vanishes of the term in $h^2$ is nonnegative.

Thus

$$\begin{aligned}&\int _{t_0}^T\Bigl (\int _{{\mathbb {R}}^d} D_{\alpha \alpha } L(x,\alpha )\beta (t,x)\cdot \beta (t,x) m(t,dx)+2\int _{{\mathbb {R}}^d} D_\alpha L(x,\alpha )\cdot \beta (t,x) \rho (t,x)dx\\&\quad +\int _{{\mathbb {R}}^d} 2 L(x,\alpha )\xi (t,x)dx+2\frac{\delta {\mathcal {F}}}{\delta m}(m(t))(\xi (t))+ \frac{\delta ^2 {\mathcal {F}}^2}{\delta m}(m(t))(\rho (t),\rho (t))\Bigr )dt \\&\quad + 2\frac{\delta {\mathcal {G}}}{\delta m}(m(T))(\xi (T))+ \frac{\delta ^2 {\mathcal {G}}}{\delta m^2}(m(T))(\rho (T),\rho (T)) \; \ge \; 0. \end{aligned}$$

Using the equation satisfied by the multiplier u and the equation satisfied by $\xi$ we find

$$\begin{aligned}&\int _{t_0}^T \int _{{\mathbb {R}}^d} (L(x,\alpha )\xi (t,x)dx+\frac{\delta {\mathcal {F}}}{\delta m}(m(t))(\xi (t)))dt+ \frac{\delta {\mathcal {G}}}{\delta m}(m(T))(\xi (T))\\&\quad = \int _{t_0}^T \int _{{\mathbb {R}}^d} ((-H(x,Du)-\alpha \cdot Du)\xi (t,x)dx\\&\qquad +\frac{\delta {\mathcal {F}}}{\delta m}(m(t))(\xi (t)))dt+ \frac{\delta {\mathcal {G}}}{\delta m}(m(T))(\xi (T)) \\&\quad = -\int _{t_0}^T\int _{{\mathbb {R}}^d} Du(t,x)\cdot \beta (t,x) \rho (t,x) dxdt\\&\quad = - \int _{t_0}^T\int _{{\mathbb {R}}^d} D_\alpha L(x,\alpha )\cdot \beta (t,x) \rho (t,x)dxdt. \end{aligned}$$

Inserting the last equality in the previous inequality yields the second-order optimality condition when $\beta$ is smooth. The general case is obtained by approximation using the estimates in the first part of the proof. $\square$