Mean-Field Pontryagin Maximum Principle

Abstract

We derive a maximum principle for optimal control problems with constraints given by the coupling of a system of ordinary differential equations and a partial differential equation of Vlasov type with smooth interaction kernel. Such problems arise naturally as Gamma-limits of optimal control problems constrained by ordinary differential equations, modeling, for instance, external interventions on crowd dynamics by means of leaders. We obtain these first-order optimality conditions in the form of Hamiltonian flows in the Wasserstein space of probability measures with forward–backward boundary conditions with respect to the first and second marginals, respectively. In particular, we recover the equations and their solutions by means of a constructive procedure, which can be seen as the mean-field limit of the Pontryagin Maximum Principle applied to the optimal control problem for the discretized density, under a suitable scaling of the adjoint variables.

Appendix: Semiconvexity Along Geodesics of \(\mathbb {H}_c\)

Throughout the section, ${\mathcal {K}}$ shall denote a convex, compact subset of ${\mathbb R}^{2d}$. The following property shall be used to prove that the subdifferential of $\mathbb {H}_c$ is nonempty.

Definition A.1

Let $\psi : {\mathcal {P}}_2({\mathbb R}^n) \rightarrow ]-\infty ,+\infty ]$ be a proper, lower semicontinuous functional. We say that $\psi$ is semiconvex along geodesics whenever, for every $\nu _0, \nu _1 \in {\mathcal {P}}_2({\mathbb R}^n)$ and every optimal transport plan $\rho \in \varPi _o(\nu _0, \nu _1)$ there exists $C \in {\mathbb R}$ for which for every $s \in [0,1]$ it holds

$$\begin{aligned}&\psi (((1-s)\pi _1 + s\pi _2)_{\#}\rho ) \\&\quad \le (1-s) \psi (\nu _0) + s \psi (\nu _1) + Cs(1-s) {\mathcal W}^2_2(\nu _0, \nu _1)). \end{aligned}$$

In order to prove the semiconvexity of $\mathbb {H}_c$, we shall establish the semiconvexity of the following functionals:

where $\hat{{\mathcal {F}}}$, $\hat{{\mathcal {G}}}$, $\hat{\ell }$, and $\hat{\omega }$ are ${\mathcal {C}}^2$ functions. The desired result will then follow by noticing that $\mathbb {H}_c(\nu ) = \hat{\mathbb {H}}_c^1(\nu ) + \hat{\mathbb {H}}_c^2(\nu )$ for $\hat{{\mathcal {F}}} = {\mathcal {F}}$, $\hat{{\mathcal {G}}} = {\mathcal {G}}$, $\hat{\ell } = -\ell \circ (\pi _1, Id )$, $\hat{\omega } = \omega \circ \pi _1$ and ${\mathcal {K}} = cl \!\left( {B(0,R_T)}\right)$.

The following simple property will be needed to prove semiconvexity of the above functionals.

Lemma A.1

Let $\nu _0,\nu _1 \in {\mathcal P}_c({\mathbb R}^{2d})$ with support contained in ${\mathcal {K}}$. Let $\rho \in \varPi (\nu _0,\nu _1)$ and set

$$\begin{aligned} \nu _s = ((1-s)\pi _1 + s\pi _2)_{\#} \rho , \end{aligned}$$

(42)

for every $s \in [0,1]$. Then, it holds

$$\begin{aligned} \mathrm {supp}(\nu _s) \subseteq {\mathcal {K}} \quad \text { for all } s \in [0,1]. \end{aligned}$$

Proof

We first notice that for every $\rho \in \varPi (\nu _0,\nu _1)$ it holds

$$\begin{aligned} \mathrm {supp}(\rho ) \subseteq {\mathcal {K}} \times {\mathcal {K}}\,. \end{aligned}$$

(43)

This follows from the equality

$$\begin{aligned} {\mathbb R}^{4d} \backslash ({\mathcal {K}} \times {\mathcal {K}}) = ({\mathbb R}^{2d} \times ({\mathbb R}^{2d} \backslash {\mathcal {K}})) \cup (({\mathbb R}^{2d} \backslash {\mathcal {K}}) \times {\mathbb R}^{2d}) \end{aligned}$$

and from the fact that both ${\mathbb R}^{2d} \times ({\mathbb R}^{2d} \backslash {\mathcal {K}}))$ and $({\mathbb R}^{2d} \backslash {\mathcal {K}}) \times {\mathbb R}^{2d}$ are $\rho$-null sets by hypothesis.

To prove Lemma A.1, it suffices to show that for all $f \in {\mathcal {C}}({\mathbb R}^{2d})$ satisfying $f \equiv 0$ on ${\mathcal {K}}$ it holds

$$\begin{aligned} \int _{{\mathbb R}^{2d}} f \hbox {d}\nu _s = 0. \end{aligned}$$

(44)

Indeed,

$$\begin{aligned} \int _{{\mathbb R}^{2d}} f \hbox {d}\nu _s&= \int _{{\mathbb R}^{4d}} f d((1-s)\pi _1 + s\pi _2)_{\#} \rho (z_0,z_1) \\&=\int _{{\mathbb R}^{4d}} f((1-s)z_0 + s z_1) d \rho (z_0,z_1) \\&=\int _{{\mathcal {K}} \times {\mathcal {K}}} f((1-s)z_0 + s z_1) d \rho (z_0,z_1), \end{aligned}$$

since, by (43), $\mathrm {supp}(\rho ) \subseteq {\mathcal {K}} \times {\mathcal {K}}$. The convexity of ${\mathcal {K}}$ implies $(1-s)z_0 + sz_1 \in {\mathcal {K}}$ for every $s \in [0,1]$, which, together with the assumption $f \equiv 0$ in ${\mathcal {K}}$, yield (44), as desired. $\square$

In what follows, we shall make use of the following, well-known result.

Remark A.1

Let ${\mathcal {K}}$ be a convex, compact subset of ${\mathbb R}^{2d}$ and let $f \in {\mathcal {C}}^2({\mathbb R}^{2d};{\mathbb R})$. Then, there exists $C_{{\mathcal {K}},f} \in {\mathbb R}$ depending only on ${\mathcal {K}}$ and f such that

$$\begin{aligned} f((1-s)x_0 + sx_1) \le (1-s)f(x_0) + s f(x_1) + C_{{\mathcal {K}},f} s(1-s) \Vert x_0 - x_1\Vert ^2, \end{aligned}$$

(45)

for every $x_0,x_1 \in {\mathbb R}^{2d}$ and $s \in [0,1]$.

We now prove the semiconvexity of $\hat{\mathbb {H}}^1_c$.

Lemma A.2

Let $\nu _0,\nu _1 \in {\mathcal P}_c({\mathbb R}^{2d})$ and let $\rho \in \varPi (\nu _0,\nu _1)$. Then, there exists $C \in {\mathbb R}$ independent of $\nu _0$ and $\nu _1$ for which

$$\begin{aligned} \hat{\mathbb {H}}^1_c(((1-s)\pi _1 + s\pi _2)_{\#}\rho ) \le (1-s) \hat{\mathbb {H}}^1_c(\nu _0) + s \hat{\mathbb {H}}^1_c(\nu _1) + Cs(1-s) {\mathcal W}^2_2(\nu _0, \nu _1) \end{aligned}$$

holds for every $s \in [0,1]$.

Proof

We may assume $\mathrm {supp}(\nu _0),\mathrm {supp}(\nu _1) \subseteq {\mathcal {K}}$ for some convex and compact set ${\mathcal {K}} \subset {\mathbb R}^{2d}$, otherwise the inequality is trivial. Hence, from Lemma A.1, it follows $\mathrm {supp}(\nu _s) \subseteq {\mathcal {K}}$ for every $s \in [0,1]$. But then, since $\hat{{\mathcal {F}}}$ and $\hat{{\mathcal {G}}}$ are both ${\mathcal {C}}^2$, the result follows as in [33, Proposition 9.3.2, Proposition 9.3.5]. $\square$

Corollary A.1

Let $\hat{\omega } \in {\mathcal {C}}^2({\mathbb R}^{2d};{\mathbb R}^{d})$, $\nu _0,\nu _1 \in {\mathcal P}_c({\mathbb R}^{2d})$, $\rho \in \varPi (\nu _0,\nu _1)$ and define $\nu _s$ as in (42) for $s \in [0,1]$. If we set

$$\begin{aligned} \xi _s = \int _{{\mathbb R}^{2d}} \hat{\omega } d \nu _s, \end{aligned}$$

(46)

then

$$\begin{aligned} \Vert \xi _s - (1-s)\xi _0 - s\xi _1\Vert \le C s (1-s) {\mathcal W}^2_2(\nu _0,\nu _1), \end{aligned}$$

for all $s\in [0,1]$, where C is independent of $\nu _0$ and $\nu _1$.

Proof

Follows from Lemma A.2 applied first to the functions $\hat{{\mathcal {F}}} \equiv 0$ and $\hat{{\mathcal {G}}} \equiv \hat{\omega }$, and then to $\hat{{\mathcal {F}}} \equiv 0$ and $\hat{{\mathcal {G}}} \equiv -\hat{\omega }$. $\square$

The semiconvexity of $\hat{\mathbb {H}}^2_c$ will be deduced as a corollary of the following estimate.

Lemma A.3

Suppose that $\hat{\ell } \in {\mathcal {C}}^2({\mathbb R}^{2d}\times {\mathbb R}^d;{\mathbb R})$, let $z_0,z_1 \in {\mathcal {K}}$ and set $z_s = (1-s)z_0 + sz_1$ for all $s \in [0,1]$. Furthermore, let $\nu _0,\nu _1 \in {\mathcal P}_c({\mathbb R}^{2d})$, $\rho \in \varPi (\nu _0,\nu _1)$ and define $\nu _s$ and $\xi _s$ as in (42) and (46) for $s \in [0,1]$. Then, for all $s \in [0,1]$, it holds

$$\begin{aligned} \hat{\ell }(z_s,\xi _s)&\le (1-s) \hat{\ell }(z_0,\xi _0) + s \hat{\ell }(z_1,\xi _1) + C_{{\mathcal {K}},\hat{\ell },\hat{\omega }} s(1-s) {\mathcal W}^2_2(\nu _0,\nu _1) \\&\quad + C_{{\mathcal {K}},\hat{\ell },\hat{\omega }} s(1-s) \Vert z_0 - z_1\Vert ^2, \end{aligned}$$

for some constant $C_{{\mathcal {K}},\hat{\ell },\hat{\omega }}$ depending only on ${\mathcal {K}},\hat{\ell }$ and $\hat{\omega }$.

Proof

Since ${\mathcal {K}}$ is compact, $z_s \in {\mathcal {K}}$ for all $s \in [0,1]$. Moreover, $(1-s)\xi _0 + s\xi _1 \in {\mathcal {K}}'$ for all $s \in [0,1]$, for some convex and compact set ${\mathcal {K}}' \subset {\mathbb R}^d$. Notice that from (45) follows

$$\begin{aligned} \begin{aligned} \hat{\ell }(z_s,(1-s)\xi _0 + s\xi _1)&\le (1-s) \hat{\ell }(z_0,\xi _0) + s \hat{\ell }(z_1,\xi _1)\\&\quad + C_{{\mathcal {K}},{\mathcal {K}}'}s(1-s)\left( \Vert z_0 - z_1\Vert ^2 + \Vert \xi _0 - \xi _1\Vert ^2 \right) , \end{aligned} \end{aligned}$$

(47)

and from the definition of $\xi _s$ and Jensen’s inequality, we get

$$\begin{aligned} \Vert \xi _0 - \xi _1\Vert ^2&\le {\mathrm {Lip}}_{{\mathcal {K}}}(\omega ) {\mathcal W}^2_1(\nu _0,\nu _1) \le {\mathrm {Lip}}_{{\mathcal {K}}}(\omega ) {\mathcal W}^2_2(\nu _0,\nu _1). \end{aligned}$$

(48)

Moreover, for every $s\in [0,1]$ it holds

$$\begin{aligned} \begin{aligned} \Vert \hat{\ell }(z_s,\xi _s) - \hat{\ell }(z_s,(1-s)\xi _0 + s\xi _1)\Vert&\le {\mathrm {Lip}}_{{\mathcal {K}} \times {\mathcal {K}}'} \Vert \xi _s - (1-s)\xi _0 - s\xi _1\Vert \\&\le {\mathrm {Lip}}_{{\mathcal {K}} \times {\mathcal {K}}'} s(1-s) C {\mathcal W}^2_2(\nu _0,\nu _1). \end{aligned} \end{aligned}$$

(49)

Hence, for every $s\in [0,1]$, using (47), (48) and (49), we get

$$\begin{aligned} \hat{\ell }(z_s,\xi _s)&= \hat{\ell }(z_s,\xi _s) - \hat{\ell }(z_s,(1-s)\xi _0 + s\xi _1) + \hat{\ell }(z_s,(1-s)\xi _0 + s\xi _1) \\&\le (1-s) \hat{\ell }(z_0,\xi _0) + s \hat{\ell }(z_1,\xi _1) + C_{{\mathcal {K}},\hat{\ell },\hat{\omega }} s(1-s) {\mathcal W}^2_2(\nu _0,\nu _1) \\&\quad + C_{{\mathcal {K}},\hat{\ell },\hat{\omega }} s(1-s) \Vert z_0 - z_1\Vert ^2. \end{aligned}$$

This concludes the proof. $\square$

Corollary A.2

Let $\nu _0,\nu _1 \in {\mathcal P}_c({\mathbb R}^{2d})$ and $\rho \in \varPi _o(\nu _0,\nu _1)$. Then, there exists $C \in {\mathbb R}$ independent of $\nu _0$ and $\nu _1$ for which

$$\begin{aligned} \hat{\mathbb {H}}^2_c(((1-s)\pi _1 + s\pi _2)_{\#}\rho ) \le (1-s) \hat{\mathbb {H}}^2_c(\nu _0) + s \hat{\mathbb {H}}^2_c(\nu _1) + Cs(1-s) {\mathcal W}^2_2(\nu _0, \nu _1) \end{aligned}$$

holds for every $s \in [0,1]$.

Proof

Notice that, by Lemma A.1, $\hat{H}^2_c(\nu _s)$ can be rewritten as

$$\begin{aligned} \hat{\mathbb {H}}^2_c(\nu _s) = \int _{{\mathcal {K}} \times {\mathcal {K}}} \hat{\ell }(z_s,\xi _s)\hbox {d}\rho (z_0,z_1), \end{aligned}$$

Furthermore, since $\rho \in \varPi _o(\nu _0,\nu _1)$ it holds

$$\begin{aligned} \int _{{\mathcal {K}} \times {\mathcal {K}}} \Vert z_0 - z_1\Vert ^2 \hbox {d}\rho (z_0,z_1) = \int _{{\mathbb R}^{4d}} \Vert z_0 - z_1\Vert ^2 \hbox {d}\rho (z_0,z_1) = {\mathcal W}^2_2(\nu _0,\nu _1), \end{aligned}$$

the thesis follows from Lemma A.3. $\square$

Proposition A.1

The functional $\mathbb {H}_c$ is semiconvex along geodesics.

Proof

Follows directly from Lemma A.2 and Corollary A.2, by noticing that $\mathbb {H}_c(\nu ) = \hat{\mathbb {H}}_c^1(\nu ) + \hat{\mathbb {H}}_c^2(\nu )$ for $\hat{{\mathcal {F}}} = {\mathcal {F}}$, $\hat{{\mathcal {G}}} = {\mathcal {G}}$, $\hat{\ell } = -\ell \circ (\pi _1, Id )$, $\hat{\omega } = \omega \circ \pi _1$ and ${\mathcal {K}} = cl \!\left( {B(0,R_T)}\right)$. $\square$