Abstract
We are interested in Pontryagin’s stochastic maximum principle of controlled McKean–Vlasov stochastic differential equations. We allow the law to be anticipating, in the sense that, the coefficients (the drift and the diffusion coefficients) depend not only of the solution at the current time t, but also on the law of the future values of the solution \(P_{X(t+\delta )}\), for a given positive constant \(\delta \). We emphasise that being anticipating w.r.t. the law of the solution process does not mean being anticipative in the sense that it anticipates the driving Brownian motion. As an adjoint equation, a new type of delayed backward stochastic differential equations (BSDE) with implicit terminal condition is obtained. By using that the expectation of any random variable is a function of its law, our BSDE can be written in a simple form. Then, we prove existence and uniqueness of the solution of the delayed BSDE with implicit terminal value, i.e. with terminal value being a function of the law of the solution itself.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
The stochastic maximum principle and the characterization of the optimal control by Pontryagin’s maximum principle have been studied for the classical case by many authors such as Bismut [9], and Bensoussan [8] and so on. This principle was extended to more general cases such as controlled MacKean–Vlasov systems in the sense that both the drift and the diffusion coefficients in their dynamics, are supposed to depend at time t on the solution X(t) and on its law \(P_{X(t)}\) as follows
We refer for example to Carmona and Delarue [13, 14], Buckdahn et al. [10], Agram et al. [2,3,4].
In this paper, we want to extend Pontryagin’s stochastic maximum principle to a more general case, the case of controlled McKean–Vlasov SDE with anticipating law, i.e., our dynamics are assumed to satisfy for a given positive constant \(\delta \), the equation
where u(t) is our control process. Here the coefficients depend at time t on both the solution X(t) and the law \(P_{X(t+\delta )}\) of the anticipating solution. We remark here that the SDE (1.1) being anticipative w.r.t. the law of the solution process does not mean being anticipative in the sense that it anticipates the driving Brownian motion B.
The performance functional for a control u is given by
for some given bounded functions g and l and we want to maximize this performance over the set \({\mathcal {U}}\) of admissible control processes (will be specified later), as follows: find \(u^{*}\in {\mathcal {U}}\) such that \(J(u^{*})=\sup _{u\in {\mathcal {U}}}J\left( u\right) \).
We define the Hamiltonian H associated to this problem to be
and we show that the couple (p, q) is the solution of the adjoint backward stochastic differential equation given by
with terminal condition
The adjoint equation (1.2), (1.3) is a new type of delayed McKean–Vlasov BSDE with implicit terminal condition, i.e., with terminal value being a function of the law of the solution itself. In order to write it in a more comprehensible form, we use the fact that the expectation of any random variable is a function of its law, and under suitable assumptions on both the driver and a terminal value, we can get existence and uniqueness of our delayed BSDE. It is a generalisation of the adjoint equation of the above mentioned problem we are interested in.
Stochastic Pontryagin’s maximum principle in both cases partial and complete information, for a case where the mean-field term is expressed by the expected value of the state, has been studied for example by Anderson and Djehiche [7], Hu et al. [5, 18] and Agram and Røse [6].
For more general mean-field problems, we refer to Lions [19], Cardaliaguet [12] and Buckdahn et al. [11].
Delayed BSDEs have been studied by Delong and Imkeller [15]; they have later been extended by the same authors to the jump case, and studied [16] by the help of the Malliavin calculus and for more details, we refer to Delong’s book [17].
For mean-field delayed BSDE, we refer to Agram [1].
To the best of our knowledge our paper is the first to study optimal control problems of mean-field SDEs with anticipating law.
The paper is organized as follows: in the next section, we give some preliminaries which will be used throughout this work. In Sect. 3, the existence and the uniqueness of McKean–Vlasov SDEs with anticipating law is investigated. Section 4 is devoted to the study of Pontryagin’s stochastic maximum principle. In the last section, we prove the existence and the uniqueness for the associated delayed McKean–Vlasov BSDEs with implicit terminal condition.
This work has been presented at seminars and conferences in Brest, Biskra, Marrakech, Mans and Oslo.
2 Framework
We introduce some notations, definitions and spaces which will be used throughout this work. Let \(\left( \Omega ,{\mathcal {F}},P\right) \) be a complete probability space, B a d-dimensional Brownian motion and \({\mathbb {F}}=\left( {\mathcal {F}}_{t}\right) _{t\ge 0}\) the Brownian filtration generated by B and completed by all P-null sets. Let \({\mathcal {P}} _{2}({\mathbb {R}}^{d}):=\{\mu \in {\mathcal {P}}({\mathbb {R}}^{d}): \int _{{\mathbb {R}^{d}}} |x|^{2}\mu (dx)<+\infty \},\) where \({\mathcal {P}}({\mathbb {R}}^{d})\) is the space of all the probability measures on \(({\mathbb {R}}^{d},{\mathcal {B}}({\mathbb {R}}^{d} ))\); recall that \({\mathcal {B}}({\mathbb {R}}^{d})\) denotes the Borel \(\sigma \)-field over \({\mathbb {R}}^{d}\). We endow \({\mathcal {P}}_{2}({\mathbb {R}}^{d})\) with the 2-Wasserstein metric \(W_{2}\) on \({\mathcal {P}}_{2}({\mathbb {R}}^{d})\): For \(\mu _{1},\mu _{2}\in {\mathcal {P}}_{2}({\mathbb {R}}^{d})\), the 2-Wasserstein distance is defined by
We also remark that, if \((\Omega ,{\mathcal {F}},P)\) is “rich enough” in the sense that
then we also have
Let \(({\tilde{\Omega }},{\tilde{{{\mathcal {F}}}}},{\tilde{P}}) :=(\Omega ,{\mathcal {F}},P) \), \(\mathbb {{\tilde{F}}}:={\mathbb {F}}\) and \(({\bar{\Omega }},{\bar{{{\mathcal {F}}}}},{\bar{P}}) =( \Omega ,{\mathcal {F}} ,P) \otimes ( {\tilde{\Omega }},{\tilde{{{\mathcal {F}}}}},{\tilde{P}}) \). For any measurable space \((E,{\mathcal {E}}) \) and any random variable \(\zeta :\)\((\Omega ,{\mathcal {F}},P) \rightarrow (E,{\mathcal {E}}) \), we put \({\tilde{\zeta }}( {\tilde{\omega }}) :=\zeta ({\tilde{\omega }}) ,\)\({\tilde{\omega }}\in {\tilde{\Omega }}=\Omega \), \(\zeta (\omega ,{\tilde{\omega }}) :=\zeta (\omega ) \), \({\tilde{\zeta }}(\omega ,{\tilde{\omega }}) :={\tilde{\zeta }}({\tilde{\omega }}) \), \((\omega ,{\tilde{\omega }}) \in \Omega \times {\tilde{\Omega }}.\) We observe that \({\tilde{\zeta }}\) on \(({\tilde{\Omega }},{\tilde{{{\mathcal {F}}}}},{\tilde{P}}) \) is a copy of \(\zeta \) on \((\Omega ,{\mathcal {F}},P) ,\) and\(\ \zeta \), \({\tilde{\zeta }}\) are i.i.d under \({\bar{P}}.\) Moreover, for \(\zeta \), \(\eta :(\Omega ,{\mathcal {F}},P) \rightarrow (E,{\mathcal {E}}) \) random variables and \(\varphi : (E^{2},{\mathcal {E}}^{2}) \rightarrow ( B,{\mathcal {B}}( {\mathbb {R}} )) \) a bounded and measurable function, we have
We recall now the notion of derivative of a function \(\varphi :{\mathcal {P}} _{2}({\mathbb {R}}^{d})\rightarrow {\mathbb {R}}\) w.r.t a probability measure \(\mu \), which was studied by Lions in his course at Collège de France in [19]; see also the notes of Cardaliaguet [12], the works by Carmona and Delarue [14] and in Buckdahn et al. [11]. We say that \(\varphi \) is differentiable at \(\mu \) if, for the lifted function \({\tilde{\varphi }}(\zeta ):=\varphi (P_{\zeta })\), \(\zeta \in L^{2}\left( \Omega ,{\mathcal {F}},P;{\mathbb {R}}^{d}\right) \), there is some \(\zeta _{0}\in L^{2}\left( \Omega ,{\mathcal {F}},P;{\mathbb {R}}^{d}\right) \) with \(P_{\zeta _{0}}=\mu \), such that \({\tilde{\varphi }}\) is differentiable in the Frèchet sense at \(\zeta _{0}\), such that there exists a linear continuous mapping \(D{\tilde{\varphi }}(\zeta _{0}):L^{2}(\Omega ,{\mathcal {F}} ,P;{\mathbb {R}}^{d}) \rightarrow {\mathbb {R}}\)\((L( L^{2}( \Omega ,{\mathcal {F}},P;{\mathbb {R}}^{d}) ;{\mathbb {R}}) )\), such that
for \(\left| \eta \right| _{L^{2}(\Omega ) }^{2} \rightarrow 0\), \(\eta \in L^{2}(\Omega ,{\mathcal {F}},P;{\mathbb {R}} ^{d}).\) With the identification that \(L(L^{2}(\Omega ,{\mathcal {F}},P;{\mathbb {R}}^{d}) ;{\mathbb {R}}) \equiv L^{2}(\Omega ,{\mathcal {F}},P;{\mathbb {R}}^{d})\), given by Riesz’ representation theorem, we can write
In Lions [19] and Cardaliaguet [12], it has been proved that there exists a Borel function \(h:{\mathbb {R}}\rightarrow {\mathbb {R}}\), such that \((D{\tilde{\varphi }})(\zeta _{0})=h(\zeta _{0})\)P-a.s. Note that \(h(\zeta _{0})\)P-a.s. uniquely determined. Consequently, h(y) is \(P_{\zeta _{0}}(dy)\)-a.e. uniquely determined. We define
Hence
Example 2.1
Given a function \(\varphi (P_{\zeta })=g\left( {\mathbb {E}}\left[ f(\zeta )\right] \right) \), for \(g,f\in C_{l,b}^{1}({\mathbb {R}})\) and \(\zeta \in L^{2}( \Omega ,{\mathcal {F}},P;{\mathbb {R}}^{d})\), then
Throughout this work, we will use also the following spaces:
-
\(S_{{\mathbb {F}}}^{2}([0,T])\) is the set of real valued \({\mathbb {F}} \)-adapted continuous processes \((X(t))_{t\in [0,T]}\) such that
$$\begin{aligned} {\Vert X\Vert }_{S_{{\mathbb {F}}}^{2}}:={{\mathbb {E}}}\Bigg [\sup _{t\in [0,T]}|X(t)|^{2}\Bigg ]<\infty . \end{aligned}$$ -
\(L_{{\mathbb {F}}}^{2}([0,T])\) is the set of real valued \({\mathbb {F}} \)-adapted processes \((Q(t))_{t\in [0,T]}\) such that
$$\begin{aligned} \Vert Q\Vert _{L_{{\mathbb {F}}}^{2}}^{2}:={{\mathbb {E}}}\Bigg [ \int _{0}^{T} |Q(t)|^{2}dt\Bigg ]<\infty . \end{aligned}$$ -
\(L^{2}({\mathcal {F}}_{t})\) is the set of real valued square integrable \({\mathcal {F}}_{t}\)-measurable random variables.
3 Solvability of the anticipated forward McKean–Vlasov equations
Let us consider the following anticipated SDE for a given positive constant \(\delta \)
The functions \(\sigma :\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \rightarrow {\mathbb {R}} ^{d\times d}\) and \(b:\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \rightarrow {\mathbb {R}} ^{d}\) are progressively measurable and are assumed to satisfy the following set of assumptions.
Assumptions (H.1): There exists \(C>0\), such that
-
1.
For all \(t\in \left[ 0,T\right] \), \(x,x^{\prime }\in {\mathbb {R}} ^{d},\mu ,\mu ^{\prime }\in {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \)
$$\begin{aligned} \left| \sigma \left( t,x,\mu \right) -\sigma \left( t,x^{\prime } ,\mu ^{\prime }\right) \right| +\left| b\left( t,x,\mu \right) -b\left( t,x^{\prime },\mu ^{\prime }\right) \right| \le C\left( \left| x-x^{\prime }\right| +W_{2}\left( \mu ,\mu ^{\prime }\right) \right) . \end{aligned}$$ -
2.
For all \(t\in \left[ 0,T\right] \), \(x,x^{\prime }\in {\mathbb {R}} ^{d}\)
$$\begin{aligned} \left| \sigma \left( t,0,P_{0}\right) \right| +\left| b\left( t,0,P_{0}\right) \right| \le C, \end{aligned}$$where \(P_{0}\) is the distribution law of zero, i.e., the Dirac measure with mass at zero.
Remark 3.1
Note that Assumption (H.1) implies that the coefficients b and \(\sigma \) are of linear growth. Indeed we have
and a similar estimate holds for b.
Proposition 3.2
Under the above Assumption (H.1), there is some \(\delta _{0}>0\), such that for all \(\delta \in \left( 0,\delta _{0}\right] \), there exists a unique solution \(X\in S_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \) of SDE (3.1); \(\delta _{0}\) depends only on the Lipschitz constant C of the coefficients b and \(\sigma \) (see (H.1)) but not on the coefficients themselves.
Proof
For \(U\in S_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) ,\) we can make the identification with the continuous process
Given \(U\in H\), we put
Then \(V\in S_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \subset H\) (with the above identification), and setting \(\Phi \left( U\right) :=V\) we define a mapping \(\Phi :H\rightarrow H.\) Fixing \(\beta >0\) (\(\beta \) will be specified later), we introduce the norm
Obviously, \((H,\left\| \cdot \right\| _{-\beta })\) is a Banach space, and the norm \(\left\| \cdot \right\| _{-\beta }\) is equivalent to the norm \(\left\| \cdot \right\| _{0}\) (obtained from \(\left\| \cdot \right\| _{-\beta }\) by taking \(\beta =0\)). We are going to prove that \(\Phi :(H,\left\| \cdot \right\| _{-\beta })\rightarrow (H,\left\| \cdot \right\| _{-\beta })\) is contracting. Indeed, we consider arbitrary \(U^{i}\in H\), \(i=1,2,\) and we put \(V^{i}:=\Phi ( U^{i}) \), \(i=1,2\). Let \({\bar{U}}:=U^{1}-U^{2}\) and \({\bar{V}}:=V^{1}-V^{2}.\) Then, applying Itô’s formula to \((e^{-\beta t}\left| {\bar{V}}\left( t\right) \right| ^{2})_{t\ge 0}\), we get from the Assumptions (H.1)
Indeed, we recall that
Hence for \(t=T\), we have
We seek suitable \(\beta >0\), \(\delta >0\) with \(\delta \le \frac{1}{\beta }\), i.e., \(\beta \delta \le 1\), in order to estimate
Choosing \(\beta :=7C\), \(\delta _{0}:=\tfrac{1}{7C}(=\frac{1}{\beta })\), we have for all \(\delta \in \left( 0,\delta _{0}\right) \):
Then
i.e.,
This proves that \(\Phi :(H,\left\| \cdot \right\| _{-\beta })\rightarrow (H,\left\| \cdot \right\| _{-\beta })\) is a contraction on the Banach space \(\left( H,\left\| \cdot \right\| _{-\beta }\right) \). Hence, there is a unique fixed point \(X\in H,\) such that \(X=\Phi \left( X\right) ,\) i.e.,
\(v\left( dt\right) \)-a.e. on \(\left[ 0,T\right] \), P-a.s., with \(v\left( dt\right) =I_{\left[ 0,T\right] }\left( t\right) dt+P_{T} \left( dt\right) \) (Recall the definition of H). For a \(v\otimes P-\)modification of X, also denoted by X, we have \(X\in S_{{\mathbb {F}}} ^{2}\left( \left[ 0,T\right] \right) \) and
\(\square \)
4 Pontryagin’s stochastic maximum principle
Let us introduce now our stochastic control problem.
4.1 Controlled stochastic differential equation
As control state space we consider a bounded convex subset U of \({\mathbb {R}} ^{d}\). A process \(u=\left( u(t)\right) _{t\in \left[ 0,T\right] }:\left[ 0,T\right] \times \Omega \rightarrow U\) which is progressively measurable is called an admissible control; \({\mathcal {U}}=L_{{\mathbb {F}}}^{0}(\left[ 0,T\right] ;U)\) is the set of all admissible controls. The dynamics of our controlled system are driven by functions \(\sigma :\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \times U\rightarrow {\mathbb {R}} ^{d\times d}\), \(b:\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \times U\rightarrow {\mathbb {R}} ^{d}\).
Assumptions (H.2): The coefficients \(\sigma \) and b are supposed to be continuous on \(\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \times U\) and Lipschitz on \( {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) ,\) uniformly w.r.t. \(u\in U\) and \(\omega \in \Omega \) i.e., there is some \(C>0\), such that for all \(\left( x,\mu \right) ,\left( x^{\prime } ,\mu ^{\prime }\right) \in {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \), \(u\in U\), we have
On the other hand, from the continuity of the coefficients on \(\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \times U\), we have
This shows that, for every \(u\in {\mathcal {U}}:=L_{{\mathbb {F}}}^{0}\left( \left[ 0,T\right] ;U\right) \) and \(\omega \in \Omega \), the coefficients \(\sigma \) and b satisfy the Assumptions (H.1). Thus, for \(\delta _{0}>0\) from Proposition 3.2, for all \(u\in {\mathcal {U}}\); \(x\in {\mathbb {R}} ^{d}\), there is a unique solution \(X^{u}\left( t\right) \in S_{{\mathbb {F}} }^{2}\left( \left[ 0,T\right] ; {\mathbb {R}} ^{d}\right) \) of the equation
4.2 Cost functional
Let us endow our control problem with a terminal cost \(g:\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \rightarrow {\mathbb {R}} \), and a running cost \(l:\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \times U\rightarrow {\mathbb {R}} \).
Assumptions (H.3): We suppose that \(g:\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \rightarrow {\mathbb {R}} \) is continuous and satisfies a linear growth assumption: For some constant \(C>0\),
Let \(l:\left[ 0,T\right] \times \Omega \times {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \times U\rightarrow {\mathbb {R}} \) be continuous and such that, for some \(C>0\), for all \(\left( x,\mu ,u\right) \in {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \times U\), we have
For any admissible control u, we define the performance functional:
A control process \(u^{*}\in {\mathcal {U}}\) is called optimal, if
Let us suppose that there is an optimal control \(u^{*}\in {\mathcal {U}}\). Our objective is to characterise the optimal control. For this let us assume some additional assumptions.
Assumptions (H.4): Let U be convex (and, hence, \({\mathcal {U}}\) is convex). The functions \(\sigma \left( \cdot ,\cdot ,\cdot ,u\right) \), \(b\left( \cdot ,\cdot ,\cdot ,u\right) \), \(l\left( \cdot ,\cdot ,\cdot ,u\right) \) and \(g\left( \cdot ,\cdot \right) \) are continuously differentiable over \( {\mathbb {R}} ^{d}\times {\mathcal {P}}_{2}\left( {\mathbb {R}} ^{d}\right) \times U\) with bounded derivatives.
Given an arbitrary but fixed control \(u\in {\mathcal {U}}\), we define
Note that, thanks to the convexity of U and \({\mathcal {U}}\), also \(u^{\theta }\in {\mathcal {U}},\theta \in \left[ 0,1\right] \). We denote by \(X^{\theta }:=X^{u^{\theta }}\) and by \(X^{*}:=X^{u^{*}}\) the solution processes corresponding to \(u^{\theta }\) and \(u^{*},\) respectively. For simplicity of the computations, we set \(d=1.\)
4.3 Variational SDE
Given \(u^{*}\in {\mathcal {U}}\) and the associated controlled state process \(X^{*}\), let \(Y=\left( Y(t)\right) _{t\in \left[ 0,T\right] }\in S_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \) be the unique solution of the following SDE
Remark 4.1
Note that SDE (4.1) is obtained by formal differentiation of Eq. (6.1) (with \(u=u^{\theta }\)) at \(\theta =0\).
From the previous section, we have the existence and the uniqueness of a solution for all \(\delta \in \left( 0,\delta _{0}^{\prime }\right] ;0<\)\(\delta _{0}^{\prime }\le \delta _{0}\) small enough.
Indeed, Eq. (4.1) is of the form
where \(\alpha _{i}:\left[ 0,T\right] \times \Omega \rightarrow {\mathbb {R}} ,i=1,2,3,4,\) are bounded progressively measurable processes and \(\beta _{j}:\left[ 0,T\right] \times \Omega \rightarrow {\mathbb {R}} ,j=1,2,\) two bounded \(({\mathcal {F}}_{t}\otimes {\tilde{{{\mathcal {F}}}}} _{T}{)}\)- progressively measurable processes. With the method used in the proof of Proposition 3.2, we get the existence of \(\delta _{0}^{^{\prime }}\in \left( 0,\delta _{0}\right] \) stated above. It turns out that Y(t) is the \(L^{2}\)-derivative of \(X^{\theta }(t)\) w.r.t. \(\theta \) at \(\theta =0.\) More precisely, the following property holds.
Lemma 4.2
\({{\mathbb {E}}}[\sup _{t\in \left[ 0,T\right] }|Y(t)-\tfrac{X^{\theta }(t)-X^{*}(t)}{\theta }|^{2}]\rightarrow 0\) as \(\left( \theta \rightarrow 0\right) \).
Proof
The proof is obtained with standard computations. For the sake of completeness, we give details in the Appendix. \(\square \)
4.4 Variational inequality
We know that if \(u^{*}\) is an optimal control, we have \(J(u^{*})\le J(u^{\theta })\), for all \(\theta \in \left[ 0,1\right] \), i.e.,
Lemma 4.3
Under Assumptions (H.3), (H.4), Lemma 4.2 and inequality (4.2), we have
Proof
From the definition of \(J(u^{*})\), we have
the fact that
and by repeating previous arguments, (4.3) is obtained. \(\square \)
4.5 Adjoint processes
Let us first recall the equation satisfied by the derivative process
where for notational convenient, we have used the short hand notations
and similarly.In order to determine the adjoint backward equation, we suppose that it has the form
for some adapted process \(\alpha \) and terminal value p(T) which we have to determine. Applying Itô’s formula to \(p\left( t\right) Y\left( t\right) ,\) we obtain
with \({\mathbb {E}}\left[ Y\left( 0\right) p\left( 0\right) \right] =0.\)
We have that
Using the above computations, we obtain
and similarly
Substituting (4.7), (4.8) into (4.5), we get
As \(X^{*}\left( t\right) =X^{*}\left( T\right) ,\)\(Y(t)=Y(T),t\ge T\), we get
Analogously,
Combining (4.9)–(4.11), we obtain
Hence, putting
and
Then, (4.12), takes the form
We are now able to determine our adjoint process, putting
and
where we denote by
Combining (4.13), (4.14) with (4.16) and (4.17), then (4.4) takes the following form
with terminal condition
We suppose that the above BSDE (4.18), (4.19) has a unique solution \((p,q)\in S_{{\mathbb {F}}}^{2}([0,T])\times L_{{\mathbb {F}}}^{2}([0,T])\). We will discuss this BSDE in the next section.
4.6 Stochastic maximum principle
We define now the Hamiltonian \(H:[0,T]\times \Omega \times {\mathbb {R}} \times {\mathcal {P}}_{2}\left( {\mathbb {R}} \right) \times U\times {\mathbb {R}} \times {\mathbb {R}} \rightarrow {\mathbb {R}} \), as
Theorem 4.4
(Maximum principle) Let \(u^{*}(t)\) be an optimal control and \(X^{*}(t)\) the corresponding trajectory. Then, we have
where \((p,q)\in S_{{\mathbb {F}}}^{2}([0,T])\times L_{{\mathbb {F}}}^{2}([0,T])\) is the solution of the adjoint equation (4.18), (4.19).
Proof
From (4.15) and (4.3) with the choice (4.16) and (4.17), we get
for all \(u\in {\mathcal {U}}\). Assume for some \(u\in {\mathcal {U}},\)
is such that
Then, for \({\tilde{u}}(t):=u(t)I_{\Gamma _{u}}(t)+u^{*}(t)I_{\Gamma _{u}^{c} }(t),t\in \left[ 0,T\right] ,{\tilde{u}}\in {\mathcal {U}}\) is such that
But this is a contradiction and proves that
dtdP-a.e, for all \(u\in {\mathcal {U}}.\) By the definition of H in (4.20), the proof is complete. \(\square \)
5 Solvability of the delayed McKean–Vlasov BSDE
We now study the BSDE which is the adjoint equation to the above control problem. We consider the BSDE
which we have seen due to our computations that it has the form
with
Let us better understand the form of this BSDE: for \(\left( t,\omega ,{\tilde{\omega }}\right) \in \left[ 0,T\right] \times \Omega \times {\tilde{\Omega }},x_{1},x_{2},x_{3},x_{4}\in {\mathbb {R}} \), putting
and in order to describe also the terminal condition of our BSDE, we consider the coefficient
We know that
By using (5.3) the BSDE (5.1), (5.2) takes the form
Definition 5.1
The BSDE \((p,q)\in S_{{\mathbb {F}}}^{2}([0,T])\times L_{{\mathbb {F}}}^{2}([0,T])\) is defined by
where
we see that \(\zeta \in L^{2}\left( \Omega ,{\mathcal {F}},P\right) \).
Remark 5.2
We call the BSDE (5.4), delayed BSDE because the driver at time t depend on both the solution at time t and on its previous value, i.e. the solution at time \(t-\delta \).
Note that \(\theta \) satisfies the following:
Assumptions (H.5):
-
1.
\(\theta :\left[ 0,T\right] \times \Omega \times {\tilde{\Omega }}\times {\mathbb {R}} ^{4}\rightarrow {\mathbb {R}} \) is jointly measurable,
-
2.
\(\theta _{t}\left( \cdot ,\cdot ,x\right) \) is \({\mathcal {F}}_{t} \otimes {\tilde{{{\mathcal {F}}}}}_{T}\)-progressively measurable, for all \(x\in {\mathbb {R}} ^{4}\),
-
3.
for all \(x,x^{\prime }\in {\mathbb {R}} ^{4}\),
$$\begin{aligned} \left| \theta _{t}\left( \omega ,{\tilde{\omega }},x\right) -\theta _{t}\left( \omega ,{\tilde{\omega }},x^{\prime }\right) \right| \le C\left| x-x^{\prime }\right| \text {, }dtP(d\omega ){\tilde{P}} (d{\tilde{\omega }})\text {-a.e.} \end{aligned}$$
Similarly, \(\vartheta \) is assumed to satisfy the following:
Assumptions (H.6):
-
1.
\(\vartheta :\left[ T-\delta ,T\right] \times \Omega \times \tilde{\Omega }\times {\mathbb {R}} ^{2}\rightarrow {\mathbb {R}} \) is jointly measurable,
-
2.
\(\vartheta \left( \cdot ,\cdot ,x\right) \) is \({\mathcal {F}}_{T} \otimes {\tilde{{{\mathcal {F}}}}}_{T}\)-measurable, for all \(\left( t,x\right) \in \left[ T-\delta ,T\right] \times {\mathbb {R}} ^{2}\),
-
3.
\(\left| \vartheta _{t}\left( \omega ,{\tilde{\omega }},0\right) \right| \le C,dtP\left( d\omega \right) {\tilde{P}}\left( d{\tilde{\omega }}\right) \)-a.e, for some constant \(C>0\),
-
4.
\(|\vartheta _{t}\left( \omega ,{\tilde{\omega }},x\right) -\vartheta _{t}(\omega ^{^{\prime }},{\tilde{\omega }}^{\prime },x^{\prime })|\le C\left| x-x^{\prime }\right| \), for all x, \(x^{\prime }\in {\mathbb {R}} ^{2}\), \(dtP(d\omega ){\tilde{P}}(d{\tilde{\omega }})\)-a.e.
However, the function \(\varphi :{\mathcal {P}}_{2}( {\mathbb {R}} )\rightarrow {\mathbb {R}} \) in a delayed BSDE (5.1), (5.2) is Lipschitz continuous. Consequently, we have the following more general form for our BSDE.
We consider arbitrary \(\theta ,\vartheta ,\varphi ,\psi ,\zeta \) with \(\theta \) satisfying the Assumption (H.5), \(\vartheta \) satisfying (H.6), \(\varphi ,\psi :{\mathcal {P}}_{2}( {\mathbb {R}} )\rightarrow {\mathbb {R}} \) being Lipschitz and \(\zeta \in L^{2}\left( \Omega ,{\mathcal {F}},P\right) \), and we study the delayed BSDE,
Remark 5.3
The adjoint BSDE we describe it above is a special case of (5.5). Indeed, for the adjoint BSDE we have:
Definition 5.4
We say that \(\left( p,q\right) \in S_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \times L_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \) is a solution of (5.5), if
and if (5.5) is satisfied.
Theorem 5.5
Under the above assumptions there is some \(\delta _{0}>0\) small enough such that for all \(\delta \in \left( 0,\delta _{0}\right] \), BSDE (5.5) has a unique solution \(\left( p,q\right) \in S_{{\mathbb {F}}}^{2}\left[ 0,T\right] \times L_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \).
Proof
We embed \(S_{{\mathbb {F}}}^{2}\left[ 0,T\right] \subset {\mathbb {R}} \times L_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \): For \(U\in S_{{\mathbb {F}}}^{2}\left[ 0,T\right] \) we put \(U\left( t\right) =U\left( 0\right) \), \(t\in \left[ -\delta ,0\right] \), and we observe that
For \(V\in L_{{\mathbb {F}}}^{2}\left[ 0,T\right] \) we use the convention that \(V\left( t\right) =0\), \(t\le 0\). Let \(\left( U,V\right) \in H=\left( {\mathbb {R}} \times L_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \times L_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \right) \), and \(\left( p,q\right) \in S_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \times L_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) \left( \subset H\right) \) the unique solution of the equation
For this observe that the terminal condition is in \(L^{2}\left( \Omega ,{\mathcal {F}},P\right) \) and the given coefficient of the BSDE is \({\mathbb {F}}\)-progressively measurable and square integrable. Let us define
For a suitable \(\beta >0\) which will be specified later, we define the norm
which is equivalent to the standard norm \(\left\| \cdot \right\| _{0}\)\(\left( \text {for }\beta =0\right) \) on H. Note that \(\left( H,\left\| \cdot \right\| _{0}\right) \) is a Banach space, and so is \((H,\left\| \cdot \right\| _{\beta })\). We show that for some \(\delta _{0}>0\), we have for all \(\delta \in \left( 0,\delta _{0}\right] \) that
is a contraction, i.e, there is a unique fixed point \(\left( p,q\right) \in H,\) such that \(\Phi \left( p,q\right) =\left( p,q\right) .\) Then \(\left( p,q\right) \) solves BSDE (5.5) and belongs in particular to \(S_{{\mathbb {F}}}^{2}\left( \left[ 0,T\right] \right) .\) Let \(\left( U^{i},V^{i}\right) \in H,i=1,2,\) and consider \(\left( p^{i},q^{i}\right) =\Phi \left( U^{i},V^{i}\right) \), i.e.,
From Itô’s formula applied to \(e^{\beta t}\left| {\bar{p}}\left( t\right) \right| ^{2}\), we obtain
Observe that, thanks to the Assumptions (H.5) and (H.6),
Hence, for some small \(\rho >0,\)
Note that
Moreover, recall that, \({\bar{V}}\left( t\right) =0\), \(t\le 0.\) On the other hand,
Letting \(0<\delta \le \rho \), we obtain
We choose now \(\rho =\tfrac{1}{8C},\beta =C_{\rho }+1\) and \(\delta _{0}\in (0,\frac{1}{8C}),\) such that \(\tfrac{1+e^{\beta \delta _{0}}}{8}\le \tfrac{1}{2}\). Then, for all \(\delta \in \left( 0,\delta _{0}\right] ,\)
i.e.,
for all \(\left( U^{1},V^{1}\right) ,\left( U^{2},V^{2}\right) \in H.\) This completes the proof. \(\square \)
References
Agram, N.: Mean-field delayed BSDEs with Jumps. arXiv preprint arXiv:1801.03364 (2018)
Agram, N., Bachouch, A., Øksendal, B., Proske, F.: Singular control optimal stopping of memory mean-field processes. SIAM J. Math. Anal. 51(1), 450–468 (2019)
Agram, N., Øksendal, B.: Stochastic control of memory mean-field processes. Appl. Math. Optim. 79(1), 181–204 (2019)
Agram, N., Øksendal, B.: Model uncertainty stochastic mean-field control. Stoch. Anal. Appl. 37(1), 36–56 (2019)
Agram, N., Hu, Y., Øksendal, B.: Mean-field backward stochastic differential equations and applications. arXiv preprint arXiv:1801.03349 (2018)
Agram, N., Røse, E.E.: Optimal control of forward-backward mean-field stochastic delayed systems. Afrika Matematika 29(1–2), 149–174 (2018)
Andersson, D., Djehiche, B.: A maximum principle for SDEs of mean-field type. Appl. Math. Optim. 63(3), 341–356 (2011)
Bensoussan, A.: Lectures on stochastic control. In: Mitter, S.K., Moro, A. (eds.) Nonlinear Filtering and Stochastic Control. Lecture Notes in Mathematics, vol. 972, pp. 1–62. Springer, Berlin, Heidelberg (1982)
Bismut, J.M.: An introductory approach to duality in optimal stochastic control. SIAM Rev. 20(1), 62–78 (1978)
Buckdahn, R., Djehiche, B., Li, J.: A general stochastic maximum principle for SDEs of mean-field type. Appl. Math. Optim. 64(2), 197–216 (2011)
Buckdahn, R., Li, J., Peng, S., Rainer, C.: Mean-field stochastic differential equations and associated PDEs. Ann. Probab. 45(2), 824–878 (2017)
Cardaliaguet, P.: Notes on mean field games, p. 120. Technical report (2010)
Carmona, R., Delarue, F., Lachapelle, A.: Control of McKean–Vlasov dynamics versus mean field games. Math. Financ. Econ. 7(2), 131–166 (2013)
Carmona, R., Delarue, F.: Forward-backward stochastic differential equations and controlled McKean–Vlasov dynamics. Ann. Probab. 43(5), 2647–2700 (2015)
Delong, Ł., Imkeller, P.: Backward stochastic differential equations with time delayed generators-results and counterexamples. Ann. Appl. Probab. 20(4), 1512–1536 (2010)
Delong, Ł., Imkeller, P.: On Malliavin’s differentiability of BSDEs with time delayed generators driven by Brownian motions and Poisson random measures. Stoch. Process. Appl. 120(9), 1748–1775 (2010)
Delong, Ł.: Backward stochastic differential equations with jumps and their actuarial and financial applications. Springer, Berlin (2013)
Hu, Y., Øksendal, B., Sulem, A.: Singular mean-field control games. Stoch. Anal. Appl. 35(5), 823–851 (2017)
Lions, P.L.: Théorie des jeux à champs moyens. Cours au college de France (2012)
Acknowledgements
I would like to thank Rainer Buckdahn for helpful discussions during my visit to Brest as well as Bernt Øksendal.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This research was carried out with support of the Norwegian Research Council, within the research project Challenges in Stochastic Control, Information and Applications (STOCONINF), project number 250768/F20.
Appendix
Appendix
Let \(\delta _{0}^{^{\prime }}\in \left( 0,\delta _{0}\right] \). For simplicity, we suppose that \(b\equiv 0\) and that \(\sigma \left( t,x,\mu ,u\right) =\sigma \left( t,\mu ,u\right) \), because the case \(\sigma \left( t,x,u\right) \) is well studied in the literature. Among the vast literature, we refer, for example, to [8]. Notice that, we have
and that Eq. (4.1) writes as follows
Then,
Let us define \(\zeta ^{\theta }\left( t\right) :=\tfrac{X^{\theta }\left( t\right) -X^{*}\left( t\right) }{\theta }\), \(\theta \in \left( 0,1\right] \). Then Eq. (6.1) becomes
Recall that \(\partial _{\mu }\sigma ,\partial _{u}\sigma \) and U are bounded. Then, with the argument given above, for some \(0<\delta _{0}^{^{\prime \prime } }\leqslant \delta _{0}^{^{\prime }}\) small enough, for all \(\delta \in [0,\delta _{0}^{^{\prime \prime }}],\)
Indeed, for suitably defined \(({\mathcal {F}}_{t}\otimes {\tilde{{{\mathcal {F}}}}} _{T}{)}\)-adapted \((\alpha ^{\theta }(s)),(\beta ^{\theta }(s))\) processes, depending on \(u^{\theta },u^{*}\); but bounded by a bound independent of \(\theta ,\)\(u^{\theta }\) and \(u^{*}\), we have
Then, for \(\beta >0\), with \(\zeta ^{\theta }(t)=0\),
where we have supposed that \(\beta \delta \le 1\). Letting \(\beta \delta \le 1\) we put \(\beta =4C\), and we suppose that \(\delta \le \tfrac{1}{6C}\). Then, for some constant \(C^{^{\prime }}\),
We can write now (6.2) as
with
We have the following estimate
where
Then
As \((\partial _{\mu }\sigma )\) is bounded and
We can conclude that \(I_{1}^{\theta }\rightarrow 0\), as \(\theta \rightarrow 0\). On the other hand, as \(\partial _{u}\sigma \) is bounded, it follows from the bounded convergence theorem that
Consequently,
Recalling that
and the equation satisfied by Y, we see that
Consequently, for \(\delta \le \delta _{0}^{^{\prime }}\),
This completes the proof.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Agram, N. Stochastic optimal control of McKean–Vlasov equations with anticipating law. Afr. Mat. 30, 879–901 (2019). https://doi.org/10.1007/s13370-019-00689-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13370-019-00689-w
Keywords
- Stochastic control
- McKean-Vlasov equations
- Anticipating law
- Delayed backward stochastic differential equations