Abstract
In this paper, we consider stochastic optimal control of systems driven by stochastic differential equations with irregular drift coefficient. We establish a necessary and sufficient stochastic maximum principle. To achieve this, we first derive an explicit representation of the first variation process (in the Sobolev sense) of the controlled diffusion. Since the drift coefficient is not smooth, the representation is given in terms of the local time of the state process. Then we construct a sequence of optimal control problems with smooth coefficients by an approximation argument. Finally, we use Ekeland’s variational principle to obtain an approximating adjoint process from which we derive the maximum principle by passing to the limit. The work is notably motivated by the optimal consumption problem of investors paying wealth tax.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Let \(T \in (0,\infty )\) be a given deterministic time horizon and \(d \in \mathbb {N}\), let \(\Omega := C([0,T],\mathbb {R}^d)\) be the canonical space of continuous paths. We denote by B the canonical process and by \(\mathbb {P}\) the Wiener measure. Equip \(\Omega \) with \((\mathcal {F}_t)_{t\in [0,T]}\), the \(\mathbb {P}\)-completion of the canonical filtration of B. Given a d-dimensional vector \(\sigma \) and a function \(b: [0,T]\times \mathbb {R}\times \mathbb {R}^m\rightarrow \mathbb {R}\), we consider a controlled diffusion of the form
and the control problem
Hereby, the performance functional J is given by
where, f and g may be seen as profit and bequest functions, respectively. The set \(\mathcal {A}\) is the set of admissible controls and is defined as the set of progressively measurable processes \(\alpha \) valued in a closed convex set \(\mathbb {A}\subseteq \mathbb {R}^m\) such that (1) admits a unique strong solution. The goal of the present article is to derive the maximum principle for the above control problem when the drift b is merely measurable in the state variable x.
The stochastic maximum principle is arguably one of the most prominent ways to tackle stochastic control problems as (2) by fully probabilistic methods. It is the direct generalization to the stochastic framework of the maximum principle of Pontryagin [47] in deterministic control. It gives a necessary condition of optimality in the form of a two-point boundary value problem and a maximum condition on the Hamiltonian. More precisely let the Hamiltonian H be defined as
and assume just for a moment the functions b, f and g to be continuously differentiable. Then, if \({\hat{\alpha }} \in \mathcal {A}\) is an optimal control, then according to the stochastic maximum principle, it holds \(H(t, X^{{\hat{\alpha }}}(t), Y(t), {\hat{\alpha }}(t)) \ge H(t, X^{{\hat{\alpha }}}(t), Y(t), a) \) \(P\otimes dt\)-a.s. for every \(a \in \mathbb {A}\) where (Y, Z) are adapted processes solving the so-called adjoint equation
Under additional convexity conditions, this necessary condition is sufficient. The interest of the maximum principle is that it reduces the solvability of the control problem (2) to that of a (scalar) variational problem, and therefore allows to derive (sometimes explicit) characterizations of optimal controls. We refer for instance to [10, 52] for proofs and historical remarks. The maximum principle has far-reaching consequences and is widely used in the stochastic control and stochastic differential game literature [11, 12, 24, 30, 34, 45]. Its use also fueled by recent progress on the theory of forward backward SDEs. We refer the reader for instance to, [16, 32, 33, 35, 46, 53] and the references therein.
The maximum principle roughly presented above naturally requires differentiability of the coefficients of the control problem, which precludes the applicability of this method to control problems with non-smooth coefficients. The effort to extend the stochastic maximum principle to problems with non-smooth coefficients started with the work of Merzedi [41] who derived a necessary condition of optimality for a problem with a Lipschitz continuous drift, but not necessarily differentiable everywhere in the state and the control variable. His result was further extended, notably to degenerate diffusion cases and singular control problems in [1,2,3,4, 9, 31, 43]. See also [50] for the infinity horizon case. To the best of our knowledge, all existing results on the stochastic maximum principle assume some level of regularity, usually Lipschitz-continuous drifts.
The present work considers the case where b is Borel measurable in x and bounded, and we will derive both necessary and the sufficient conditions of optimality. At this point, an immediate natural question is: What form should the adjoint equation take in this case? The starting point of our argument is the following simple observation: When b is differentiable, the adjoint equation is explicitly solvable, with the solution given by
where the process
is the first variation process (in the Sobolev sense) of the dynamical system \(X^{{\hat{\alpha }},x}\) solving (1) with initial condition \(X^{{\hat{\alpha }},x}_0 = x\). This suggests the form of the adjoint process when b is not differentiable, since it is well-known that despite the roughness of the drift b, the dynamical system \(X^{{\hat{\alpha }},x}\) is still differentiable with respect to x (at least in the Sobolev sense), due to Brownian regularization [42] and therefore admits a Sobolev differentiable flow. The crux of our argument will be to make use of this Sobolev differential stochastic flow to define the adjoint process (rather than the adjoint equation) in the non-smooth case to prove necessary and sufficient conditions of optimality.
Throughout this work the functions f and g are assumed to be continuously differentiable with first derivatives of linear growth. In particular, we will assume
and
The main results of this work are the following necessary and sufficient conditions in the Pontryagin stochastic maximum principle.
Theorem 1.1
Assume that b satisfies \(b(t,x,a):= b_1(t,x) + b_2(t,x,a)\) where \(b_1\) is a bounded, Borel measurable function and \(b_2\) is bounded measurable, and continuously differentiable in its second and third variables with bounded derivatives. Let \({\hat{\alpha }} \in \mathcal {A}\) be an optimal control and let \(X^{{\hat{\alpha }}}\) be the associated optimal trajectory. Then the flow \(\Phi ^{{\hat{\alpha }}}\) of \(X^{{\hat{\alpha }}}\) is well-defined and it holds
where \(Y^{{\hat{\alpha }}}\) is the adjoint process given by
Theorem 1.2
Let the conditions of Theorem 1.1 be satisfied, further assume that g and \((x,a)\mapsto H(t,x,y,a)\) are concave. Let \({\hat{\alpha \in }} \mathbb {A}\) satisfy
with Y given by (5). Then, \({\hat{\alpha }}\) is an optimal control.
Theorems 1.1 and 1.2 constitute sharp improvements over existing results as far as regularity of the drift is concerned, since it assumes measurable drifts as opposed to Lipschitz-continuous in the literature. We will elaborate on the conditions imposed in the above theorems in Sect. 4. Let us at this point remark that the result remains true when assuming \(b_2\) Lipschitz-continuous (see Remark (2.2). We do not do so to ease the presentation and focus on the non-smoothness issue. However, the techniques of the proof presented here do not seem to extend to the random volatility case because the various applications of Girsanov’s theorem might fail in this case. It is conceivable that a technique based on Zvonkin’s transform allows to derive a maximum principle in the non-constant volatility case. However using this method, the roughness of the drift coefficient does not enable to show the differentiability with respect to the initial condition which is key in the proof of the maximum principle. In addition, observe that when b is smooth, our results correspond exactly to the classical version of the stochastic maximum principle. The only difference here being the fact that the process \(\Phi ^{{\hat{\alpha }}}\) seems abstract, as it is obtained from an existence result (of the first variation process). It turns out that when the drift is not smooth, the flow \(\Phi ^{{\hat{\alpha }}}\) still admits an explicit representation much similar to (3), but in terms of the local time of the controlled process. This representation will be investigated in the present case of controlled diffusions with rough drifts in the “Appendix” (see Theorem A.1) and will be used in the proof of the maximum principle. Further note that the explicit representation of the flow is a result of independent interest. We will not discuss it further to avoid losing focus from the paper’s subject.
1.1 Motivation: Optimal Consumption Under Wealth Tax Payment
This problem is motivated by the optimal consumption problem of a financial agent paying wealth tax. In fact, since the seminal work of Merton [39], this problem has attracted much attention and enormous progress has been made. Optimal investment and consumption in complete and incomplete markets is well understood and several methods have been developed. Let us refer for instance to [21, 25,26,27] for just a few milestones. The impetus for the present work is to extend this literature (especially the optimal consumption problem of Cuoco and Cvitanić [13, 14]) to the very practical situation of optimal consumption of an individual (e.g. a retiree) living off of their investment in the stock market while paying wealth taxes.
We consider the classical financial model of Cuoco [13] with m stocks \(S = (S^1,\dots ,S^m)\) with cumulative dividend processes \(D = (D^1,\dots , D^m)\) such that \(S+D\) is the Itô diffusion
In addition there is a bond with rate \(r=0\). The agent is endowed with an initial wealth \(x_0>0\) and a nonnegative stochastic income process y such that
for some \(K_y>0\). Further assume that the agent also consumes at rate c(t). If \(\theta \) represents the dollar amount invested at time t, the wealth of the agent evolves as
As in [13, 14], we assume that the agent fixes an investment strategy \(\theta \) (which we assume for simplicity to be constant) and looks for the optimal consumption plan \({\hat{c}}\). This problem is fully solved in [13] when general constraints are put on the admissibility of c.
In the present work, we will further assume that the agent pays wealth taxes. In most countries and states, tax categories are set with respect to the tax payer’s wealth. To simplify the exposition, we will assume that only two tax categories are given; low and high: The agent pays \(\ell \) if their wealth is below a given threshold e and h otherwise. Hence, the wealth process now takes the form
That is, the agent’s wealth is the sum of their initial endowment and their trading gains minus cumulative withdrawals and cumulative tax paid. The problem faced by the agent is thus
with \(b_1(t,x) = \ell 1_{\{x\le e\}}+ h1_{\{x> e\}}\), \(b_2(t,x,c):= y_t + \theta \mu _t - c \), \({\tilde{\sigma }}: \theta \sigma \) and a utility function \(U:[0,T]\times \mathbb {R}\) that is assumed to be increasing, strictly concave, continuously differentiable in the second argument, and continuous in the first.
The problem here is the fact that, due to tax payment, the drift of the state process X is discontinuous. This stochastic control problem falls within the scope of our maximum principle discussed above, which then allows to provide a fully probabilistic characterization of the solution of this problem via the associated adjoint process.
To the best of our knowledge, optimal consumption problems under with wealth tax payment have not been considered so far. Note however the works of [6, 7] on optimal control under capital gain taxes using dynamic programming.
The remainder of the article is dedicated to the proofs of Theorems 1.1 and 1.2. The necessary condition is proved in the next section and the sufficient condition is proved in Sect. 3. In this section, we also present and example where, in addition to providing a characterization, our maximum principle allows to derive explicit solution to a control problem with non-smooth coefficients. In Sect. 4 we discuss the conditions imposed in the main theorems. The paper ends with an “Appendix” on explicit representations of the flow of SDEs with measurable and random drifts.
2 The Necessary Condition for Optimality
The goal of this section is to prove Theorem 1.1. Let us first precise the definition of the set of admissible controls. Let \(\mathbb {A}\subseteq \mathbb {R}^m\) be a closed convex subset of \(\mathbb {R}^m\). The set of admissible controls is defined as:
for some \(M>0\). This set is clearly non-empty even when \(b_1\) is not trivial. In fact, \(\mathcal {A}\) already includes a large class of controls usually considered in the literature. Let us illustrate this with two examples:
Example 2.1
-
Markovian controls If one considers controls of the form \(\alpha _t =\varphi (t,X_t)\) for a measurable function \(\varphi \), then the SDE (1) admits a unique strong solution, see e.g. [42] or [22].
-
Open loop controls Consider the set \(\mathcal {A}'\) defined as: The set of progressively measurable processes \(\alpha :[0,T]\times \Omega \rightarrow \mathbb {A}\) which are Malliavin differentiable (with Malliavin derivative \(D_s\alpha (t)\)), with
$$\begin{aligned} \mathbb {E}\Big [\int _0^T|\alpha (t)|^2\,\textrm{d}t \Big ] + \sup _{s\in [0,T]}\mathbb {E}\Big [\Big (\int _0^T|D_s\alpha (t)|^2\,\textrm{d}t\Big )^4 \Big ] <\infty \end{aligned}$$and such that there are constants \(C,\eta >0\) (possibly depending on \(\alpha \)) such that
$$\begin{aligned} \mathbb {E}[|D_s \alpha (t) - D_{s'}\alpha (t)|^4] \le C|s-s'|^\eta . \end{aligned}$$It follows from [38, Theorem 1.2] that if the drift satisfies the conditions of Theorem 1.1, then the SDE (1) is uniquely solvable for every \(\alpha \in \mathcal {A}'\).
For later reference, note that for every \(\alpha \in \mathcal {A}\) it holds \(E[\sup _{t\in [0,T]}|X^\alpha (t)|^p]<\infty \) for every \(p\ge 1\).
In the rest of the article, we let \(b_n\) be a sequence of functions defined by
such that \(b_{1,n}: [0, T ] \times \mathbb {R} \rightarrow \mathbb {R}, n \ge 1\) are smooth functions with compact support and converging a.e. to \(b_1\). Since \(b_1\) is bounded, the sequence \(b_{1,n}\) can also be taken bounded. We denote by \(X^{\alpha }_n\) the solution of the SDE (1) with drift b replaced by \(b_n\). This process is clearly well-defined since \(b_n\) is a Lipschitz continuous function. Similarly, we denote respectively by \(J_n\) and \(V_n\) the performance and the value function of the problem when the drift b is replaced by \(b_n\). That is, we put
and
Furthermore, we denote by \(\delta \) the distance
The general idea of the proof will be to start by showing that an optimal control for the problem (2) is also optimal for an appropriate perturbation of the approximating problem with value \(V_n(x_0)\). This is due to the celebrated variational principle of Ekeland. This maximum principle for control problems with smooth drifts will involve the state process \(X_n^{{\hat{\alpha _n}}}\) and its flow \(\Phi ^{{\hat{\alpha _n}}}_n\). The last and most demanding step is to pass to the limit and show some form of “stability” of the maximum principle.
Remark 2.2
When \(b_2\) is not continuously differentiably but Lipschitz-continuous, one also approximates it by smooth functions \(b_{2,n}\) which have uniformly bounded derivatives, i.e. such that \(\sup _n|\partial _xb_{2,n}|<\infty \). The rest of the proof is then the same.
We first address this limit step by a few intermediary technical lemmas that will be brought together to prove Theorem 1.2 at the end of this section.
Lemma 2.3
We have the following bounds:
-
(i)
For every sequence \((\alpha _n)_n\) in \(\mathcal {A}\), it holds \(\sup _n\mathbb {E}\big [\sup _{t \in [0,T]}|X^{\alpha _n}_n(t)|^2 \big ]<\infty \).
-
(ii)
For every \(\alpha _1,\alpha _2 \in \mathcal {A}\) it holds that
$$\begin{aligned} \mathbb {E}\big [| X^{\alpha _1}_n(t) - X^{\alpha _2}(t)|^2 \big ]\le & {} C\Big ( \delta (\alpha _1,\alpha _2)^4 + \Big (\int _0^T\frac{1}{\sqrt{2\pi s}}e^{\frac{|x_0|^2}{2s}}\int _{\mathbb {R}^d}\\{} & {} \big |b_{1,n} (s,\sigma y)-b_{1} (s,\sigma y)\big |^4e^{-\frac{|y|^2}{4s}}\textrm{d}y\,\textrm{d}s\Big )^{1/2}\Big ). \end{aligned}$$ -
(iii)
Given \(k \in \mathbb {N}\), for every sequence \((\alpha _n)_{n\ge 1}\) in \(\mathcal {A}\) and \(\alpha \in \mathcal {A}\) such that \(\delta (\alpha _n,\alpha )\rightarrow 0\), it holds that
$$\begin{aligned} \mathbb {E}\big [| X^{\alpha _n}_k(t) - X^{\alpha }_k(t)|^2 \big ] \rightarrow 0. \end{aligned}$$
Proof
The proof of (i) is standard, it follows from the linear growth property of \(b_n\) uniformly in n, i.e. \(|b_n(t,x,a)| \le C(1 + |x| + |a|)\) for some \(C>0\) and all \(n\ge 1\).
Let us turn to the proof of (ii). Adding and subtracting the same term and using the fundamental theorem of calculus, we arrive at
where \(\Lambda _n(\lambda ,t)\) is the process given by \(\Lambda _n(\lambda ,t):= \lambda X^{\alpha _1}_n(t) + (1 - \lambda )X^{\alpha _2}(t)\). Therefore, we obtain that \(X^{\alpha _1}_n - X^{\alpha _2}\) admits the representation
Hence, taking the expectation on both sides above and then using twice the Cauchy–Schwarz inequality, we have that
By the Lipschitz continuity of \(b_2\), the last term on the right hand side is estimated as
Moreover, denoting
the second integral on the right side of (8) can be further estimated as follows:
for some constant \(C>0\) and the probability measure \(\mathbb {Q}\) is the measure with density
Note that we used the Cauchy–Schwarz inequality and then the fact that b is bounded to get \(\mathbb {E}[(\frac{\,\textrm{d}\mathbb {Q}}{\,\textrm{d}\mathbb {P}})^{-1}]\le C\). By the Girsanov’s theorem, under the measure \(\mathbb {Q}\), the process \((X^{\alpha _2}(t) - x_0)\sigma ^\top /|\sigma |^2 \) is a Brownian motion. Thus, it follows that
and using the density of Brownian motion, we have for every \(p\ge 1\)
By the Fubini’s theorem, this shows that
Let us now turn our attention to the first term in (8). Since \(\Lambda _{n}(\lambda ,t)\) takes the form
where \(b^{\lambda ,\alpha _2}(s)\) is the short hand notation for \(\lambda b_{n}(s, X^{\alpha _1}_n(s),\alpha _1(s)) + (1-\lambda )b(s, X^{\alpha _2}(s),{\alpha _2}(s))\). We use the Jensen’s inequality, the Girsanov’s theorem as above and Lipschitz continuity of \(b_2\) to get
with \(\,\textrm{d}\mathbb {Q}^\lambda = \mathcal {E}\big (\frac{\sigma ^\top }{|\sigma |^2}\int _0^Tb^{\lambda ,\alpha _2}(s)\textrm{d}B(s) \big )\,\textrm{d}\mathbb {P}\), and where we used the fact that \(b^{\lambda ,\alpha _2}\) is bounded. Since the sequence \((b_{1,n})_n\) is uniformly bounded, it follows from Lemma A.3 that
Therefore, putting together (8), (9), (11) and (13) concludes the proof.
Since \(b_k\) is Lipschitz continuous the convergence (ii) follows by classical arguments, the proof is omitted. \(\square \)
Lemma 2.4
Let \(\alpha \in \mathcal {A}\) and let \(\alpha _n\) be a sequence of admissible controls such that \(\delta (\alpha _n,\alpha )\rightarrow 0\). Then, it holds
-
(i)
\(| J_k(\alpha _n) - J_k(\alpha ) | \rightarrow 0\) as \(n\rightarrow \infty \) for every \(k \in \mathbb {N}\) fixed. In particular, the function \(J_k:(\mathcal {A},\delta ) \rightarrow \mathbb {R}\) is continuous.
-
(ii)
\(|J_n(\alpha ) - J(\alpha )| \le C\varepsilon _n\) for some \(C>0\) with \(\varepsilon _n\downarrow 0\).
Proof
-
(i)
The continuity of \(J_k\) easily follows by continuity of f and g. In fact, we have
$$\begin{aligned} |J_k(\alpha _n) - J_k(\alpha )|&\le \mathbb {E}\Big [|g(X^{\alpha _n}_k(T)) - g(X^{\alpha }_k(T))| + \int _0^T|f(t, X^{\alpha _n}_k(t), \alpha _n(t)) \\&\quad - f(t, X^{\alpha }_k(t), \alpha (t)) |\,\textrm{d}t \Big ]\rightarrow 0, \end{aligned}$$where the convergence follows by the dominated convergence and Lemma 2.3.
-
(ii)
is also a direct consequence of Lemma 2.3. In fact, by the linear growth of \(\partial _xf\) and \(\partial _xg\) we have
$$\begin{aligned}&| J_n(\alpha ) - J(\alpha ) |\le \mathbb {E}\Big [|g(X^{\alpha }_n(T)) - g(X^{\alpha }(T))| + \int _0^T|f(t, X^{\alpha }_n(t), \alpha (t)) \\&\qquad - f(t, X^{\alpha }(t), \alpha (t)) |\,\textrm{d}t \Big ]\\&\quad \le \mathbb {E}\Big [\int _0^1|\partial _xg(\lambda X^{\alpha }_n(T)) + (1-\lambda )X^{\alpha }(T))|\,\textrm{d}\lambda |X^{\alpha }_n(T) - X^{\alpha }(T)|\Big ]\\&\qquad + \mathbb {E}\Big [\int _0^T\int _0^1|\partial _{x}f\big (t, \lambda X^{\alpha }_n(t)+(1-\lambda )X^{\alpha }(t),\alpha (t) \big )|\,\textrm{d}\lambda |X^{\alpha }_n(t) - X^{\alpha }(t)|\,\textrm{d}t \Big ]\\&\quad \le C\mathbb {E}\Big [1+\sup _{t\in [0,T]}\big ( |X^{\alpha }_n(t)|^2 + |X^\alpha (t)|^2\big ) \Big ]^{1/2}\Big (\sup _{t \in [0,T]}\mathbb {E}\Big [|X^\alpha _n(t) - X^\alpha (t)|^2\Big ]\Big )^{1/2}, \end{aligned}$$
where we used Cauchy–Schwarz inequality and Fubini’s theorem. Therefore, by Lemma 2.3(i) we have
where the second inequality follows from Lemma 2.3. \(\square \)
The next lemma pertains to the stability of the adjoint process with respect to the drift and the control process. This result is based on similar stability properties for stochastic flows. Given \(x \in \mathbb {R}\) and the solution \(X^{\alpha ,x}\) of the SDE (1) with initial condition \(X^{\alpha ,x}_t = x\), the first variation process of \(X^{\alpha ,x}\) is the derivative \(\Phi ^\alpha (t,s)\) of the function \(x\mapsto X^{\alpha ,x}(s)\). Existence and properties of this Sobolev differentiable flow have been extensively studied by Kunita [29] for equations with sufficiently smooth coefficients. In particular, when the drift b is Lipschitz and continuously differentiable, the function \(\Phi ^\alpha (t,s)\) exists and, for almost every \(\omega \), is the (classical) derivative of \(x\mapsto X^{\alpha ,x}(s)\). The case of measurable (deterministic) drifts is studied by Mohammed et. al. [42] and extended to measurable and random drifts in [38]. These works show that, when b is measurable, then \(X^{\alpha ,\cdot }(s)\in L^2(\Omega , W^{1,p}(U))\) for every \(s \in [t,T]\) and \(p>1\), where \(W^{1,p}(U)\) is the usual Sobolev space and U an open and bounded subset of \(\mathbb {R}\). That is, \(\Phi ^\alpha (t,s)\) exists and is the weak derivative of \(X^{\alpha ,\cdot }\).
The proof of the stability result will make use of an explicit representation of the process \(\Phi ^\alpha \) with respect to the local time-space integral. Recall that for \(a\in \mathbb {R}\) and \(X=\{X(t),t\ge 0\}\) a continuous semimartingale, the local time \(L^{X}(t,a)\) of X at a is defined by the Tanaka–Meyer formula as
where \({{\,\textrm{sgn}\,}}(x)=-1_{(-\infty ,0]}(x)+1_{(0,+\infty )}(x)\). The local time-space integral plays a crucial role in the representations of the Sobolev derivative of the flows of the solution to the SDE (1). It is defined for functions in the space \((\mathcal {H}_x, \Vert \cdot \Vert ^x)\) defined (see e.g. [17]) as the space of Borel measurable functions \(f:[0,T]\times \mathbb {R}\rightarrow \mathbb {R} \) with the norm
Since \(b_1\) is bounded, we obviously have \(b_1 \in \mathcal {H}^x\) for every x. Moreover, it follows from [19] (see also [5]) that for every continuous semimartingale X the local time-space integral of \(f\in \mathcal {H}^x\) with respect to \(L^{X}(t,z)\) is well defined and satisfies
for every continuous function (in space) \(f \in \mathcal {H}^x\) admitting a continuous derivative \(\partial _xf(s,\cdot )\) (see [19, Lemma 2.3]). This representation allows to derive the following:
Lemma 2.5
For every \(\alpha \in \mathcal {A}\) and \(c\ge 0\), it holds
Proof
First observe that for every \(n \in \mathbb {N}\), it follows by the Cauchy–Schwarz inequality that
where \(\mathbb {Q}\) is the probability measure given as in (10) with \(\alpha _2\) therein replaced by \(\alpha \). Hence, since \((X^{\alpha ,x}-x_0)\sigma ^\top /|\sigma |^2\) is a Brownian motion under \(\mathbb {Q}\), it follows by (14) that
for some constant \({\overline{C}}>0\) which does not depend on n, where this latter inequality follows by Lemma A.3. Since \(b_1\) is bounded and \(b_{1,n}\) converges to \(b_1\) pointwise, it follows by [19, Theorem 2.2] that \(\int _{\mathbb {R}}b_{1,n}\left( u,z\right) L^{X^{\alpha ,x}}(\textrm{d}u,\textrm{d}z) \rightarrow \int _{\mathbb {R}}b_1\left( u,z\right) L^{X^{\alpha ,x}}(\textrm{d}u,\textrm{d}z) \) as n goes to infinity. Thus, it follows by the continuity of the exponential function and dominated convergence that
\(\square \)
We are now ready to prove stability of the flow and of the adjoint processes.
Lemma 2.6
Let \(\alpha \in \mathcal {A}\) and \(\alpha _n\) be a sequence of admissible controls such that \(\delta (\alpha _n,\alpha )\rightarrow 0\). Then, the processes \(X^{\alpha _n}_n\) and \(X^{\alpha }\) admit Sobolev differentiable flows denoted \(\Phi ^{\alpha _n}_n\) and \(\Phi ^{\alpha }\), respectively and for every \(0\le t\le s\le T\) it holds
-
(i)
\(\mathbb {E}\big [|\Phi ^{\alpha _n}_n(t,s) - \Phi ^\alpha (t,s) |^2 \big ] \rightarrow 0\) as \(n\rightarrow \infty \),
-
(ii)
\(\mathbb {E}\big [| Y^{\alpha _n}_n(t) - Y^\alpha (t)| \big ] \rightarrow 0\) as \(n\rightarrow \infty \),
where \(Y^\alpha \) is the adjoint process defined as
and \(Y^{\alpha _n}_n\) is defined similarly, with \((X^{\alpha },\alpha , \Phi ^\alpha )\) replaced by \((X^{\alpha _n}_n,\alpha _n, \Phi ^{\alpha _n}_n)\).
Proof
The existence of the process \(\Phi ^{\alpha _n}_n\) is standard, it follows for instance by [28]. The existence of the flow \(\Phi ^{\alpha }\) follows by [38, Theorem 1.3]. We start by proving the first convergence claim. As explained above, these processes admit explicit representations in terms of the space-time local time process. It fact, it follows from Theorem A.1 that \(\Phi ^\alpha \) admits the representation
and \(\Phi _n^{\alpha _n}\) admits the same representation with \((b_1, X^{\alpha ,x},\alpha )\) replaced by \((b_{1,n}, X^{\alpha _n,x}, \alpha _n)\). Using these explicit representations and the Hölder inequality, we have
Splitting up the terms in power 4, then applying the Hölder and Young inequalities we continue the estimations as
It follows from Lemma 2.5 that \(I_1\) and \(I_{5,n}\) are bounded. Since \(\partial _xb_2\) is bounded, it follows that \(I_{2,n}\) and \(I_{4,n}\) are also bounded with bounds independent on n. Let us now show that \(I_{3,n}\) and \(I_{6,n} \) converge to zero. We show only the convergence of \(I_{6,n}\) since that of \(I_{3,n}\) will follow (at least for a subsequence) from Lemma 2.3 and dominated convergence since \(\partial _xb_{2}\) is continuous and bounded.
To that end, further define the processes \(A_n^{\alpha _n}\) and \(A^{\alpha }\) by
In order to show that \(A_n^{\alpha _n}\) converges to \(A^{\alpha }\) in \(L^2\), we will show that \(A_n^{\alpha _n}\) converges weakly to \(A^{\alpha }\) in \(L^2\) and that \(E[|A_n^{\alpha _n}|^2]\) converges to \(E[|A^{\alpha }|^2]\) in \(\mathbb {R}\). We first prove the weak convergence. Since the set
spans a dense subspace in \(L^2(\Omega )\), in order to the show weak convergence, it is enough to show that
Denote by \({\tilde{X}}_n^{\alpha _n, x}\) and \({\tilde{X}}^{\alpha ,x}\) the processes given by
and
Observe that these processes are well-defined, since we have \(\tilde{X}^{{\tilde{\alpha }},x}(t,\omega ) = X^{\alpha ,x}(t,\omega + \varphi )\) and \({\tilde{X}}_n^{{\tilde{\alpha }}_n,x}(t,\omega ) = X_n^{\alpha _n,x}(t,\omega + \varphi )\). Using the Cameron–Martin–Girsanov theorem as in the proof of Lemma 2.5, we have
where \({\tilde{u}}(s, x,\alpha (\omega )): = u(s, x,\alpha (\omega +\varphi ))\). Let us set
Next, add and subtract the term \(\mathcal {E}\Big (\int _0^T\Big \{{\tilde{u}}_n(s,x+\sigma \cdot B(s),\alpha _n (s))+\sigma \cdot {\dot{\varphi }}(s)\Big \}\textrm{d}B(s)\Big )e^{\int _s^{t}\int _{\mathbb {R}}b_{1}\left( u,z\right) L^{|\sigma \Vert B^x_\sigma }(\textrm{d}u,\textrm{d}z)}\) and then use the inequality \(|e^x-e^y|\le |x-y||e^x+e^y|\) to obtain
Therefore, an application of the Hölder’s inequality yields the estimate
Using Lemma A.2, it follows that \(J_{2,n}\) converges to zero, and by the dominated convergence, the definition of u, the inequality \(|e^x-e^y|\le |x-y||e^x+e^y|\) and once more Lemma A.2, \(J_{5,n}\) also converges to zero. Thanks to Lemma A.3 and boundedness of \(b_{1,n}\) (respectively \(b_1\)), the term \(J_{3,n}\) (respectively \(J_{4,n}\)) is bounded. The bound of \(J_{1,n}\) follows by the uniform boundedness of \(u_n\).
It remains to show that \(\mathbb {E}[|A_n^{\alpha _n}(t)|^2]\) converges to \(\mathbb {E}[|A^{\alpha }(t)|^2]\) in \(\mathbb {R}\). Using the Girsanov transform as in the proof of Lemma 2.5, we have
and
Therefore using once more \(|e^x-e^y|\le |x-y||e^x+e^y|\) and the Cauchy–Schwarz inequality
Now, introducing the random variables
and
we continue the above estimations as
By Lemma A.2, \(F_{1,n}\) converges to zero in \(L^2(\Omega )\). Using similar arguments as in [5, Lemma A.3], one can show that \(V_n\) converges to zero in \(L^2(\Omega )\) by the boundedness of \(u_n\) and the definition of the distance \(\delta \). Observe however that in this case, \(u_n\) depends on \(\alpha _n\) and not on \(\alpha \) as in [5, Lemma A.3]. Nevertheless using the fact that \(b_{1,n}\), \(b_1\) and \(b_2\) are bounded and Lipschitz in the second variable, one can show by the dominated convergence theorem and similar reasoning as in (9) that the overall term converges to zero. It is also worth mentioning that the other terms are uniformly bounded by application of either the Girsanov theorem and/or Lemma A.3 to the uniformly bounded sequences \((u_n)_{n\ge 1},(b_{1,n})_{n\ge 1}\) and the bounded functions \(u, b_{1}\).
Let us now turn our attention to the proof of (ii). Compute the difference \(Y_{n}^{\alpha _n}(t)-Y^\alpha (t)\), add and subtract the terms \(\Phi ^{\alpha }(t,T) \partial _xg(X_n^{\alpha _n}(T))\) and \(\int _t^T\Phi ^{\alpha }(t,u) \partial _xf(u,X_n^{\alpha _n}(u), \alpha _n(u))\,\textrm{d}u\) and then apply Hölder’s inequality to obtain
for some constant \(C_T\) depending only on T. Since the process \(\Phi ^{\alpha }\) is square integrable, (see [38, Theorem 1.3]) it follows by boundedness and continuity of \(\partial _xg,\partial _xf\) as well as Lemma 2.3 that the first and third terms converge to zero as n goes to infinity. Moreover, by
linear growth of \(\partial _xf\) and \(\partial _xg\), Lemma 2.3(i) and the \(L^2\) convergence of \(\Phi _n^{\alpha _n}(t,u)\) to \(\Phi ^{\alpha }(t,u)\) given in part (i), we conclude that the second and last terms in (22) converge to zero, which shows (ii). \(\square \)
Proof
(of Theorem 1.1) Let \({\hat{\alpha }}\) be an optimal control and \(n\ge 1\) fixed. Observe that by the linear growth assumption on f, g the function \(J_n\) is bounded from above. By Lemma 2.4 the function \(J_n\) is also continuous on \((\mathcal {A},\delta )\) and there exists \(\varepsilon _n\) such that
That is, \(J_n({\hat{\alpha }}) \le \inf _{\alpha \in \mathcal {A}}J_n(\alpha ) + 2\varepsilon _n\). Thus, by Ekeland’s variational principle, see e.g. [20], there is a control \({\hat{\alpha _n}} \in \mathcal {A}\) such that \(\delta ({\hat{\alpha }}, {\hat{\alpha _n}})\le (2\varepsilon _n)^{1/2}\) and
In other words, putting \(J^\varepsilon _n(\alpha ):= J_n(\alpha ) + (2\varepsilon _n)^{1/2}\delta ({\hat{\alpha _n}},\alpha )\), the control process \({\hat{\alpha _n}}\) is optimal for the problem with cost function \(J^\varepsilon _n\).
Now, let \(\beta \in \mathcal {A}\) be an arbitrary control and \(\varepsilon >0\) a fixed constant. By convexity of \(\mathbb {A}\), it follows that \({\hat{\alpha _n}} + \varepsilon \eta \in \mathcal {A}\), with \(\eta := \beta - {\hat{\alpha _n}}\). Thus, since \(b_n\) is sufficiently smooth, it is standard that the functional \(J_n\) is Gâteau differentiable (see [10, Lemma 4.8]) and its Gâteau derivative in the direction \(\eta \) is given by
where \(V_n\) is the stochastic process solving the linear equation
On the other hand using triangular inequality, we have
Therefore, \(J^\varepsilon _n\) is also Gâteau differentiable and since \({\hat{\alpha _n}}\) is optimal for \(J^\varepsilon _n\), we have
for a constant \(C_M>0\) depending on the constant M (introduced in the definition of \(\mathcal {A}\)). The inequality follows since \({\hat{\alpha _n\in }} \mathcal {A}\), and \(H_n\) is the Hamiltonian of the problem with drift \(b_n\) given by
and \((Y^{{\hat{\alpha _n}}}_n, Z^{{\hat{\alpha _n}}}_n)\) the adjoint processes satisfying
By standard arguments, we can thus conclude that
Recalling that \(b_{1,n}\) does not depend on \(\alpha \), this amounts to (recall definition of \(b_n\) in (7))
We will now take the limit on both sides above as n goes to infinity. It follows by Lemma 2.3 and Lemma 2.6 respectively that \(X_n^{{\hat{\alpha _n}}}(t) \rightarrow X^{{\hat{\alpha }}}(t)\) and \(Y^{{\hat{\alpha _n}}}_n(t) \rightarrow Y^{{\hat{\alpha }}}(t)\) \(\mathbb {P}\)-a.s. for every \(t\in [0,T]\). Since \({\hat{\alpha _n\rightarrow }} \alpha \), we therefore conclude that
This shows (4), which concludes the proof. \(\square \)
3 The Sufficient Condition for Optimality
3.1 Proof of Theorem 1.2
Let us now turn to the proof of the sufficient condition of optimality. Since we will need to preserve the concavity of H assumed in Theorem 1.2 after approximation, we specifically assume that the function \(b_n\) is defined by standard mollification. Therefore, \(H_n(t,x,y,a):= f(t,x,a)+ b_n(t,x,a)y\) is a mollification of H and thus remains concave.
Proof
(of Theorem 1.2) Let \({\hat{a}} \in \mathcal {A}\) satisfy (6) and \(\alpha '\) an arbitrary element of \(\mathcal {A}\). We would like to show that \(J({\hat{a}}) \ge J(\alpha ')\). Let \(n \in \mathbb {N}\) be arbitrarily chosen. By definition, we have
where we used the definition of \(H_n\) and the fact that g is concave. Since \(Y_n^{{\hat{a}}}\) satisfies
it follows by the martingale representation and the Itô’s formula that there is a square integrable progressive process \((Y^{{\hat{a}}}_n,Z^{{\hat{a}}}_n)\) such that \(Y_n^{{\hat{a}}}\) satisfies the (linear) equation
Recall that since \(b_n\) is smooth, so is \(H_n\). Therefore, by the Itô’s formula once again we have
Since the stochastic integral above is a local martingale, a standard localization argument allows to take expectation on both sides to get that
where the latter inequality follows by concavity of \(H_n\).
Coming back to the expression of interest \(J({\hat{a}}) - J(\alpha ')\), we have
Since \(b_{1,n}\) does not depend on \(\alpha \), we have that \(\partial _\alpha H_n(u, X_n^{{\hat{a}}}(u), Y_n^{{\hat{a}}}(u), {\hat{a}}(u)) = \partial _\alpha b_{2}(u, X^{{\hat{a}}}_n(u),{\hat{a}}(u))Y^{{\hat{a}}}_n(u) + \partial _\alpha f(u, X^{{\hat{a}}}_n(u),{\hat{a}}(u))\). Therefore, taking the limit as n goes to infinity, it follows by Lemmas 2.3, 2.4 and 2.6 that it holds
Since \({\hat{a}}\) satisfies (6), we therefore conclude that \(J({\hat{a}}) \ge J(\alpha ')\). \(\square \)
3.2 Example: Stochastic Predicted Miss Problem
Let us consider the following Stochastic predicted miss problem which first appeared in the works of Davis [15] and Gelder [51] as a conjecture of optimal laws for problems with finite fuel constraints. It was first rigorously solved by Beneš [8]. More precisely, let us consider the following optimal control problem, with the cost function given by
where the state process is the controlled SDE
and the control variable \(\alpha (t)\) takes values in \(\mathbb {A}=[-1,1]\). We are interested in the control problem
As pointed earlier, this problem was first rigorously solved in [8] and then in [23] in the linear case. Further it was also considered in [4] in the case of Lipschitz coefficients. Our goal here is to provide an explicit (feedback) solution to this control problem when the function \(b_1\) is only measurable. Hence, we considered the following conditions:
-
(A)
The function \(g:\mathbb {R}\rightarrow \mathbb {R}\) is even, continuously differentiable and increasing on \(x > 0\) and concave. Moreover, it holds \(|g(x)|\le K(1+|x|^p)\) for all \(x \in \mathbb {R}\) and some \(p\ge 1\).
-
(B)
The function \(b_1:\mathbb {R}\rightarrow \mathbb {R}\) is odd and bounded; and the function \(b_2:[0,T]\times \mathbb {R}\rightarrow \mathbb {R}\) is odd, bounded, differentiable and Lipschitz-continuous in its second variable, uniform in the first one.
Proposition 3.1
If the conditions (A) and (B) are satisfied, then the control problem (24) is given in feedback form via
Proof
Since the reward function g is even, maximizing it is similar to maximizing |X(T)|. Thus, we should take \(\alpha \) such that \(xb_2\alpha >0\). This can be seen applying Itô–Tanaka formula to |X(t)|. Thus, we make the Ansatz
with the notation \({\hat{X}}:= X^{{\hat{\alpha }}}\). We will use our maximum principle to show that this yields an optimal control.
In fact, by Theorem 1.2, it suffices to show that \({\hat{\alpha }}(t)\) maximizes the Hamiltonian \(H(t,{\hat{X}},Y,\alpha ):= \big (b_1({\hat{X}}(t)) + b_2(t, {\hat{X}}(t))\alpha \big )\cdot Y_t\), where Y denotes the adjoint process. Since the maximizer of this function is given by
it thus remains to show that for \({\hat{\alpha }}\) given by (25) it holds \({{\,\textrm{sgn}\,}}(Y(t)) = {{\,\textrm{sgn}\,}}({\hat{X}}(t))\).
Recall that the adjoint process takes the form
Since \(b_1\) is time-independent, the process \(\Phi \) takes the form
where \(L^{{\hat{X}}}_T\) denotes the local time of the process X at time T.
This follows by Theorem A.1 where, due to the Bouleau-Yor formula for semimartingales (see [48, Theorem 77, page 227] or [49, Exercise 1.28 page 236]) we can replace \(\int _0^T\int _{\mathbb {R}}b_1(z)L^{{\hat{X}}}(\,\textrm{d}u, \,\textrm{d}z)\) by \(\int _{\mathbb {R}}b_1(z)L^{{\hat{X}}}_T(\,\textrm{d}z)\). Next, let us define the function \({\widetilde{b}}_1\) by
Then \({\widetilde{b}}_1\) admits a bounded derivative \(b_1\) almost everywhere. Using again the Bouleau-Yor formula for continuous semimartingales we have
Substituting the above into (27), yields
Next we show that \({{\,\textrm{sgn}\,}}(Y(t)) = {{\,\textrm{sgn}\,}}({\hat{X}}(t))\). By (25) it holds that
Now observe that the function \( x\mapsto b_1(x)+|b_2(t,x)|{{\,\textrm{sgn}\,}}(x)\) is odd and \({\tilde{B}}(t)=-B(t)\) is again a Brownian motion with the same law as B. Thus,
showing that \(-{\hat{X}}\) is a weak solution of the controlled SDE. By the weak uniqueness, it follows that \({\hat{X}}(s)\) and \(-{\hat{X}}(s)\) have the same distribution given \(s\ge \tau \) with
Now, we claim that
Indeed, by the weak uniqueness, we know that \(({\hat{X}},W)\) and \((-{\hat{X}},{\tilde{W}})\) have the same distribution.
Then using the facts that \(\partial _xg, b_2\) are odd and \({\widetilde{b}}_1\) is even, we obtain
where the latter equality follows from the fact that \(\tilde{B}(t)=-B(t)\) has the same law as B(t) as a process. Thus, we have \(2I_1=0\) and the claim follows.
Coming back to the adjoint process, we have
For \(T<\tau \), we have that \({{\,\textrm{sgn}\,}}({\hat{X}}(s))={{\,\textrm{sgn}\,}}({\hat{X}}(t))\) for all \(t\le s\le T\). If follows that the term inside the expectation is zero or has the same sign as \({\hat{X}}(t)\) (by properties of \(\partial _xg\)). Thus, it holds \({{\,\textrm{sgn}\,}}(Y(t))={{\,\textrm{sgn}\,}}({\hat{X}}(t))\), which yields the result. \(\square \)
4 Concluding Remarks
Let us conclude the paper by briefly discussing our assumptions. The condition \(b=b_1+b_2\) seems essential to derive existence and uniqueness results of the controlled system. For instance, the crucial bound (13) derived in [5, 37] is unknown when \(b_1\) depends on \(\alpha \). This condition is also vital in obtaining the explicit representation of the Sobolev derivative of the flows of the solution to the SDE in terms of its local time. This representation cannot be expected in multidimensions due to the non-commutativity of matrices and the local time. Therefore, much stronger (regularity) conditions are needed to derive the maximum principle in this case (see for example [1, 2, 4]). Note in addition that the boundedness assumption on b is made mostly to simplify the presentation. The results should also hold with b of linear growth in the spatial variable, albeit with more involved computations and with T small enough, since the flow in this case is expected to exist in small time.
Given the drift b, some known conditions on the control \(\alpha \) that guaranty existence and uniqueness of the strong solution to the SDE (1) satisfied by the controlled process are given in Example 2.1. These conditions involve the Malliavin derivative of \(\alpha \). Let us remark that the Malliavin differentiability of the control is not an uncommon assumption. This condition appears implicitly in the works [36, 40, 44] on the stochastic maximum principle where the coefficients are required to be at least two times differentiable with bounded derivatives.
Data Availability
Data sharing not applicable to this article as no datasets were generated or analysed during the current study
References
Bahlali, K., Chighoub, F., Djehiche, B., Mezerdi, B.: Optimality necessary conditions in singular stochastic control problems with nonsmooth data. J. Math. Anal. Appl. 355, 479–494 (2009)
Bahlali, K., Djehiche, B., Mezerdi, B.: On the stochastic maximum principle in optimal control of degenerate diffusions with Lipschitz coefficients. Appl. Math. Optim. 56, 364–378 (2007)
Bahlali, K., Mezerdi, B., Ouknine, Y.: The maximum principle for optimal control of diffusions with non-smooth coefficients. Stoch. Stoch. Rep. 57(3–4), 303–316 (1996)
Bahlali, S., Djehiche, B., Mezerdi, B.: The relaxed stochastic maximum principle in singular optimal control of diffusions. SIAM J. Control Optim. 46(2), 427–444 (2007)
Banos, D., Meyer-Brandis, T., Proske, F., Duedahl, S.: Computing deltas without derivatives. Finance Stoch. 21(2), 509–549 (2017)
Ben-Tahar, I., Soner, H.M., Touzi, N.: The dynamic programming equation for the problem of optimal investment under capital gains taxes. SIAM J. Financ. Math. 1, 366–395 (2010)
Ben-Tahar, I., Soner, H.M., Touzi, N.: The dynamic programming equation for the problem of optimal investment under capital gains taxes. SIAM J. Control Optim. 48(5), 1779–1801 (2007)
Beneš, V.E.: Full “bang’’ to reduce predicted miss is optimal. SIAM J. Control Optim. 14(1), 62–84 (1976)
Bossy, M., Cissé, M., Talay, D.: Stochastic representations of derivatives of solutions of one-dimensional parabolic variational inequalities with Neumann boundary conditions. Ann. Inst. H. Poincaré Probab. Stat. 47(2), 395–424 (2011)
Carmona, R.: Lectures on BSDEs, Stochastic Control and Stochastic Differential Games with Financial Applications, Financial Mathematics, vol. 1. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2016)
Carmona, R., Delarue, F.: Forward–backward stochastic differential equations and controlled McKean–Vlasov dynamics. Ann. Probab. 43(5), 2647–2700 (2015)
Carmona, R., Fouque, J.-P., Sun, L.-H.: Mean field games and systemic risk. Commun. Math. Sci. 13(4), 911–933 (2015)
Cuoco, D.: Optimal consumption and equilibrium prices with portfolio constraints and stochastic income. J. Econ. Theory 72, 33–73 (1997)
Cuoco, D., Cvitanić, J.: Optimal consumption choices for a “large’’ investor. J. Econ. Dyn. Contr. 22, 401–436 (1998)
Davis, R.C.: Stochastic final-value control systems with a fuel constraint. J. Math. Anal. Appl. 21, 62–78 (1968)
Delarue, F.: On the existence and uniqueness of solutions to FBSDEs in a non-degenerate case. Stoch. Proc. Appl. 99, 209–286 (2002)
Eisenbaum, N.: Integration with respect to local time. Potential Anal. 13, 303–328 (2000)
Eisenbaum, N.: Local time-space stochastic calculus for Lévy processes. Stoch. Proc. Appl. 116, 757–778 (2006)
Eisenbaum, N.: Local time-space stochastic calculus for reversible semimartingales. Séminaire de Probabilités XL 137–146 (2007)
Ekeland, I.: Non convex minimization problems. Bull. Am. Math. Soc. 1, 443–474 (1979)
Fleming, W.H., Zariphopoulou, T.: An optimal invesetment/concumption model with borrowing. Math. Oper. Res. 16, 802–822 (1991)
Gyöngy, I., Martinez, T.: On stochastic differential equations with locally unbounded drift. Czechoslov. Math. J. 51(4), 763–783 (2001)
Haussmann, U.G.: Some examples of optimal stochastic controls or: the stochastic maximum principle at work. SIAM Rev. 23, 292–307 (1981)
Heyne, G., Kupper, M., Tangpi, L.: Portfolio optimization under nonlinear utility. Int. J. Theor. Appl. Finance 19(5), 1650029 (2016)
Hu, Y., Imkeller, P., Müller, M.: Utility maximization in incomplete markets. Ann. Appl. Probab. 15(3), 1691–1712 (2005)
Karatzas, I., Lehczky, J., Shreve, S.: Optimal portfolio and consumption decisions for a “small investor’’ on a finite horizon. SIAM J. Control Optim. 25, 1557–1586 (1987)
Kramkov, D., Schachermayer, W.: The asysmptotic elasticity of utility functions and optimal investment in incomplete market. Ann. Appl. Probab. 9(3), 904–950 (1999)
Kunita, H.: Some extensions of Itô’s formula. Séminaire de Probabilités XV 1979(80), 118–141 (1981)
Kunita, H.: Stochastic Flows and Stochastic Differential Equations. Cambridge University Press (1990)
Laurière, M., Tangpi, L.: Convergence of large population games to mean field games with interaction through the controls. SIAM J. Math. Anal. 54(3), 3535–3574 (2022)
Lépingle, D., Nualart, D., Sanz, M.: Dérivation stochastique de diffusions réfléchie. Ann. Inst. H. Poincaré Probab. Stat. 25(3), 283–305 (1989)
Luo, P., Menoukeu-Pamen, O., Tangpi, L.: Strong solutions to forward backward stochastic differential equations with measurable coefficients. Stoch. Proc. Appl. 144, 1–22 (2022)
Luo, P., Tangpi, L.: Solvability of FBSDEs with diagonally quadradic generators. Stoch. Dyn. 17(6), 1750043 (2017)
Luo, P., Tangpi, L.: Laplace principle for large population games with control interaction. Preprint (2021)
Ma, J., Zhang, J.: On weak solutions of forward–backward SDEs. Probab. Theory Relat. Field 151, 475–507 (2011)
Menoukeu-Pamen, O.: Maximum principles of Markov regime-switching forward–backward stochastic differential equations with jumps and partial information. J. Optim. Theory and Appl. 175, 373–410 (2017)
Menoukeu-Pamen, O., Meyer-Brandis, T., Nilssen, T., Proske, F., Zhang, T.: A variational approach to the construction and Malliavin differentiability of strong solutions of SDE’s. Math. Ann. 357(2), 761–799 (2013)
Menoukeu-Pamen, O., Tangpi, L.: Strong solutions of some one-dimensional SDEs with random and unbounded drifts. SIAM J. Math. Anal. 51, 4105–4141 (2019)
Merton, R.C.: Optimum consumption and portfolio rules in a continuous-time model. J. Econ. Theory 3, 373–413 (1971)
Meyer-Brandis, T., Øksendal, B., Zhou, X.: A mean-field stochastic maximum principle via Malliavin calculus. In: Stochastic: An International Journal of Probability and Stochastic Processes. Special Issue: The Mark H.A. Davis festschrift: Stochastics, Control and Finance, vol. 84, pp. 643–666 (2012)
Mezerdi, B.: Necessary conditions for optimality for a diffusion with a non-smooth drift. Stochastics 24, 305–326 (1988)
Mohammed, S.E.A., Nilssen, T., Proske, F.: Sobolev differentiable stochastic flows for SDE’s with singular coeffcients: applications to the stochastic transport equation. Ann. Probab. 43(3), 1535–1576 (2015)
N’Zi, M., Ouknine, Y., Sulem, A.: Regularity and representation of viscosity solutions of partial differential equations via backward stochastic differential equations. Stoch. Process. Appl. 116(9), 1319–1339 (2006)
Øksendal, B., Sulem, A.: Maximum principles for optimal control of forward-backward stochastic differential equations with jumps. SIAM J. Control Optim. 48(5), 2845–2976 (2009)
Peng, S.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28, 966–979 (1990)
Peng, S., Wu, Z.: Fully coupled forward–backward stochastic differential equations and applications to optimal control. SIAM J. Control Optim. 37(3), 825–843 (1999)
Pontryagin, L.: Mathematical Theory of Optimal Processes. CRC Press (1962)
Protter, P.E.: Stochastic Integration and Differential Equations. Springer-Verlag (2004)
Revuz, D., Yor, M.: Continuous Martingales and Brownian Motion, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], 3rd edn, vol. 293. Springer-Verlag, Berlin (1999)
Socgnia, V.K., Menoukeu-Pamen, O.: An infinite horizon stochastic maximum principle for discounted control problem with Lipschitz coefficients. J. Math. Anal. Appl. 422(1), 684–711 (2015)
Van Gelder, A., Dunn, J., Mendelsohn, J.: The final value optimal stochastic control problem with bounded controller. In: Proceedings of JAAC (1966)
Yong, J., Zhou, X.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999)
Zhang, J.: Backward Stochastic Differential Equations—From Linear to Fully Nonlinear Theory. Springer, New York (2017)
Zhang, X.: Stochastic differential equations with Sobolev diffusions and singular drift and applications. Ann. Appl. Probab. 26(5), 2697–2732 (2016)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Mihai Sirbu.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Olivier Menoukeu-Pamen and Ludovic Tangpi thank the anonymous referees for helpful comments and suggestions. Menoukeu-Pamen acknowledges financial support by the Alexander von Humboldt Foundation, under the program financed by the German Federal Ministry of Education and Research entitled German Research Chair No 01DG15010. Tangpi acknowledges financial support by the NSF Grant DMS-2005832 and the NSF CAREER Award DMS-2143861.
Appendix A. Representation of the Differential Flow by Time-Space Local Time
Appendix A. Representation of the Differential Flow by Time-Space Local Time
It is well-known that conditions on the coefficients can be given under which solutions of stochastic differential equations admit a stochastic differential flow of diffeomorphisms. Such flows have been extensively investigated in the work of Kunita [29] for equations with sufficiently smooth coefficients. When the drift merely measurable, it turns out (see e.g. [37, 42, 54]) that flows still exists, at least in the Sobolev sense.
In this appendix, we show that the first variation process admits an explicit representation. The difficulty here is the lack of regularity of the drift, around which we get using local time-space integration. This representation has been obtained in [5] assuming that the drift \(b=b_1+b_2\) is deterministic with \(b_1\) bounded and measurable and \(b_2\) Lipschitz-continuous.
Theorem A.1
Suppose that b is as in Theorem 1.1 and \(\alpha \in \mathcal {A}\). For every \(0\le s\le t\le T\), the first variation \(\Phi ^{\alpha , x}(t,s)\) of the unique strong solution to the SDE (1) admits the representation
Here \(\int _s^t\int _{\mathbb {R}}b_1(u,z)L^{X^x}(\textrm{d}u,\textrm{d}z)\) is the integration with respect to the time-space local time of \(X^x\) and \(b'_2\) is the derivative with respect to the second parameter.
Proof
We know from [5, 38] that under the condition of the Theorem, the SDE (1) has a Sobolev differentiable flow denoted \(\Phi ^{\alpha ,x}\). In particular, it is shown in these references that \(\Phi ^{\alpha ,x}_n(t,s)\) converges to \(\Phi ^{\alpha ,x}(t,s)\) weakly in \(L^2(U\times \Omega )\).
Thus, in order to show the representation (32), it suffices to show that \(\Phi ^{\alpha ,x}_n(t,s)\) converges to
weakly in \(L^2(U\times \Omega )\). Since the set
spans a dense subspace in \(L^2(U\times \Omega )\), it is therefore enough to show that
Let \(\varphi \in C^1_b([0,T],\mathbb {R}^d)\), recall that for every n, the process \({\tilde{X}}^{{\tilde{\alpha }},x}_n:=X^{\tilde{\alpha },x}_n(\omega +\varphi )\), with \({\tilde{\alpha }}(\omega )=\alpha (\omega +\varphi )\) satisfies the SDE
We have by using the Cameron–Martin theorem, the fact that \(|e^x-e^y|\le |x-y||e^x+e^y|\), the Hölder inequality and boundedness of \(b_2^\prime \) that
where the last inequality follows from the boundedness of \(b_2\) and \(b'_2\). By Lemma 2.5, we have that \(\mathbb {E}[ e^{2\int _s^{t}\int _{\mathbb {R}}b_{1,n}\left( u,z\right) L^{\tilde{X}_n^{{\tilde{\alpha }},x}}(\textrm{d}u,\textrm{d}z)}]\) is bounded. The second term on the right side of the above converges to zero since one can show as in Lemma 2.3 that \({\tilde{X}}^{n,\tilde{\alpha },x}(s)\) converges strongly to \({\tilde{X}}^{{\tilde{\alpha }},x}(s)\) in \(L^2\) and \(b_2^\prime \) is bounded and continuous.
We now show that the second term converges to zero. We will show both the weak convergence and the convergence in mean square. Using the Cameron–Martin–Girsanov theorem as above for every \(\varphi _1 \in C^1_b([0,T],\mathbb {R}^d)\) we have
Therefore, using the inequality \(|e^x-e^y|\le |x-y||e^x+e^y|\) and the Hölder’s inequality we obtain
Lemma A.2, shows that \(J_{2,n}\) converges to zero, and the convergence to zero of \(J_{5,n}\) follows by the dominated convergence. Thanks to Lemma A.3 and the boundedness of \(b_{1,n}\) and \(b_1\), respectively, the term \(J_{3,n}\) (respectively \(J_{4,n}\)) is bounded. The bound of \(J_{1,n}\) follows by the uniform boundedness of \(u_n\).
Set \(A_n^{\alpha }(t)=e^{\int _s^{t}\int _{\mathbb {R}}b_{1,n}\left( u,z\right) L^{\tilde{X}_n^{{\tilde{\alpha }},x}}(\textrm{d}u,\textrm{d}z)}\) and \(A^{\alpha }(t)=e^{\int _s^{t}\int _{\mathbb {R}}b_1\left( u,z\right) L^{\tilde{X}^{{\tilde{\alpha }},x}}(\textrm{d}u,\textrm{d}z)}\). It remains to show the convergence of the second moment, i.e. \(\mathbb {E}[|A_n^{\alpha }(t)|^2]\) converges to \(\mathbb {E}[|A^{\alpha }(t)|^2]\) in \(\mathbb {R}\). This follows as in the proof of Lemma 2.6. The desired result follows. \(\square \)
We know from [18, Theorem 2.1] that the local time-space integral of \(f \in {{\mathcal {H}}}^0\) admits the decomposition
\(0\le t\le T\), a.s., where \({\widehat{B}}\) is the time-reversed Brownian motion, that is
In addition, the process \(W=\{W(t),\,\,\,0\le t\le T\}\) is an independent Brownian motion with respect to the filtration \(\mathcal {F}_t^{{\widehat{B}}}\) generated by \({\widehat{B}}_t\), and satisfies:
Lemma A.2
Let \(\varphi \in C^1_b([0,T],\mathbb {R}^d)\) and define \(F_{1,n}\) by
Then \(\mathbb {E}[|F_{1,n}|^2]\) converges to zero as n goes to \(\infty \).
Proof
Using the local time-space decomposition (36), the Minkowski integral inequality with the measure \(\nu (\sigma )=\int _{\sigma }\frac{\textrm{d}s}{2\sqrt{T-s}}\), the Hölder and the Burkholder–Davis–Gundy inequalities, we get
Now using the Cauchy–Schwartz inequality and the fact that \(E[B^4(t)]=3t^2\), we can continue the estimation as
Each term above converges to zero. We give the detail only for the first term. The treatment of the two other terms is analogous. Given \(p>1\), using the density of the Brownian motion, we have as in the proof of Lemma 2.3 (see (11))
Since \(b_{1,n}\) converges to \(b_1\), it follows from the dominated convergence theorem that each term in the above inequality converge to zero. \(\square \)
The following Lemma corresponds to [5, Lemma A.2] and it gives the exponential bound of the local time-space integral of a bounded function
Lemma A.3
Let \(b:[0,T]\times \mathbb {R} \rightarrow \mathbb {R}\) be a bounded and measurable function. Then for \(t\in [0,T],\, \lambda \in \mathbb {R}\) and compact subset \(K\subset \mathbb {R}\), we have
where C is an increasing function and \(L^{B^x}(\textrm{d}s,\textrm{d}y)\) denotes integration with respect to the local time of the Brownian motion \(B^x\) in both time and space. In addition, if \(b_n\) is an approximating sequence of b such that the \(b_n\) are uniformly bounded by \(\Vert b\Vert _{\infty }\) then the above bound still hold true with the bound independent of n.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Menoukeu-Pamen, O., Tangpi, L. Maximum Principle for Stochastic Control of SDEs with Measurable Drifts. J Optim Theory Appl 197, 1195–1228 (2023). https://doi.org/10.1007/s10957-023-02209-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-023-02209-0
Keywords
- Stochastic maximum principle
- Singular drifts
- Sobolev differentiable flow
- Ekeland’s variational principle