THE PATHWISE-DETERMINED MAXIMUM PRINCIPLE AND SYMMETRIC INTEGRALS

Abstract

For stochastic differential equations with the Stratonovich integral and with a controlled drift, a connection is found between the stochastic maximum principle with the averaged cost functional and the trajectory-deterministic maximum principle with the trajectory cost functional. It is shown that for stochastic differential equations with symmetric integral and controlled drift, the trajectory maximum principle allows one to find the optimal nonanticipating control.

Appendices

Appendix A. Symmetric Integrals and Pathwise Equations with Symmetric Integrals

Symmetric integrals are a generalization of the Stratonovich integrals with respect to the Wiener process W(s) (see [5, 6] for details).

Let V(s) , $s \in [0, +\infty )$, be an arbitrary continuous function. A symmetric integral is called

$$\begin{aligned} {\int _{0}^{t}} f (s, V (s)) * dV (s) = \lim \limits _{n \rightarrow \infty } {\int _{0}^{t}}f (s, V^{(n)} (s)) (V^{(n)})'(s)\,ds, \end{aligned}$$

(A.1)

where $V^{(n)}(s)$ are broken lines constructed according to the partition $\{t_{k}^{(n)} \}$ of the segment [0, t] such that $\max \limits _{k}(t_{k}^{(n)} - t_{k-1}^{(n)})\rightarrow 0$ as $n\rightarrow \infty$.

We say that a pair of functions (V(s), f(s, u)) satisfies the condition (S) , if the following assumptions hold:

(a) V(s) , $s \in [0, t]$, is a continuous function.

(b) The function f(s, u) , $s \in [0, t]$, is right continuous and has bounded variation for almost all u.

(c) The function |f| (t, u) is a locally summable function with respect to u for each t, here |f| (t, u) is a full variation of the function $s \rightarrow f (s, u)$ on the segment [0, t] .

(d) $\ \ \int _{0}^ t {\textbf {1}} (s: V (s) = u) | f | (ds, u) = 0$ for almost all u, where ${\textbf {1}} (A)$ is an indicator of the set A, that is, a function equal to 1 on A and 0 out A.

Let the pair of functions (V(s), f(s, u)) satisfy the condition (S) , then the symmetric integral $\int _{0}^ t f (s, V (s)) * dV (s)$ exists [5, 6].

Some properties of symmetric integrals

1. In the definition of a symmetric integral, broken lines can be replaced by smooth functions constructed as follows. For each n, fix the point $t_{k}^{(n)}$. In the neighborhood of $[t_{k}^{(n)} - \epsilon _ {n, k}, t_{k}^{(n)} +\epsilon _{n, k}] \in [t_{k-1}^{(n)} t_{k + 1}^{(n)}]$, $\epsilon _ {n, k}> 0$, the angle is replaced by an arc in the following way. The top of the arc must be at $\left( t_{k}^{(n)}, V^{(n)} (t_{k}^{(n)}) \right)$ and the new function and its derivative must be continuous; we can assume that $\sum _{k}\epsilon _{n, k} <2^{- n}$.

2. Let the pair of functions (V(s), f(s, u)) satisfy the condition (S) , then

$$\begin{aligned} \int _{t_{0}}^{t} f(s,V(s))*dV(s){=}\int _{V(t_{0})}^{V(t)} f(t,u)\,du{-}\int _{R}\int _{t_{0}}^{t}\kappa (u,V(t_{0}),V(s))f(ds,u)\,du, \end{aligned}$$

(A.2)

where $\kappa (u,a,b)=sign(b-a){\textbf {1}}(a\wedge b<v<a\vee b)$.

3. Let the function F(t, u) have continuous partial derivatives $F'_{t}$ and $F'_{u}$, then there exists a symmetric integral ${\int _{0}^{t}} F'_{u} (s, V (s)) * dV (s)$ and the formula

$$\begin{aligned} F (t, V (t)) - F (0, V (0)) = {\int _{0}^{t}} F_{s}'(s, V (s)) \, ds + {\int _{0}^{t}} F_{u}' (s, V (s) ) * dV (s). \end{aligned}$$

(A.3)

is valid. Thus, the formula well known for the Stratonovich integral is true for symmetric integrals.

Suppose that the functions f(s, u) and $f'_{u} (s, u)$ are continuous with respect to the variable s. Then the path integral ${\int _{0}^{t}} f (s), W (s)) * dW (s)$ coincides with the stochastic Stratonovich integral ${\int _{0}^{t}} f (s, W (s)) * dW (s)$ for almost all trajectories of Wiener process. Therefore, we will apply the same notation for both types of integrals.

Symmetric integrals generate a new class of pathwise equations. We will consider a pathwise equation with a symmetric integral [5, 6]

$$\begin{aligned} \xi (t) - \xi (0) = {\int _{0}^{t}} \sigma (s, V (s), \xi (s)) * dV (s) + {\int _{0}^{t}} b (s, V (s), \xi (s))\,ds, \ \ \xi (0) = \xi _{0}, \end{aligned}$$

(A.4)

$t \in [0, T]$, here V(s) is an arbitrary continuous function (or the trajectory of a random process).

By the solution of equation (A.4) we mean any function $\xi (s) = \varphi (s, V (s)), s \in [0, T]$, $\xi (0) = \xi _{0}$, satisfying the following conditions:

• a pair of functions $(V (s), \sigma (s, u, \varphi (s, u)))$ satisfies the condition (S) on [0, T] ;

• the function $b (s, V (s), \xi (s))$ summable on the segment [0, T] ;

• the function $\xi (s)$ turns equation (A.4) into identity.

The function $\varphi (s, u)$ is called a solution structure of equation (A.4).

It is important that equation (A.4) is a deterministic equation, therefore, if V(s) is a random process with continuous sample paths, then equation (A.4) with symmetric integrals is a path equation. Note that the coefficients and solution of the pathwise equation are not required to be predictable.

On the other hand, if $V (t)=W(t)$ is the Wiener process and the coefficients $\sigma (t,v,\varphi )$ and $b(t,v,\varphi )$ are smooth and nonrandom, then equation (A.4) coincides with the Stratonovich equation for almost all trajectories of Wiener process.

The transition to pathwise equations allows us to construct a solution method and identify the solution structure of a wide class of equations with symmetric integral.

Theorem 3

Let the function V(t) , $t\in [0, T]$, be a continuous function. Suppose that $\sigma (t, v, \varphi )$ and $b (t, v, \varphi )$ are jointly continuous and the function $\varphi (t, v)$, $\varphi (0, V (0)) = \xi (0)$, has the continuous derivatives $\varphi '_{t} (t, v)$, $\varphi '_{v} (t, v)$.

If $\varphi (t, v)$ satisfies the system of equations:

$$\begin{aligned} \varphi '_{t} (t, V (t)) = b (t, V (t), \varphi (t, V (t))), \ \ t \in [0, T], \end{aligned}$$

(A.5)

$$\begin{aligned} \varphi '_{v} (t, V (t)) = \sigma (t, V (t), \varphi (t, V (t))), \ \ t\in [0, T], \end{aligned}$$

(A.6)

then the symmetric integral

$$\begin{aligned} {\int _{0}^{t}} \sigma (s, V (s), \varphi (s, V (s))) * dV (s) \end{aligned}$$

exists for all $t \in [0, T]$ and $\xi (t) = \varphi (t, V (t))$ is a solution of equation (A.4).

Theorem 4

Let the functions $\sigma (t, v, \varphi )$, $b (t, v, \varphi )$, and their partial derivatives $\sigma '_{t} (t, v, \varphi )$, $\sigma '_{\varphi } (t, v, \varphi )$, and $b_{\varphi }'(t, v, \varphi )$ be continuous. Suppose a solution of the Cauchy problem (7.4) exists, then it is unique.

The system of equations (A.5), (A.6) is an unusual system of equations along the trajectory of the function V(s) , so there are no well-developed methods for solving such equations. In this regard, it seems reasonable to reduce the problem of solving system (A.5), (A.6) to the problem of solving the chain of equations

$$\begin{aligned} \varphi '_{v}(t,v){=}\sigma (t,v,\varphi (t,v)), \ \ \varphi '_{t}(t,V(t)){=}b(t,V(t),\varphi (t,V(t))), \ \ \varphi (0,V(0)){=}\xi (0). \end{aligned}$$

(A.7)

In this case, we can use the methods of the theory of ordinary differential equations. Indeed, the general solution $\varphi (t, v) = \varphi ^{*}(t, v, C (t))$ of the first equation from (A.7), if it exists, depends on an arbitrary function C(t) and by differentiability theorem with respect to the parameter, the function C(t) is a smooth function.

The second relation from (A.7) can be considered as an ODE with respect to the unknown function C(t) . Really, substituting the function $\varphi ^{*} (t, v, C (t))$ into the second equation from (A.7), we get the Cauchy problem for the unknown function C(t) :

$$\begin{aligned} (\varphi ^{*})'_{t} (t, V (t), C (t)) + (\varphi ^{*})'_{C} (t, V (t), C (t)) C'(t) = b (t, V (t), \varphi ^{*}(t, V (t), C (t))), \end{aligned}$$

the initial condition has the form $\varphi ^{*}(0, V (0), C (0)) = \xi (0)$.

Obviously, the solution of chain (A.7), if one exists, allows us to find a solution of system (A.5), (A.6). It means, that the problem of the existence of the solution of equation (A.4) can be reduced to the problem of clarifying the solvability conditions for both ODEs of chain (A.7).

In particular, if the coefficients of the equation have a simpler form: $\sigma = \sigma (t, \varphi )$, $b= b (t, \varphi )$, then the first equation of (A.7) is the equation with separable variables, and a general solution of equation (A.7) is represented as $\varphi (t, v + C (t))$. The function C(t) is the solution of the Cauchy problem

$$\begin{aligned} \sigma (t,\varphi (t,V(t)+C(t)))C'(t)= {b(t,\varphi (t,V(t)+C(t)))-\varphi '_{t}(t,V(t)+C(t))}, \end{aligned}$$

$\ \ \ \varphi (0,V(0)+C(0))=\xi _{0}$.

Appendix B. Proof of the passage to the limit in the boundary value problem of the maximum principle with symmetric integrals

Let $V^{(n)} (t)$, $t \in [0, T]$, be a sequence of smooth approximations of broken lines introduced earlier. Consider sequence of boundary value problems of the maximum principle

$$\begin{aligned} {\left\{ \begin{array}{ll} x_{n}'(t)=B(t,x_{n}(t)+V_{n}(t),u_{n}(t)), \ \ x_{n}(0)=x_{0}, \\ d{\psi _{n}}(t) = \left[ (F_{0})'_{y}(t,y_{n}(t),u_{n}(t))-{\psi _{n}}(t)(B)'_{y}(t,y_{n}(t),u_{n}(t))\right. \\ \left. +(\Lambda _{1})'_{y}(t,y_{n}(t) )\right] dt+ (\Lambda _{2})'_{y}(t,y_{n}(t) )*dV_{n}(t), \\ \psi _{n}(T)=-(G_{0})'_{x}(y_{n}(0),y_{n}(T)), \ y_{n}(t)=x_{n}(t)+V_{n}(t), \end{array}\right. } \end{aligned}$$

(B.1)

here $u_{n}(t)=u(t,y_{n}(t),\psi _{n}(t))$, and boundary value problem

$$\begin{aligned} {\left\{ \begin{array}{ll} x(t)=B(t,x(t)+V(t),u(t)), \ \ x(0)=x_{0}, \\ d{\psi }(t) = \left[ (F_{0})'_{y}(t,y(t),u(t))-{\psi }(t)(B)'_{y}(t,y(t),u(t))\right. \\ \left. +(\Lambda _{1})'_{y}(t,y(t) )\right] dt+ (\Lambda _{2})'_{y}(t,y(t) )*dV(t), \\ \psi (T)=-(G_{0})'_{x}(y(0),y(T)), \ y(t)=x(t)+V(t), \end{array}\right. } \end{aligned}$$

(B.2)

where $u(t)=u(t,y(t),\psi (t))$. Recall that since there are no restrictions on controls of the form (2.4), the control $u (t, y, \psi )$ can be found for both problems from the same condition (4.4).

Suppose $\phi (s, \bar{q})$, $s \in [0, T]$, $\bar{q} \in Q$ is an arbitrary function. By definition, put $M_{\phi }(s,Q)=\sup \{ \phi (s,\bar{q}) : \bar{q}\in Q\}$. We will need the following version of Gronwall’s lemma.

Suppose that the functions u(t) , f(t) , $t \in [t_{0}, T]$, are non-negative, continuous, and satisfy the inequality

$$\begin{aligned} u(t)\le C+{\int _{0}^{t}} f(s)u(s)\,ds, \ C>0, \ t \in [t_{0}, T], \end{aligned}$$

then

$$\begin{aligned} u(t)\le C\exp \left\{ \int _{t_{0}}^{t} f(s)\,ds\right\} , \ t\in [t_{0},T]. \end{aligned}$$

Let the condition $({\Lambda })$ be true. Suppose that the solutions $(x_{n} (t), \psi _{n} (t))$ of boundary value problems (B.1) exist. We show that we can construct a subsequence of solutions $(x_{n_{m}} (t), \psi _{n_{m}} (t))$, $t \in [0, T]$ such that this subsequence converges uniformly to the solution $(x ( t), \psi (t))$ as $n_{m} \rightarrow \infty$. To do this, it is enough to check that the sequence $\{x_{n} (t), {\psi _{n}} (t) \}$ is uniformly bounded and equicontinuous. Indeed, using for each n the boundedness of the function B(t, z, u) , we get:

$$\begin{aligned} |x_{n}(t)-x_{0}|{\le \int _{0}^{T}} |B(s,x_{n}(s)+V_{n}(s),u_{n}(s))|\,ds\le {\int _{0}^{T}}M_{B}(s)\,ds. \end{aligned}$$

Further, suppose $t_{1}> t_{0}$, then

$$\begin{aligned} |x_{n}(t_{1})-x_{n}(t_{0})|\le \int _{t_{0}}^{t_{1}} |B(s,x_{n}(s)+V_{n}(s),u_{n}(s))|\,ds\le \int _{t_{0}}^{t_{1}}M_{B}(s)\,ds. \end{aligned}$$

Hence, the sequence $\{{x_{n}} (t)\}$ is uniformly bounded and equicontinuous.

Consider a sequence of functions $\{{\psi _{n}} (t) \}$. Denote by

$$\begin{aligned} J_{n} = \int _{t_{0}}^{t} (\Lambda _{2})'_{y} (s, x_{n} (s) + V_{n} (s)) * dV_{n} (s) \end{aligned}$$

the second term in the equation for the adjoint variable of system (B.1). In view of formula (A.1), we have:

$$\begin{aligned} J_{n} = \int _{t_{0}}^{t}(\Lambda _{2})'_{y}(s,x_{n}(s)+ V_{n}(s) )*dV_{n}(s) =\int _{V_{n}(t_{0})}^{V_{n}(t)}(\Lambda _{2})'_{v}(t,x_{n}(t)+v )\,dv \end{aligned}$$

$$\begin{aligned} -\int _{t_{0}}^{t}\int _{V_{n}(t_{0})}^{V_{n}(s)}(\Lambda _{2})''_{v s}(s,x_{n}(s)+v )\,dv\,ds= \Lambda _{2}(t,x_{n}(t)+V_{n}(t))-\Lambda _{2}(t,x_{n}(t)+V_{n}(t_{0})) \end{aligned}$$

(B.3)

$$\begin{aligned} -\int _{t_{0}}^{t}\left[ (\Lambda _{2})'_{s}(s,x_{n}(s)+V_{n}(s))-(\Lambda _{2})'_{s}(s,x_{n}(s)+V_{n}(t_{0}))\right] ds. \end{aligned}$$

Hence,

$$\begin{aligned} J_{n}\le 2\sup _{s\in [t_{0},T]}M_{\Lambda _{2}}(s)+2\int _{t_{0}}^{T} M_{(\Lambda _{2})'_{s}}(s)\,ds. \end{aligned}$$

Denote $R(t,y,\psi ,u)=(F_{0})'_{y}(t,y,u)-{\psi }(B)'_{y}(t,y,u) +(\Lambda _{1})'_{y}(t,y ).$ Then for any $t_{1}> t_{0}$ we obtain

$$\begin{aligned} \int _{t_{0}}^{t_{1}}|R(s,x_{n}(s)+V_{n}(s),\psi _{n}(s),u_{n}(s))|\,ds \end{aligned}$$

$$\begin{aligned} \le \int _{t_{0}}^{t_{1}}[M_{(F_{0})'_{y}}(s)+|{\psi _{n}(s)}|M_{(B)'_{y}}(s) + M_{(\Lambda _{1})'_{y}}(s)]\,ds. \end{aligned}$$

(B.4)

Consequently,

$$\begin{aligned} |\psi _{n}(t)|\le C^{*}+{\int _{0}^{t}}[|{\psi _{n}(s)}|M_{(B)'_{y}}(s) + M_{(\Lambda _{1})'_{y}}(s)]\,ds, \end{aligned}$$

where

$$\begin{aligned} C^{*}{=}|\psi _{n}(0)|{+}2\sup _{s\in [0,T]}M_{\Lambda _{2}}(s){+} {\int _{0}^{T}}\left[ M_{(F_{0})'_{y}}(s){+}M_{(\Lambda _{1})'_{y}}(s)+2M_{(\Lambda _{2})'_{s}}(s)\right] ds. \end{aligned}$$

Hence, in view of the above-mentioned Granwall lemma we get

$$\begin{aligned} |\psi _{n}(t)|\le C^{*}\exp \left\{ \int _{t_{0}}^{t} M_{(B)'_{y}}(s) \,ds\right\} \le \bar{C}, \end{aligned}$$

(B.5)

where

$$\begin{aligned} \bar{C}=C^{*}\exp \left\{ {\int _{0}^{T}} M_{(B)'_{y}}(s)\, ds\right\} . \end{aligned}$$

So, the family $\{{\psi _{n}} (t) \}$ is uniformly bounded, we show that this is equicontinuous. Let $t_{1}> t_{0}$, then

$$\begin{aligned} |\psi _{n}(t_{1})-\psi _{n}(t_{0})|\le \int _{t_{0}}^{t_{1}}|R(s,x_{n}(s)+V_{n}(s),\psi _{n}(s),u_{n}(s))| \,ds+|d_{n}|, \end{aligned}$$

where

$$\begin{aligned} d_{n}= \int _{t_{0}}^{t_{1}}(\Lambda _{2})'_{y}(s,x_{n}(s)+V_{n}(s) )*dV_{n}(s). \end{aligned}$$

Using inequalities (B.4) and (B.5), we get:

$$\begin{aligned} \int _{t_{0}}^{t_{1}}|R(s,x_{n}(s){+}V_{n}(s),\psi _{n}(s),u_{n}(s))| \,ds \end{aligned}$$

$$\begin{aligned} \le \int _{t_{0}}^{t_{1}}[M_{(F_{0})'_{y}}(s){+}\bar{C}M_{(B)'_{y}}(s) {+} M_{(\Lambda _{1})'_{y}}(s)]\,ds. \end{aligned}$$

Further, from relation (B.3), taking into account the condition (L) , we have

$$\begin{aligned} |d_{n}|\le \int _{m(t_{0},t_{1})}^{M(t_{0},t_{1})}|(\Lambda _{2})'_{y}(t_{0},x_{n}(t_{0})+v)|\,dv \end{aligned}$$

$$\begin{aligned} +\int _{m(t_{0},t_{1})}^{M(t_{0},t_{1})}\int _{t_{0}}^{t_{1}}|((\Lambda _{2})''_{s y}(s,x_{n}(s)+v))'_{s}|\,ds\,dv \end{aligned}$$

$$\begin{aligned} \le [M(t_{0},t_{1})-m(t_{0},t_{1})]\left\{ \sup _{ t y}\{|(\Lambda _{2})'_{y}(t,y)|\}+[t_{2}-t_{1}]\sup _{ t y}\{|(\Lambda _{2})''_{s y}(t,y)|\}\right\} , \end{aligned}$$

where $M(t_{0},t_{1})=\max \{V(s):s\in [t_{0},t_{1}]\}$, $m(t_{0},t_{1})=\min \{V(s):s\in [t_{0},t_{1}]\}$. So, the family $\{x_{n} (t),\psi _{n} (t)\}$ is uniformly bounded and equicontinuous. Therefore, according to the Arzelo-Ascoli theorem, there is a subsequence $(x_{n_{m}} (t), \psi _{n_{m}} (t))$ converging uniformly to some continuous functions $(x (t), \psi (t ))$.

Let us show that $(\tilde{x} (t), \tilde{\psi }(t))$ is the solution to the Cauchy problem (B.2). Indeed, in view of the continuity and the boundedness of the function B(t, z, u) and the piecewise continuity of the function $u (s, y, \psi )$, we have:

$$\begin{aligned} \tilde{x}(t)-x(0){=} \lim \limits _{m\rightarrow \infty }{\int _{0}^{t}} B(s,x_{n_{m}}(s)+V_{n_{m}}(s),u(s,x_{n_{m}}(s)+V_{n_{m}}(s),\psi _{n_{m}}(s)))\,ds \end{aligned}$$

$$\begin{aligned} ={\int _{0}^{t}} \lim \limits _{m\rightarrow \infty }B(s,x_{n_{m}}(s)+ V_{n_{m}}(s),u(s,x_{n_{m}}(s)+V_{n_{m}}(s),\psi _{n_{m}}(s)))\,ds \end{aligned}$$

$$\begin{aligned} ={\int _{0}^{t}} B(s,\tilde{x}(s)+V(s),u(s))\,ds. \end{aligned}$$

Therefore, using the Lebesgue theorem on the passage to the limit, we obtain

$$\begin{aligned} \tilde{\psi }(T)-\psi (t)= \tilde{\psi }(T)-\lim \limits _{m\rightarrow \infty }\psi _{n_{m}}(t) \end{aligned}$$

$$\begin{aligned} = {\int _{t}^{T}}\lim \limits _{m\rightarrow \infty } \left[ (F_{0})'_{y}(s,y_{n_{m}}(s),u_{n_{m}}(s))- {\psi _{n_{m}}}(s)(B)'_{y}(s,y_{n_{m}}(s),u_{n_{m}}(s))\right. \end{aligned}$$

$$\begin{aligned} \left. +(\Lambda _{1})'_{y}(s,y_{n_{m}}(s))\right] \,ds +\lim \limits _{m\rightarrow \infty }J_{n_{m}}. \end{aligned}$$

The fact that the last term is equal to ${\int _{t}^{T}} (\Lambda _{2})'_{y} (s, y (s)) * dV (s)$, is checked using relation (B.3) and formula (A.3). Indeed, we have:

$$\begin{aligned} \lim \limits _{m\rightarrow \infty }J_{n_{m}}= \Lambda _{2}(t,x(t)+V(t))-\Lambda _{2}(t,x(t)+V(t_{0})) \end{aligned}$$

$$\begin{aligned} -\int _{t_{0}}^{t}\left[ (\Lambda _{2})'_{s}(s,x(s)+V(s))-(\Lambda _{2})'_{s}(s,x(s))+V(t_{0}))\right] \,ds \end{aligned}$$

$$\begin{aligned} ={\int _{t}^{T}}(\Lambda _{2})'_{y}(s,y(s) )*dV(s). \end{aligned}$$

So $(\tilde{x} (t), \tilde{\psi }(t))$ is the solution of system of equations (B.2).