Abstract
In this paper we study a general optimal liquidation problem with a control-dependent stopping time which is the first time the stock holding becomes zero or a fixed terminal time, whichever comes first. We prove a stochastic maximum principle (SMP) which is markedly different in its Hamiltonian condition from that of the standard SMP with fixed terminal time. We present a simple example in which the optimal solution satisfies the SMP in this paper but fails the standard SMP in the literature.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Two main approaches to solving stochastic optimal control problems are the partial differential equation (PDE) method based on the dynamic programming principle (DPP) and the backward stochastic differential equation (BSDE) method based on the stochastic maximum principle (SMP), see Fleming and Soner [9], Yong and Zhou [19] for expositions. Most literature is on fixed terminal time problems. When the underlying state process is a controlled diffusion process, one may find the optimal solution with the Hamilton–Jacobi–Bellman (HJB) equation (a nonlinear PDE) or the BSDE (a coupled forward-backward system connected with the Hamiltonian condition). When the underlying state process is a controlled jump-diffusion process, one may find the optimal solution with the HJB partial differential difference equation or the BSDE, see Oksental and Sulem [13] for model formulations and solution methodologies. When the terminal time is not fixed but a random stopping time that is determined directly by a decision-maker, one has an optimal stopping problem or a combined optimal control/stopping problem and may find the optimal solution with the HJB variational equation or the reflected BSDE, see Pham [14] for an excellent concise introduction of this and other topics above, see also Barbu and Röckner [2], Cordoni et al. [6], Diomande and Maticiuc [8], Popier and Zhou [16] for results on existence and uniqueness of forward backward and delayed stochastic differential equations and second order BSDEs and applications.
In this paper we investigate a stochastic optimal control problem with a random terminal time. In contrast to optimal stopping problem, the terminal time is indirectly determined by control strategies. Specifically, we consider an optimal liquidation problem in which the terminal time is determined by the first time the stock holding becomes zero or a fixed terminal time, whichever comes first. The objective is to maximize the expected cash value of the liquidation at the terminal time subject to some other underlying state process (stock price, volatility, etc.) dynamics. Such a model cannot be cast into the framework of the optimal stopping problem as stopping time is not directly controlled nor the jump-diffusion model with the fixed terminal time as stopping time is random. For a Markovian model, one can show that the value function satisfies the HJB equation with the boundary condition when the stock holding is zero as well as the terminal condition at the fixed terminal time, see for example Cartea et al. [4]. Since the HJB equation is a nonlinear PDE, one can find the closed-form solution or show the existence of a classical solution only for some specific models and has to rely on the viscosity solution concept for general models. For a non-Markovian model or with control constraints, the HJB approach loses its tractability. On the other hand, the SMP approach provides an alternative way of solving the problem for general, possibly non-Markovian, models. However, the standard SMP only applies to the problem with fixed terminal time. The first huddle we must overcome is to find the form of the BSDE and the associated Hamiltonian condition for the problem with random control-dependent terminal time.
There have been some efforts in the literature to address optimal trading. Ankirchner et al. [1] use the BSDE approach to solving the singular terminal state problem in which the terminal inventory is forced to be zero at terminal time, differently from our setting in which this constraint is weakened. Horst and Naujokat [10] state a version of the SMP for optimal trading strategy with the spread driven by a jump diffusion process. Pham [15] studies a model with multiple stopping times, a special form of the jump-diffusion process, and characterizes the value function with a system of backward recursive dynamic programming equations and the optimal control with progressive enlargement of filtration.
Cordoni and Di Persio [5] study a similar model to [15] and derive a system of backward recursive BSDEs that are similar to the standard SMP over adjacent stopping times intervals. There is, however, a key difference in the stopping time definitions in Pham [15] and Cordoni and Di Persio [5]. The former is independent of controls and is given by some driving jump processes whereas the latter depends on controls and is given by the first time the underlying controlled state process hits some deterministic boundaries. In the standard derivation of SMP with fixed terminal time (c.f. Bensoussan [3] and Pham [14]), one may use the optimality condition and the variation of the optimal control to derive the BSDE and the Hamiltonian condition. When the terminal time is a stopping time depending on control, then variational analysis involves changes in terminal time as well as underlying state variables, unlike that for the standard SMP that only involves changes in underlying state variables. This aspect is the most arduous difficulty that needs to be overcome in the proof of the SMP with control dependent terminal time. Cordoni and Di Persio [5] prove the SMP without discussing the possibility of changes of terminal stopping times due to changes of controls, see for example [5, Eq. (15)] in the proof of necessary SMP and [5, Eq. (24)] in the proof of sufficient SMP, which implies the SMP in [5] is only valid for a model with stopping times independent of controls, same as that of [15], but invalid for control-dependent stopping times, that is, the BSDE and the Hamiltonian condition in [5] are not applicable to the stopping time definition there.
The main contribution of this paper is to give the SMP in the presence of random control-dependent terminal time, the first in the literature to the best knowledge of the authors. The main theorem (Theorem 2.2) states that the adjoint process satisfies a standard BSDE [see (2.7)] with the terminal time the optimal stopping time, not necessarily the fixed terminal time, and the Hamiltonian function is also a standard one [see (2.9)], but the Hamiltonian condition is markedly different from that of the standard SMP with fixed terminal time [see (2.13)], specifically, we need to add a supplementary nonlinear term that is not additively separable between the optimal control and any other controls. This additional term counts for the random control-dependent terminal time. We give a simple example to show that the optimal solution satisfies the SMP in Theorem 2.2 but not the standard SMP, e.g., the one in [14]. The SMP in this paper only applies to the optimal liquidation problem, but the idea of the variational analysis involving controlled stopping time may be explored further for more general models with other applications, for example, the optimal path planning problem that steers autonomous vehicles to navigate between any two points while optimizes energy, time, etc., see Lee et al. [12] and Subramani et al [17].
The rest of the paper is organized as follows. In Sect. 2 we describe the model and state the main result (Theorem 2.2) that is the SMP with random control-dependent terminal time. In Sect. 3 we present an example to illustrate the main result and show the standard SMP does not hold. In Sect. 4 we prove Theorem 2.2. Section 5 concludes.
2 Model Setup
Let \((\Omega , {\mathcal {F}}, ({\mathcal {F}}_t)_{t\in [0,T]}, \mathbb {P})\) be a filtered probability space, where \(({\mathcal {F}}_t)_{t\in [0,T]}\) is the natural filtration generated by an m-dimensional standard Brownian motion \(\mathbf {W}\), augmented by all \(\mathbb {P}\)-null sets. Let T be the fixed terminal time. Let \((\pi _t)_{t\in [0,T]}\) denote the rate of selling the stock, which is a decision (control) variable selected by the agent and is said admissible if it is a progressively measurable, non-negative, right-continuous and square integrable process. Denote by \({\mathcal {A}}\) the set of all admissible control processes. We consider \(\pi \) to be the liquidation rate of the inventory \(Q_t\), defined as
Let \((\mathbf {X}_t)_{t\in [0,T]}\) be an \(\mathbb {R}^n\) valued stochastic process satisfying the following stochastic differential equation (SDE):
with initial condition \(\mathbf {X}_0=\mathbf {x}\), where \({\varvec{\mu }}:[0,T]\times \mathbb {R}^n\rightarrow \mathbb {R}^n\), \({\varvec{\sigma }}:[0,T]\times \mathbb {R}^n\rightarrow \mathbb {R}^{n\times m}\) are two continuous functions. \(\mathbf {X}\) represents the market information such as stock price, volatility, etc., and is not influenced by the control process \(\pi \) (liquidation without price impact). The optimal liquidation problem is defined by
where \(g:\mathbb {R}^n\times \mathbb {R}\rightarrow \mathbb {R}\) and \(f:[0,T]\times [0,\infty )\times \mathbb {R}^n\times \mathbb {R}\rightarrow \mathbb {R}\) are two continuously differentiable functions representing the terminal and running payoffs, \(\tau ^\pi \) is a stopping time defined by
the first time when all stock is liquidated or fixed terminal time T, whichever comes first.
To simplify the notation we consider a one-dimensional process X, but all results can be obtained in the multi-dimensional case. We denote the state space of the pair \((X_r,Q_r)\) as \({\mathcal {O}}:=\mathbb {R}\times [0,q_0]\). In the following, with a slight abuse of notations, we denote \(\pi _r\) as \(\pi _r\mathbb {1}_{r\le \tau ^\pi }\), which equals 0 after \(\tau ^\pi \). Similarly, \(Q^\pi _r=0\) for \(r> \tau ^\pi \). Whenever we refer to a time interval [a, b), if \(a\ge b\), then we consider it to be an empty set.
To state and prove a necessary SMP for problem (2.3), we follow the procedure in Bensoussan [3]. Assume that c is the optimal control and Q and \(\tau \) are the corresponding optimal inventory and stopping time, defined in (2.1) and (2.4) respectively. If \(Q_0=0\) then \(\tau =0\) and there is nothing to discuss. We assume \(Q_0>0\) which implies \(\tau >0\). We next define the variation of the optimal control c. For fixed \(t\in [0,\tau )\) and \({\bar{c}} \ge 0\), we have \(q:=Q_t>0\). Choose \(\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right) \), that is, \(t<t+\theta <T\) and \(q-{{\bar{c}}}\theta >0\), consider a variation of c as follows:
where \(\mathbb {1}_S\) is an indicator that equals 1 if S is true and 0 otherwise, and for \(r\ge t\),
The control \(c^{\theta ,{\bar{c}}, t}\) in (2.5) is an admissible control, see Lemma 4.1. Let \(Q^{\theta ,{\bar{c}}, t}_r\) be the corresponding inventory under the control \(c^{\theta ,{\bar{c}}, t}\), given by (2.1), and \(\tau ^{\theta ,{\bar{c}}, t}\) be the first hitting time r when the inventory \(Q^{\theta ,{\bar{c}}, t}_r\) gets to 0, given by (2.4).
Since \(c^{\theta ,{\bar{c}}, t}_r=c_r\) for \(r\in [0, t)\), we have \(Q^{\theta ,{\bar{c}}, t}_r=Q_r\) for \(r\in [0, t)\), in particular, \(Q^{\theta ,{\bar{c}}, t}_t=Q_t\) and \(\tau ^{\theta ,{\bar{c}}, t}>t\). In fact, with the condition on \(\theta \), \(Q^{\theta ,{\bar{c}}, t}_{(t+\theta )\wedge \tau } = Q_t - {{\bar{c}}}((t+\theta )\wedge \tau -t)\ge q-{{\bar{c}}}\theta >0\), so \(\tau ^{\theta ,{\bar{c}}, t}>(t+\theta )\wedge \tau \). The term \(\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\) represents the difference of the total liquidation on the time interval \([t,(t+\theta ) \wedge \tau ]\) with the constant control \({{\bar{c}}}\) and with the optimal control c, which determines the relation of \(\tau \) and \(\tau ^{\theta ,{\bar{c}}, t}\). If \(\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }>0\) then \(t+\theta<\tau ^{\theta ,{\bar{c}}, t}<\tau \) when \(t+\theta<\tau <T\), specifically, \(\tau ^{\theta ,{\bar{c}}, t}\) is the time r when the optimal inventory \(Q_r\) is equal to \(\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\), see Fig. 1a. If \(\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }<0\) then \(\tau ^{\theta ,{\bar{c}}, t}>\tau \), see Fig. 1b. Note that we denote \(c_r^{\theta ,{\bar{c}}, t}\) as \(c_r^{\theta ,{\bar{c}}, t}\mathbb {1}_{r\le \tau ^{\theta ,{\bar{c}}, t}}\), which equals 0 after \(\tau ^{\theta ,{\bar{c}}, t}\). We see that \(c_r^{\theta ,{\bar{c}}, t}\) is well defined in (2.5). Indeed, when \(\tau \ge \tau ^{\theta ,{\bar{c}}, t}\), the last term in (2.5) disappears, while if \(\tau <\tau ^{\theta ,{\bar{c}}, t}\), the quantity \(\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\) is negative, making the last term in (2.5) a non-negative term.
Let \((Y_r,Z_r)_{r\in [0,\tau ]}\) be a solution of the following BSDE:
The processes (Y, Z) are conventional in the usual SMP formulation (c.f. Bensoussan [3] and Pham [14]). BSDE (2.7) has a random terminal time \(\tau \), in contrast to the standard BSDE with fixed terminal time. This type of BSDEs has been studied in the literature, c.f. Darling and Pardoux [7] and in Wu [18].
We assume \(\mu , \sigma , f, g\) satisfy the following conditions for some positive constant K.
Assumption 2.1
For any \(t,t'\in [0,T]\), \(\pi ,\pi '\ge 0\), \(x,x'\in \mathbb {R}\), \(q,q'\ge 0\),
Define the Hamiltonian as
Denote by
We now state the stochastic maximum principle for problem (2.3).
Theorem 2.2
Let Assumption 2.1 be satisfied. Let \((c_r)_{r\in [0,T]}\) be the optimal control for problem (2.3), satisfying
Let \((Q_r)_{r\in [0,T]}\) and \((X_r)_{r\in [0,T]}\) be the corresponding solutions to SDEs (2.1) and (2.2) with \(Q_0>0\), \(\tau >0\) the corresponding stopping time in (2.4), and \((Y_r, Z_r)_{r\in [0,\tau ]}\) the solution to BSDE (2.7). Assume that there exist \(\mathbb {R}\)-valued functions \({\bar{g}}(t,{\bar{c}},x,q)\) and \({\bar{f}}(t,{\bar{c}},x,q)\) so that for any \(t\in [0,\tau )\), \((x,q)\in {\mathcal {O}}\) and \({\bar{c}}\ge 0\),
where \(\mathbb {E}^t[\cdot ]=\mathbb {E}[\cdot | X_t=x, \ Q_t=q]\) is the conditional expectation at time t. Then, c necessarily satisfies for \(t\in [0,\tau )\), \({\bar{c}}\ge 0\),
where \({\mathcal {G}}(t,{\bar{c}},x,q)\) is defined as
and \(\Lambda (t,{\bar{c}})\) as
Remark 2.3
The definition of \({\bar{g}}\) in (2.11) is asymmetric in the arguments of functions g. We may define \({\bar{g}}\) in a symmetric way as
To get an analogy of the Hamiltonian condition in Theorem 2.2, we have to define \({\bar{g}}\) as in (2.11). This point is illustrated in (4.33) in the proof of Theorem 2.2. Note also that stopping time \(\tau \) is determined by the optimal control c as in (2.4) and is therefore given. The definitions of \(\tau \) and \(\tau ^{\theta , {\bar{c}}, t}\) are unrelated but \(\tau ^{\theta , {\bar{c}}, t}\) converges to \(\tau \) in \(L^1\) and almost surely as \(\theta \rightarrow 0\), see Lemma 4.5. One interesting question raised by one of the reviewers is that if the limit \(\lim _{\theta \rightarrow 0} \frac{\tau ^{\theta , {\bar{c}},t}-\tau }{\theta }\) exists or not. The answer in general is negative. This can be seen by the following simple example. Assume \(t=0\) and \(q_0=T^2/2\). Assume the optimal control is \(c_t=T-t\) for \(t\in [0,T]\), which gives \(Q_r=q_0-\int _0^r c_s ds = (1/2)(T-r)^2\) for \(r\in [0,T]\) and \(Q_r=0\) if and only if \(r=\tau :=T\). Now consider a perturbation with \({\bar{c}}>T\) and \(0<\theta <T^2/(2{{\bar{c}}})\), which gives \(Q^{\theta , {\bar{c}},t}_r=q_0-{{\bar{c}}} r>0\) for \(r\in [0,\theta ]\) and \(Q^{\theta , {\bar{c}},t}_r=q_0-\int _0^r c^{\theta , {\bar{c}},t}_s ds = -{\bar{c}}\theta +T\theta - \theta ^2/2 + (T-r)^2/2\) for \(r\in [\theta , T]\) and \(Q^{\theta , {\bar{c}},t}_r=0\) if and only if \(r=\tau ^{\theta , {\bar{c}},t}:=T-\sqrt{\theta ^2-2T\theta +2{\bar{c}}\theta }\). We have \(\tau ^{\theta , {\bar{c}},t}\rightarrow \tau \) as \(\theta \rightarrow 0\) but
which shows the limit does not exist.
Remark 2.4
The standard SMP (c.f. Pham [14]) and Theorem 2.2 cannot be recovered from each other as they solve different problems. However, their statements are similar and only differ for the additional term \({\mathcal {G}}\) in (2.14).
Remark 2.5
The same result as Theorem 2.2 can be obtained in the case when the admissible set is bounded by above as well, i.e. when \(\pi \) is required to be in \(\pi \in [0,b]\) with \(0<b\le +\infty \). Although the proof does not change, the only remark we want to point out is on the admissibility of control \(c^{\theta ,{\bar{c}}, t}\). Since \({\bar{c}}\in [0,b]\) and \(c_r\in [0,b]\) for every \(r\in [0,T]\), then \(-\frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }=\frac{1}{\theta }\int _t^{(t+\theta )\wedge \tau }(c_r-{\bar{c}})dr\le \frac{(t+\theta )\wedge \tau -t}{\theta }b\le b\).
3 Example
In this section we describe an example to show that the usual SMP is not satisfied whereas Theorem 2.2 is satisfied. We consider an optimal liquidation problem with no market impact on trade and no terminal execution. In particular, let \(g=0\) and \(f(\pi ,x)=\pi x\). Assume the admissible control can only take values in the interval \([0,c^+]\). Let \(t\in [0,T]\) and \(Q_t=q>0\) be fixed. The stock price X satisfies the SDE: for \(r\in [t,T]\) and \(X_t=x>0\),
The agent aims to maximise his final cash value that is the cumulated liquidation wealth up to the stopping time at which the agent runs out the liquidating stocks and there is no residual value for any remaining stocks at horizon time T. The value function to this problem is defined by
where \(\tau ^\pi =T\wedge \min \{r\ge t\; | \; Q^\pi _r=0\}\) is the first hitting time of \(Q^\pi \) to zero or the fixed terminal time T, whichever comes first. We define an admissible control strategy c as follows: for any \(r\in [t,T]\),
The inventory \(Q_r\) in (2.1) is given by
We also have that for any \(r\in [t,T]\), \(Q_r\le c^+(T-r) \;\Leftrightarrow \; q\le c^+(T-t)\). Using the above expression for \(Q_r\), it is easy to check that the first hitting time of \(Q_r=0\) is
Since the stopping time \(\tau \) is equal to the terminal time T, it may look like the control in (3.2) is the same control we would have found in the usual setting without the stopping time. However, the optimal control in the usual case without stopping time would have been equal to \(c^+\). When \(q< c^+(T-t)\), at time \(r=t+\frac{q}{c^+}< T\), the inventory would have reached zero and from that time onward the inventory would have become negative, making the control \(c^+\) not feasible for our problem, see Remark 3.2.
Substituting (3.2) and \(\tau =T\) into (3.1), we can easily show that the value function associated with control (3.2) is equal to
Proposition 3.1
Function \(v^c\) in (3.3) coincides with the value function v of problem (3.1) and the strategy c in (3.2) is the optimal control.
Proof
By definition of \(v^c(t,x,q)\), we have that for any \(t\in [0,T]\), \(x>0\) and \(q>0\)
We now show that \(v^c\ge v\). Define
Simple calculus shows that
It can be easily verified that w satisfies the following HJB equation, for \(t\in [0,T]\) and \(q>0\)
w satisfies the boundary condition \(w(t,0)=0\) for any \(t\in [0,T]\) and the terminal condition \(w(T,q)=0\) for any \(q>0\).
Denote by \(c^*_r\) the optimal control for the value function v(t, x, q) in (3.1) and \(Q^*_r\) and \(\tau ^*\) the corresponding inventory and stopping time respectively. Using (2.1), for any \(r\in [t,T]\), we have
We have used (3.6) in the last inequality. Finally, from \(X_t=x\), \(Q_t^*=q\), \(v^c(t,x,q)=xw(t,q)\), using the stochastic integration by parts and (3.7), we have
However, using boundary and terminal condition \(v^c(t,x,0)=0\) for any \(t\in [0,T]\) and \(x>0\) and \(v^c(T,x,q)=0\) for any \(x>0\) and \(q>0\), we conclude that \(v^c(\tau ^*,X_{\tau ^*},Q^*_{\tau ^*})=0\), as either \(\tau ^*=T\) or \(Q^*_{\tau ^*}=0\). Therefore, taking conditional expectations on both sides of (3.8) and using the optional sampling theorem, we have that the expected value of the random variable \(\int _t^{\tau ^*} w(r,Q^*_r) X_r dW_r\) is equal to 0 and we get
Combining (3.4) and the previous expression, we conclude the proof. \(\square \)
Remark 3.2
In the standard version of the SMP (cf. Pham [14]) there should be 2 adjoint processes in the BSDE, referring respectively to processes X and Q. However, since the process X does not depend on control \(\pi \), the terms regarding the adjoint process referring to X can be removed from the Hamiltonian and, noting that \(g=0\) and \(f(\pi ,x)=\pi x\), those referring to Q are identically equal to 0. The necessary condition of the standard SMP is equivalent to
However, from (3.1) we get that the maximal point of \(f(\cdot ,x)\) is \({\bar{\pi }}=c^+\). We have shown that if \(q< c^+(T-t)\), then the optimal strategy is \(c_r=\frac{q}{T-t}\) for \(t\le r\le T\) as in (3.2), which is less than \({\bar{\pi }}=c^+\), a contradiction to (3.9) and the standard SMP.
We next verify that optimal control in (3.2) satisfies Theorem 2.2. We need to show that (2.13) holds true. Firstly, we observe that the model setup satisfies Assumption 2.1. Using the fact that \(\mu = 0\) and \(g=0\), we show that (2.13) holds true by proving that for any \({\bar{c}}\in [0,c^+]\), \(t\in [0,T)\),
We first find the expression for \({\bar{f}}(t,{\bar{c}},x,q)\) in the following proposition.
Proposition 3.3
Let \(t\in [0,T)\) be fixed and \((c_r)_{r\in [t,T]}\) be the optimal control in (3.2). Then, for any \({\bar{c}}\in [0,c^+]\), \(x>0\) and \(q>0\)
Proof
We consider any \(\theta \in (0,T-t)\), so that \(\tau =T> t+\theta \). Using the fact that \(c_r\) in (3.2) is constant in time, we have for any \(r\in [t+\theta , \tau ]\)
If \(q>c^+(T-t)\), then (3.2) implies that \(c_t=c^+\) and from (3.11),
Here we have used the fact that \({\bar{c}}\) is an admissible control and so \({\bar{c}}\in [0,c^+]\). The above expression implies that \(\tau ^{\theta ,{\bar{c}}, t}=T\) a.s.. On the other hand, if \(q\le c^+(T-t)\), then \(c_t=\frac{q}{T-t}\) and from (3.11), \(Q^{\theta ,{\bar{c}}, t}_T=-\theta ({\bar{c}}-c_t)\). Hence, if \({\bar{c}}\le c_t, Q^{\theta ,{\bar{c}}, t}_T\ge 0\) and so \(\tau ^{\theta ,{\bar{c}}, t}=T\) a.s.. If \({\bar{c}}> c_t, Q^{\theta ,{\bar{c}}, t}_T< 0\) and so \(\tau ^{\theta ,{\bar{c}}, t}<T\) a.s., so by setting (3.11) equal to 0 we get that \(\tau ^{\theta ,{\bar{c}}, t}=T-\theta \left( \frac{{\bar{c}}}{c_t}-1\right) \) a.s., where \(\theta \left( \frac{{\bar{c}}}{c_t}-1\right) >0\), since \({\bar{c}}>c_t\).
In conclusion, if \(q>c^+(T-t)\), then \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}_{\max }=T\) and we have that \({\bar{f}}=0\) from definition (2.12). If \(q\le c^+(T-t)\), then we consider two sub-cases. If \({\bar{c}}\le c_t\), then \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}_{\max }=T\), making \({\bar{f}}=0\) again. If \({\bar{c}}>c_t\), then \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=T-\theta \left( \frac{{\bar{c}}}{c_t}-1\right) \) and \(\tau ^{\theta ,{\bar{c}}, t}_{\max }=T\) and so
This concludes the proof of the proposition. \(\square \)
To prove (3.10), we split the proof of (3.10) in two parts. If \(q\le c^+(T-t)\), then the left side of (3.10) is equal to \({\bar{c}} X_t-c_t X_t-({\bar{c}}-c_t)X_t\mathbb {1}_{{\bar{c}}\ge c_t}=X_t\cdot \min \left( {\bar{c}}-c_t, \ 0\right) \le 0\). If \(q> c^+(T-t)\), then the left side of (3.10) is equal to \({\bar{c}} X_t-c_t X_t=({\bar{c}}-c^+)X_t\le 0\) as \({\bar{c}}\le c^+\). Hence, (3.10) is satisfied for any \(c^+\ge {\bar{c}}\ge 0\), and Theorem 2.2 holds.
Remark 3.4
The main purpose of this example is to show that the standard SMP cannot be applied when the terminal time is an indirectly controlled stopping time but our necessary SMP can accommodate that. It is in general difficult to find the optimal solution using Theorem 2.2 that has no particular advantage to the DPP for the example, but this is the first step in addressing the indirectly controlled random terminal time problem with the SMP which has the potential for solving the non-Markovian model, see Sect. 5 for possible further research.
4 Proof of Theorem 2.2
In this section we consider all assumptions of Theorem 2.2 are satisfied. (2.10) implies that \(\mathbb {E}\left[ \sup _{r\in [0,T]}c_r\right] <\infty \). As mentioned in the model setup, for any fixed time \(t\in [0,\tau )\), we have \(Q_t=q>0\) and so \(\tau >t\) a.s.. We consider a partition of the whole event space \(\{\tau >t\}\), which helps us in stating and proving some preliminary results that are needed in the proof of Theorem 2.2. As general hints for better understanding, we remind that \(\tau \) is defined so that \(Q_\tau =0\) if \(\tau <T\), \(Q_\tau \ge 0\) if \(\tau =T\), and \(Q_r>0\) if \(r\in [t,\tau )\). Similarly, \(Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}=0\) if \(\tau ^{\theta ,{\bar{c}}, t}<T\), \(Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}\ge 0\) if \(\tau ^{\theta ,{\bar{c}}, t}=T\), and \(Q^{\theta ,{\bar{c}}, t}_r>0\) if \(r\in [t,\tau ^{\theta ,{\bar{c}}, t})\). We first observe, using (2.5), that if \(\theta \in \left( 0, (T-t)\wedge \frac{q}{{\bar{c}}}\right) \), then for any \(r\in [t,(t+\theta )\wedge \tau ]\),
Therefore, if \(\theta \in \left( 0, (T-t)\wedge \frac{q}{{\bar{c}}}\right) \), then
Let \(t\in [0,\tau )\), \(0< \theta <(T-t)\wedge \frac{q}{{\bar{c}}} \), \({\bar{c}}\ge 0\) be fixed, we define the following partitions of \(\{\tau >t\}\):
We now present the properties of the different cases \(E^{\theta ,{\bar{c}}, t}_{i}\), for any \(i\in \{1,2,3\}\). In particular, for each of the events we show a scheme for the different values of quantities \(c^{\theta ,{\bar{c}}, t}\) and \(Q^{\theta ,{\bar{c}}, t}\) in each of the time spans. These schemes help in understanding some steps in the proof of lemmas below.
-
(1)
On the event \(E^{\theta ,{\bar{c}}, t}_{1}\):
From previous scheme we conclude that on the event \(E^{\theta ,{\bar{c}}, t}_{1}\)
$$\begin{aligned} 0=Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=Q_{\tau ^{\theta ,{\bar{c}}, t}}-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta } \ \Rightarrow \ Q_{\tau ^{\theta ,{\bar{c}}, t}}=\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }, \text { which also implies that } \gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }>0, \end{aligned}$$(4.2)since by definition of \(\tau \), for any \(r\in [t,\tau )\), \(Q_r>0\).
$$\begin{aligned}&\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \le \max \left( \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| ,\ |Q_{\tau ^{\theta ,{\bar{c}}, t}}|\right) =\sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| , \nonumber \\&\qquad \forall r\in [t,T], \end{aligned}$$(4.3)$$\begin{aligned}&Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_{\tau }\le Q_{\tau ^{\theta ,{\bar{c}}, t}}= \gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }. \end{aligned}$$(4.4) -
(2)
On the event \(E^{\theta ,{\bar{c}}, t}_{2}\): If \(\tau > t+\theta \)
If \(\tau \le t+\theta \), from (2.6), \(\gamma ^{\theta ,{\bar{c}}, t}_\tau =\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\)
From previous scheme we conclude that on the event \(E^{\theta ,{\bar{c}}, t}_{2}\),
$$\begin{aligned} Q^{\theta ,{\bar{c}}, t}_\tau =Q_\tau -\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }, \text { which implies } \gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }<0, \end{aligned}$$(4.5)since by definition of \(\tau ^{\theta ,{\bar{c}}, t}\), for any \(r\in [t,\tau ^{\theta ,{\bar{c}}, t})\), \(Q^{\theta ,{\bar{c}}, t}_r>0\). Moreover,
$$\begin{aligned}&\begin{aligned} \text {if } \tau ^{\theta ,{\bar{c}}, t}<T, \ 0=Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\left( 1-\frac{\tau ^{\theta ,{\bar{c}}, t}-\tau }{\theta }\right) \ \Rightarrow \ \tau ^{\theta ,{\bar{c}}, t}=\tau +\theta ,\\ \text {if } \tau ^{\theta ,{\bar{c}}, t}=T, \ 0\le Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\left( 1-\frac{\tau ^{\theta ,{\bar{c}}, t}-\tau }{\theta }\right) \ \Rightarrow \ \tau ^{\theta ,{\bar{c}}, t}\le \tau +\theta , \end{aligned} \end{aligned}$$(4.6)$$\begin{aligned}&\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \le \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| , \qquad \forall r\in [t,T], \end{aligned}$$(4.7)$$\begin{aligned}&\text {if } \tau ^{\theta ,{\bar{c}}, t}<T, \ Q_{\tau }^{\theta ,{\bar{c}}, t}-Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta },\nonumber \\&\text {if } \tau ^{\theta ,{\bar{c}}, t}=T, \ Q_{\tau }^{\theta ,{\bar{c}}, t}-Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}\le -\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }. \end{aligned}$$(4.8) -
(3)
On the event \(E^{\theta ,{\bar{c}}, t}_{3}\):
From previous scheme we conclude that on the event \(E^{\theta ,{\bar{c}}, t}_{3}\)
$$\begin{aligned}&\begin{aligned}&\text {if } Q_T=0, \ 0\le Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta },\\&\text {if } \tau =\tau ^{\theta ,{\bar{c}}, t}<T, \ 0= Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta },\\&\text {if } Q_T>0, \ 0\le Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=Q_{T}-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }, \end{aligned} \end{aligned}$$(4.9)$$\begin{aligned}&\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \le \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| , \qquad \forall r\in [t,T], \end{aligned}$$(4.10)$$\begin{aligned}&Q_{\tau }-Q_{\tau ^{\theta ,{\bar{c}}, t}}=0. \end{aligned}$$(4.11)
From previous schemes we derive the following Lemmas.
Lemma 4.1
Let \(t\in [0,\tau )\) be fixed, let \({\bar{c}}\ge 0\) and let \(\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right) \). Then the control \(c^{\theta ,{\bar{c}}, t}\) in (2.5) is admissible.
Proof
Firstly, we observe that control \(c^{\theta ,{\bar{c}}, t}_r\) is non-negative for any \(r\in [t,\tau )\). If \(\tau ^{\theta ,{\bar{c}}, t}> \tau \), i.e. if we are in the event \(E^{\theta ,{\bar{c}}, t}_2\), then using (4.5) we get that \(\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }<0\) and so the control \(c^{\theta ,{\bar{c}}, t}_r\) is non-negative for any \(r\ge \tau \) as well. Progressive measurability, right-continuity and square integrability of \(c^{\theta ,{\bar{c}}, t}\) immediately follow. \(\square \)
Lemma 4.2
Let \(t\in [0,\tau )\) be fixed, let \({\bar{c}}\ge 0\) and let \(\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right) \). Then
Proof
Looking at schemes on pages 9-10, it follows that on the event \(E_1^{\theta ,{\bar{c}}, t}\), \(\tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )=\tau ^{\theta ,{\bar{c}}, t}\), on the event \(E_2^{\theta ,{\bar{c}}, t}\cap \{ \tau >t+\theta \}\), \(\tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )=\tau \), on the event \(E_2^{\theta ,{\bar{c}}, t}\cap \{ \tau \le t+\theta \}\), \(\tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )=t+\theta \) and on the event \(E_3^{\theta ,{\bar{c}}, t}\), \(\tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )=\tau ^{\theta ,{\bar{c}}, t}=\tau \). Then, the result immediately follows.
\(\square \)
Lemma 4.3
Let \(t\in [0,\tau )\) and \({\bar{c}}\ge 0\) be fixed. Then
Proof
Let \(\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right) \) be fixed. From definition of \(\gamma ^{\theta ,{\bar{c}}, t}_r\) in (2.6) we immediately see that
Merging (4.3), (4.7) and (4.10), we see that
Therefore, merging (4.14) and previous expression we get that
We conclude the proof by using (2.10). \(\square \)
Lemma 4.4
Let \({\bar{c}}\ge 0\) and \(t\in [0,\tau )\) be fixed. Then
Proof
We firstly prove (4.15). We have that
In previous calculations we used that for any \(n\ge {\bar{n}}:=\left\lceil \frac{1}{(T-t)\wedge \frac{q}{{\bar{c}}}}\right\rceil \), the sequence of events \(\left\{ \tau \le t+\frac{1}{n}\right\} \) is decreasing. This concludes proof of (4.15).
We now prove (4.16). Using definition of Q, we have that under event \(E_1^{\theta ,{\bar{c}}, t}\), \(Q_{\tau }=Q_{\tau ^{\theta ,{\bar{c}}, t}}-\int _{\tau ^{\theta ,{\bar{c}}, t}}^{\tau } c_r dr\le Q_{\tau ^{\theta ,{\bar{c}}, t}}\). Moreover, if \(Q_T>0\), then it necessarily implies that \(\tau =T\). Using (4.2) we have that
In previous calculations we used that the sequence of events \(\Big (\Big \{Q_\tau \le \int _t^{t+\frac{1}{n}}\Big | {\bar{c}}-c_r \Big |dr\Big \}Big)_{n\ge {\bar{n}}}\) is decreasing and using right-continuity of c, \(\int _t^{t+\frac{1}{n}}\left| {\bar{c}}-c_r \right| dr\) converges to 0 a.s., as \(n\rightarrow \infty \). This concludes proof of (4.16).
We now prove (4.17). Using (4.2), we get
In previous calculations we used right-continuity of process c and that the sequence of events \(\left\{ c_t> \inf _{r\in \left[ t,t+\frac{1}{n}\right] }c_r\right\} _{n\ge {\bar{n}}}\) is decreasing. This concludes proof of (4.17).
We now prove (4.18). Using (4.6), we get
In previous calculations we used that the sequence of events \(\left\{ \tau +\frac{1}{n}\ge T\right\} _{n\ge {\bar{n}}}\) is decreasing. This concludes proof of (4.18).
We now prove (4.19). Using (4.9), we get
In previous calculations we used right-continuity of process c and that the sequence of events \(\left\{ c_t\le \sup _{r\in \left[ t,t+\frac{1}{n}\right] }c_r\right\} _{n\ge {\bar{n}}}\) is decreasing. This concludes the proof of (4.19). (4.20) can be proved similarly. \(\square \)
Lemma 4.5
Let \(t\in [0,\tau )\) and \({\bar{c}}\ge 0\) be fixed. Then
Proof
We firstly prove that \(\lim _{\theta \rightarrow 0} \tau ^{\theta ,{\bar{c}}, t}=\tau \) pointwise \(\mathbb {P}\)-almost everywhere. We assume on the contrary there exists a non-null event \({\mathcal {E}}\), so that \(\lim _{\theta \rightarrow 0} \left| \tau ^{\theta ,{\bar{c}}, t}-\tau \right| >0 \) on \({\mathcal {E}}\), which means that
Using that under event \(E^{\theta ,{\bar{c}}, t}_{1}\), \(\tau >\tau ^{\theta ,{\bar{c}}, t}\) and so \(\left| \tau -\tau ^{\theta ,{\bar{c}}, t}\right| > \gamma \) implies that \(\tau -\tau ^{\theta ,{\bar{c}}, t}> \gamma \), which implies \(Q_{\tau -\gamma }=Q_{\tau ^{\theta ,{\bar{c}}, t}}-\int _{\tau ^{\theta ,{\bar{c}}, t}}^{\tau -\gamma } c_r dr\le Q_{\tau ^{\theta ,{\bar{c}}, t}}=\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\). Moreover, using that under event \(E^{\theta ,{\bar{c}}, t}_{2}\), \(\tau ^{\theta ,{\bar{c}}, t}=(\tau +\theta )\wedge T\), \(\left| \tau -\tau ^{\theta ,{\bar{c}}, t}\right| > \gamma \) implies that \(\theta \ge (\tau +\theta )\wedge T-\tau > \gamma \), which is never verified, as \(\theta<{\bar{\theta }}<\gamma \). Moreover, under event \(E^{\theta ,{\bar{c}}, t}_{3}\), we have that \(\tau ^{\theta ,{\bar{c}}, t}=\tau \), which never satisfies \(\left| \tau -\tau ^{\theta ,{\bar{c}}, t}\right| > \gamma \). Therefore, we have that (4.22) implies that
Reminding that \(\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta } =\int _t^{(t+\theta )\wedge \tau }({\bar{c}}-c_r)dr\le \int _t^{(t+\theta )\wedge \tau }{\bar{c}}dr\le {\bar{c}}\theta \), expression (4.23) implies that \(Q_{\tau -\gamma }=0\) on \({\mathcal {E}}\), which contradicts definition of \(\tau \), as \(\tau \) should be the first time in which \(Q_r\) hits 0. Therefore, we conclude that \({\mathcal {E}}\) must be a set with 0 measure, which implies \(\mathbb {P}^t\)-almost everywhere pointwise convergence of \(\tau ^{\theta ,{\bar{c}}, t}\) to \(\tau \). To prove (4.21) we observe that \(\left| \tau ^{\theta ,{\bar{c}}, t}-\tau \right| \le T\), independently on \(\theta \). Applying the dominated convergence theorem (DCT) we get (4.21). \(\square \)
Lemma 4.6
Let \(t\in [0,\tau )\), \({\bar{c}}\ge 0\) and \(p\in [1,2)\) be fixed. Then
Proof
Let \(t\in [0,\tau )\) and \(\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right) \) be fixed. We firstly observe that
which is \(L^p\)-integrable thanks to assumption (2.10). Moreover, we have that
Therefore, by using right-continuity of control c and mean-value theorem, we conclude that the pointwise limit of the expression inside the expectation in (4.24) is 0. Finally, by using DCT we conclude the proof of (4.24).
Moreover, looking at schemes in pages pages 9-10, we can immediately see that
Therefore, by applying (4.24) and previous expression into (4.25) we prove the Lemma. \(\square \)
Lemma 4.7
Let \(t\in [0,\tau )\), \({\bar{c}}\ge 0\) and \(p\in [1,2)\) be fixed. Then
Proof
Let \(t\in [0,\tau )\) and \(\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right) \) be fixed. Using schemes in pp. 9–10, (4.2), (4.4), (4.6), (4.8), using Hölder’s inequality (with coefficients \(\frac{p+2}{2p}\) and \(\frac{p+2}{2-p}\)), reminding that \(E^{\theta ,{\bar{c}}, t}_{2}\) implies that \(Q_T=0\), we get
Taking the limit of the above expression and using (4.14), (4.16), (4.17), (4.18), (4.19), (4.20) and (4.24) we conclude the proof of the Lemma. \(\square \)
Lemma 4.8
Let \(t\in [0,\tau )\) and \({\bar{c}}\ge 0\) be fixed. Then,
Proof
Let \(\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right) \) be fixed. By using (4.4), (4.8) and (4.11), we have that
Taking the limit of the above expression and using (4.12) we conclude the proof of (4.26). We now prove (4.27). By (4.7) and (4.10), we have that
Taking the limit of the above expression and using (4.12) we conclude the proof of (4.27). \(\square \)
Lemma 4.9
For any \((x,q),(x,q')\in {\mathcal {O}}\) with \(q\ne q'\), we have that
Proof
We observe that
and so using Lipschitz continuity of \(\partial _q g\) in Assumption 2.8, we get
This proves the lemma. \(\square \)
We introduce a process used in the proof of Theorem 2.2. Define \((\xi _r)_{r\in [t,T]}\) as
\(\xi \) is the same as process \(\zeta \) in Bensoussan [3]. The corresponding part of z in Bensoussan [3] for process Q would be constantly equal to \({\bar{c}}-c_t\), as it can be inferred with a simple calculus.
Our proof relies on the arguments in Bensoussan [3] with one key difference. Due to the presence of the control-dependent stopping time \(\tau \) in our setting, it makes necessary the introduction of the stopping time \(\tau ^{\theta ,{\bar{c}}, t}\) as well. This complicates all the proofs and makes necessary many adjustments, especially on those results in Bensoussan [3] that concern terminal time T that must be adapted to \(\tau \) or \(\tau ^{\theta ,{\bar{c}}, t}\) accordingly.
Lemma 4.10
Let \(t\in [0,\tau )\) and \({\bar{c}}\ge 0\) be fixed. Then,
Proof
From (4.28) and using boundedness of \(\partial _q f\), we have that
Therefore, by taking the limit of previous expression and applying (4.21), we get the first limit in the statement. Moreover, from definition of \(Q_r\), we have
Therefore, by taking the limit of previous expression, using (2.10) and applying (4.21), we finish the proof of the Lemma. \(\square \)
Lemma 4.11
Let \(t\in [0,\tau )\) and \({\bar{c}}\ge 0\) be fixed. Then
Proof
Let \(\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right) \) be fixed. We denote for any \(r\in [t,T]\)
The proof of this lemma will be divided in 3 steps. In step 1 we prove that
In Step 2 we prove that,
In Step 3 we prove that
In Bensoussan [3] there is no difference between Steps 2 and 3, while in our case we need to consider them both. However, the structure of our proof resembles the one in the reference. Once the proof of the 3 steps is completed, we conclude the proof of the Lemma as follows. By merging the limits above in Steps 2 and 3, we have
Step 1. From (4.28), reminding that \(c_t^{\theta ,{\bar{c}}, t}={\bar{c}}\) and definition of \({\tilde{f}}\) we have that for any \(r\in [t,t+\theta ]\),
By taking \(r=t+\theta \) in previous expression, so that the second last line disappears and using Assumption 2.1, boundedness of \(\partial _q f\) and Hölder’s inequality, we get
Using DCT, \(\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| c_r-c_t\right| ^2\right] \) and \(\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| c_r^{\theta ,{\bar{c}}, t}-c_t^{\theta ,{\bar{c}}, t}\right| ^2\right] \) converge to 0 as \(\theta \rightarrow 0\). Indeed, c and \(c^{\theta ,{\bar{c}}, t}\) are right-continuous and thanks to (2.10) and (4.14), the arguments of the expectations converge to 0 a.s. and they are bounded by \(2\sup _{r\in [t,T]}|c_r|^2\) and \(2\sup _{r\in [t,T]}|c_r^{\theta ,{\bar{c}}, t}|^2\), which are \(L^1\)-integrable processes. \(\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| X_r-X_t\right| ^2\right] \) converges to 0 using standard arguments in SDE theory (c.f. Krylov [11, Corollary 2.5.12]). Moreover, using \(L^2\)-integrability of c and \(c^{\theta ,{\bar{c}}, t}\) and standard arguments in SDE theory (c.f. Krylov [11, Corollary 2.5.12]), we get that \(\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left( 2|X_r|+2|c_r^{\theta ,{\bar{c}}, t}|\right) ^2\right] \) and \(\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left( 2|X_r|+2|c_r|\right) ^2\right] \) are bounded independently of \(\theta \). Moreover, by definition of \(Q_r\),
which converges to 0, thanks to (2.10). Using (4.13), we have that \(\mathbb {E}^t\Big [\Big (\sup _{r\in [t,t+\theta ]}\Big |Q_r^{\theta ,{\bar{c}}, t}-Q_r\Big |\Big )\Big ]\) converges to 0. Finally, we observe that \(\frac{2}{\theta }\int _t^{t+\theta } |s-t|ds=\theta \). Therefore, by taking limit of (4.30) we conclude the proof of Step 1.
Step 2. From (4.29), using Assumption 2.1 and boundedness of \(\partial _q f\), we get a similar expression to (4.30). Step 2 can be proved in a similar way to Step 1, the main difference is the term \(\mathbb {E}^t\left[ \left| \frac{\tau ^{\theta ,{\bar{c}}, t}_{\min }-t}{\theta }-1\right| \mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }\le t+\theta }\right] \). However, by reminding that by (4.1), \(\left\{ \tau ^{\theta ,{\bar{c}}, t}_{\min }\le t+\theta \right\} =\left\{ \tau \le t+\theta \right\} \), that under event \(\tau ^{\theta ,{\bar{c}}, t}_{\min }\le t+\theta \), then \(\left| \frac{\tau ^{\theta ,{\bar{c}}, t}_{\min }-t}{\theta }-1\right| \le 1\) and by using (4.15), we conclude that
This concludes the proof of Step 2.
Step 3. From (4.28) and definition of \({\tilde{f}}\) we have that for any \(r\in [t+\theta , T]\),
Therefore, by applying Assumption 2.1, then boundedness and Lipschitz continuity of \(\partial _q f\) follows, we have that
Therefore, from previous expression and using Lemma 4.2, we get that
By taking limit of the above expression for \(\theta \rightarrow 0\), by using (4.12) and (4.24) together with Step 1, we conclude the proof of Step 3 and the proof of the Lemma as well. \(\square \)
Proof of Theorem 2.2. Let \(t\in [0,\tau )\) be fixed. Since control c is optimal, it necessarily follows that for any \({\bar{c}}\ge 0\) and for any \(\theta >0\)
where
and \(\tau ^\pi =T\wedge \min \{r\ge t\; | \; Q_r^\pi =0\}\). We write Q as \(Q^c\) and \(\tau \) as \(\tau ^c\) as in Theorem 2.2.
Therefore, we necessarily have that for any \({\bar{c}}\ge 0\)
provided the limit exists. By definition of \(v^\pi \), reminding that when \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}\), then for \(r\ge \tau ^{\theta ,{\bar{c}}, t}\), \({\hat{Q}}^{\theta ,{\bar{c}}, t}_r=Q_r\) and \({\hat{c}}^{\theta ,{\bar{c}}, t}_r=c_r\) and when \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau \), then for \(r\ge \tau \), \({\hat{Q}}^{\theta ,{\bar{c}}, t}_r=Q_r^{\theta ,{\bar{c}}, t}\) and \({\hat{c}}^{\theta ,{\bar{c}}, t}_r=c_r^{\theta ,{\bar{c}}, t}\)
The first line on the right-hand side of (4.32) can be written as
Reminding that when \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}\), then \({\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}=Q_{\tau ^{\theta ,{\bar{c}}, t}}\) and \({\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau }=Q_\tau \) and when \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau \), then \({\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}=Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}\) and \({\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau }=Q^{\theta ,{\bar{c}}, t}_{\tau }\), then we have that the last element on the right-hand side of (4.33) is equal to 0. The first element on the right-hand side of (4.33) is equal to \(-{\bar{g}}(t,{\bar{c}},x,q)\) by its definition (2.11). We define \({\tilde{g}}\) for any \((x,q)\in {\mathcal {O}}, (x,q')\in {\mathcal {O}}\) as
From Assumption 2.8 we have that \({\tilde{g}}\) is bounded by \(K(1+|x|)\). The second element on the right-hand side of (4.33) is equal to
Using Hölder’s inequality, boundedness of \({\tilde{g}}\) and (4.25), we get
Here we used standard arguments of SDE theory, i.e. \(\mathbb {E}^t\left[ \sup _{r\in [t,T]}|X_r|^4\right] <\infty \). Moreover, using Lemma 4.9 together with definition of \({\tilde{g}}\), we get that
where in the last line we used (4.13) in Lemma 4.3. Moreover, using Lipschitz continuity of \(\partial _q g\) Lemma (4.10) and that either \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}\) or \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau \), we get that
Hence, merging the last three expressions above into (4.34), we get
The third element on the right-hand side of (4.33) is equal to
Using Hölder’s inequality, boundedness of \({\tilde{g}}\) and Lemma 4.7, we get
Here we used standard arguments of SDE theory, i.e. \(\mathbb {E}^t\left[ \sup _{r\in [0,T]}|X_r|^4\right] <\infty \). Moreover, using Lemma 4.9 together with definition of \({\tilde{g}}\), we get that
where in the last line we used (4.26) in Lemma 4.8. Moreover, using Lipschitz continuity of \(\partial _q g\) in Assumption 2.8, we have that
where in the last equality we have used (4.27) in Lemma 4.8. Hence, merging the last three expressions above into (4.36), we get
Combining (2.11), (4.35) and the above expression into (4.33), we conclude that the first line of the right-hand side of (4.32) is equal to
The second and third lines of right-hand side of (4.32) can be written as
Using Lemma 4.11, we have that
Using Lemma 4.10 ad reminding that either \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}\) or \(\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau \), we have that
Using (2.12), the third limit on the right-hand side of (4.38) converges to \({\bar{f}}(t,{\bar{c}},x,q)\). Combining the above two expressions and (2.12) into (4.38), we get that
Then, merging (4.32) with (4.37) and the above expression, also noting \(X_t=x\) and \(Q_t=q\), we get
where \({\mathcal {G}}(t,{\bar{c}},x,q)\) is defined in (2.14).
However, from (2.7) and (4.28), we have
In the last equality we have used the Optional Stopping Theorem, which ensures that \(\int _t^\cdot Z_r dW_r\) is a martingale, whose conditional expectation is 0. Substituting the above expression into (4.39) and using the optimality condition (4.31), we get that for any \({\bar{c}}\ge 0\) and \(t\in [0,\tau )\)
Since the argument of the first conditional expectation is \({\mathcal {F}}^t\)-measurable, using \({\mathcal {H}}\) in (2.9), we get the Hamiltonian condition (2.13). This concludes the proof of Theorem 2.2.
5 Conclusions
In this paper we have proved a new SMP (Theorem 2.2) for an optimal liquidation problem with control-dependent terminal time, which is markedly different in the Hamiltonian condition from that of the standard SMP. We have given a simple example to show that the optimal solution satisfies the SMP in Theorem 2.2 but not the standard SMP in the literature. This is only the first step in the direction of SMP for control-dependent stopping time problems and there remain many open questions to be answered, for example, existence of pointwise limits (2.11) and (2.12), sufficient SMP for optimality, a jump diffusion control-dependent model for \(\mathbf {X}\) process, and applications to concrete financial scenarios. We leave these and other questions for future research.
References
Ankirchner, S., Jeanblanc, M., Kruse, T.: BSDEs with singular terminal condition and a control problem with constraints. SIAM J. Control. Optim. 52(2), 893–913 (2014)
Barbu, V., Röckner, M.: Backward uniqueness of stochastic parabolic like equations driven by Gaussian multiplicative noise. Stoch. Process. Appl. 126(7), 2163–2179 (2016)
Bensoussan, A.: Lectures on stochastic control: Variational methods in stochastic control. In: Mitter, S.K., Moro, A. (eds.) Nonlinear Filtering and Stochastic Control. Lecture Notes in Mathematics, pp. 1–62. Springer, Berlin (1982)
Cartea, A., Jaimungal, S., Penalva, J.: Algorithmic and High-frequency Trading. Cambridge Press, Cambridge (2015)
Cordoni, F., Di Persio, L.: A maximum principle for a stochastic control problem with multiple random terminal times. Math. Eng. 2, 557–583 (2020)
Cordoni, F., Di Persio, L., Maticiuc, L., Zǎlinescu, A.: A stochastic approach to path-dependent nonlinear Kolmogorov equations via BSDEs with time-delayed generators and applications to finance. Stoch. Process. Appl. 130(3), 1669–1712 (2020)
Darling, R.W.R., Pardoux, E.: Backwards SDE with random terminal time and applications to semilinear elliptic PDE. Ann. Probab. 25(3), 1135–1159 (1997)
Diomande, B., Maticiuc, L.: Multivalued stochastic delay differential equations and related stochastic control problems. Quaest. Math. 40(6), 769–802 (2017)
Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Springer, Berlin (2006)
Horst, U., Naujokat, F.: When to cross the spread? Trading in two-sided limit order books. SIAM J. Financ. Math. 5(1), 278–315 (2014)
Krylov, N.V.: Controlled Diffusion Processes. Springer, New York (1980)
Lee, T., Kim, H., Chung, H., Bang, Y., Myung, H.: Energy efficient path planning for a marine surface vehicle considering heading angle. Ocean Eng. 107, 118–131 (2015)
Oksendal, B., Sulem, A.: Applied Stochastic Control of Jump Diffusions. Springer, Berlin (2005)
Pham, H.: Continuous-Time Stochastic Control and Optimization with Financial Applications. Springer, Berlin (2009)
Pham, H.: Stochastic control under progressive enlargement of filtrations and applications to multiple defaults risk management. Stoch. Process. Appl. 120(9), 1795–1820 (2010)
Popier, A., Zhou, C.: Second-order BSDE under monotonicity condition and liquidation problem under uncertainty. Ann. Appl. Probab. 29(3), 1685–1739 (2019)
Subramani, D.N., Wei, Q.J., Lermusiaux, P.F.: Stochastic time-optimal path-planning in uncertain, strong, and dynamic flows. Comput. Methods Appl. Mech. Eng. 333, 218–237 (2018)
Wu, Z.: Fully coupled FBSDE with Brownian motion and Poisson process in stopping time duration. J. Aust. Math. Soc. 74(2), 249–266 (2003)
Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, Berlin (1999)
Acknowledgements
The authors are very grateful to two anonymous reviewers whose constructive comments and suggestions have helped to improve the paper of the previous three versions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
H. Zheng: Supported in part by the EPSRC (UK) Grant (EP/V008331/1)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Cesari, R., Zheng, H. Stochastic Maximum Principle for Optimal Liquidation with Control-Dependent Terminal Time. Appl Math Optim 85, 43 (2022). https://doi.org/10.1007/s00245-022-09848-1
Accepted:
Published:
DOI: https://doi.org/10.1007/s00245-022-09848-1
Keywords
- Stochastic maximum principle
- Control-dependent terminal time
- Optimal liquidation
- Variational analysis
- Backwards stochastic differential equations