Stochastic Maximum Principle for Optimal Liquidation with Control-Dependent Terminal Time

Abstract

In this paper we study a general optimal liquidation problem with a control-dependent stopping time which is the first time the stock holding becomes zero or a fixed terminal time, whichever comes first. We prove a stochastic maximum principle (SMP) which is markedly different in its Hamiltonian condition from that of the standard SMP with fixed terminal time. We present a simple example in which the optimal solution satisfies the SMP in this paper but fails the standard SMP in the literature.

1 Introduction

Two main approaches to solving stochastic optimal control problems are the partial differential equation (PDE) method based on the dynamic programming principle (DPP) and the backward stochastic differential equation (BSDE) method based on the stochastic maximum principle (SMP), see Fleming and Soner [9], Yong and Zhou [19] for expositions. Most literature is on fixed terminal time problems. When the underlying state process is a controlled diffusion process, one may find the optimal solution with the Hamilton–Jacobi–Bellman (HJB) equation (a nonlinear PDE) or the BSDE (a coupled forward-backward system connected with the Hamiltonian condition). When the underlying state process is a controlled jump-diffusion process, one may find the optimal solution with the HJB partial differential difference equation or the BSDE, see Oksental and Sulem [13] for model formulations and solution methodologies. When the terminal time is not fixed but a random stopping time that is determined directly by a decision-maker, one has an optimal stopping problem or a combined optimal control/stopping problem and may find the optimal solution with the HJB variational equation or the reflected BSDE, see Pham [14] for an excellent concise introduction of this and other topics above, see also Barbu and Röckner [2], Cordoni et al. [6], Diomande and Maticiuc [8], Popier and Zhou [16] for results on existence and uniqueness of forward backward and delayed stochastic differential equations and second order BSDEs and applications.

In this paper we investigate a stochastic optimal control problem with a random terminal time. In contrast to optimal stopping problem, the terminal time is indirectly determined by control strategies. Specifically, we consider an optimal liquidation problem in which the terminal time is determined by the first time the stock holding becomes zero or a fixed terminal time, whichever comes first. The objective is to maximize the expected cash value of the liquidation at the terminal time subject to some other underlying state process (stock price, volatility, etc.) dynamics. Such a model cannot be cast into the framework of the optimal stopping problem as stopping time is not directly controlled nor the jump-diffusion model with the fixed terminal time as stopping time is random. For a Markovian model, one can show that the value function satisfies the HJB equation with the boundary condition when the stock holding is zero as well as the terminal condition at the fixed terminal time, see for example Cartea et al. [4]. Since the HJB equation is a nonlinear PDE, one can find the closed-form solution or show the existence of a classical solution only for some specific models and has to rely on the viscosity solution concept for general models. For a non-Markovian model or with control constraints, the HJB approach loses its tractability. On the other hand, the SMP approach provides an alternative way of solving the problem for general, possibly non-Markovian, models. However, the standard SMP only applies to the problem with fixed terminal time. The first huddle we must overcome is to find the form of the BSDE and the associated Hamiltonian condition for the problem with random control-dependent terminal time.

There have been some efforts in the literature to address optimal trading. Ankirchner et al. [1] use the BSDE approach to solving the singular terminal state problem in which the terminal inventory is forced to be zero at terminal time, differently from our setting in which this constraint is weakened. Horst and Naujokat [10] state a version of the SMP for optimal trading strategy with the spread driven by a jump diffusion process. Pham [15] studies a model with multiple stopping times, a special form of the jump-diffusion process, and characterizes the value function with a system of backward recursive dynamic programming equations and the optimal control with progressive enlargement of filtration.

Cordoni and Di Persio [5] study a similar model to [15] and derive a system of backward recursive BSDEs that are similar to the standard SMP over adjacent stopping times intervals. There is, however, a key difference in the stopping time definitions in Pham [15] and Cordoni and Di Persio [5]. The former is independent of controls and is given by some driving jump processes whereas the latter depends on controls and is given by the first time the underlying controlled state process hits some deterministic boundaries. In the standard derivation of SMP with fixed terminal time (c.f. Bensoussan [3] and Pham [14]), one may use the optimality condition and the variation of the optimal control to derive the BSDE and the Hamiltonian condition. When the terminal time is a stopping time depending on control, then variational analysis involves changes in terminal time as well as underlying state variables, unlike that for the standard SMP that only involves changes in underlying state variables. This aspect is the most arduous difficulty that needs to be overcome in the proof of the SMP with control dependent terminal time. Cordoni and Di Persio [5] prove the SMP without discussing the possibility of changes of terminal stopping times due to changes of controls, see for example [5, Eq. (15)] in the proof of necessary SMP and [5, Eq. (24)] in the proof of sufficient SMP, which implies the SMP in [5] is only valid for a model with stopping times independent of controls, same as that of [15], but invalid for control-dependent stopping times, that is, the BSDE and the Hamiltonian condition in [5] are not applicable to the stopping time definition there.

The main contribution of this paper is to give the SMP in the presence of random control-dependent terminal time, the first in the literature to the best knowledge of the authors. The main theorem (Theorem 2.2) states that the adjoint process satisfies a standard BSDE [see (2.7)] with the terminal time the optimal stopping time, not necessarily the fixed terminal time, and the Hamiltonian function is also a standard one [see (2.9)], but the Hamiltonian condition is markedly different from that of the standard SMP with fixed terminal time [see (2.13)], specifically, we need to add a supplementary nonlinear term that is not additively separable between the optimal control and any other controls. This additional term counts for the random control-dependent terminal time. We give a simple example to show that the optimal solution satisfies the SMP in Theorem 2.2 but not the standard SMP, e.g., the one in [14]. The SMP in this paper only applies to the optimal liquidation problem, but the idea of the variational analysis involving controlled stopping time may be explored further for more general models with other applications, for example, the optimal path planning problem that steers autonomous vehicles to navigate between any two points while optimizes energy, time, etc., see Lee et al. [12] and Subramani et al [17].

The rest of the paper is organized as follows. In Sect. 2 we describe the model and state the main result (Theorem 2.2) that is the SMP with random control-dependent terminal time. In Sect. 3 we present an example to illustrate the main result and show the standard SMP does not hold. In Sect. 4 we prove Theorem 2.2. Section 5 concludes.

2 Model Setup

Let $(\Omega , {\mathcal {F}}, ({\mathcal {F}}_t)_{t\in [0,T]}, \mathbb {P})$ be a filtered probability space, where $({\mathcal {F}}_t)_{t\in [0,T]}$ is the natural filtration generated by an m-dimensional standard Brownian motion $\mathbf {W}$, augmented by all $\mathbb {P}$-null sets. Let T be the fixed terminal time. Let $(\pi _t)_{t\in [0,T]}$ denote the rate of selling the stock, which is a decision (control) variable selected by the agent and is said admissible if it is a progressively measurable, non-negative, right-continuous and square integrable process. Denote by ${\mathcal {A}}$ the set of all admissible control processes. We consider $\pi$ to be the liquidation rate of the inventory $Q_t$, defined as

$$\begin{aligned} Q_t^\pi =q_0-\int ^t_0 \pi _r \; dr . \end{aligned}$$

(2.1)

Let $(\mathbf {X}_t)_{t\in [0,T]}$ be an $\mathbb {R}^n$ valued stochastic process satisfying the following stochastic differential equation (SDE):

$$\begin{aligned} d\mathbf {X}_t={\varvec{\mu }}(t,\mathbf {X}_t)dt+{\varvec{\sigma }}(t,\mathbf {X}_t)d\mathbf {W}_t, \end{aligned}$$

(2.2)

with initial condition $\mathbf {X}_0=\mathbf {x}$, where ${\varvec{\mu }}:[0,T]\times \mathbb {R}^n\rightarrow \mathbb {R}^n$, ${\varvec{\sigma }}:[0,T]\times \mathbb {R}^n\rightarrow \mathbb {R}^{n\times m}$ are two continuous functions. $\mathbf {X}$ represents the market information such as stock price, volatility, etc., and is not influenced by the control process $\pi$ (liquidation without price impact). The optimal liquidation problem is defined by

$$\begin{aligned} \sup _{\pi \in {\mathcal {A}}}\mathbb {E}\left[ g( \mathbf {X}_{\tau ^\pi }, Q_{\tau ^\pi }^\pi )+ \int _0^{\tau ^\pi } f(r, \pi _r, \mathbf {X}_r, Q_r^\pi ) \;dr \right] , \end{aligned}$$

(2.3)

where $g:\mathbb {R}^n\times \mathbb {R}\rightarrow \mathbb {R}$ and $f:[0,T]\times [0,\infty )\times \mathbb {R}^n\times \mathbb {R}\rightarrow \mathbb {R}$ are two continuously differentiable functions representing the terminal and running payoffs, $\tau ^\pi$ is a stopping time defined by

$$\begin{aligned} \tau ^\pi =T\wedge \min \left\{ r\ge 0\; | \; Q_r^\pi =0\right\} , \end{aligned}$$

(2.4)

the first time when all stock is liquidated or fixed terminal time T, whichever comes first.

To simplify the notation we consider a one-dimensional process X, but all results can be obtained in the multi-dimensional case. We denote the state space of the pair $(X_r,Q_r)$ as ${\mathcal {O}}:=\mathbb {R}\times [0,q_0]$. In the following, with a slight abuse of notations, we denote $\pi _r$ as $\pi _r\mathbb {1}_{r\le \tau ^\pi }$, which equals 0 after $\tau ^\pi$. Similarly, $Q^\pi _r=0$ for $r> \tau ^\pi$. Whenever we refer to a time interval [a, b), if $a\ge b$, then we consider it to be an empty set.

To state and prove a necessary SMP for problem (2.3), we follow the procedure in Bensoussan [3]. Assume that c is the optimal control and Q and $\tau$ are the corresponding optimal inventory and stopping time, defined in (2.1) and (2.4) respectively. If $Q_0=0$ then $\tau =0$ and there is nothing to discuss. We assume $Q_0>0$ which implies $\tau >0$. We next define the variation of the optimal control c. For fixed $t\in [0,\tau )$ and ${\bar{c}} \ge 0$, we have $q:=Q_t>0$. Choose $\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right)$, that is, $t<t+\theta <T$ and $q-{{\bar{c}}}\theta >0$, consider a variation of c as follows:

$$\begin{aligned} c^{\theta ,{\bar{c}}, t}_r:=c_r\mathbb {1}_{r\in [0, t)} + {\bar{c}}\mathbb {1}_{r\in [t, (t+\theta )\wedge \tau )}+c_r\mathbb {1}_{r\in [t+\theta ,\tau )}-\frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }\mathbb {1}_{r\in [\tau , +\infty )}, \end{aligned}$$

(2.5)

where $\mathbb {1}_S$ is an indicator that equals 1 if S is true and 0 otherwise, and for $r\ge t$,

$$\begin{aligned} \gamma ^{\theta ,{\bar{c}}, t}_r:=\int _t^{r\wedge \tau }({\bar{c}}-c_s)ds. \end{aligned}$$

(2.6)

The control $c^{\theta ,{\bar{c}}, t}$ in (2.5) is an admissible control, see Lemma 4.1. Let $Q^{\theta ,{\bar{c}}, t}_r$ be the corresponding inventory under the control $c^{\theta ,{\bar{c}}, t}$, given by (2.1), and $\tau ^{\theta ,{\bar{c}}, t}$ be the first hitting time r when the inventory $Q^{\theta ,{\bar{c}}, t}_r$ gets to 0, given by (2.4).

Since $c^{\theta ,{\bar{c}}, t}_r=c_r$ for $r\in [0, t)$, we have $Q^{\theta ,{\bar{c}}, t}_r=Q_r$ for $r\in [0, t)$, in particular, $Q^{\theta ,{\bar{c}}, t}_t=Q_t$ and $\tau ^{\theta ,{\bar{c}}, t}>t$. In fact, with the condition on $\theta$, $Q^{\theta ,{\bar{c}}, t}_{(t+\theta )\wedge \tau } = Q_t - {{\bar{c}}}((t+\theta )\wedge \tau -t)\ge q-{{\bar{c}}}\theta >0$, so $\tau ^{\theta ,{\bar{c}}, t}>(t+\theta )\wedge \tau$. The term $\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }$ represents the difference of the total liquidation on the time interval $[t,(t+\theta ) \wedge \tau ]$ with the constant control ${{\bar{c}}}$ and with the optimal control c, which determines the relation of $\tau$ and $\tau ^{\theta ,{\bar{c}}, t}$. If $\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }>0$ then $t+\theta<\tau ^{\theta ,{\bar{c}}, t}<\tau$ when $t+\theta<\tau <T$, specifically, $\tau ^{\theta ,{\bar{c}}, t}$ is the time r when the optimal inventory $Q_r$ is equal to $\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }$, see Fig. 1a. If $\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }<0$ then $\tau ^{\theta ,{\bar{c}}, t}>\tau$, see Fig. 1b. Note that we denote $c_r^{\theta ,{\bar{c}}, t}$ as $c_r^{\theta ,{\bar{c}}, t}\mathbb {1}_{r\le \tau ^{\theta ,{\bar{c}}, t}}$, which equals 0 after $\tau ^{\theta ,{\bar{c}}, t}$. We see that $c_r^{\theta ,{\bar{c}}, t}$ is well defined in (2.5). Indeed, when $\tau \ge \tau ^{\theta ,{\bar{c}}, t}$, the last term in (2.5) disappears, while if $\tau <\tau ^{\theta ,{\bar{c}}, t}$, the quantity $\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }$ is negative, making the last term in (2.5) a non-negative term.

Fig. 1

Graphical examples of c (in blue) and $c^{\theta ,{\bar{c}}, t}$ (in red)

Let $(Y_r,Z_r)_{r\in [0,\tau ]}$ be a solution of the following BSDE:

$$\begin{aligned} {\left\{ \begin{array}{ll} -dY_r=\partial _q f(r, c_r,X_r,Q_r) dr-Z_r dW_r\\ Y_\tau =\partial _q g( X_\tau , Q_\tau ). \end{array}\right. } \end{aligned}$$

(2.7)

The processes (Y, Z) are conventional in the usual SMP formulation (c.f. Bensoussan [3] and Pham [14]). BSDE (2.7) has a random terminal time $\tau$, in contrast to the standard BSDE with fixed terminal time. This type of BSDEs has been studied in the literature, c.f. Darling and Pardoux [7] and in Wu [18].

We assume $\mu , \sigma , f, g$ satisfy the following conditions for some positive constant K.

Assumption 2.1

For any $t,t'\in [0,T]$, $\pi ,\pi '\ge 0$, $x,x'\in \mathbb {R}$, $q,q'\ge 0$,

$$\begin{aligned} \begin{aligned}&\left| \mu (t,x)-\mu (t,x') \right| + \left| \sigma (t,x)-\sigma (t,x') \right| \le K\left| x-x' \right| ,\\&\left| \mu (t,x) \right| + \left| \sigma (t,x) \right| \le K\left( |x|+1 \right) ,\\&\left| g(x,q)-g(x,q') \right| \le K(1+\left| x \right| )\left| q-q' \right| ,\\&\left| f(t,\pi ,x,q)-f(t',\pi ,x,q') \right| \le K\left( \left| q-q' \right| +\left| t-t' \right| \right) ,\\&\left| f(t,\pi ,x,q)-f(t,\pi ',x',q) \right| \le K\left( \left| x-x' \right| +\left| \pi -\pi ' \right| \right) \left( 1+|x|+|x'| +|\pi |+|\pi '| \right) ,\\&\left| \partial _q f(t,\pi ,x,q)-\partial _q f(t,\pi ,x,q') \right| +\left| \partial _q g(x,q)-\partial _q g(x,q') \right| \le K|q-q'|. \end{aligned} \end{aligned}$$

(2.8)

Define the Hamiltonian as

$$\begin{aligned} {\mathcal {H}}(t,\pi ,x,q,y):=-\pi y+f(t,\pi ,x,q). \end{aligned}$$

(2.9)

Denote by

$$\begin{aligned} \begin{aligned}&\tau _{\min }^{\theta ,{\bar{c}}, t}:=\min \left( \tau , \tau ^{\theta ,{\bar{c}}, t}\right) ,\ \tau _{\max }^{\theta ,{\bar{c}}, t}:=\max \left( \tau , \tau ^{\theta ,{\bar{c}}, t}\right) ,\\&{\hat{Q}}^{\theta ,{\bar{c}}, t}_r:=\max \left( Q_r, Q^{\theta ,{\bar{c}}, t}_r\right) ,\ {\hat{c}}^{\theta ,{\bar{c}}, t}_r:=\max \left( c_r, c^{\theta ,{\bar{c}}, t}_r\right) . \end{aligned} \end{aligned}$$

We now state the stochastic maximum principle for problem (2.3).

Theorem 2.2

Let Assumption 2.1 be satisfied. Let $(c_r)_{r\in [0,T]}$ be the optimal control for problem (2.3), satisfying

$$\begin{aligned} \mathbb {E}\left[ \sup _{r\in [0,T]}c_r^2\right] <\infty . \end{aligned}$$

(2.10)

Let $(Q_r)_{r\in [0,T]}$ and $(X_r)_{r\in [0,T]}$ be the corresponding solutions to SDEs (2.1) and (2.2) with $Q_0>0$, $\tau >0$ the corresponding stopping time in (2.4), and $(Y_r, Z_r)_{r\in [0,\tau ]}$ the solution to BSDE (2.7). Assume that there exist $\mathbb {R}$-valued functions ${\bar{g}}(t,{\bar{c}},x,q)$ and ${\bar{f}}(t,{\bar{c}},x,q)$ so that for any $t\in [0,\tau )$, $(x,q)\in {\mathcal {O}}$ and ${\bar{c}}\ge 0$,

$$\begin{aligned} {\bar{g}}(t,{\bar{c}},x,q)&=\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \frac{g(X_{\tau },Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})-g(X_{\tau ^{\theta ,{\bar{c}}, t}},Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})}{\theta }\right] , \end{aligned}$$

(2.11)

$$\begin{aligned} {\bar{f}}(t,{\bar{c}},x,q)&=\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \frac{\text {sign}(\tau -\tau ^{\theta ,{\bar{c}}, t})}{\theta }\int _{\tau _{\min }^{\theta ,{\bar{c}}, t}}^{\tau _{\max }^{\theta ,{\bar{c}}, t}}f\left( r, {\hat{c}}^{\theta ,{\bar{c}}, t}_r, X_r,{\hat{Q}}^{\theta ,{\bar{c}}, t}_r\right) \; dr\right] , \end{aligned}$$

(2.12)

where $\mathbb {E}^t[\cdot ]=\mathbb {E}[\cdot | X_t=x, \ Q_t=q]$ is the conditional expectation at time t. Then, c necessarily satisfies for $t\in [0,\tau )$, ${\bar{c}}\ge 0$,

$$\begin{aligned} {\mathcal {H}}(t,{\bar{c}},X_t,Q_t,Y_t)-{\mathcal {H}}(t,c_t,X_t,Q_t,Y_t)+{\mathcal {G}}(t,{\bar{c}},X_t,Q_t)\le 0 \quad \text {a.s.}, \end{aligned}$$

(2.13)

where ${\mathcal {G}}(t,{\bar{c}},x,q)$ is defined as

$$\begin{aligned} {\mathcal {G}}(t,{\bar{c}},x,q):=({\bar{c}}-c_t)\mathbb {E}^t\left[ \partial _q g(X_\tau , Q_\tau ) \mathbb {1}_{\Lambda (t,{\bar{c}})}\right] -{\bar{g}}(t,{\bar{c}},x,q)-{\bar{f}}(t,{\bar{c}},x,q) \end{aligned}$$

(2.14)

and $\Lambda (t,{\bar{c}})$ as

$$\begin{aligned} \Lambda (t,{\bar{c}}):=\left( \{Q_T=0\}\cap \{{\bar{c}}\ge c_t\}\right) \cup \left( \{\tau<T\}\cap \{{\bar{c}}< c_t\}\right) . \end{aligned}$$

Remark 2.3

The definition of ${\bar{g}}$ in (2.11) is asymmetric in the arguments of functions g. We may define ${\bar{g}}$ in a symmetric way as

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \frac{g(X_{\tau },Q_\tau )-g(X_{\tau ^{\theta ,{\bar{c}}, t}},Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})}{\theta }\right] . \end{aligned}$$

To get an analogy of the Hamiltonian condition in Theorem 2.2, we have to define ${\bar{g}}$ as in (2.11). This point is illustrated in (4.33) in the proof of Theorem 2.2. Note also that stopping time $\tau$ is determined by the optimal control c as in (2.4) and is therefore given. The definitions of $\tau$ and $\tau ^{\theta , {\bar{c}}, t}$ are unrelated but $\tau ^{\theta , {\bar{c}}, t}$ converges to $\tau$ in $L^1$ and almost surely as $\theta \rightarrow 0$, see Lemma 4.5. One interesting question raised by one of the reviewers is that if the limit $\lim _{\theta \rightarrow 0} \frac{\tau ^{\theta , {\bar{c}},t}-\tau }{\theta }$ exists or not. The answer in general is negative. This can be seen by the following simple example. Assume $t=0$ and $q_0=T^2/2$. Assume the optimal control is $c_t=T-t$ for $t\in [0,T]$, which gives $Q_r=q_0-\int _0^r c_s ds = (1/2)(T-r)^2$ for $r\in [0,T]$ and $Q_r=0$ if and only if $r=\tau :=T$. Now consider a perturbation with ${\bar{c}}>T$ and $0<\theta <T^2/(2{{\bar{c}}})$, which gives $Q^{\theta , {\bar{c}},t}_r=q_0-{{\bar{c}}} r>0$ for $r\in [0,\theta ]$ and $Q^{\theta , {\bar{c}},t}_r=q_0-\int _0^r c^{\theta , {\bar{c}},t}_s ds = -{\bar{c}}\theta +T\theta - \theta ^2/2 + (T-r)^2/2$ for $r\in [\theta , T]$ and $Q^{\theta , {\bar{c}},t}_r=0$ if and only if $r=\tau ^{\theta , {\bar{c}},t}:=T-\sqrt{\theta ^2-2T\theta +2{\bar{c}}\theta }$. We have $\tau ^{\theta , {\bar{c}},t}\rightarrow \tau$ as $\theta \rightarrow 0$ but

$$\begin{aligned} \lim _{\theta \rightarrow 0} \frac{\tau ^{\theta , {\bar{c}},t}-\tau }{\theta }=-\lim _{\theta \rightarrow 0} \frac{\sqrt{\theta ^2-2T\theta +2{\bar{c}}\theta }}{\theta }=-\lim _{\theta \rightarrow 0} \sqrt{1+2\frac{{\bar{c}}-T}{\theta }}=-\infty , \end{aligned}$$

which shows the limit does not exist.

Remark 2.4

The standard SMP (c.f. Pham [14]) and Theorem 2.2 cannot be recovered from each other as they solve different problems. However, their statements are similar and only differ for the additional term ${\mathcal {G}}$ in (2.14).

Remark 2.5

The same result as Theorem 2.2 can be obtained in the case when the admissible set is bounded by above as well, i.e. when $\pi$ is required to be in $\pi \in [0,b]$ with $0<b\le +\infty$. Although the proof does not change, the only remark we want to point out is on the admissibility of control $c^{\theta ,{\bar{c}}, t}$. Since ${\bar{c}}\in [0,b]$ and $c_r\in [0,b]$ for every $r\in [0,T]$, then $-\frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }=\frac{1}{\theta }\int _t^{(t+\theta )\wedge \tau }(c_r-{\bar{c}})dr\le \frac{(t+\theta )\wedge \tau -t}{\theta }b\le b$.

3 Example

In this section we describe an example to show that the usual SMP is not satisfied whereas Theorem 2.2 is satisfied. We consider an optimal liquidation problem with no market impact on trade and no terminal execution. In particular, let $g=0$ and $f(\pi ,x)=\pi x$. Assume the admissible control can only take values in the interval $[0,c^+]$. Let $t\in [0,T]$ and $Q_t=q>0$ be fixed. The stock price X satisfies the SDE: for $r\in [t,T]$ and $X_t=x>0$,

$$\begin{aligned} dX_r=X_rdW_r. \end{aligned}$$

The agent aims to maximise his final cash value that is the cumulated liquidation wealth up to the stopping time at which the agent runs out the liquidating stocks and there is no residual value for any remaining stocks at horizon time T. The value function to this problem is defined by

$$\begin{aligned} v(t,x,q)&=\sup _{\pi \in {\mathcal {A}}}\mathbb {E}^t\left[ \int _t^{\tau ^\pi } \pi _r X_r \;dr \right] , \end{aligned}$$

(3.1)

where $\tau ^\pi =T\wedge \min \{r\ge t\; | \; Q^\pi _r=0\}$ is the first hitting time of $Q^\pi$ to zero or the fixed terminal time T, whichever comes first. We define an admissible control strategy c as follows: for any $r\in [t,T]$,

$$\begin{aligned} c_r= {\left\{ \begin{array}{ll}\frac{q}{T-t} &{} \text {if } q\le c^+(T-t),\\ c^+ &{} \text {if } q>c^+(T-t). \end{array}\right. } \end{aligned}$$

(3.2)

The inventory $Q_r$ in (2.1) is given by

$$\begin{aligned} Q_r={\left\{ \begin{array}{ll}\frac{q}{T-t}(T-r) &{} \text {if } q\le c^+(T-t),\\ q-(r-t)c^+ &{} \text {if } q>c^+(T-t). \end{array}\right. } \end{aligned}$$

We also have that for any $r\in [t,T]$, $Q_r\le c^+(T-r) \;\Leftrightarrow \; q\le c^+(T-t)$. Using the above expression for $Q_r$, it is easy to check that the first hitting time of $Q_r=0$ is

$$\begin{aligned} \tau =T \quad \text { a.s.}. \end{aligned}$$

Since the stopping time $\tau$ is equal to the terminal time T, it may look like the control in (3.2) is the same control we would have found in the usual setting without the stopping time. However, the optimal control in the usual case without stopping time would have been equal to $c^+$. When $q< c^+(T-t)$, at time $r=t+\frac{q}{c^+}< T$, the inventory would have reached zero and from that time onward the inventory would have become negative, making the control $c^+$ not feasible for our problem, see Remark 3.2.

Substituting (3.2) and $\tau =T$ into (3.1), we can easily show that the value function associated with control (3.2) is equal to

$$\begin{aligned} v^c(t,x,q)={\left\{ \begin{array}{ll}qx &{} \text {if } q\le c^+(T-t),\\ xc^+ (T-t) &{} \text {if } q>c^+(T-t). \end{array}\right. } \end{aligned}$$

(3.3)

Proposition 3.1

Function $v^c$ in (3.3) coincides with the value function v of problem (3.1) and the strategy c in (3.2) is the optimal control.

Proof

By definition of $v^c(t,x,q)$, we have that for any $t\in [0,T]$, $x>0$ and $q>0$

$$\begin{aligned} v^c(t,x,q)=\mathbb {E}^t\left[ \int _t^\tau c_rX_r dr\right] \le \sup _{\pi \in {\mathcal {A}}}\mathbb {E}^t\left[ \int _t^{\tau ^\pi } \pi _rX_r dr\right] = v(t,x,q). \end{aligned}$$

(3.4)

We now show that $v^c\ge v$. Define

$$\begin{aligned} w(t,q)={\left\{ \begin{array}{ll}q &{} \text {if } q\le c^+(T-t),\\ c^+ (T-t) &{} \text {if } q>c^+(T-t). \end{array}\right. } \end{aligned}$$

(3.5)

Simple calculus shows that

$$\begin{aligned} \partial _q w(t,q)={\left\{ \begin{array}{ll}1 &{} \text {if } q\le c^+(T-t),\\ 0 &{} \text {if } q>c^+(T-t), \end{array}\right. }\qquad \partial _t w(t,q)={\left\{ \begin{array}{ll}0 &{} \text {if } q\le c^+(T-t),\\ -c^+ &{} \text {if } q>c^+(T-t). \end{array}\right. } \end{aligned}$$

It can be easily verified that w satisfies the following HJB equation, for $t\in [0,T]$ and $q>0$

$$\begin{aligned} \partial _t w+\sup _{\pi \in [0,c^+]} \left[ \pi -\pi \partial _qw\right] =0. \end{aligned}$$

(3.6)

w satisfies the boundary condition $w(t,0)=0$ for any $t\in [0,T]$ and the terminal condition $w(T,q)=0$ for any $q>0$.

Denote by $c^*_r$ the optimal control for the value function v(t, x, q) in (3.1) and $Q^*_r$ and $\tau ^*$ the corresponding inventory and stopping time respectively. Using (2.1), for any $r\in [t,T]$, we have

$$\begin{aligned} dw(r,Q^*_r)= & {} \partial _t w(r,Q^*_r)dr+\partial _qw(r,Q^*_r)dQ^*_r=\left[ \partial _t w(r,Q^*_r)-\partial _qw(r,Q^*_r)c^*_r\right] dr\nonumber \\\le & {} -c^*_rdr. \end{aligned}$$

(3.7)

We have used (3.6) in the last inequality. Finally, from $X_t=x$, $Q_t^*=q$, $v^c(t,x,q)=xw(t,q)$, using the stochastic integration by parts and (3.7), we have

$$\begin{aligned} \begin{aligned} v^c(\tau ^*,X_{\tau ^*},Q^*_{\tau ^*})&=v^c(t,X_t,Q^*_t)+\int _t^{\tau ^*} X_r dw(r,Q^*_r) +\int _t^{\tau ^*} w(r,Q^*_r) dX_r \\&\le v^c(t,x,q)-\int _t^{\tau ^*} X_r c^*_r dr+\int _t^{\tau ^*} w(r,Q^*_r) X_r dW_r. \end{aligned} \end{aligned}$$

(3.8)

However, using boundary and terminal condition $v^c(t,x,0)=0$ for any $t\in [0,T]$ and $x>0$ and $v^c(T,x,q)=0$ for any $x>0$ and $q>0$, we conclude that $v^c(\tau ^*,X_{\tau ^*},Q^*_{\tau ^*})=0$, as either $\tau ^*=T$ or $Q^*_{\tau ^*}=0$. Therefore, taking conditional expectations on both sides of (3.8) and using the optional sampling theorem, we have that the expected value of the random variable $\int _t^{\tau ^*} w(r,Q^*_r) X_r dW_r$ is equal to 0 and we get

$$\begin{aligned} v^c(t,x,q)\ge \mathbb {E}^t\left[ \int _t^{\tau ^*} c^*_rX_r\; dr\right] = v(t,x,q). \end{aligned}$$

Combining (3.4) and the previous expression, we conclude the proof. $\square$

Remark 3.2

In the standard version of the SMP (cf. Pham [14]) there should be 2 adjoint processes in the BSDE, referring respectively to processes X and Q. However, since the process X does not depend on control $\pi$, the terms regarding the adjoint process referring to X can be removed from the Hamiltonian and, noting that $g=0$ and $f(\pi ,x)=\pi x$, those referring to Q are identically equal to 0. The necessary condition of the standard SMP is equivalent to

$$\begin{aligned} f(\pi ,X_t)\le f(c_t,X_t), \quad \forall t\in [0,T], \ \forall \pi \in [0,c^+]. \end{aligned}$$

(3.9)

However, from (3.1) we get that the maximal point of $f(\cdot ,x)$ is ${\bar{\pi }}=c^+$. We have shown that if $q< c^+(T-t)$, then the optimal strategy is $c_r=\frac{q}{T-t}$ for $t\le r\le T$ as in (3.2), which is less than ${\bar{\pi }}=c^+$, a contradiction to (3.9) and the standard SMP.

We next verify that optimal control in (3.2) satisfies Theorem 2.2. We need to show that (2.13) holds true. Firstly, we observe that the model setup satisfies Assumption 2.1. Using the fact that $\mu = 0$ and $g=0$, we show that (2.13) holds true by proving that for any ${\bar{c}}\in [0,c^+]$, $t\in [0,T)$,

$$\begin{aligned} f({\bar{c}},X_t)-f(c_{t},X_t)-{\bar{f}}(t,{\bar{c}},X_t,Q_t)\le 0. \end{aligned}$$

(3.10)

We first find the expression for ${\bar{f}}(t,{\bar{c}},x,q)$ in the following proposition.

Proposition 3.3

Let $t\in [0,T)$ be fixed and $(c_r)_{r\in [t,T]}$ be the optimal control in (3.2). Then, for any ${\bar{c}}\in [0,c^+]$, $x>0$ and $q>0$

$$\begin{aligned} {\bar{f}}(t,{\bar{c}},x,q)={\left\{ \begin{array}{ll}({\bar{c}}-c_t) x \mathbb {1}_{{\bar{c}}\ge c_t} &{} \text {if } q\le c^+(T-t),\\ 0 &{} \text {if } q>c^+(T-t). \end{array}\right. } \end{aligned}$$

Proof

We consider any $\theta \in (0,T-t)$, so that $\tau =T> t+\theta$. Using the fact that $c_r$ in (3.2) is constant in time, we have for any $r\in [t+\theta , \tau ]$

$$\begin{aligned} Q^{\theta ,{\bar{c}}, t}_r=q-{\bar{c}}\theta -\int _{t+\theta }^r c_s ds=q-c_t(r-t)+\theta (c_t-{\bar{c}}). \end{aligned}$$

(3.11)

If $q>c^+(T-t)$, then (3.2) implies that $c_t=c^+$ and from (3.11),

$$\begin{aligned} Q^{\theta ,{\bar{c}}, t}_T> \theta (c^+-{\bar{c}})\ge 0. \end{aligned}$$

Here we have used the fact that ${\bar{c}}$ is an admissible control and so ${\bar{c}}\in [0,c^+]$. The above expression implies that $\tau ^{\theta ,{\bar{c}}, t}=T$ a.s.. On the other hand, if $q\le c^+(T-t)$, then $c_t=\frac{q}{T-t}$ and from (3.11), $Q^{\theta ,{\bar{c}}, t}_T=-\theta ({\bar{c}}-c_t)$. Hence, if ${\bar{c}}\le c_t, Q^{\theta ,{\bar{c}}, t}_T\ge 0$ and so $\tau ^{\theta ,{\bar{c}}, t}=T$ a.s.. If ${\bar{c}}> c_t, Q^{\theta ,{\bar{c}}, t}_T< 0$ and so $\tau ^{\theta ,{\bar{c}}, t}<T$ a.s., so by setting (3.11) equal to 0 we get that $\tau ^{\theta ,{\bar{c}}, t}=T-\theta \left( \frac{{\bar{c}}}{c_t}-1\right)$ a.s., where $\theta \left( \frac{{\bar{c}}}{c_t}-1\right) >0$, since ${\bar{c}}>c_t$.

In conclusion, if $q>c^+(T-t)$, then $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}_{\max }=T$ and we have that ${\bar{f}}=0$ from definition (2.12). If $q\le c^+(T-t)$, then we consider two sub-cases. If ${\bar{c}}\le c_t$, then $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}_{\max }=T$, making ${\bar{f}}=0$ again. If ${\bar{c}}>c_t$, then $\tau ^{\theta ,{\bar{c}}, t}_{\min }=T-\theta \left( \frac{{\bar{c}}}{c_t}-1\right)$ and $\tau ^{\theta ,{\bar{c}}, t}_{\max }=T$ and so

$$\begin{aligned} {\bar{f}}(t,{\bar{c}},x,q)&=\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \frac{1}{\theta }\int _{T-\theta \left( \frac{{\bar{c}}}{c_t}-1\right) }^{T}c_r X_r\; dr\right] =\lim _{\theta \rightarrow 0}\frac{\theta \left( \frac{{\bar{c}}}{c_t}-1\right) }{\theta }c_tx=({\bar{c}}-c_t) x . \end{aligned}$$

This concludes the proof of the proposition. $\square$

To prove (3.10), we split the proof of (3.10) in two parts. If $q\le c^+(T-t)$, then the left side of (3.10) is equal to ${\bar{c}} X_t-c_t X_t-({\bar{c}}-c_t)X_t\mathbb {1}_{{\bar{c}}\ge c_t}=X_t\cdot \min \left( {\bar{c}}-c_t, \ 0\right) \le 0$. If $q> c^+(T-t)$, then the left side of (3.10) is equal to ${\bar{c}} X_t-c_t X_t=({\bar{c}}-c^+)X_t\le 0$ as ${\bar{c}}\le c^+$. Hence, (3.10) is satisfied for any $c^+\ge {\bar{c}}\ge 0$, and Theorem 2.2 holds.

Remark 3.4

The main purpose of this example is to show that the standard SMP cannot be applied when the terminal time is an indirectly controlled stopping time but our necessary SMP can accommodate that. It is in general difficult to find the optimal solution using Theorem 2.2 that has no particular advantage to the DPP for the example, but this is the first step in addressing the indirectly controlled random terminal time problem with the SMP which has the potential for solving the non-Markovian model, see Sect. 5 for possible further research.

4 Proof of Theorem 2.2

In this section we consider all assumptions of Theorem 2.2 are satisfied. (2.10) implies that $\mathbb {E}\left[ \sup _{r\in [0,T]}c_r\right] <\infty$. As mentioned in the model setup, for any fixed time $t\in [0,\tau )$, we have $Q_t=q>0$ and so $\tau >t$ a.s.. We consider a partition of the whole event space $\{\tau >t\}$, which helps us in stating and proving some preliminary results that are needed in the proof of Theorem 2.2. As general hints for better understanding, we remind that $\tau$ is defined so that $Q_\tau =0$ if $\tau <T$, $Q_\tau \ge 0$ if $\tau =T$, and $Q_r>0$ if $r\in [t,\tau )$. Similarly, $Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}=0$ if $\tau ^{\theta ,{\bar{c}}, t}<T$, $Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}\ge 0$ if $\tau ^{\theta ,{\bar{c}}, t}=T$, and $Q^{\theta ,{\bar{c}}, t}_r>0$ if $r\in [t,\tau ^{\theta ,{\bar{c}}, t})$. We first observe, using (2.5), that if $\theta \in \left( 0, (T-t)\wedge \frac{q}{{\bar{c}}}\right)$, then for any $r\in [t,(t+\theta )\wedge \tau ]$,

$$\begin{aligned} Q^{\theta ,{\bar{c}}, t}_r=q-\int _{t}^r c^{\theta ,{\bar{c}}, t}_s ds=q-{\bar{c}}\left( r-t\right) \ge q-{\bar{c}}\theta >0. \end{aligned}$$

Therefore, if $\theta \in \left( 0, (T-t)\wedge \frac{q}{{\bar{c}}}\right)$, then

$$\begin{aligned} \tau ^{\theta ,{\bar{c}}, t} >(t+\theta )\wedge \tau \end{aligned}$$

(4.1)

Let $t\in [0,\tau )$, $0< \theta <(T-t)\wedge \frac{q}{{\bar{c}}}$, ${\bar{c}}\ge 0$ be fixed, we define the following partitions of $\{\tau >t\}$:

$$\begin{aligned} \begin{aligned} E^{\theta ,{\bar{c}}, t}_{1}&:=\left\{ t<t+\theta<\tau ^{\theta ,{\bar{c}}, t}<\tau \right\} ,\\ E^{\theta ,{\bar{c}}, t}_{2}&:=\left\{ t<\tau<\tau ^{\theta ,{\bar{c}}, t} \right\} ,\\ E^{\theta ,{\bar{c}}, t}_{3}&:=\left\{ t< t+\theta <\tau =\tau ^{\theta ,{\bar{c}}, t} \right\} . \end{aligned} \end{aligned}$$

We now present the properties of the different cases $E^{\theta ,{\bar{c}}, t}_{i}$, for any $i\in \{1,2,3\}$. In particular, for each of the events we show a scheme for the different values of quantities $c^{\theta ,{\bar{c}}, t}$ and $Q^{\theta ,{\bar{c}}, t}$ in each of the time spans. These schemes help in understanding some steps in the proof of lemmas below.

• (1) On the event $E^{\theta ,{\bar{c}}, t}_{1}$:

  

  From previous scheme we conclude that on the event $E^{\theta ,{\bar{c}}, t}_{1}$

  $$\begin{aligned} 0=Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=Q_{\tau ^{\theta ,{\bar{c}}, t}}-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta } \ \Rightarrow \ Q_{\tau ^{\theta ,{\bar{c}}, t}}=\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }, \text { which also implies that } \gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }>0, \end{aligned}$$

  (4.2)

  since by definition of $\tau$, for any $r\in [t,\tau )$, $Q_r>0$.

  $$\begin{aligned}&\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \le \max \left( \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| ,\ |Q_{\tau ^{\theta ,{\bar{c}}, t}}|\right) =\sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| , \nonumber \\&\qquad \forall r\in [t,T], \end{aligned}$$

  (4.3)
  $$\begin{aligned}&Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_{\tau }\le Q_{\tau ^{\theta ,{\bar{c}}, t}}= \gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }. \end{aligned}$$

  (4.4)

• (2) On the event $E^{\theta ,{\bar{c}}, t}_{2}$: If $\tau > t+\theta$

  

  If $\tau \le t+\theta$, from (2.6), $\gamma ^{\theta ,{\bar{c}}, t}_\tau =\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }$

  

  From previous scheme we conclude that on the event $E^{\theta ,{\bar{c}}, t}_{2}$,

  $$\begin{aligned} Q^{\theta ,{\bar{c}}, t}_\tau =Q_\tau -\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }, \text { which implies } \gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }<0, \end{aligned}$$

  (4.5)

  since by definition of $\tau ^{\theta ,{\bar{c}}, t}$, for any $r\in [t,\tau ^{\theta ,{\bar{c}}, t})$, $Q^{\theta ,{\bar{c}}, t}_r>0$. Moreover,

  $$\begin{aligned}&\begin{aligned} \text {if } \tau ^{\theta ,{\bar{c}}, t}<T, \ 0=Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\left( 1-\frac{\tau ^{\theta ,{\bar{c}}, t}-\tau }{\theta }\right) \ \Rightarrow \ \tau ^{\theta ,{\bar{c}}, t}=\tau +\theta ,\\ \text {if } \tau ^{\theta ,{\bar{c}}, t}=T, \ 0\le Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\left( 1-\frac{\tau ^{\theta ,{\bar{c}}, t}-\tau }{\theta }\right) \ \Rightarrow \ \tau ^{\theta ,{\bar{c}}, t}\le \tau +\theta , \end{aligned} \end{aligned}$$

  (4.6)
  $$\begin{aligned}&\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \le \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| , \qquad \forall r\in [t,T], \end{aligned}$$

  (4.7)
  $$\begin{aligned}&\text {if } \tau ^{\theta ,{\bar{c}}, t}<T, \ Q_{\tau }^{\theta ,{\bar{c}}, t}-Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta },\nonumber \\&\text {if } \tau ^{\theta ,{\bar{c}}, t}=T, \ Q_{\tau }^{\theta ,{\bar{c}}, t}-Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}\le -\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }. \end{aligned}$$

  (4.8)

• (3) On the event $E^{\theta ,{\bar{c}}, t}_{3}$:

  

  From previous scheme we conclude that on the event $E^{\theta ,{\bar{c}}, t}_{3}$

  $$\begin{aligned}&\begin{aligned}&\text {if } Q_T=0, \ 0\le Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta },\\&\text {if } \tau =\tau ^{\theta ,{\bar{c}}, t}<T, \ 0= Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta },\\&\text {if } Q_T>0, \ 0\le Q_{\tau ^{\theta ,{\bar{c}}, t}}^{\theta ,{\bar{c}}, t}=Q_{T}-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }, \end{aligned} \end{aligned}$$

  (4.9)
  $$\begin{aligned}&\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \le \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| , \qquad \forall r\in [t,T], \end{aligned}$$

  (4.10)
  $$\begin{aligned}&Q_{\tau }-Q_{\tau ^{\theta ,{\bar{c}}, t}}=0. \end{aligned}$$

  (4.11)

From previous schemes we derive the following Lemmas.

Lemma 4.1

Let $t\in [0,\tau )$ be fixed, let ${\bar{c}}\ge 0$ and let $\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right)$. Then the control $c^{\theta ,{\bar{c}}, t}$ in (2.5) is admissible.

Proof

Firstly, we observe that control $c^{\theta ,{\bar{c}}, t}_r$ is non-negative for any $r\in [t,\tau )$. If $\tau ^{\theta ,{\bar{c}}, t}> \tau$, i.e. if we are in the event $E^{\theta ,{\bar{c}}, t}_2$, then using (4.5) we get that $\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }<0$ and so the control $c^{\theta ,{\bar{c}}, t}_r$ is non-negative for any $r\ge \tau$ as well. Progressive measurability, right-continuity and square integrability of $c^{\theta ,{\bar{c}}, t}$ immediately follow. $\square$

Lemma 4.2

Let $t\in [0,\tau )$ be fixed, let ${\bar{c}}\ge 0$ and let $\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right)$. Then

$$\begin{aligned} c_r^{\theta ,{\bar{c}}, t}-c_r&=0, \quad \forall r\in \left[ t+\theta , \tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )\right] ,\\ Q_r^{\theta ,{\bar{c}}, t}-Q_r&=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }, \quad \forall r\in \left[ t+\theta , \tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )\right] . \end{aligned}$$

Proof

Looking at schemes on pages 9-10, it follows that on the event $E_1^{\theta ,{\bar{c}}, t}$, $\tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )=\tau ^{\theta ,{\bar{c}}, t}$, on the event $E_2^{\theta ,{\bar{c}}, t}\cap \{ \tau >t+\theta \}$, $\tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )=\tau$, on the event $E_2^{\theta ,{\bar{c}}, t}\cap \{ \tau \le t+\theta \}$, $\tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )=t+\theta$ and on the event $E_3^{\theta ,{\bar{c}}, t}$, $\tau ^{\theta ,{\bar{c}}, t}_{\min }\vee (t+\theta )=\tau ^{\theta ,{\bar{c}}, t}=\tau$. Then, the result immediately follows.

$\square$

Lemma 4.3

Let $t\in [0,\tau )$ and ${\bar{c}}\ge 0$ be fixed. Then

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| \right] =0, \end{aligned}$$

(4.12)

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \sup _{r\in [t,T]}\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \right] =0. \end{aligned}$$

(4.13)

Proof

Let $\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right)$ be fixed. From definition of $\gamma ^{\theta ,{\bar{c}}, t}_r$ in (2.6) we immediately see that

$$\begin{aligned} \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| \le \theta \left( {\bar{c}}+\sup _{r\in [t,t+\theta ]}c_r\right) \quad \text {a.s.}. \end{aligned}$$

(4.14)

Merging (4.3), (4.7) and (4.10), we see that

$$\begin{aligned} \left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \le \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| , \quad \forall r\in [t, T]. \end{aligned}$$

Therefore, merging (4.14) and previous expression we get that

$$\begin{aligned} \mathbb {E}^t\left[ \sup _{r\in [t,T]}\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \right] \le \theta \left( {\bar{c}}+\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}c_r\right] \right) . \end{aligned}$$

We conclude the proof by using (2.10). $\square$

Lemma 4.4

Let ${\bar{c}}\ge 0$ and $t\in [0,\tau )$ be fixed. Then

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {P}\left( \{\tau \le t+\theta \}\right) =0, \end{aligned}$$

(4.15)

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{1}\cap \{Q_T>0\}\right) =0, \end{aligned}$$

(4.16)

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{1}\cap \{{\bar{c}}< c_t\}\right) =0, \end{aligned}$$

(4.17)

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{2}\cap \{\tau ^{\theta ,{\bar{c}}, t}=T\}\right) =0, \end{aligned}$$

(4.18)

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{3}\cap \{{\bar{c}}>c_t\}\cap \{Q_T=0\}\right) =0, \end{aligned}$$

(4.19)

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{3}\cap \{ \tau<T \}\cap \{ {\bar{c}}< c_t \}\right) =0. \end{aligned}$$

(4.20)

Proof

We firstly prove (4.15). We have that

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {P}\left( \{\tau \le t+\theta \}\right)&=\lim _{n\rightarrow \infty }\mathbb {P}\left( \left\{ \tau \le t+\frac{1}{n}\right\} \right) =\mathbb {P}\left( \bigcap _{n\ge {\bar{n}}}\left\{ \tau \le t+\frac{1}{n}\right\} \right) \\&= \mathbb {P}\left( \{\tau \le t\}\right) =0. \end{aligned} \end{aligned}$$

In previous calculations we used that for any $n\ge {\bar{n}}:=\left\lceil \frac{1}{(T-t)\wedge \frac{q}{{\bar{c}}}}\right\rceil$, the sequence of events $\left\{ \tau \le t+\frac{1}{n}\right\}$ is decreasing. This concludes proof of (4.15).

We now prove (4.16). Using definition of Q, we have that under event $E_1^{\theta ,{\bar{c}}, t}$, $Q_{\tau }=Q_{\tau ^{\theta ,{\bar{c}}, t}}-\int _{\tau ^{\theta ,{\bar{c}}, t}}^{\tau } c_r dr\le Q_{\tau ^{\theta ,{\bar{c}}, t}}$. Moreover, if $Q_T>0$, then it necessarily implies that $\tau =T$. Using (4.2) we have that

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{1}\cap \{Q_T>0\}\right)&\le \lim _{\theta \rightarrow 0}\mathbb {P}\left( \{Q_\tau \le Q_{\tau ^{\theta ,{\bar{c}}, t}}=\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\}\cap \{Q_\tau>0\}\right) \\&\le \lim _{n\rightarrow \infty }\mathbb {P}\left( \left\{ Q_\tau \le \sup _{r\in \left[ t,t+\frac{1}{n}\right] }\left| \gamma ^{\frac{1}{n},{\bar{c}}, t}_r \right| \right\} \cap \{Q_\tau>0\}\right) \\&=\mathbb {P}\left( \bigcap _{n\ge {\bar{n}}}\left\{ Q_\tau \le \int _t^{t+\frac{1}{n}}\left| {\bar{c}}-c_r \right| dr\right\} \cap \{Q_\tau>0\}\right) \\&= \mathbb {P}\left( \left\{ Q_\tau =0\right\} \cap \{Q_\tau >0\}\right) =0. \end{aligned} \end{aligned}$$

In previous calculations we used that the sequence of events $\Big (\Big \{Q_\tau \le \int _t^{t+\frac{1}{n}}\Big | {\bar{c}}-c_r \Big |dr\Big \}Big)_{n\ge {\bar{n}}}$ is decreasing and using right-continuity of c, $\int _t^{t+\frac{1}{n}}\left| {\bar{c}}-c_r \right| dr$ converges to 0 a.s., as $n\rightarrow \infty$. This concludes proof of (4.16).

We now prove (4.17). Using (4.2), we get

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{1}\cap \{{\bar{c}}< c_t\}\right)&\le \lim _{\theta \rightarrow 0}\mathbb {P}\left( \left\{ \theta {\bar{c}}-\int _t^{t+\theta }c_s ds>0\right\} \cap \{{\bar{c}}<c_t\}\right) \\&\le \lim _{n\rightarrow \infty }\mathbb {P}\left( \left\{ {\bar{c}}> \inf _{r\in \left[ t,t+\frac{1}{n}\right] }c_r\right\} \cap \{{\bar{c}}<c_t\}\right) \\&=\mathbb {P}\left( \bigcap _{n\ge {\bar{n}}}\left\{ {\bar{c}}> \inf _{r\in \left[ t,t+\frac{1}{n}\right] }c_r\right\} \cap \{{\bar{c}}<c_t\}\right) \\&= \mathbb {P}\left( \{{\bar{c}}\ge c_t\}\cap \{{\bar{c}}<c_t\}\right) =0. \end{aligned} \end{aligned}$$

In previous calculations we used right-continuity of process c and that the sequence of events $\left\{ c_t> \inf _{r\in \left[ t,t+\frac{1}{n}\right] }c_r\right\} _{n\ge {\bar{n}}}$ is decreasing. This concludes proof of (4.17).

We now prove (4.18). Using (4.6), we get

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{2}\cap \{\tau ^{\theta ,{\bar{c}}, t}=T\}\right)&\le \lim _{\theta \rightarrow 0}\mathbb {P}\left( \left\{ \tau +\theta \ge \tau ^{\theta ,{\bar{c}}, t} \right\} \cap \{\tau<T\}\cap \{\tau ^{\theta ,{\bar{c}}, t}=T\}\right) \\&=\lim _{n\rightarrow \infty }\mathbb {P}\left( \left\{ \tau +\frac{1}{n}\ge T\right\} \cap \{\tau<T\}\right) \\&=\mathbb {P}\left( \bigcap _{n\ge {\bar{n}}}\left\{ \tau +\frac{1}{n}\ge T\right\} \cap \{\tau<T\}\right) \\&=\mathbb {P}\left( \left\{ \tau \ge T\right\} \cap \{\tau <T\}\right) =0. \end{aligned} \end{aligned}$$

In previous calculations we used that the sequence of events $\left\{ \tau +\frac{1}{n}\ge T\right\} _{n\ge {\bar{n}}}$ is decreasing. This concludes proof of (4.18).

We now prove (4.19). Using (4.9), we get

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {P}\left( E^{\theta ,{\bar{c}}, t}_{3}\cap \{{\bar{c}}>c_t\}\cap \{Q_T=0\}\right)&\le \lim _{\theta \rightarrow 0}\mathbb {P}\left( \{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\le 0\}\cap \{{\bar{c}}>c_t\}\right) \\&\le \lim _{n\rightarrow \infty }\mathbb {P}\left( \left\{ {\bar{c}}\le \sup _{r\in \left[ t,t+\frac{1}{n}\right] }c_r\right\} \cap \{{\bar{c}}>c_t\}\right) \\&=\mathbb {P}\left( \bigcap _{n\ge {\bar{n}}}\left\{ {\bar{c}}\le \sup _{r\in \left[ t,t+\frac{1}{n}\right] }c_r\right\} \cap \{{\bar{c}}>c_t\}\right) \\&= \mathbb {P}\left( \{{\bar{c}}\le c_t\}\cap \{{\bar{c}}>c_t\}\right) =0. \end{aligned} \end{aligned}$$

In previous calculations we used right-continuity of process c and that the sequence of events $\left\{ c_t\le \sup _{r\in \left[ t,t+\frac{1}{n}\right] }c_r\right\} _{n\ge {\bar{n}}}$ is decreasing. This concludes the proof of (4.19). (4.20) can be proved similarly. $\square$

Lemma 4.5

Let $t\in [0,\tau )$ and ${\bar{c}}\ge 0$ be fixed. Then

$$\begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left| \tau ^{\theta ,{\bar{c}}, t}-\tau \right| \right] =0. \end{aligned}$$

(4.21)

Proof

We firstly prove that $\lim _{\theta \rightarrow 0} \tau ^{\theta ,{\bar{c}}, t}=\tau$ pointwise $\mathbb {P}$-almost everywhere. We assume on the contrary there exists a non-null event ${\mathcal {E}}$, so that $\lim _{\theta \rightarrow 0} \left| \tau ^{\theta ,{\bar{c}}, t}-\tau \right| >0$ on ${\mathcal {E}}$, which means that

$$\begin{aligned} \exists \gamma>0 \ \text { s.t. } \forall {\bar{\theta }}\in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\wedge \gamma \right) , \ \exists \theta \in (0,{\bar{\theta }}) \text { s.t. } \left| \tau -\tau ^{\theta ,{\bar{c}}, t}\right| > \gamma \ \text { on } {\mathcal {E}}. \end{aligned}$$

(4.22)

Using that under event $E^{\theta ,{\bar{c}}, t}_{1}$, $\tau >\tau ^{\theta ,{\bar{c}}, t}$ and so $\left| \tau -\tau ^{\theta ,{\bar{c}}, t}\right| > \gamma$ implies that $\tau -\tau ^{\theta ,{\bar{c}}, t}> \gamma$, which implies $Q_{\tau -\gamma }=Q_{\tau ^{\theta ,{\bar{c}}, t}}-\int _{\tau ^{\theta ,{\bar{c}}, t}}^{\tau -\gamma } c_r dr\le Q_{\tau ^{\theta ,{\bar{c}}, t}}=\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }$. Moreover, using that under event $E^{\theta ,{\bar{c}}, t}_{2}$, $\tau ^{\theta ,{\bar{c}}, t}=(\tau +\theta )\wedge T$, $\left| \tau -\tau ^{\theta ,{\bar{c}}, t}\right| > \gamma$ implies that $\theta \ge (\tau +\theta )\wedge T-\tau > \gamma$, which is never verified, as $\theta<{\bar{\theta }}<\gamma$. Moreover, under event $E^{\theta ,{\bar{c}}, t}_{3}$, we have that $\tau ^{\theta ,{\bar{c}}, t}=\tau$, which never satisfies $\left| \tau -\tau ^{\theta ,{\bar{c}}, t}\right| > \gamma$. Therefore, we have that (4.22) implies that

$$\begin{aligned} \exists \gamma >0 \ \text { s.t. } \forall {\bar{\theta }}\in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\wedge \gamma \right) , \ \exists \theta \in (0,{\bar{\theta }}) \text { s.t. } Q_{\tau -\gamma }\le \gamma ^{\theta ,{\bar{c}}, t}_{t+\theta } \ \text { on } {\mathcal {E}}. \end{aligned}$$

(4.23)

Reminding that $\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta } =\int _t^{(t+\theta )\wedge \tau }({\bar{c}}-c_r)dr\le \int _t^{(t+\theta )\wedge \tau }{\bar{c}}dr\le {\bar{c}}\theta$, expression (4.23) implies that $Q_{\tau -\gamma }=0$ on ${\mathcal {E}}$, which contradicts definition of $\tau$, as $\tau$ should be the first time in which $Q_r$ hits 0. Therefore, we conclude that ${\mathcal {E}}$ must be a set with 0 measure, which implies $\mathbb {P}^t$-almost everywhere pointwise convergence of $\tau ^{\theta ,{\bar{c}}, t}$ to $\tau$. To prove (4.21) we observe that $\left| \tau ^{\theta ,{\bar{c}}, t}-\tau \right| \le T$, independently on $\theta$. Applying the dominated convergence theorem (DCT) we get (4.21). $\square$

Lemma 4.6

Let $t\in [0,\tau )$, ${\bar{c}}\ge 0$ and $p\in [1,2)$ be fixed. Then

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \left| {\bar{c}}-c_t-\frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }\right| ^p\right] =0, \end{aligned}$$

(4.24)

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \left| \frac{Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}-Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}}{\theta }+{\bar{c}}-c_t\right| ^p\right] =0. \end{aligned}$$

(4.25)

Proof

Let $t\in [0,\tau )$ and $\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right)$ be fixed. We firstly observe that

$$\begin{aligned} \begin{aligned} \left| \frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }\right| \le \frac{1}{\theta }\int _t^{t+\theta } |{\bar{c}}-c_s|ds\le |{\bar{c}}|+\sup _{s\in [t,T]}|c_s|, \end{aligned} \end{aligned}$$

which is $L^p$-integrable thanks to assumption (2.10). Moreover, we have that

$$\begin{aligned} \begin{aligned} \left| {\bar{c}}-c_t-\frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }\right| =\left| {\bar{c}}-c_t-\frac{1}{\theta }\int _t^{t+\theta } ({\bar{c}}-c_s)ds\right| . \end{aligned} \end{aligned}$$

Therefore, by using right-continuity of control c and mean-value theorem, we conclude that the pointwise limit of the expression inside the expectation in (4.24) is 0. Finally, by using DCT we conclude the proof of (4.24).

Moreover, looking at schemes in pages pages 9-10, we can immediately see that

$$\begin{aligned} Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}-Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}=-\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }. \end{aligned}$$

Therefore, by applying (4.24) and previous expression into (4.25) we prove the Lemma. $\square$

Lemma 4.7

Let $t\in [0,\tau )$, ${\bar{c}}\ge 0$ and $p\in [1,2)$ be fixed. Then

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \left| \frac{{\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau }{\theta }-({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right| ^p\right] =0. \end{aligned}$$

Proof

Let $t\in [0,\tau )$ and $\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right)$ be fixed. Using schemes in pp. 9–10, (4.2), (4.4), (4.6), (4.8), using Hölder’s inequality (with coefficients $\frac{p+2}{2p}$ and $\frac{p+2}{2-p}$), reminding that $E^{\theta ,{\bar{c}}, t}_{2}$ implies that $Q_T=0$, we get

$$\begin{aligned}&\mathbb {E}^t\left[ \left| \frac{{\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau }{\theta }-({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right| ^p\right] \\&\quad =\mathbb {E}^t\left[ \left| \frac{Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_\tau }{\theta }-({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{1}}\right] \\&\qquad +\mathbb {E}^t\left[ \left| \frac{Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-Q^{\theta ,{\bar{c}}, t}_{\tau }}{\theta }-({\bar{c}}-c_t)\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{2}}\right] \\&\qquad +\mathbb {E}^t\left[ \left| -({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{3}}\right] \\&\quad \le \mathbb {E}^t\left[ \left| \frac{Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_\tau }{\theta }\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{1}\cap \{ Q_T>0 \}}\right] \\&\qquad +\mathbb {E}^t\left[ \left| \frac{Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_\tau }{\theta }-({\bar{c}}-c_t)\mathbb {1}_{\tau<T}\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{1}\cap \{ Q_T=0 \}\cap \{ {\bar{c}}< c_t \}}\right] \\&\qquad +\mathbb {E}^t\left[ \left| \frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }-({\bar{c}}-c_t)\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{1}\cap \{ Q_T=0 \}\cap \{ {\bar{c}}\ge c_t \}}\right] \\&\qquad +\mathbb {E}^t\left[ \left| \frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }-({\bar{c}}-c_t)\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{2}\cap \{ \tau ^{\theta ,{\bar{c}}, t}<T \}}\right] \\&\qquad +\mathbb {E}^t\left[ \left| \frac{Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-Q^{\theta ,{\bar{c}}, t}_{\tau }}{\theta }-({\bar{c}}-c_t)\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{2}\cap \{ \tau ^{\theta ,{\bar{c}}, t}=T \}}\right] \\&\qquad +\mathbb {E}^t\left[ \left| -({\bar{c}}-c_t)\mathbb {1}_{\{ Q_T=0 \}\cap \{ {\bar{c}}\ge c_t \}}\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{3}}\right] \\&\qquad +\mathbb {E}^t\left[ \left| -({\bar{c}}-c_t)\mathbb {1}_{\{ \tau<T \}\cap \{ {\bar{c}}< c_t \}}\right| ^p\mathbb {1}_{E^{\theta ,{\bar{c}}, t}_{3}}\right] \\&\quad \le \left( \mathbb {E}^t\left[ \left| \frac{\sup _{r\in [t,t+\theta ]}\gamma ^{\theta ,{\bar{c}}, t}_r}{\theta }\right| ^{\frac{p+2}{2}}\right] \right) ^{\frac{2p}{p+2}}\bigg (\mathbb {P}^t\left( E^{\theta ,{\bar{c}}, t}_{1}\cap \{ Q_T>0 \}\right) ^{\frac{2-p}{p+2}}\\&\qquad +2^{p-1}\mathbb {P}^t\left( E^{\theta ,{\bar{c}}, t}_{1}\cap \{ Q_T=0 \}\cap \{ {\bar{c}}< c_t \}\right) ^{\frac{2-p}{p+2}} \\&\qquad +2^{p-1}\mathbb {P}^t\left( E^{\theta ,{\bar{c}}, t}_{2}\cap \{ \tau ^{\theta ,{\bar{c}}, t}=T \}\right) ^{\frac{2-p}{p+2}}\bigg )\\&\qquad +\mathbb {E}^t\left[ \left| \frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }-({\bar{c}}-c_t)\right| ^p\right] +|{\bar{c}}-c_t|^p\bigg (2^{p-1}\mathbb {P}^t\\&\qquad \left( E^{\theta ,{\bar{c}}, t}_{1}\cap \{ Q_T=0 \}\cap \{ {\bar{c}}< c_t \} \right) +2^{p-1}\mathbb {P}^t\left( E^{\theta ,{\bar{c}}, t}_{2}\cap \{ \tau ^{\theta ,{\bar{c}}, t}=T \} \right) \\&\qquad +\mathbb {P}^t\left( E^{\theta ,{\bar{c}}, t}_{3}\cap \{ Q_T=0 \}\cap \{ {\bar{c}}> c_t \}\right) +\mathbb {P}^t\left( E^{\theta ,{\bar{c}}, t}_{3}\cap \{ \tau<T \}\cap \{ {\bar{c}}< c_t \}\right) \bigg ). \end{aligned}$$

Taking the limit of the above expression and using (4.14), (4.16), (4.17), (4.18), (4.19), (4.20) and (4.24) we conclude the proof of the Lemma. $\square$

Lemma 4.8

Let $t\in [0,\tau )$ and ${\bar{c}}\ge 0$ be fixed. Then,

$$\begin{aligned}&\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left| {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right| \right] =0, \end{aligned}$$

(4.26)

$$\begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \left| {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau -Q_\tau \right| \right] =0. \end{aligned}$$

(4.27)

Proof

Let $\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right)$ be fixed. By using (4.4), (4.8) and (4.11), we have that

$$\begin{aligned} \begin{aligned} \mathbb {E}^t\left[ \left| {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right| \right]&\le \mathbb {E}^t\left[ \left| Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_\tau \right| \mathbb {1}_{E^{\theta ,{\bar{c}}, t}_1}\right] +\mathbb {E}^t\left[ \left| Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-Q^{\theta ,{\bar{c}}, t}_\tau \right| \mathbb {1}_{E^{\theta ,{\bar{c}}, t}_2}\right] \\&\le \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| \right] . \end{aligned} \end{aligned}$$

Taking the limit of the above expression and using (4.12) we conclude the proof of (4.26). We now prove (4.27). By (4.7) and (4.10), we have that

$$\begin{aligned} \begin{aligned} \mathbb {E}^t\left[ \left| {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau -Q_\tau \right| \right]&\le \mathbb {E}^t\left[ \left| Q^{\theta ,{\bar{c}}, t}_{\tau }-Q_\tau \right| \mathbb {1}_{E^{\theta ,{\bar{c}}, t}_2}\right] +\mathbb {E}^t\left[ \left| Q^{\theta ,{\bar{c}}, t}_\tau -Q_\tau \right| \mathbb {1}_{E^{\theta ,{\bar{c}}, t}_3}\right] \\&\le \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| \gamma ^{\theta ,{\bar{c}}, t}_r\right| \right] . \end{aligned} \end{aligned}$$

Taking the limit of the above expression and using (4.12) we conclude the proof of (4.27). $\square$

Lemma 4.9

For any $(x,q),(x,q')\in {\mathcal {O}}$ with $q\ne q'$, we have that

$$\begin{aligned} \left| \frac{g(x,q)-g(x,q')}{q-q'}-\partial _q g\left( x,q'\right) \right| \le K |q-q'|. \end{aligned}$$

Proof

We observe that

$$\begin{aligned} \begin{aligned} \frac{g(x,q)-g(x,q')}{q-q'}=\int _0^1\partial _q g\left( x, q'+\lambda (q-q')\right) \; d\lambda \end{aligned} \end{aligned}$$

and so using Lipschitz continuity of $\partial _q g$ in Assumption 2.8, we get

$$\begin{aligned} \begin{aligned} \left| \frac{g(x,q)-g(x,q')}{q-q'}-\partial _q g\left( x,q'\right) \right|&=\left| \int _0^1\partial _q g\left( x, q'+\lambda (q-q')\right) \; d\lambda -\partial _q g\left( x,q'\right) \right| \\&\le \int _0^1\left| \partial _q g\left( x, q'+\lambda (q-q')\right) -\partial _q g\left( x,q'\right) \right| d\lambda \\&\le \frac{K}{2}|q-q'|. \end{aligned} \end{aligned}$$

This proves the lemma. $\square$

We introduce a process used in the proof of Theorem 2.2. Define $(\xi _r)_{r\in [t,T]}$ as

$$\begin{aligned} \xi _r := f(t, {\bar{c}}, X_{t}, Q_{t})-f(t, c_{t}, X_{t}, Q_{t}) - \int _t^r ({\bar{c}}-c_t) \partial _q f(s, c_s,X_s,Q_s)ds. \nonumber \\ \end{aligned}$$

(4.28)

$\xi$ is the same as process $\zeta$ in Bensoussan [3]. The corresponding part of z in Bensoussan [3] for process Q would be constantly equal to ${\bar{c}}-c_t$, as it can be inferred with a simple calculus.

Our proof relies on the arguments in Bensoussan [3] with one key difference. Due to the presence of the control-dependent stopping time $\tau$ in our setting, it makes necessary the introduction of the stopping time $\tau ^{\theta ,{\bar{c}}, t}$ as well. This complicates all the proofs and makes necessary many adjustments, especially on those results in Bensoussan [3] that concern terminal time T that must be adapted to $\tau$ or $\tau ^{\theta ,{\bar{c}}, t}$ accordingly.

Lemma 4.10

Let $t\in [0,\tau )$ and ${\bar{c}}\ge 0$ be fixed. Then,

$$\begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left| \xi _{\tau ^{\theta ,{\bar{c}}, t}}-\xi _{\tau }\right| \right] =0,\\ \lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left| Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_{\tau }\right| \right] =0. \end{aligned}$$

Proof

From (4.28) and using boundedness of $\partial _q f$, we have that

$$\begin{aligned} \mathbb {E}^t\left[ \left| \xi _{\tau ^{\theta ,{\bar{c}}, t}}-\xi _{\tau }\right| \right]&=\mathbb {E}^t\left[ \left| -\int _{\tau }^{\tau ^{\theta ,{\bar{c}}, t}}({\bar{c}}-c_t) \partial _q f(r, c_r,X_r,Q_r)dr\right| \right] \\&\le K|{\bar{c}}-c_t|\mathbb {E}^t\left[ \left| \tau ^{\theta ,{\bar{c}}, t}-\tau \right| \right] . \end{aligned}$$

Therefore, by taking the limit of previous expression and applying (4.21), we get the first limit in the statement. Moreover, from definition of $Q_r$, we have

$$\begin{aligned} \mathbb {E}^t\left[ \left| Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_{\tau }\right| \right]&=\mathbb {E}^t\left[ \left| -\int _{\tau }^{\tau ^{\theta ,{\bar{c}}, t}}c_r dr\right| \right] \\&\le \left( \mathbb {E}^t\left[ \left| \int _t^T c_r^2 dr\right| \right] \right) ^{1/2}\left( \mathbb {E}^t\left[ \left| \tau ^{\theta ,{\bar{c}}, t}-\tau \right| \right] \right) ^{1/2}. \end{aligned}$$

Therefore, by taking the limit of previous expression, using (2.10) and applying (4.21), we finish the proof of the Lemma. $\square$

Lemma 4.11

Let $t\in [0,\tau )$ and ${\bar{c}}\ge 0$ be fixed. Then

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \frac{1}{\theta }\int _t^{\tau ^{\theta ,{\bar{c}}, t}_{\min }} \left( f\left( s,c_s^{\theta ,{\bar{c}}, t}, X_s,Q_s^{\theta ,{\bar{c}}, t}\right) -f(s,c_s,X_s,Q_s)\right) ds-\xi _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right] =0. \end{aligned}$$

Proof

Let $\theta \in \left( 0,(T-t)\wedge \frac{q}{{\bar{c}}}\right)$ be fixed. We denote for any $r\in [t,T]$

$$\begin{aligned} {\tilde{f}}_r^{\theta ,{\bar{c}}, t}:=\frac{1}{\theta }\int _t^r\left( f\left( s,c_s^{\theta ,{\bar{c}}, t},X_s,Q_s^{\theta ,{\bar{c}}, t}\right) - f(s,c_s,X_s,Q_s)\right) ds -\xi _r. \end{aligned}$$

The proof of this lemma will be divided in 3 steps. In step 1 we prove that

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \left| {\tilde{f}}_{t+\theta }^{\theta ,{\bar{c}}, t}\right| \right] =0. \end{aligned}$$

In Step 2 we prove that,

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ {\tilde{f}}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}\mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }\le t+\theta }\right] =0. \end{aligned}$$

In Step 3 we prove that

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ {\tilde{f}}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}\mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }> t+\theta }\right] =0. \end{aligned}$$

In Bensoussan [3] there is no difference between Steps 2 and 3, while in our case we need to consider them both. However, the structure of our proof resembles the one in the reference. Once the proof of the 3 steps is completed, we conclude the proof of the Lemma as follows. By merging the limits above in Steps 2 and 3, we have

$$\begin{aligned} \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ {\tilde{f}}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}\right] =\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ {\tilde{f}}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}\mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }\le t+\theta }\right] +\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ {\tilde{f}}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}\mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }> t+\theta }\right] =0. \end{aligned}$$

Step 1. From (4.28), reminding that $c_t^{\theta ,{\bar{c}}, t}={\bar{c}}$ and definition of ${\tilde{f}}$ we have that for any $r\in [t,t+\theta ]$,

$$\begin{aligned} \begin{aligned} {\tilde{f}}_r^{\theta ,{\bar{c}}, t}&=\frac{1}{\theta }\int _t^r\left( f\left( s,c_s^{\theta ,{\bar{c}}, t}, X_s, Q_s^{\theta ,{\bar{c}}, t}\right) -f(s,c_s^{\theta ,{\bar{c}}, t},X_s,Q_s)\right) ds\\&\quad +\frac{1}{\theta }\int _t^r\left( f\left( s,c_s^{\theta ,{\bar{c}}, t}, X_s,Q_s\right) -f(t,c_t^{\theta ,{\bar{c}}, t},X_t,Q_t)\right) ds\\&\quad +\frac{1}{\theta }\int _t^r\left( f(t,c_t,X_t,Q_t)-f\left( s,c_s, X_s,Q_s\right) \right) ds\\&\quad +f(t,c_t,X_t,Q_t)\left( 1-\frac{r-t}{\theta }\right) +f(t,c_t^{\theta ,{\bar{c}}, t},X_t,Q_t)\left( \frac{r-t}{\theta }-1\right) \\&\quad +({\bar{c}}-c_t)\int _t^r\partial _qf(s,c_s,X_s,Q_s)ds. \end{aligned} \end{aligned}$$

(4.29)

By taking $r=t+\theta$ in previous expression, so that the second last line disappears and using Assumption 2.1, boundedness of $\partial _q f$ and Hölder’s inequality, we get

$$\begin{aligned} \begin{aligned}&\mathbb {E}^t\left[ \left| {\tilde{f}}_{t+\theta }^{\theta ,{\bar{c}}, t}\right| \right] \\&\le K\Bigg (\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| Q_r^{\theta ,{\bar{c}}, t}-Q_r\right| \right] +2\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| Q_r-Q_t\right| \right] +\theta |{\bar{c}}-c_t|\\&\quad +\left( \left( \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| X_r-X_t\right| ^2\right] \right) ^{1/2}+\left( \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| c_r^{\theta ,{\bar{c}}, t}-c_t^{\theta ,{\bar{c}}, t}\right| ^2\right] \right) ^{1/2}\right) \cdot \\&\quad \cdot \left( \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left( 1+2|X_r|+2|c_r^{\theta ,{\bar{c}}, t}|\right) ^2\right] \right) ^{1/2} + \frac{2}{\theta }\int _t^{t+\theta } |s-t|ds\\&\quad +\left( \left( \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| X_r-X_t\right| ^2\right] \right) ^{1/2}+\left( \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| c_r-c_t\right| ^2\right] \right) ^{1/2}\right) \cdot \\&\quad \cdot \left( \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left( 1+2|X_r|+2|c_r|\right) ^2\right] \right) ^{1/2}\Bigg ). \end{aligned} \end{aligned}$$

(4.30)

Using DCT, $\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| c_r-c_t\right| ^2\right]$ and $\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| c_r^{\theta ,{\bar{c}}, t}-c_t^{\theta ,{\bar{c}}, t}\right| ^2\right]$ converge to 0 as $\theta \rightarrow 0$. Indeed, c and $c^{\theta ,{\bar{c}}, t}$ are right-continuous and thanks to (2.10) and (4.14), the arguments of the expectations converge to 0 a.s. and they are bounded by $2\sup _{r\in [t,T]}|c_r|^2$ and $2\sup _{r\in [t,T]}|c_r^{\theta ,{\bar{c}}, t}|^2$, which are $L^1$-integrable processes. $\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| X_r-X_t\right| ^2\right]$ converges to 0 using standard arguments in SDE theory (c.f. Krylov [11, Corollary 2.5.12]). Moreover, using $L^2$-integrability of c and $c^{\theta ,{\bar{c}}, t}$ and standard arguments in SDE theory (c.f. Krylov [11, Corollary 2.5.12]), we get that $\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left( 2|X_r|+2|c_r^{\theta ,{\bar{c}}, t}|\right) ^2\right]$ and $\mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left( 2|X_r|+2|c_r|\right) ^2\right]$ are bounded independently of $\theta$. Moreover, by definition of $Q_r$,

$$\begin{aligned} \mathbb {E}^t\left[ \sup _{r\in [t,t+\theta ]}\left| Q_r-Q_t\right| \right] =\mathbb {E}^t\left[ \int _t^{t+\theta }\left| c_r\right| dr\right] \le \sqrt{\theta }\left( \mathbb {E}^t\left[ \int _t^{T}c_r^2 dr\right] \right) ^{1/2}, \end{aligned}$$

which converges to 0, thanks to (2.10). Using (4.13), we have that $\mathbb {E}^t\Big [\Big (\sup _{r\in [t,t+\theta ]}\Big |Q_r^{\theta ,{\bar{c}}, t}-Q_r\Big |\Big )\Big ]$ converges to 0. Finally, we observe that $\frac{2}{\theta }\int _t^{t+\theta } |s-t|ds=\theta$. Therefore, by taking limit of (4.30) we conclude the proof of Step 1.

Step 2. From (4.29), using Assumption 2.1 and boundedness of $\partial _q f$, we get a similar expression to (4.30). Step 2 can be proved in a similar way to Step 1, the main difference is the term $\mathbb {E}^t\left[ \left| \frac{\tau ^{\theta ,{\bar{c}}, t}_{\min }-t}{\theta }-1\right| \mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }\le t+\theta }\right]$. However, by reminding that by (4.1), $\left\{ \tau ^{\theta ,{\bar{c}}, t}_{\min }\le t+\theta \right\} =\left\{ \tau \le t+\theta \right\}$, that under event $\tau ^{\theta ,{\bar{c}}, t}_{\min }\le t+\theta$, then $\left| \frac{\tau ^{\theta ,{\bar{c}}, t}_{\min }-t}{\theta }-1\right| \le 1$ and by using (4.15), we conclude that

$$\begin{aligned} \begin{aligned}&\lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ {\tilde{f}}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}\mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min } \le t+\theta }\right] \\&\le \left( \left| f(t,{\bar{c}},X_t,Q_t)\right| +\left| f(t,c_t,X_t,Q_t)\right| \right) \lim _{\theta \rightarrow 0} \mathbb {P}\left( \left\{ \tau \le t+\theta \right\} \right) =0. \end{aligned} \end{aligned}$$

This concludes the proof of Step 2.

Step 3. From (4.28) and definition of ${\tilde{f}}$ we have that for any $r\in [t+\theta , T]$,

$$\begin{aligned} {\tilde{f}}_r^{\theta ,{\bar{c}}, t}&={\tilde{f}}_{t+\theta }^{\theta ,{\bar{c}}, t}+\frac{1}{\theta }\int _{t+\theta }^r\left( f\left( s,c_s^{\theta ,{\bar{c}}, t}, X_s, Q_s^{\theta ,{\bar{c}}, t}\right) -f(s,c_s,X_s,Q_s^{\theta ,{\bar{c}}, t})\right) ds\\&\quad +\int _{t+\theta }^r\int _0^1\left( {\bar{c}}-c_t-\frac{Q_s-Q_s^{\theta ,{\bar{c}}, t}}{\theta }\right) \\&\quad \times \partial _q f\left( s, c_s, X_s, Q_s+\lambda \left( Q_s^{\theta ,{\bar{c}}, t}-Q_s\right) \right) d\lambda ds\\&\quad +\int _{t+\theta }^r\int _0^1({\bar{c}}-c_t)\Bigg (\partial _q f(s,c_s, X_s, Q_s) \\&\quad -\partial _q f\left( s, c_s, X_s, Q_s+\lambda \left( Q_s^{\theta ,{\bar{c}}, t}-Q_s\right) \right) \Bigg )d\lambda ds. \end{aligned}$$

Therefore, by applying Assumption 2.1, then boundedness and Lipschitz continuity of $\partial _q f$ follows, we have that

$$\begin{aligned} \left| {\tilde{f}}_r^{\theta ,{\bar{c}}, t}\right|&\le \left| {\tilde{f}}_{t+\theta }^{\theta ,{\bar{c}}, t}\right| +\frac{K}{\theta }\int _{t+\theta }^r\left| c_s^{\theta ,{\bar{c}}, t}-c_s\right| ds\\&\quad +K\int _{t+\theta }^r\left| \frac{Q_s-Q_s^{\theta ,{\bar{c}}, t}}{\theta }-({\bar{c}}-c_t)\right| ds\\&\quad +\frac{K|{\bar{c}}-c_t|}{2}\int _{t+\theta }^r\left| Q_s^{\theta ,{\bar{c}}, t}-Q_s\right| ds. \end{aligned}$$

Therefore, from previous expression and using Lemma 4.2, we get that

$$\begin{aligned} \left| \mathbb {E}^t\left[ {\tilde{f}}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\theta ,{\bar{c}}, t}\mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }>t+\theta }\right] \right|&\le \mathbb {E}^t\left[ \left| {\tilde{f}}_{t+\theta }^{\theta ,{\bar{c}}, t}\right| \right] +K\mathbb {E}^t\left[ \frac{\mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }> t+\theta }}{\theta }\int _{t+\theta }^{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\left| c_s^{\theta ,{\bar{c}}, t}-c_s\right| ds\right] \\&\quad +K\mathbb {E}^t\left[ \mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }> t+\theta }\int _{t+\theta }^{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\left| \frac{Q_s-Q_s^{\theta ,{\bar{c}}, t}}{\theta }-({\bar{c}}-c_t)\right| ds\right] \\&\quad +K\frac{|{\bar{c}}-c_t|}{2}\mathbb {E}^t\left[ \mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }> t+\theta }\int _{t+\theta }^{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\left| Q_s^{\theta ,{\bar{c}}, t}-Q_s\right| ds\right] \\&\le \mathbb {E}^t\left[ \left| {\tilde{f}}_{t+\theta }^{\theta ,{\bar{c}}, t}\right| \right] +KT\mathbb {E}^t\left[ \left| \frac{\gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }}{\theta }-({\bar{c}}-c_t)\right| \right] \\&\quad +KT\frac{|{\bar{c}}-c_t|}{2}\mathbb {E}^t\left[ \left| \gamma ^{\theta ,{\bar{c}}, t}_{t+\theta }\right| \right] . \end{aligned}$$

By taking limit of the above expression for $\theta \rightarrow 0$, by using (4.12) and (4.24) together with Step 1, we conclude the proof of Step 3 and the proof of the Lemma as well. $\square$

Proof of Theorem 2.2. Let $t\in [0,\tau )$ be fixed. Since control c is optimal, it necessarily follows that for any ${\bar{c}}\ge 0$ and for any $\theta >0$

$$\begin{aligned} v^{c^{\theta ,{\bar{c}}, t}}(t, x,q)\le v^c(t,x,q), \end{aligned}$$

where

$$\begin{aligned} v^\pi (t,x,q) =\mathbb {E}^t\left[ g( X_{\tau ^\pi }, Q_{\tau ^\pi }^\pi )+ \int _t^{\tau ^\pi } f(r, \pi _r, X_r, Q_r^\pi ) \;dr\right] \end{aligned}$$

and $\tau ^\pi =T\wedge \min \{r\ge t\; | \; Q_r^\pi =0\}$. We write Q as $Q^c$ and $\tau$ as $\tau ^c$ as in Theorem 2.2.

Therefore, we necessarily have that for any ${\bar{c}}\ge 0$

$$\begin{aligned} \lim _{\theta \rightarrow 0} \frac{v^{c^{\theta ,{\bar{c}}, t}}(t, x,q)-v^c(t,x,q)}{\theta }\le 0, \end{aligned}$$

(4.31)

provided the limit exists. By definition of $v^\pi$, reminding that when $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}$, then for $r\ge \tau ^{\theta ,{\bar{c}}, t}$, ${\hat{Q}}^{\theta ,{\bar{c}}, t}_r=Q_r$ and ${\hat{c}}^{\theta ,{\bar{c}}, t}_r=c_r$ and when $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau$, then for $r\ge \tau$, ${\hat{Q}}^{\theta ,{\bar{c}}, t}_r=Q_r^{\theta ,{\bar{c}}, t}$ and ${\hat{c}}^{\theta ,{\bar{c}}, t}_r=c_r^{\theta ,{\bar{c}}, t}$

$$\begin{aligned} \begin{aligned}&\lim _{\theta \rightarrow 0} \frac{v^{c^{\theta ,{\bar{c}}, t}} (t, x,q)-v^c(t,x,q)}{\theta }\\&=\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{g(X_{\tau ^{\theta ,{\bar{c}}, t}}, Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})-g(X_\tau , Q_\tau )}{\theta }\right] \\&+\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{1}{\theta }\int _{t}^{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\left( f(r, c^{\theta ,{\bar{c}}, t}_r, X_r,Q^{\theta ,{\bar{c}}, t}_t)-f(r, c_r, X_r,Q_r)\right) dr\right] \\&+\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ -\frac{\text {sign}(\tau -\tau ^{\theta ,{\bar{c}}, t})}{\theta }\int _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\tau ^{\theta ,{\bar{c}}, t}_{\max }}f(r, {\hat{c}}^{\theta ,{\bar{c}}, t}_r, X_r,{\hat{Q}}^{\theta ,{\bar{c}}, t}_r) dr\right] . \end{aligned} \end{aligned}$$

(4.32)

The first line on the right-hand side of (4.32) can be written as

$$\begin{aligned} \begin{aligned}&\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{g(X_{\tau ^{\theta ,{\bar{c}}, t}}, Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})-g(X_\tau , Q_\tau )}{\theta }\right] =\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{g(X_{\tau ^{\theta ,{\bar{c}}, t}}, Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})-g(X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})}{\theta }\right] \\&+\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{g(X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }})-g(X_\tau , Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }})}{\theta }\right] +\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{g(X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})-g(X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau )}{\theta }\right] \\&+\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{g(X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})-g(X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }})+g(X_\tau , Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }})-g(X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})+g(X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau )-g(X_\tau , Q_\tau )}{\theta }\right] . \end{aligned} \end{aligned}$$

(4.33)

Reminding that when $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}$, then ${\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}=Q_{\tau ^{\theta ,{\bar{c}}, t}}$ and ${\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau }=Q_\tau$ and when $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau$, then ${\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}=Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}$ and ${\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau }=Q^{\theta ,{\bar{c}}, t}_{\tau }$, then we have that the last element on the right-hand side of (4.33) is equal to 0. The first element on the right-hand side of (4.33) is equal to $-{\bar{g}}(t,{\bar{c}},x,q)$ by its definition (2.11). We define ${\tilde{g}}$ for any $(x,q)\in {\mathcal {O}}, (x,q')\in {\mathcal {O}}$ as

$$\begin{aligned} {\tilde{g}}(x,q,q'):={\left\{ \begin{array}{ll} \frac{g(x,q)-g(x,q')}{q-q'} &{}\text {if } q\ne q',\\ \partial _q g(x,q') &{}\text {if } q=q'. \end{array}\right. } \end{aligned}$$

From Assumption 2.8 we have that ${\tilde{g}}$ is bounded by $K(1+|x|)$. The second element on the right-hand side of (4.33) is equal to

$$\begin{aligned}&\lim _{\theta \rightarrow 0} \mathbb {E}^t \left[ {\tilde{g}}\left( X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}, Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }} \right) \frac{Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}-Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}}{\theta }\right] \nonumber \\&\quad =\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ {\tilde{g}}\left( X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}, Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }} \right) \left( \frac{Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}-Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}}{\theta }+{\bar{c}}-c_t\right) \right] \nonumber \\&\quad \quad -\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ ({\bar{c}}-c_t)\left( {\tilde{g}}\left( X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}, Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }} \right) -\partial _q g\left( X_\tau ,Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right) \right) \right] \nonumber \\&\quad \quad - ({\bar{c}}-c_t) \lim _{\theta \rightarrow 0}\mathbb {E}^t\left[ \partial _q g\left( X_\tau ,Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right) -\partial _q g\left( X_\tau ,Q_\tau \right) \right] \nonumber \\&\quad \quad -({\bar{c}}-c_t) \mathbb {E}^t\left[ \partial _q g\left( X_\tau ,Q_\tau \right) \right] . \end{aligned}$$

(4.34)

Using Hölder’s inequality, boundedness of ${\tilde{g}}$ and (4.25), we get

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t&\left[ \left| {\tilde{g}}\left( X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}, Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }} \right) \left( \frac{Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}-Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}}{\theta }+{\bar{c}}-c_t\right) \right| \right] \\&\le K\left( \mathbb {E}^t\left[ \left( 1+\left| X_\tau \right| \right) ^4\right] \right) ^{\frac{1}{4}}\lim _{\theta \rightarrow 0} \left( \mathbb {E}^t\left[ \left| \frac{Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}-Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}}{\theta }+{\bar{c}}-c_t\right| ^{\frac{4}{3}}\right] \right) ^{\frac{3}{4}}=0. \end{aligned} \end{aligned}$$

Here we used standard arguments of SDE theory, i.e. $\mathbb {E}^t\left[ \sup _{r\in [t,T]}|X_r|^4\right] <\infty$. Moreover, using Lemma 4.9 together with definition of ${\tilde{g}}$, we get that

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0}&\mathbb {E}^t\left[ |{\bar{c}}-c_t|\left| {\tilde{g}}\left( X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}, Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }} \right) -\partial _q g\left( X_{\tau }, Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right) \right| \right] \\&\qquad \le \frac{K}{2}|{\bar{c}}-c_t|\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left| Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}-Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }} \right| \mathbb {1}_{Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\ne Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}}\right] =0, \end{aligned} \end{aligned}$$

where in the last line we used (4.13) in Lemma 4.3. Moreover, using Lipschitz continuity of $\partial _q g$ Lemma (4.10) and that either $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}$ or $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau$, we get that

$$\begin{aligned} \begin{aligned}&\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left| \partial _q g\left( X_\tau ,Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right) -\partial _q g\left( X_\tau ,Q_\tau \right) \right| \right] \\&\le K\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left| Q_{\tau ^{\theta ,{\bar{c}}, t}}-Q_\tau \right| \mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}}\right] =0. \end{aligned} \end{aligned}$$

Hence, merging the last three expressions above into (4.34), we get

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{g(X_\tau , Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }})-g(X_\tau , Q_{\tau ^{\theta ,{\bar{c}}, t}_{\min }})}{\theta }\right] =-({\bar{c}}-c_t) \mathbb {E}^t\left[ \partial _q g\left( X_\tau ,Q_\tau \right) \right] . \end{aligned} \end{aligned}$$

(4.35)

The third element on the right-hand side of (4.33) is equal to

$$\begin{aligned} \begin{aligned}&\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ {\tilde{g}}\left( X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}, {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) \frac{{\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau }{\theta }\right] \\&=\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ {\tilde{g}}\left( X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}, {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) \left( \frac{{\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau }{\theta }-({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right) \right] \\&\quad +\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left( {\tilde{g}}\left( X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}, {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) -\partial _q g\left( X_\tau ,{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) \right) ({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right] \\&\quad +\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left( \partial _q g\left( X_\tau ,{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) -\partial _q g\left( X_\tau ,Q_\tau \right) \right) ({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right] \\&\quad +({\bar{c}}-c_t) \mathbb {E}^t\left[ \partial _q g\left( X_\tau ,Q_\tau \right) \mathbb {1}_{\Lambda (t,{\bar{c}})}\right] . \end{aligned}\nonumber \\ \end{aligned}$$

(4.36)

Using Hölder’s inequality, boundedness of ${\tilde{g}}$ and Lemma 4.7, we get

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t&\left[ \left| {\tilde{g}}\left( X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}, {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) \left( \frac{{\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau }{\theta }-({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right) \right| \right] \\&\le K\left( \mathbb {E}^t\left[ \left( 1+\left| X_\tau \right| \right) ^4\right] \right) ^{\frac{1}{4}}\\&\quad \lim _{\theta \rightarrow 0} \left( \mathbb {E}^t\left[ \left| \frac{{\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau }{\theta }-({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\right| ^{\frac{4}{3}}\right] \right) ^{\frac{3}{4}}=0. \end{aligned} \end{aligned}$$

Here we used standard arguments of SDE theory, i.e. $\mathbb {E}^t\left[ \sup _{r\in [0,T]}|X_r|^4\right] <\infty$. Moreover, using Lemma 4.9 together with definition of ${\tilde{g}}$, we get that

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0}&\mathbb {E}^t\left[ |{\bar{c}}-c_t|\left| {\tilde{g}}\left( X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}, {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) -\partial _q g\left( X_{\tau }, {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) \right| \mathbb {1}_{\Lambda (t,{\bar{c}})}\right] \\&\le \frac{K}{2}|{\bar{c}}-c_t|\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left| {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}-{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right| \mathbb {1}_{{\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}}\ne {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau }\right] =0, \end{aligned} \end{aligned}$$

where in the last line we used (4.26) in Lemma 4.8. Moreover, using Lipschitz continuity of $\partial _q g$ in Assumption 2.8, we have that

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t&\left[ \left| ({\bar{c}}-c_t)\mathbb {1}_{\Lambda (t,{\bar{c}})}\left( \partial _q g\left( X_\tau ,{\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau \right) -\partial _q g\left( X_\tau ,Q_\tau \right) \right) \right| \right] \\&\le K|{\bar{c}}-c_t|\lim _{\theta \rightarrow 0}\mathbb {E}\left[ \left| {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau -Q_\tau \right| \right] =0, \end{aligned} \end{aligned}$$

where in the last equality we have used (4.27) in Lemma 4.8. Hence, merging the last three expressions above into (4.36), we get

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t&\left[ \frac{g(X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})-g(X_\tau , {\hat{Q}}^{\theta ,{\bar{c}}, t}_\tau )}{\theta }\right] =({\bar{c}}-c_t) \mathbb {E}^t\left[ \partial _q g\left( X_\tau ,Q_\tau \right) \mathbb {1}_{\Lambda (t,{\bar{c}})}\right] . \end{aligned} \end{aligned}$$

Combining (2.11), (4.35) and the above expression into (4.33), we conclude that the first line of the right-hand side of (4.32) is equal to

$$\begin{aligned} \begin{aligned}&\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{g(X_{\tau ^{\theta ,{\bar{c}}, t}}, Q^{\theta ,{\bar{c}}, t}_{\tau ^{\theta ,{\bar{c}}, t}})-g(X_\tau , Q_\tau )}{\theta }\right] \\&=-{\bar{g}}(t,{\bar{c}},x,q)-\mathbb {E}^t\left[ ({\bar{c}}-c_t) \partial _q g(X_\tau , Q_\tau )\right] +({\bar{c}}-c_t)\mathbb {E}^t\left[ \partial _q g(X_\tau , Q_\tau ) \mathbb {1}_{\Lambda (t,{\bar{c}})}\right] . \end{aligned} \end{aligned}$$

(4.37)

The second and third lines of right-hand side of (4.32) can be written as

$$\begin{aligned} \begin{aligned}&\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{1}{\theta }\int _{t}^{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\left( f(r, c^{\theta ,{\bar{c}}, t}_r, X_r,Q^{\theta ,{\bar{c}}, t}_r)-f(r, c_r, X_r,Q_r)\right) dr\right] \\&\qquad \qquad -\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{\text {sign}(\tau -\tau ^{\theta ,{\bar{c}}, t})}{\theta }\int _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\tau ^{\theta ,{\bar{c}}, t}_{\max }}f(r, {\hat{c}}^{\theta ,{\bar{c}}, t}_r, X_r,{\hat{Q}}^{\theta ,{\bar{c}}, t}_r) dr\right] \\&\qquad =\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{1}{\theta }\int _{t}^{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\left( f(r, c^{\theta ,{\bar{c}}, t}_r, X_r,Q^{\theta ,{\bar{c}}, t}_r)-f(r, c_r, X_r,Q_r)\right) dr-\xi _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right] \\&\qquad \quad +\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \xi _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right] -\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{\text {sign}(\tau -\tau ^{\theta ,{\bar{c}}, t})}{\theta }\int _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\tau ^{\theta ,{\bar{c}}, t}_{\max }}f(r, {\hat{c}}^{\theta ,{\bar{c}}, t}_r, X_r,{\hat{Q}}^{\theta ,{\bar{c}}, t}_r) dr\right] . \end{aligned} \end{aligned}$$

(4.38)

Using Lemma 4.11, we have that

$$\begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{1}{\theta }\int _{t}^{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\left( f(r, c^{\theta ,{\bar{c}}, t}_r, X_r,Q^{\theta ,{\bar{c}}, t}_r)-f(r, c_r, X_r,Q_r)\right) dr-\xi _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right] =0. \end{aligned}$$

Using Lemma 4.10 ad reminding that either $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}$ or $\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau$, we have that

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \xi _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\right]&=\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \left( \xi _{\tau ^{\theta ,{\bar{c}}, t}}-\xi _{\tau }\right) \mathbb {1}_{\tau ^{\theta ,{\bar{c}}, t}_{\min }=\tau ^{\theta ,{\bar{c}}, t}}\right] + \mathbb {E}^t\left[ \xi _{\tau }\right] =\mathbb {E}^t\left[ \xi _{\tau }\right] . \end{aligned} \end{aligned}$$

Using (2.12), the third limit on the right-hand side of (4.38) converges to ${\bar{f}}(t,{\bar{c}},x,q)$. Combining the above two expressions and (2.12) into (4.38), we get that

$$\begin{aligned} \begin{aligned}&\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{1}{\theta }\int _{t}^{\tau ^{\theta ,{\bar{c}}, t}_{\min }}\left( f(r, c^{\theta ,{\bar{c}}, t}_r, X_r,Q^{\theta ,{\bar{c}}, t}_r)-f(r, c_r, X_r,Q_r)\right) dr\right] \\&\qquad \qquad -\lim _{\theta \rightarrow 0} \mathbb {E}^t\left[ \frac{\text {sign}(\tau -\tau ^{\theta ,{\bar{c}}, t})}{\theta }\int _{\tau ^{\theta ,{\bar{c}}, t}_{\min }}^{\tau ^{\theta ,{\bar{c}}, t}_{\max }}f(r, {\hat{c}}^{\theta ,{\bar{c}}, t}_r, X_r,{\hat{Q}}^{\theta ,{\bar{c}}, t}_r) dr\right] \\&\qquad =\mathbb {E}^t\left[ \xi _\tau \right] -{\bar{f}}(t,{\bar{c}},x,q). \end{aligned} \end{aligned}$$

Then, merging (4.32) with (4.37) and the above expression, also noting $X_t=x$ and $Q_t=q$, we get

$$\begin{aligned} \begin{aligned} \lim _{\theta \rightarrow 0} \frac{v^{c^{\theta ,{\bar{c}}, t}}(t, x,q)-v^c(t,x,q)}{\theta }&= \mathbb {E}^t\left[ -({\bar{c}}-c_t) \partial _q g(X_\tau , Q_\tau ) +\xi _\tau \right] +{\mathcal {G}}(t,{\bar{c}},X_t,Q_t), \end{aligned} \end{aligned}$$

(4.39)

where ${\mathcal {G}}(t,{\bar{c}},x,q)$ is defined in (2.14).

However, from (2.7) and (4.28), we have

$$\begin{aligned} \begin{aligned} \mathbb {E}^t&\left[ -({\bar{c}}-c_t)\partial _q g(X_\tau , Q_\tau ) +\xi _\tau \right] = \mathbb {E}^t\left[ -({\bar{c}}-c_t) Y_\tau +\xi _\tau \right] \\&= \mathbb {E}^t\left[ -({\bar{c}}-c_t) Y_t -({\bar{c}}-c_t)\int _t^\tau dY_r+\xi _t+\int _t^\tau d\xi _r\right] \\&=\mathbb {E}^t\left[ -({\bar{c}}-c_t) Y_t + f(t, {\bar{c}}, X_{t}, Q_{t})-f(t, c_{t}, X_{t}, Q_{t})\right] . \end{aligned} \end{aligned}$$

In the last equality we have used the Optional Stopping Theorem, which ensures that $\int _t^\cdot Z_r dW_r$ is a martingale, whose conditional expectation is 0. Substituting the above expression into (4.39) and using the optimality condition (4.31), we get that for any ${\bar{c}}\ge 0$ and $t\in [0,\tau )$

$$\begin{aligned} \begin{aligned}&\mathbb {E}^t\left[ -({\bar{c}}-c_t) Y_t + f(t, {\bar{c}}, X_{t}, Q_{t})-f(t, c_{t}, X_{t}, Q_{t})\right] +{\mathcal {G}}(t,{\bar{c}},X_t,Q_t)\le 0. \end{aligned} \end{aligned}$$

Since the argument of the first conditional expectation is ${\mathcal {F}}^t$-measurable, using ${\mathcal {H}}$ in (2.9), we get the Hamiltonian condition (2.13). This concludes the proof of Theorem 2.2.

5 Conclusions

In this paper we have proved a new SMP (Theorem 2.2) for an optimal liquidation problem with control-dependent terminal time, which is markedly different in the Hamiltonian condition from that of the standard SMP. We have given a simple example to show that the optimal solution satisfies the SMP in Theorem 2.2 but not the standard SMP in the literature. This is only the first step in the direction of SMP for control-dependent stopping time problems and there remain many open questions to be answered, for example, existence of pointwise limits (2.11) and (2.12), sufficient SMP for optimality, a jump diffusion control-dependent model for $\mathbf {X}$ process, and applications to concrete financial scenarios. We leave these and other questions for future research.