1 Introduction

Impulse control constitutes a versatile framework for controlling real-life stochastic systems. In this type of control, a decision-maker determines intervention times and instantaneous after-intervention states of the controlled process. By doing so, one can affect a continuous time phenomenon in a discrete time manner. Consequently, impulse control attracted considerable attention in the mathematical literature; see e.g. [7, 13, 31] for classic contributions and [6, 14, 24, 26] for more recent results. In addition to generic mathematical properties, impulse control problems were studied with reference to specific applications including i.a. controlling exchange rates, epidemics, and portfolios with transaction costs; see e.g. [23, 30, 32] and references therein.

When looking for an optimal impulse control strategy, one must decide on the optimality criterion. Recently, considerable attention was paid to the so-called risk-sensitive functional given, for any \(\gamma \in \mathbb {R}\), by

$$\begin{aligned} \mu ^\gamma (Z):={\left\{ \begin{array}{ll} \frac{1}{\gamma }\ln \mathbb {E}[\exp (\gamma Z)], &{} \gamma \ne 0,\\ \mathbb {E}[Z], &{} \gamma =0, \end{array}\right. } \end{aligned}$$
(1.1)

where Z is a (random) payoff corresponding to a chosen control strategy; see [19] for a seminal contribution. This functional with \(\gamma =0\) corresponds to the usual linear criterion and the case \(\gamma <0\) is associated with risk-averse preferences; see [8] for a comprehensive overview. Also, the functional with \(\gamma >0\) could be linked to the asymptotics of the power utility function; see [36] for details. Recent comprehensive discussion on the long-run version with \(\mu ^\gamma \) could be found in [10]. We refer also to [28] and references therein for a discussion on the connection between (1.1) and the duality of the large deviations-based criteria.

In this paper we focus on the use of the functional \(\mu ^\gamma \) with \(\gamma >0\). More specifically, we consider the impulse control problem for some continuous time Markov process and construct a solution to the associated Bellman equation which characterises an optimal impulse control strategy. To do this, we study the family of impulse control problems in bounded domains and then extend the analysis to the generic locally compact state space. This idea was used in [2], where PDEs techniques were applied to obtain the characterisation of the controlled diffusions in the risks-sensitive setting. A similar approximation for the the average cost per unit time problem was considered in [37].

The main contribution of this paper is a construction of a solution to the Bellman equation associated with the problem, see Theorem 5.1 for details. It should be noted that we get a bounded solution even though the state space could be unbounded and we assume virtually no ergodicity conditions for the uncontrolled process. Also, note that present results for \(\gamma >0\) complement our recent findings on the impulse control with the risk-averse preferences; see [29] for the dyadic case and [20] for the continuous time framework. Nevertheless, it should be noted that the techniques for \(\gamma <0\) and \(\gamma >0\) are substantially different and it is not possible to directly transform the results in one framework to the other; see e.g. [21, 25] for further discussion.

The structure of this paper is as follows. In Sect. 2 we formally introduce the problem, discuss the assumptions and, in Theorem 2.3, provide a verification argument. Next, in Sect. 3 we consider an auxiliary dyadic problem in a bounded domain and in Theorem 3.1 we construct a solution to the corresponding Bellman equation. This is used in Sect. 4 where we extend our analysis to the unbounded domain with the dyadic time-grid; see Theorem 4.2 for the main result. Next, in Sect. 5 we finally construct a solution to the Bellman equation for the original problem; see Theorem 5.1. Finally, in Appendix A we discuss some properties of the optimal stopping problems that are used in this paper.

2 Preliminaries

Let \(X=(X_t)_{t\ge 0}\) be a continuous time standard Feller–Markov process on a filtered probability space \((\Omega , \mathcal {F}, (\mathcal {F}_t), \mathbb {P})\). The process X takes values in a locally compact separable metric space E endowed with a metric \(\rho \) and the Borel \(\sigma \)-field \(\mathcal {E}\). With any \(x\in E\) we associate a probability measure \(\mathbb {P}_x\) describing the evolution of the process X starting in x; see Section 1.4 in [33] for details. Also, we use \(\mathbb {E}_x\), \(x\in E\), and \(P_t(x,A):=\mathbb {P}_x[X_t\in A]\), \(t\ge 0\), \(x\in E\), \(A\in \mathcal {E}\), for the corresponding expectation operator and the transition probability, respectively. By \(\mathcal {C}_b(E)\) we denote the family of continuous bounded real-valued functions on E. Also, to ease the notation, by \(\mathcal {T}\), \(\mathcal {T}_x\), and \(\mathcal {T}_{x,b}\) we denote the families of stopping times, \(\mathbb {P}_x\) a.s. finite stopping times, and \(\mathbb {P}_x\) a.s. bounded stopping times, respectively. Also, for any \(\delta >0\), by \(\mathcal {T}^\delta \subset \mathcal {T}\), \(\mathcal {T}_x^\delta \subset \mathcal {T}_x\), and \(\mathcal {T}_{x,b}^\delta \subset \mathcal {T}_{x,b}\), we denote the respective subfamilies of dyadic stopping times, i.e., those taking values in the set \(\{0,\delta , 2\delta , \ldots \}\cup \{\infty \}\). Finally, note that in this paper we follow the conventions \(\mathbb {N}:=\{0,1,2, \ldots \}\) and \(\mathbb {R}_{-}:=(-\infty ,0]\).

Throughout this paper we fix some compact set \(U\subseteq E\) and we assume that a decision-maker is allowed to shift the controlled process to U. This is done with the help of an impulse control strategy, i.e. a sequence \(V:=(\tau _i,\xi _i)_{i=1}^\infty \), where \((\tau _i)\) is an increasing sequence of stopping times and \((\xi _i)\) is a sequence of \(\mathcal {F}_{\tau _i}\)-measurable after-impulse states with values in U. With any starting point \(x\in E\) and a strategy V we associate a probability measure \(\mathbb {P}_{(x,V)}\) for the controlled process Y. Under this measure, the process starts at x and follows its usual (uncontrolled) dynamics up to the time \(\tau _1\). Then, it is immediately shifted to \(\xi _1\) and starts its evolution again, etc. More formally, we consider a countable product of filtered spaces \((\Omega , \mathcal {F}, (\mathcal {F}_t))\) and a coordinate process \((X_t^1, X_t^2, \ldots )\). Then, we define the controlled process Y as \(Y_t:=X_t^i\), \(t\in [\tau _{i-1},\tau _i)\) with the convention \(\tau _0\equiv 0\). Under the measure \(\mathbb {P}_{(x,V)}\) we get \(Y_{\tau _i}=\xi _i\); we refer to Chapter V in [31] for the construction details; see also Appendix in [12] and Section 2 in [34]. A strategy \(V=(\tau _i,\xi _i)_{i=1}^\infty \) is called admissible if for any \(x\in E\) we get \(\mathbb {P}_{(x,V)}[\lim _{n\rightarrow \infty }\tau _n=\infty ]=1\). The family of admissible impulse control strategies is denoted by \(\mathbb {V}\). Also, note that, to simplify the notation, by \(Y_{\tau _i^-}:=X_{\tau _i}^i\), \(i\in \mathbb {N}_{*}\), we denote the state of the process right before the ith impulse (yet, possibly, after the jump).

In this paper we study the asymptotics of the impulse control problem given by

$$\begin{aligned} \sup _{V\in \mathbb {V}}J(x,V), \quad x\in E, \end{aligned}$$
(2.1)

where, for any \(x\in E\) and \(V\in \mathbb {V}\), we set

$$\begin{aligned} J(x,V) :=\liminf _{T\rightarrow \infty } \frac{1}{T} \ln \mathbb {E}_{(x,V)}\left[ \exp \left( \int _0^T f(Y_s)ds+\sum _{i=1}^\infty 1_{\{\tau _i\le T\}}c(Y_{\tau _i^-},\xi _i)\right) \right] , \end{aligned}$$
(2.2)

with f denoting the running reward function and c being the shift-cost function, respectively. Note that this could be seen as a long-run standardised version of the functional (1.1) with \(\gamma >0\) applied to the impulse control framework. Here, the standardisation refers to the fact that we do not use directly the parameter \(\gamma \) (apart from its sign). Also, the problem is of the long-run type, i.e. the utility is averaged over time which improves the stability of the results.

The analysis in this paper is based on the approximation of the problem in a bounded domain. Thus, we fix a sequence \((B_m)_{m\in \mathbb {N}}\) of compact sets satisfying \(B_m\subset B_{m+1}\) and \(E=\bigcup _{m=0}^\infty B_m\). Also, we assume that \(U\subset B_0\). Next, we assume the following conditions.

(\(\mathcal {A}1\)):

(Reward/cost functions). The map \(f:E\mapsto \mathbb {R}_{-}\) is a continuous and bounded. Also, the map \(c:E\times U \mapsto \mathbb {R}_{-}\) is continuous, bounded, and strictly non-positive, and satisfies the triangle inequality, i.e. for some \(c_0<0\), we have

$$\begin{aligned} 0>c_0\ge c(x,\xi )\ge c(x,\eta )+c(\eta ,\xi ), \quad x\in E,\, \xi ,\eta \in U. \end{aligned}$$
(2.3)

Also, we assume that c satisfies the uniform limit at infinity condition

$$\begin{aligned} \lim _{\Vert x\Vert , \Vert y\Vert \rightarrow \infty }\sup _{\xi \in U} |c(x,\xi )-c(y,\xi )|=0. \end{aligned}$$
(2.4)
(\(\mathcal {A}2\)):

(Transition probability continuity). For any \(t>0\), the transition probability \(P_t\) is continuous with respect to the total variation norm, i.e. for any sequence \((x_n)\subset E\) converging to \(x\in E\), we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\sup _{A\in \mathcal {E}}|P_t(x_n,A)-P_t(x,A)|=0. \end{aligned}$$
(\(\mathcal {A}3\)):

(Distance control). For any compact set \(\Gamma \subset E\), \(t_0>0\), and \(r_0>0\), we have

$$\begin{aligned} \lim _{r\rightarrow \infty }M_{\Gamma }(t_0,r)=0,\qquad \lim _{t\rightarrow 0} M_{\Gamma }(t,r_0) =0, \end{aligned}$$
(2.5)

where \(M_{\Gamma }(t,r):= \sup _{x\in \Gamma } \mathbb {P}_x[\sup _{s\in [0,t]} \rho (X_s,X_0)\ge r]\), \(t,r>0\).

(\(\mathcal {A}4\)):

(Recurrence of open sets). For any \(m\in \mathbb {N}\), \(x\in B_m\), \(\delta >0\), and any open set \(\mathcal {O}\subset B_m\), we have

$$\begin{aligned} \mathbb {P}_x\left[ \cup _{i=1}^\infty \{X_{i\delta }\in \mathcal {O}\}\right] =1. \end{aligned}$$

Also, we assume that for any \(x\in E\), \(\delta >0\), and \(m\in \mathbb {N}\), we have

$$\begin{aligned} \mathbb {P}_x[\tau _{B_m}<\infty ]=1 \end{aligned}$$
(2.6)

where \(\tau _{B_m}:=\delta \inf \{k\in \mathbb {N}:X_{k\delta }\notin B_m\}\).

Before we proceed, let us comment on these assumptions.

Assumption (A1) states typical reward/cost functions conditions. In particular, the non-positivity assumption for f is merely a technical normalisation. Indeed, for a generic \(\tilde{f}\in \mathcal {C}_b(E)\) we may set \(f(\cdot ):=\tilde{f}(\cdot )-\Vert \tilde{f}\Vert \le 0\) to get

$$\begin{aligned} J^{f}(x,V) = J^{\tilde{f}}(x,V)-\Vert \tilde{f}\Vert , \quad x\in E, \, V\in \mathbb {V}, \end{aligned}$$

where \(J^{f}\) denotes the version of the functional J from (2.2) corresponding to the running reward function f. Next, the conditions for c are standard requirements for the shift-cost functions in the impulse control setting. In particular, inequality (2.3) implies that a decision maker considering an impulse from x to \(\eta \) followed by an immediate impulse from \(\eta \) to \(\xi \) should directly shift the process from x to \(\xi \). This condition is used in Theorem 3.1. Also, (2.4) states that, at infinity, the cost function is almost constant. This is used to extract a (globally) uniformly convergent subsequence of a specific function sequence; see the proofs of Theorem 4.2 and Theorem 5.1. Finally, note that all the assumptions regarding the shift-cost functions are satisfied e.g. for c of the form \(c(x,\xi )=h(\rho (x,\xi ))+c_0\), \(x\in E\), \(\xi \in U\), where \(c_0<0\), the map \(h:\mathbb {R}\rightarrow \mathbb {R}_{-}\) is continuous, bounded, non-increasing and superadditive (i.e. satisfying \(h(x+y)\ge h(x)+h(y)\), \(x,y\in \mathbb {R}\)), and \(\rho \) denotes the underlying metric on E. For example, we may set \(h(x):=-\min (x,K)\), \(x\in \mathbb {R}\), for some constant \(K>0\).

Assumption (A2) states that the transition probabilities \(\mathbb {P}_t(x,\cdot )\) are continuous with respect to the total variation norm. Note that this directly implies that the transition semi-group associated to X is strong Feller, i.e. for any \(t>0\) and a bounded measurable map \(h:E\mapsto \mathbb {R}\), the map \(x\mapsto \mathbb {E}_x[h(X_t)]\) is continuous and bounded.

Assumption (A3) quantifies distance control properties of the underlying process. It states that, for a fixed time horizon, the process with a high probability stays close to its starting point and, with a fixed radius, with a high probability it does not leave the corresponding ball with a sufficiently short time horizon. Note that these properties are automatically satisfied if the transition semi-group is \(\mathcal {C}_0\)-Feller; see Proposition 2.1 in [26] and Proposition 6.4 in [5] for details.

Assumption (A4) states a form of the recurrence property of the process X. It requires that the process visits a sufficiently rich family of sets with unit probability.

It should be noted that the process-related Assumptions (A2)–(A4) are satisfied e.g for non-degenerate ergodic diffusions. Here, the non-degeneracy refers to the existence of a continuous and bounded density \(p_t\) with respect to some measure \(\nu _t\) such that the transition probability satisfies

$$\begin{aligned} P_t(x,A)=\int _A p_t(x,y)\nu _t(dy), \quad t>0, \,x\in E,\, A\in \mathcal {E}. \end{aligned}$$

This directly implies (A2). Next, using Theorem 6.7.2 from [1] we get that diffusions (and, more generally, solutions to stochastic differential equations driven by Lévy processes) are \(\mathcal {C}_0\)-Feller, which combined with Proposition 2.1 in [26] and Proposition 6.4 in [5] shows that (A3) is satisfied. Finally, the ergodicity guarantees (A4).

To solve (2.1), we show the existence of a solution to the impulse control Bellman equation, i.e. a function \(w\in \mathcal {C}_b(E)\) and a constant \(\lambda \in \mathbb {R}\) satisfying

$$\begin{aligned} w(x)=\sup _{\tau \in \mathcal {T}_{x,b}} \ln \mathbb {E}_x\left[ \exp \left( \int _0^\tau (f(X_s)-\lambda )ds +Mw(X_\tau )\right) \right] , \quad x\in E, \end{aligned}$$
(2.7)

where the operator M is given by

$$\begin{aligned} Mh(x):=\sup _{\xi \in U}(c(x,\xi )+h(\xi )), \quad h\in \mathcal {C}_b(E), \, x\in E; \end{aligned}$$

note that in (2.7), the uncontrolled Markov process is considered.

We start with a simple observation giving a lower bound for the constant \(\lambda \) from (2.7). To do this, we define the semi-group type by

$$\begin{aligned} r(f):=\lim _{t\rightarrow \infty } \frac{1}{t}\ln \sup _{x\in E} \mathbb {E}_x\left[ e^{\int _0^t f(X_s)ds}\right] . \end{aligned}$$
(2.8)

We refer to e.g. Proposition 1 in [35] and the discussion following Formula (10.2.2) in [18] for further properties of r(f).

Lemma 2.1

Let \((w,\lambda )\) be a solution to (2.7). Then, we get \(\lambda \ge r(f)\).

Proof

From (2.7), for any \(T\ge 0\), we get

$$\begin{aligned} w(x)\ge \ln \mathbb {E}_x\left[ e^{\int _0^T (f(X_s)-\lambda )ds +Mw(X_T)}\right] . \end{aligned}$$

Thus, using the boundedness of w and Mw, we get

$$\begin{aligned} \Vert w\Vert \ge \sup _{x\in E}\ln \mathbb {E}_x\left[ e^{\int _0^T (f(X_s)-\lambda )ds }\right] -\Vert Mw\Vert . \end{aligned}$$

Consequently, dividing both hand-sides by T and letting \(T\rightarrow \infty \), we get \(0\ge r(f-\lambda )\), which concludes the proof. \(\square \)

Let us now link a solution to (2.7) with the optimal value and an optimal strategy for (2.1). To ease the notation, we recursively define the strategy \(\hat{V}:=(\hat{\tau }_i,\hat{\xi }_i)_{i=1}^\infty \) for \(i\in \mathbb {N}{\setminus }\{0\}\) by

$$\begin{aligned} {\left\{ \begin{array}{ll} \hat{\tau }_{i}&{}:=\inf \{t\ge \hat{\tau }_{i-1}:w(X_t^i)=Mw(X_t^i)\},\\ \hat{\xi }_{i}&{}:= \mathop {\mathrm {arg\,max}}\limits _{\xi \in U}\left( c(X_{\hat{\tau }_{i}}^i,\xi )+w(\xi )\right) 1_{\{\hat{\tau }_{i}<\infty \}}+\xi _0 1_{\{\hat{\tau }_{i}=\infty \}}, \end{array}\right. } \end{aligned}$$
(2.9)

where \(\hat{\tau }_0:=0\) and \(\xi _0\in U\) is some fixed point. First, we show that \(\hat{V}\) is a proper strategy.

Proposition 2.2

The strategy \(\hat{V}\) given by (2.9) is admissible.

Proof

To ease the notation, we define \(N(0,T):=\sum _{i=1}^\infty 1_{\{\hat{\tau }_i\le T\}}\), \(T\ge 0\). We fix some \(T> 0\) and \(x\in E\), and show that we get

$$\begin{aligned} \mathbb {P}_{(x,\hat{V})}[N(0,T)=\infty ]=0. \end{aligned}$$
(2.10)

Recalling (2.9), on the event \(A:=\{\lim _{i\rightarrow \infty }\hat{\tau }_i<+\infty \}\), for any \(n\in \mathbb {N}\), \(n\ge 1\), we get \(w(X_{\hat{\tau }_n}^n)=Mw(X_{\hat{\tau }_n}^n)=c(X_{\hat{\tau }_n}^n,X_{\hat{\tau }_n}^{n+1})+w(X_{\hat{\tau }_n}^{n+1})\). Also, recalling that \(c(x,\xi )\le c_0<0\), \(x\in E\), \(\xi \in U\), for any \(n\in \mathbb {N}\), \(n\ge 1\), we have \(w(X_{\hat{\tau }_n}^{n+1})-w(X_{\hat{\tau }_n}^{n})=-c(X_{\hat{\tau }_n}^{n},X_{\hat{\tau }_n}^{n+1})\ge -c_0>0\). Using this observation and Assumption (A3), we estimate the distance between consecutive impulses which will be used to prove (2.10). More specifically, for any \(k,m\in \mathbb {N}\), \(k,m\ge 1\), we get

$$\begin{aligned}{} & {} \sum _{n=k}^{k+m-2}(w(X_{\hat{\tau }_n}^{n+1})-w(X_{\hat{\tau }_{n+1}}^{n+1}))+(w(X_{\hat{\tau }_{k+m-1}}^{k+m})-w(X_{\hat{\tau }_k}^{k+1})) \nonumber \\{} & {} \quad = w(X_{\hat{\tau }_k}^{k+1})+\sum _{n=k+1}^{k+m-1}(w(X_{\hat{\tau }_n}^{n+1})-w(X_{\hat{\tau }_n}^n))-w(X_{\hat{\tau }_k}^{k+1})\nonumber \\{} & {} \quad = \sum _{n=k+1}^{k+m-1}(w(X_{\hat{\tau }_n}^{n+1})-w(X_{\hat{\tau }_n}^n))\ge -(m-1)c_0; \end{aligned}$$
(2.11)

it should be noted that the specific values for k and m will be determined later. Using the continuity of w we may find \(K>0\) such that \(\sup _{x,y\in U}(w(x)-w(y))\le K\). Let \(m\in \mathbb {N}\) be big enough to get \(-(m-1)\frac{c_0}{2}>K\). Thus, noting that \(X_{\hat{\tau }_{k+m-1}}^{k+m}, X_{\hat{\tau }_k}^{k+1}\in U\), we have \((w(X_{\hat{\tau }_{k+m-1}}^{k+m})-w(X_{\hat{\tau }_k}^{k+1}))\le K<-(m-1)\frac{c_0}{2}\). Consequently, recalling (2.11), on A, we get

$$\begin{aligned} \sum _{n=k}^{k+m-2}(w(X_{\hat{\tau }_n}^{n+1})-w(X_{\hat{\tau }_{n+1}}^{n+1}))\ge -(m-1)\frac{c_0}{2}. \end{aligned}$$
(2.12)

Recalling the compactness of U and the continuity of w we may find \(r>0\) such that for any \(x\in U\) and \(y\in E\) satisfying \(\rho (x,y)<r\) we get \(|w(x)-w(y)|<-\frac{c_0}{2}\). Let us now consider the family of events

$$\begin{aligned} B_k:=\bigcap _{n=k}^{k+m-2}\{\rho (X_{\hat{\tau }_n}^{n+1},X_{\hat{\tau }_{n+1}}^{n+1})<r\}, \quad k\in \mathbb {N},\, k\ge 1, \end{aligned}$$
(2.13)

and note that, for any \(k\in \mathbb {N}\), \(k\ge 1\), on \(B_k\cap A\) we have \(\sum _{n=k}^{k+m-2}(w(X_{\hat{\tau }_n}^{n+1})-w(X_{\hat{\tau }_{n+1}}^{n+1}))< -(m-1)\frac{c_0}{2}\). Thus, recalling (2.12), for any \(k\in \mathbb {N}\), \(k\ge 1\), we get \(\mathbb {P}_{(x_0,\hat{V})}[ B_k\cap A]=0\) and, in particular, we have

$$\begin{aligned} \mathbb {P}_{(x_0,\hat{V})}[B_k\cap \{N(0,T)=\infty \}]=0. \end{aligned}$$
(2.14)

Let us now show that \(\limsup _{k\rightarrow \infty }\mathbb {P}_{(x_0,\hat{V})}[B_k^c\cap \{N(0,T)=\infty \}]=0\). Noting that \(\{N(0,T)=\infty \}=\{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\), for any \(t_0>0\) and \(k\in \mathbb {N}\), \(k\ge 1\), we get

$$\begin{aligned}&\mathbb {P}_{(x_0,\hat{V})}\left[ B_k^c\cap \{N(0,T)=\infty \}\right] \nonumber \\&\le \mathbb {P}_{(x_0,\hat{V})}\left[ \left( \bigcup _{n=k}^{k+m-2}\{\rho (X_{\hat{\tau }_n}^{n+1},X_{\hat{\tau }_{n+1}}^{n+1})\ge r\}\cap \{\hat{\tau }_{n+1}-\hat{\tau }_n\le t_0\}\right) \cap \{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\right] \nonumber \\&+\mathbb {P}_{(x_0,\hat{V})}\left[ \left( \bigcup _{n=k}^{k+m-2}\{\rho (X_{\hat{\tau }_n}^{n+1},X_{\hat{\tau }_{n+1}}^{n+1})\ge r\}\cap \{\hat{\tau }_{n+1}-\hat{\tau }_n> t_0\}\right) \cap \{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\right] \nonumber \\&\le \mathbb {P}_{(x_0,\hat{V})}\left[ \bigcup _{n=k}^{k+m-2}\{\sup _{t\in [0,t_0]}\rho (X_{\hat{\tau }_{n}}^{n+1},X_{\hat{\tau }_n+t}^{n+1})\ge r\}\cap \{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\right] \nonumber \\&+\mathbb {P}_{(x_0,\hat{V})}\left[ \bigcup _{n=k}^{k+m-2}\{\hat{\tau }_{n+1}-\hat{\tau }_n> t_0\}\cap \{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\right] . \end{aligned}$$
(2.15)

Using Assumption (A3), for any \(\varepsilon >0\), we may find \(t_0>0\), such that

$$\begin{aligned} \sup _{x\in U} \mathbb {P}_x\left[ \sup _{t\in [0,t_0]}\rho (X_0,X_t)\ge r\right] \le \frac{\varepsilon }{m-1}. \end{aligned}$$
(2.16)

Thus, using the strong Markov property and noting that \(X_{\hat{\tau }_n}^{n+1}\in U\), for any \(k\in \mathbb {N}\), \(k\ge 1\), we get

$$\begin{aligned}{} & {} \mathbb {P}_{(x_0,\hat{V})}\left[ \bigcup _{n=k}^{k+m-2}\{\sup _{t\in [0,t_0]}\rho (X_{\hat{\tau }_{n}}^{n+1},X_{\hat{\tau }_n+t}^{n+1})\ge r\}\cap \{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\right] \nonumber \\{} & {} \quad \le \sum _{n=k}^{k+m-2}\mathbb {P}_{(x_0,\hat{V})}\left[ \{\sup _{t\in [0,t_0]}\rho (X_{\hat{\tau }_{n}}^{n+1},X_{\hat{\tau }_n+t}^{n+1})\ge r\}\cap \{\hat{\tau }_n\le T\}\right] \nonumber \\{} & {} \quad =\sum _{n=k}^{k+m-2}\mathbb {P}_{(x_0,\hat{V})}\left[ \{\hat{\tau }_n\le T\} \mathbb {P}_{X_{\hat{\tau }_n}^{n+1}}\left[ \sup _{t\in [0,t_0]}\rho (X_{0},X_{t})\ge r \right] \right] \le \varepsilon . \end{aligned}$$
(2.17)

Recalling that \(\varepsilon >0\) was arbitrary, for any \(k\in \mathbb {N}\), \(k\ge 1\), we get

$$\begin{aligned} \mathbb {P}_{(x_0,\hat{V})}\left[ \bigcup _{n=k}^{k+m-2}\{\sup _{t\in [0,t_0]}\rho (X_{\hat{\tau }_{n}}^{n+1},X_{\hat{\tau }_n+t}^{n+1})\ge r\}\cap \{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\right] =0. \end{aligned}$$
(2.18)

Now, to ease the notation, let \(C_k:=\bigcup _{n=k}^{\infty }\{\hat{\tau }_{n+1}-\hat{\tau }_n> t_0\}\cap \{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\), \(k\in \mathbb {N}\), \(k\ge 1\), and note that \(C_{k+1}\subset C_k\), \(k\in \mathbb {N}\), \(k\ge 1\). We show that

$$\begin{aligned} \lim _{k\rightarrow \infty }\mathbb {P}_{(x_0,\hat{V})}\left[ C_k\right] =0. \end{aligned}$$

For the contradiction, assume that \(\lim _{k\rightarrow \infty }\mathbb {P}_{(x_0,\hat{V})}\left[ C_k\right] >0\). Consequently, we get \(\mathbb {P}_{(x_0,\hat{V})}\left[ \bigcap _{k=1}^\infty C_k\right] >0\). Note that for any \(\omega \in \bigcap _{k=1}^\infty C_k\) we have \(\lim _{i\rightarrow \infty }\hat{\tau }_i(\omega )\le T\). In particular, we may find \(i_0\in \mathbb {N}\) such that for any \(n\ge i_0\) we get \(\hat{\tau }_{n+1}(\omega )-\hat{\tau }_n(\omega )\le \frac{t_0}{2}\). This leads to the contradiction as from the fact that \(\omega \in \bigcap _{k=1}^\infty C_k\) we also get

$$\begin{aligned} \omega \in \bigcap _{k=1}^\infty \bigcup _{n=k}^{\infty }\{\hat{\tau }_{n+1}-\hat{\tau }_n> t_0\}\subset \bigcup _{n=i_0}^{\infty }\{\hat{\tau }_{n+1}-\hat{\tau }_n> t_0\}. \end{aligned}$$

Consequently, we get \(\lim _{k\rightarrow \infty }\mathbb {P}_{(x_0,\hat{V})}\left[ C_k\right] =0\) and, in particular, we get

$$\begin{aligned} \limsup _{k\rightarrow \infty }\mathbb {P}_{(x_0,\hat{V})}\left[ \bigcup _{n=k}^{k+m-2}\{\hat{\tau }_{n+1}-\hat{\tau }_n> t_0\}\cap \{\lim _{i\rightarrow \infty }\hat{\tau }_i\le T\}\right] \le \lim _{k\rightarrow \infty }\mathbb {P}_{(x_0,\hat{V})}\left[ C_k\right] =0. \end{aligned}$$

Hence, recalling (2.15) and (2.18), we get

$$\begin{aligned} \limsup _{k\rightarrow \infty }\mathbb {P}_{(x_0,\hat{V})}\left[ B_k^c\cap \{N(0,T)=\infty \}\right] =0. \end{aligned}$$

Thus, recalling (2.14), for any \(k\in \mathbb {N}\), \(k\ge 1\), we obtain

$$\begin{aligned} \mathbb {P}_{(x_0,\hat{V})}\left[ N(0,T)=\infty \right] =\mathbb {P}_{(x_0,\hat{V})}\left[ B_k^c\cap \{N(0,T)=\infty \}\right] , \end{aligned}$$

and letting \(k\rightarrow \infty \), we conclude the proof of (2.10). \(\square \)

Now, we show the verification result linking (2.7) with the optimal value and an optimal strategy for (2.1).

Theorem 2.3

Let \((w,\lambda )\) be a solution to (2.7) with \(\lambda >r(f)\). Then, we get

$$\begin{aligned} \lambda = \sup _{V\in \mathbb {V}} J(x,V) = J(x,\hat{V}), \quad x\in E, \end{aligned}$$

where the strategy \(\hat{V}\) is given by (2.9).

Proof

The proof is based on the argument from Theorem 4.4 in [20] thus we show only an outline. First, we show that \(\lambda =J(x,\hat{V})\), \(x\in E\), where the strategy \(\hat{V}\) is given by (2.9). Let us fix \(x\in E\). Then, combining the argument used in Lemma 7.1 in [5] and Proposition A.3, we get that the process

$$\begin{aligned} e^{\int _0^{\hat{\tau }_1\wedge T} (f(X_s^1)-\lambda )ds +w(X^1_{\hat{\tau }_1\wedge T})}, \quad T\ge 0, \end{aligned}$$

is a \(\mathbb {P}_{(x,\hat{V})}\)-martingale. Noting that on the event \(\{\hat{\tau }_{k+1}<T\}\) we get \(w(X^{k+1}_{\hat{\tau }_{k+1}})=Mw(X^{k+1}_{\hat{\tau }_{k+1}})=c(X^{k+1}_{\hat{\tau }_{k+1}},\hat{\xi }_{k+1})+w(\hat{\xi }_{k+1})\), \(k\in \mathbb {N}\), for any \(n\in \mathbb {N}\) we recursively get

$$\begin{aligned} e^{w(x)}&= {{\,\mathrm{\mathbb {E}}\,}}_{(x,\hat{V})} \left[ e^{\int _0^{\hat{\tau }_1\wedge T} (f(Y_s)-\lambda )ds +w(X^1_{\hat{\tau }_1\wedge T})}\right] \nonumber \\&={{\,\mathrm{\mathbb {E}}\,}}_{(x,\hat{V})} \left[ e^{\int _0^{\hat{\tau }_1\wedge T} (f(Y_s)-\lambda )ds +1_{\{\hat{\tau }_1< T\}} c(X^1_{\hat{\tau }_1},X^2_{\hat{\tau }_1})+1_{\{\hat{\tau }_1< T\}}w(X^2_{\hat{\tau }_1})+1_{\{\hat{\tau }_1\ge T\}} w(X_T^1) }\right] \nonumber \\&= {{\,\mathrm{\mathbb {E}}\,}}_{(x,\hat{V})} \left[ e^{\int _0^{\hat{\tau }_n\wedge T} (f(Y_s)-\lambda )ds +\sum _{i=1}^n 1_{\{\hat{\tau }_i< T\}} c(X^{i}_{\hat{\tau }_i},X^{i+1}_{\hat{\tau }_i})}\times \right. \nonumber \\&\left. \times e^{\sum _{i=1}^n 1_{\{\hat{\tau }_{i-1}< T\le \hat{\tau }_{i}\}} w(X_T^{i})+1_{\{\hat{\tau }_n<T\}} w(X_{\hat{\tau }_n}^{n+1})}\right] . \end{aligned}$$
(2.19)

Recalling Proposition 2.2 we get \(\hat{\tau }_n\rightarrow \infty \) as \(n\rightarrow \infty \). Thus, letting \(n\rightarrow \infty \) in (2.19) and using Lebesgue’s dominated convergence theorem we get

$$\begin{aligned} e^{w(x)}={{\,\mathrm{\mathbb {E}}\,}}_{(x,\hat{V})} \left[ e^{\int _0^{T} (f(Y_s)-\lambda )ds +\sum _{i=1}^\infty 1_{\{\hat{\tau }_i< T\}} c(X^{i}_{\hat{\tau }_i},X^{i+1}_{\hat{\tau }_i})+\sum _{i=1}^\infty 1_{\{\hat{\tau }_{i-1}< T\le \hat{\tau }_{i}\}} w(X_T^{i})}\right] . \end{aligned}$$

Thus, recalling the boundedness of w, taking the logarithm of both sides, dividing by T, and letting \(T\rightarrow \infty \) we obtain

$$\begin{aligned} \lambda = \liminf _{T\rightarrow \infty }{{\,\mathrm{\mathbb {E}}\,}}_{(x,\hat{V})} \left[ e^{\int _0^{T} f(Y_s)ds +\sum _{i=1}^\infty 1_{\{\hat{\tau }_i< T\}} c(X^{i}_{\hat{\tau }_i},X^{i+1}_{\hat{\tau }_i})}\right] . \end{aligned}$$

Second, let us fix some \(x\in E\) and an admissible strategy \(V=(\xi _i,\tau _i)_{i=1}^\infty \in \mathbb {V}\). We show that \(\lambda \ge J(x,V)\). Using the argument from Lemma 7.1 in [5] and Proposition A.3, we get that the process

$$\begin{aligned} e^{\int _0^{\tau _1\wedge T} (f(X_s^1)-\lambda )ds +w(X^1_{\tau _1\wedge T})}, \quad T\ge 0, \end{aligned}$$

is a \(\mathbb {P}_{(x,V)}\)-supermartingale. Noting that on the event \(\{\tau _{k+1}<T\}\) we have

$$\begin{aligned} w(X^{k+1}_{\tau _{k+1}})\ge Mw(X^{k+1}_{\tau _{k+1}})\ge c(X^{k+1}_{\tau _{k+1}},\xi _{k+1})+w(\xi _{k+1}), \quad k\in \mathbb {N}, \end{aligned}$$

for any \(n\in \mathbb {N}\) we recursively get

$$\begin{aligned} e^{w(x)}&\ge {{\,\mathrm{\mathbb {E}}\,}}_{(x,{V})} \left[ e^{\int _0^{\tau _1\wedge T} (f(Y_s)-\lambda )ds +w(X^1_{\tau _1\wedge T})}\right] \nonumber \\&\ge {{\,\mathrm{\mathbb {E}}\,}}_{(x,{V})} \left[ e^{\int _0^{\tau _1\wedge T} (f(Y_s)-\lambda )ds +1_{\{\tau _1< T\}} c(X^1_{\tau _1},X^2_{\tau _1})+1_{\{\tau _1< T\}}w(X^2_{\tau _1})+1_{\{\tau _1\ge T\}} w(X_T^1) }\right] \nonumber \\&\ge {{\,\mathrm{\mathbb {E}}\,}}_{(x,{V})} \left[ e^{\int _0^{\tau _n\wedge T} (f(Y_s)-\lambda )ds +\sum _{i=1}^n 1_{\{\tau _i< T\}} c(X^{i}_{\tau _i},X^{i+1}_{\tau _i})}\times \right. \nonumber \\&\left. \times e^{\sum _{i=1}^n 1_{\{\tau _{i-1}< T\le \tau _{i}\}} w(X_T^{i})+1_{\{\tau _n<T\}} w(X_{\tau _n}^{n+1})}\right] . \end{aligned}$$
(2.20)

Recalling the admissibility of V, we get \(\tau _n\rightarrow \infty \) as \(n\rightarrow \infty \). Thus, letting \(n\rightarrow \infty \) in (2.20) and using Fatou’s lemma, we get

$$\begin{aligned} e^{w(x)}\ge {{\,\mathrm{\mathbb {E}}\,}}_{(x,{V})} \left[ e^{\int _0^{T} (f(Y_s)-\lambda )ds +\sum _{i=1}^\infty 1_{\{\tau _i< T\}} c(X^{i}_{\tau _i},X^{i+1}_{\tau _i})+\sum _{i=1}^\infty 1_{\{\tau _{i-1}< T\le \tau _{i}\}} w(X_T^{i})}\right] . \end{aligned}$$

Thus, taking the logarithm of both sides, dividing by T, and letting \(T\rightarrow \infty \), we get

$$\begin{aligned} \lambda \ge \liminf _{T\rightarrow \infty }{{\,\mathrm{\mathbb {E}}\,}}_{(x,{V})} \left[ e^{\int _0^{T} f(Y_s)ds +\sum _{i=1}^\infty 1_{\{\tau _i< T\}} c(X^{i}_{\tau _i},X^{i+1}_{\tau _i})}\right] , \end{aligned}$$

which concludes the proof. \(\square \)

In the following sections we construct a solution to (2.7). In the construction we approximate the underlying problem using the dyadic time-grid. Also, we consider a version of the problem in the bounded domain.

3 Dyadic Impulse Control in a Bounded Set

In this section we consider a version of (2.1) with a dyadic-time-grid and obligatory impulses when the process leaves some compact set. In this way, we construct a solution to the bounded-domain dyadic counterpart of (2.7). More specifically, let us fix some \(\delta >0\) and \(m\in \mathbb {N}\). We show the existence of a map \(w_{\delta }^m\in \mathcal {C}_b(B_m)\) and a constant \(\lambda _\delta ^m\in \mathbb {R}\) satisfying

$$\begin{aligned} w_{\delta }^m(x)=\sup _{\tau \in \mathcal {T}_{x,b}^{\delta }}\ln \mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _\delta ^m )ds+Mw_\delta ^m(X_{\tau \wedge \tau _{B_m}})}\right] , \quad x\in B_m. \end{aligned}$$
(3.1)

In fact, we start with the analysis of an associated one-step equation. More specifically, we show the existence of a constant \(\lambda _{\delta }^m\in \mathbb {R}\) and a map \(w_{\delta }^m\in \mathcal {C}_b(B_m)\) satisfying

$$\begin{aligned} w_{\delta }^m(x)&= \max \left( \ln \mathbb {E}_x\left[ e^{\int _0^\delta (f(X_s)-\lambda _\delta ^m)ds+1_{\{X_{\delta }\in B_m\}}w_{\delta }^m(X_{\delta })+1_{\{X_{\delta }\notin B_m\}}Mw_{\delta }^m(X_{\delta })}\right] \right. ,\nonumber \\&M w_{\delta }^m(x)\Big ), \quad x\in B_m,\nonumber \\ w_{\delta }^m(x)&= Mw_{\delta }^m(x), \quad x\notin B_m; \end{aligned}$$
(3.2)

see Theorem 3.1 for details. Then, we link (3.2) with (3.1) in Theorem 3.4.

In the proof of Theorem 3.1 we use the Krein–Rutman theorem to get the existence of a positive eigenvalue with a non-negative eigenfunction to the specific non-linear operator associated with (3.2). This technique was primarily used in the context of diffusions; see e.g. [3, 4, 9] and references therein. See also [38] for the use with discrete time risk-sensitive Markov decision processes. It should be noted that, due to the difficulty of the verification of the theorem assumptions (including the complete continuity of a suitable operator), this approach is applied primarily in the compact state space setting and the extension to a non-compact space requires some additional arguments.

Theorem 3.1

There exists a constant \(\lambda _{\delta }^m>0\) and a map \(w_{\delta }^m\in \mathcal {C}_b(B_m)\) such that (3.2) is satisfied and we get \(\sup _{\xi \in U} w^m_\delta (\xi )=0\).

Proof

The idea of the proof is to use the Krein-Rutman theorem to get an eigenvalue and an eigenvector of a suitable operator. More specifically, we consider a cone of non-negative continuous and bounded functions \(\mathcal {C}^+_b(B_m)\subset \mathcal {C}_b(B_m)\) and, for any \(h\in \mathcal {C}^+_b(B_m)\), we define the operators

$$\begin{aligned} \tilde{M}h(x)&:=\sup _{\xi \in U} e^{c(x,\xi )}h(\xi ), \quad x\in E,\\ \tilde{P}_{\delta }^m h(x)&:=\mathbb {E}_x\left[ e^{\int _0^\delta f(X_s)ds}\left( 1_{\{X_{\delta }\in B_m\}}h(X_{\delta })+1_{\{X_{\delta }\notin B_m\}}\tilde{M}h(X_{\delta })\right) \right] , \quad x\in B_m, \\ \tilde{T}_{\delta }^m h(x)&:=\max \left( \tilde{P}_{\delta }^m h(x), \tilde{M} \tilde{P}_{\delta }^m h(x)\right) , \quad x\in B_m. \end{aligned}$$

Now, we use the Krein-Rutman theorem to show that \(\tilde{T}_{\delta }^m\) admits a positive eigenvalue and a non-negative eigenfunction; see Theorem 4.3 in [11] for details. We start with verifying the assumptions. First, note that \(\tilde{T}_{\delta }^m\) is positively homogeneous, monotonic increasing, and we have

$$\begin{aligned} \tilde{T}_{\delta }^m \mathbbm {1}(x)\ge e^{-\delta \Vert f\Vert -\Vert c\Vert }\mathbbm {1}(x), \quad x\in B_m, \end{aligned}$$

where \(\mathbbm {1}\) denotes the function identically equal to 1 on \(B_m\). Also, using Assumption (A2), we get that \(\tilde{T}_{\delta }^m\) transforms \(\mathcal {C}^+_b(B_m)\) into itself and it is continuous with respect to the supremum norm. Let us now show that \(\tilde{T}_{\delta }^m\) is in fact completely continuous. To see this, let \((h_n)_{n\in \mathbb {N}}\subset \mathcal {C}^+_b(B_m)\) be a bounded (by some constant \(K>0\)) sequence; using the Arzelà-Ascoli theorem we show that it is possible to find a convergent subsequence of \((\tilde{T}_{\delta }^m h_n)_{n\in \mathbb {N}}\). Note that, for any \(n\in \mathbb {N}\), we get

$$\begin{aligned} \Vert \tilde{T}_{\delta }^m h_n\Vert \le e^{\delta \Vert f\Vert }K, \end{aligned}$$

hence \((\tilde{T}_{\delta }^m h_n)\) is uniformly bounded. Next, let us fix some \(\varepsilon >0\), \(x\in B_m\), and \((x_k)\subset B_m\) such that \(x_k\rightarrow x\) as \(k\rightarrow \infty \). Also, to ease the notation, for any \(n\in \mathbb {N}\), we set \(H_n(x):=1_{\{x\in B_m\}}h_n(x)+1_{\{x\notin B_m\}}\tilde{M}h_n(x)\), \(x\in E\), and note that \(H_n\) are measurable functions bounded by 2K uniformly in \(n\in \mathbb {N}\). Then, for any \(n,k\in \mathbb {N}\), we get

$$\begin{aligned} | \tilde{T}_{\delta }^m h_n(x)-\tilde{T}_{\delta }^m h_n(x_k)|&\le \left| \mathbb {E}_x\left[ e^{\int _0^\delta f(X_s)ds}H_n(X_\delta )\right] - \mathbb {E}_{x_k}\left[ e^{\int _0^\delta f(X_s)ds}H_n(X_\delta )\right] \right| \nonumber \\&+ |\tilde{M}\tilde{P}_{\delta }^m h_n(x)-\tilde{M}\tilde{P}_{\delta }^m h_n({x_k})|. \end{aligned}$$
(3.3)

Also, using Assumption (A1), we may find \(k\in \mathbb {N}\) big enough such that, for any \(n\in \mathbb {N}\), we obtain

$$\begin{aligned} |\tilde{M}\tilde{P}_{\delta }^m h_n(x)-\tilde{M}\tilde{P}_{\delta }^m h_n({x_k})|\le e^{\delta \Vert f\Vert }K \sup _{\xi \in U} |e^{c(x,\xi )}-e^{c(x_k,\xi )}|\le \frac{\varepsilon }{2}. \end{aligned}$$
(3.4)

Next, note that for any \(u\in (0,\delta )\) and \(n,k\in \mathbb {N}\), we get

$$\begin{aligned}&\left| \mathbb {E}_x\left[ e^{\int _0^\delta f(X_s)ds}H_n(X_\delta )\right] - \mathbb {E}_{x_k}\left[ e^{\int _0^\delta f(X_s)ds}H_n(X_\delta )\right] \right| \nonumber \\&\le \left| \mathbb {E}_x\left[ \left( e^{\int _0^\delta f(X_s)ds}-e^{\int _u^\delta f(X_s)ds}\right) H_n(X_\delta )\right] \right| \nonumber \\&+ \left| \mathbb {E}_{x_k}\left[ \left( e^{\int _0^\delta f(X_s)ds}-e^{\int _u^\delta f(X_s)ds}\right) H_n(X_\delta )\right] \right| \nonumber \\&+ \left| \mathbb {E}_{x_k}\left[ e^{\int _u^\delta f(X_s)ds}H_n(X_\delta )\right] -\mathbb {E}_{x}\left[ e^{\int _u^\delta f(X_s)ds}H_n(X_\delta )\right] \right| . \end{aligned}$$
(3.5)

Also, using the inequality \(|e^{y}-e^{z}|\le e^{\max (y,z)}|y-z|\), \(y,z\in \mathbb {R}\), we may find \(u>0\) small enough such that, for any \(n,k\in \mathbb {N}\), we get

$$\begin{aligned} \left| \mathbb {E}_{x_k}\left[ \left( e^{\int _0^\delta f(X_s)ds}-e^{\int _u^\delta f(X_s)ds}\right) H_n(X_\delta )\right] \right| \le 2Ke^{\delta \Vert f\Vert }u\Vert f\Vert \le \frac{\varepsilon }{6}. \end{aligned}$$
(3.6)

Next, setting \(F_n^u(x):=\mathbb {E}_x\left[ e^{\int _0^{\delta -u}f(X_s)ds}H_n(X_{\delta -u})\right] \), \(n\in \mathbb {N}\), \(x\in E\), and using the Markov property combined with Assumption (A2), we may find \(k\in \mathbb {N}\) big enough such that for any \(n\in \mathbb {N}\), we get

$$\begin{aligned}{} & {} \left| \mathbb {E}_{x_k}\left[ e^{\int _u^\delta f(X_s)ds}H_n(X_\delta )\right] -\mathbb {E}_{x}\left[ e^{\int _u^\delta f(X_s)ds}H_n(X_\delta )\right] \right| = |\mathbb {E}_{x_k}[F_n^u(X_u)]-\mathbb {E}_{x}[F_n^u(X_u)]|\\{} & {} \quad \le 2K e^{\delta \Vert f\Vert } \sup _{A\in \mathcal {E}}|P_u(x_k,A)-P_u(x,A)|\le \frac{\varepsilon }{6}. \end{aligned}$$

Thus, recalling (3.5)–(3.6), we get that for \(k\in \mathbb {N}\) big enough and any \(n\in \mathbb {N}\), we get \(\left| \mathbb {E}_x\left[ e^{\int _0^\delta f(X_s)ds}H_n(X_\delta )\right] - \mathbb {E}_{x_k}\left[ e^{\int _0^\delta f(X_s)ds}H_n(X_\delta )\right] \right| \le \frac{\varepsilon }{2}\). This combined with (3.3)–(3.4) shows \( | \tilde{T}_{\delta }^m h_n(x)-\tilde{T}_{\delta }^m h_n(x_k)|\le \varepsilon \) for \(k\in \mathbb {N}\) big enough and any \(n\in \mathbb {N}\), which proves the equicontinuity of the family \((\tilde{T}_{\delta }^m h_n)_{n\in \mathbb {N}}\). Consequently, using the Arzelà-Ascoli theorem, we may find a uniformly (in \(x\in B_m\)) convergent subsequence of \((\tilde{T}_{\delta }^m h_n)_{n\in \mathbb {N}}\) and the operator \(\tilde{T}_{\delta }^m\) is completely continuous. Thus, using the Krein-Rutman theorem we conclude that there exists a constant \(\tilde{\lambda }_{\delta }^m>0\) and a non-zero map \(h_{\delta }^m\in \mathcal {C}^+_b(B_m)\) such that

$$\begin{aligned} \tilde{T}_{\delta }^m h_{\delta }^m(x)=\tilde{\lambda }_{\delta }^m h_{\delta }^m(x), \quad x\in B_m. \end{aligned}$$
(3.7)

After a possible normalisation, we assume that \(\sup _{\xi \in U}h_{\delta }^m(\xi )=1\).

Let us now show that \(h_{\delta }^m(x)>0\), \(x\in B_m\). To see this, let us define \(D:=e^{-\delta \Vert f\Vert } \frac{1}{\tilde{\lambda }_{\delta }^m}\) and let \(\mathcal {O}_h\subset B_m\) be an open set such that

$$\begin{aligned} \inf _{x\in O_h}h^m_{\delta }(x)>0; \end{aligned}$$
(3.8)

note that this set exists thanks to the continuity of \(h_{\delta }^m\) and the fact that \(h_{\delta }^m\) is non-zero. Next, using (3.7), we have

$$\begin{aligned} h_{\delta }^m(x)\ge D\mathbb {E}_x\left[ 1_{\{X_\delta \in \mathcal {O}_h\}}h_{\delta }^m(X_\delta )+1_{\{X_\delta \in B_m\setminus \mathcal {O}_h\}}h_{\delta }^m(X_\delta )\right] , \quad x\in B_m. \end{aligned}$$

Then, for any \(n\in \mathbb {N}\), we inductively get

$$\begin{aligned} h_{\delta }^m(x)&\ge D\mathbb {E}_x[1_{\{X_{\delta } \in \mathcal {O}_h \}}h_{\delta }^m(X_{\delta })] \\&+\sum _{i=2}^n D^i\mathbb {E}_x\left[ 1_{\{X_{\delta } \in B_m\setminus \mathcal {O}_h,X_{2\delta } \in B_m\setminus \mathcal {O}_h, \ldots , X_{(i-1)\delta } \in B_m\setminus \mathcal {O}_h, X_{i\delta } \in \mathcal {O}_h \}}h_{\delta }^m(X_{i\delta })\right] \\&+ D^n\mathbb {E}_x\left[ 1_{\{X_{\delta } \in B_m\setminus \mathcal {O}_h,X_{2\delta } \in B_m\setminus \mathcal {O}_h, \ldots , X_{i\delta } \in B_m\setminus \mathcal {O}_h\}}h_{\delta }^m(X_{n\delta })\right] , \, x\in B_m. \end{aligned}$$

Thus, letting \(n\rightarrow \infty \) and using Assumption (A4) combined with (3.8), we show \(h_{\delta }^m(x)>0\) for any \(x\in B_m\).

Next, we define \(w^m_\delta (x):=\ln h^m_\delta (x)\), \(x\in B_m\), and \(\lambda ^m_\delta :=\frac{1}{\delta }\ln \tilde{\lambda }^m_\delta \). Thus, from (3.7), we get that the pair \((w^m_\delta , \lambda ^m_\delta )\) satisfies

$$\begin{aligned} \tilde{T}_{\delta }^m e^{w_{\delta }^m}(x)=e^{\delta \lambda _{\delta }^m} e^{w_{\delta }^m(x)}, \quad x\in B_m, \quad \text {and}\quad \sup _{\xi \in U}w_{\delta }^m(\xi )=0. \end{aligned}$$

In fact, using (2.3) from Assumption (A1) and the argument from Theorem 3.1 in [20], we have

$$\begin{aligned} w_{\delta }^m(x)&= \max \left( \ln \mathbb {E}_x\left[ e^{\int _0^\delta (f(X_s)-\lambda _\delta ^m)ds+1_{\{X_{\delta }\in B_m\}}w_{\delta }^m(X_{\delta })+1_{\{X_{\delta }\notin B_m\}}Mw_{\delta }^m(X_{\delta })}\right] \right. ,\nonumber \\&M w_{\delta }^m(x)\Big ), \quad x\in B_m. \end{aligned}$$

Finally, we extend the definition of \(w^m_{\delta }\) to the full space E by setting

$$\begin{aligned} w^m_{\delta }(x):=Mw^m_{\delta }(x), \quad x\notin B_m; \end{aligned}$$

note that the definition is correct since, at the right-hand side, we need to evaluate \(w^m_{\delta }\) only at the points from \(U\subset B_0\subset B_m\) and this map is already defined there. \(\square \)

As we show now, Eq. (3.2) may be linked to a specific martingale characterisation.

Proposition 3.2

Let \((w_{\delta }^m, \lambda ^m_\delta )\) be a solution to (3.2). Then, for any \(x\in B_m\), we get that the process

$$\begin{aligned} z_{\delta }^m(n):=e^{\int _0^{(n\delta )\wedge \tau _{B_m}} (f(X_s)-\lambda _\delta ^m )ds+w_\delta ^m(X_{(n\delta )\wedge \tau _{B_m}})}, \quad n\ge 0, \end{aligned}$$

is a \(\mathbb {P}_x\)-supermartingale. Also, the process

$$\begin{aligned} z_{\delta }^m(n\wedge (\hat{\tau }^m_\delta /\delta )), \quad n\in \mathbb {N}, \end{aligned}$$

is a \(\mathbb {P}_x\)-martingale, where \(\hat{\tau }^m_\delta :=\delta \inf \{k\in \mathbb {N}:w_\delta ^m(X_{k\delta })=M w_\delta ^m(X_{k\delta })\}\).

Proof

To ease the notation, we show the proof only for \(\delta =1\); the general case follows the same logic. Let us fix \(m,n\in \mathbb {N}\) and \(x\in B_m\). Then, using the fact \(w_{1}^m(y)=Mw_{1}^m(y)\), \(x\notin B_m\), and the inequality

$$\begin{aligned} e^{w_{1}^m(y)} \ge \mathbb {E}_y\left[ e^{\int _0^1 (f(X_s)-\lambda _1^m)ds+1_{\{X_{1}\in B_m\}}w_{1}^m(X_{1})+1_{\{X_{1}\notin B_m\}}Mw_{1}^m(X_{1})}\right] , \quad y\in B_m, \end{aligned}$$

we have

$$\begin{aligned} \mathbb {E}_x[z_{1}^m&(n+1)|\mathcal {F}_n]=1_{\{\tau _{B_m}\le n\}}e^{\int _0^{\tau _{B_m}} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{ \tau _{B_m}})}\\&\quad +1_{\{\tau _{B_m}> n\}}e^{\int _0^{n} (f(X_s)-\lambda _1^m )ds}\times \\&\quad \times \mathbb {E}_{X_n}[e^{\int _0^{1} (f(X_s)-\lambda _1^m )ds+1_{\{X_1\in B_m\}}w_1^m(X_{1})+1_{\{X_1\notin B_m\}}w_1^m(X_{1})}]\\&=1_{\{\tau _{B_m}\le n\}}e^{\int _0^{n\wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{n\wedge \tau _{B_m}})}\\&\quad +1_{\{\tau _{B_m}> n\}}e^{\int _0^{n\wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds}\times \\&\quad \times \mathbb {E}_{X_n}[e^{\int _0^{1} (f(X_s)-\lambda _1^m )ds+1_{\{X_1\in B_m\}}w_1^m(X_{1})+1_{\{X_1\notin B_m\}}Mw_1^m(X_{1})}]\\&\le e^{\int _0^{n\wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{n\wedge \tau _{B_m}})} = z_1^m(n), \end{aligned}$$

which shows the supermartingale property of \((z_{1}^m(n))\). Next, note that on the set \(\{\tau _{B_m}\wedge \hat{\tau }^m_1>n\}\) we get

$$\begin{aligned} e^{w_1^m(X_n)}=\mathbb {E}_{X_n}\left[ e^{\int _0^1 (f(X_s)-\lambda _1^m)ds+1_{\{X_{1}\in B_m\}}w_{1}^m(X_{1})+1_{\{X_{1}\notin B_m\}}Mw_{1}^m(X_{1})}\right] . \end{aligned}$$

Thus, we have

$$\begin{aligned} \mathbb {E}_x[z_{1}^m((n+1)&\wedge \hat{\tau }^m_1)|\mathcal {F}_n]=1_{\{\tau _{B_m}\wedge \hat{\tau }^m_1\le n\}}e^{\int _0^{\tau _{B_m}\wedge \hat{\tau }^m_1} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{ \tau _{B_m}\wedge \hat{\tau }^m_1})}\\&\quad +1_{\{\tau _{B_m}\wedge \hat{\tau }^m_1> n\}}e^{\int _0^{n} (f(X_s)-\lambda _1^m )ds}\times \\&\quad \times \mathbb {E}_{X_n}[e^{\int _0^{1} (f(X_s)-\lambda _1^m )ds+1_{\{X_1\in B_m\}}w_1^m(X_{1})+1_{\{X_1\notin B_m\}}Mw_1^m(X_{1})}]\\&= e^{\int _0^{n\wedge \tau _{B_m}\wedge \hat{\tau }^m_1} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{n\wedge \tau _{B_m}\wedge \hat{\tau }^m_1})} = z_1^m(n\wedge \hat{\tau }^m_1), \end{aligned}$$

which concludes the proof. \(\square \)

Let us denote by \(\mathbb {V}_{\delta ,m}\) the family of impulse control strategies with impulse times in the time-grid \(\{0,\delta , 2\delta , \ldots \}\) and obligatory impulses when the controlled process exits the set \(B_m\) at some multiple of \(\delta \). Using a martingale characterisation of (3.2), we get that \(\lambda ^m_\delta \) is the optimal value of the impulse control problem with impulse strategies from \(\mathbb {V}_{\delta ,m}\); see Theorem 3.3 To show this result, we introduce a strategy \(\hat{V}:=(\hat{\tau }_i,\hat{\xi }_i)_{i=1}^\infty \in \mathbb {V}_{\delta ,m}\) defined recursively, for \(i=1, 2,\ldots \), by

$$\begin{aligned} \hat{\tau }_i&:=\hat{\sigma }_i\wedge \tau _{B_m}^i,\nonumber \\ \hat{\sigma }_i&:=\delta \inf \{n\ge \hat{\tau }_{i-1}/\delta :n\in \mathbb {N}, \, w^m_\delta (X_{n\delta }^i)=Mw^m_\delta (X_{n\delta }^i)\},\nonumber \\ \tau _{B_m}^i&:=\delta \inf \{n\ge \hat{\tau }_{i-1}/\delta :n\in \mathbb {N},\, X_{n\delta }^i\notin B_m\},\nonumber \\ \hat{\xi }_i&:=\mathop {\mathrm {arg\,max}}\limits _{\xi \in U}(c(X_{\hat{\tau }_i}^i,\xi )+w^m_\delta (\xi ))1_{\{\hat{\tau }_{i}<\infty \}}+\xi _0 1_{\{\hat{\tau }_{i}=\infty \}}, \end{aligned}$$
(3.9)

where \(\hat{\tau }_0:=0\) and \(\xi _0\in U\) is some fixed point.

Theorem 3.3

Let \((w_{\delta }^m, \lambda ^m_\delta )\) be a solution to (3.2). Then, for any \(x\in B_m\), we get

$$\begin{aligned} \lambda ^m_\delta =\sup _{V\in \mathbb {V}_{\delta ,m}}\liminf _{n\rightarrow \infty }\frac{1}{ n\delta }\ln \mathbb {E}_{(x,V)}\left[ e^{\int _0^{n\delta } f(Y_s)ds+\sum _{i=1}^\infty 1_{\{\tau _i\le n\delta \}}c(Y_{\tau _i^-},\xi _i)}\right] . \end{aligned}$$

Also, the strategy \(\hat{V}\) defined in (3.9) is optimal.

Proof

The proof follows the lines of the proof of Theorem 2.3 and is omitted for brevity. \(\square \)

Next, we link (3.2) with an infinite horizon optimal stopping problem under the non-degeneracy assumption.

Theorem 3.4

Let \((w_{\delta }^m, \lambda ^m_\delta )\) be a solution to (3.2) with \(\lambda ^m_\delta >r(f)\). Then, we get that \((w_{\delta }^m, \lambda ^m_\delta )\) satisfies (3.1).

Proof

As in the proof of Proposition 3.2, we consider only \(\delta =1\); the general case follows the same logic.

First, note that for any \(x\in B_m\), \(n\in \mathbb {N}\), and \(\tau \in \mathcal {T}^\delta _x\), using Proposition 3.2 and Doob’s optional stopping theorem, we have

$$\begin{aligned} e^{w_{1}^m(x)}\ge \mathbb {E}_x\left[ e^{\int _0^{n\wedge \tau \wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{n\wedge \tau \wedge \tau _{B_m}})}\right] . \end{aligned}$$

Also, recalling the boundedness of \(w^m_1\), using Proposition A.2, and letting \(n\rightarrow \infty \), we get

$$\begin{aligned} e^{w_{1}^m(x)}\ge \mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{ \tau \wedge \tau _{B_m}})}\right] . \end{aligned}$$

Next, noting that \(w_1^m(X_{\tau \wedge \tau _{B_m}})\ge Mw_1^m(X_{\tau \wedge \tau _{B_m}})\), and taking the supremum over \(\tau \in \mathcal {T}^\delta _x\), we get

$$\begin{aligned} e^{w_{1}^m(x)}\ge \sup _{\tau \in \mathcal {T}^\delta _x}\mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+Mw_1^m(X_{ \tau \wedge \tau _{B_m}})}\right] . \end{aligned}$$

Second, using again Proposition 3.2, for any \(x\in B_m\) and \(n\in \mathbb {N}\), we get

$$\begin{aligned} w_{1}^m(x)= \ln \mathbb {E}_x\left[ e^{\int _0^{n\wedge \hat{\tau }^m_\delta \wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{n\wedge \hat{\tau }^m_\delta \wedge \tau _{B_m}})}\right] . \end{aligned}$$

Using again the boundedness of \(w^m_1\) and Proposition A.2, and letting \(n\rightarrow \infty \), we get

$$\begin{aligned} w_{1}^m(x)= \ln \mathbb {E}_x\left[ e^{\int _0^{\hat{\tau }^m_\delta \wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+w_1^m(X_{ \hat{\tau }^m_\delta \wedge \tau _{B_m}})}\right] . \end{aligned}$$

In fact, noting that \(w_1^m(X_{ \hat{\tau }^m_\delta \wedge \tau _{B_m}})=Mw_1^m(X_{ \hat{\tau }^m_\delta \wedge \tau _{B_m}})\), we obtain

$$\begin{aligned} w_{1}^m(x)= \ln \mathbb {E}_x\left[ e^{\int _0^{\hat{\tau }^m_\delta \wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+Mw_1^m(X_{ \hat{\tau }^m_\delta \wedge \tau _{B_m}})}\right] , \end{aligned}$$

thus we get

$$\begin{aligned} e^{w_{1}^m(x)}= \sup _{\tau \in \mathcal {T}^\delta _x}\mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+Mw_1^m(X_{ \tau \wedge \tau _{B_m}})}\right] . \end{aligned}$$

Finally, using Proposition A.4, we have

$$\begin{aligned} e^{w_{1}^m(x)}= \sup _{\tau \in \mathcal {T}^\delta _{x,b}}\mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _1^m )ds+Mw_1^m(X_{ \tau \wedge \tau _{B_m}})}\right] , \end{aligned}$$

which concludes the proof. \(\square \)

Remark 3.5

In Theorem 3.4 we showed that, if \(\lambda ^m_\delta >r(f)\), a solution to the one-step equation (3.2) is uniquely characterised by the optimal stopping value function (3.1). If \(\lambda ^m_\delta \le r(f)\), the problem is degenerate and, in particular, we cannot use the uniform integrability result from Proposition A.2. In fact, in this case it is even possible that the one-step Bellman equation admits multiple solutions and the optimal stopping characterisation does not hold; see e.g. Theorem 1.13 in [27] for details.

4 Dyadic Impulse Control

In this section we consider a dyadic full-domain version of (2.1). We construct a solution to the associated Bellman equation which will be later used to find a solution to (2.7). The argument uses a bounded domain approximation from Sect. 3. More specifically, throughout this section we fix some \(\delta >0\) and show the existence of a function \(w_\delta \in \mathcal {C}_b(E)\) and a constant \(\lambda _\delta \in \mathbb {R}\), which are a solution to the dyadic Bellman equation of the form

$$\begin{aligned} w_{\delta }(x)=\sup _{\tau \in \mathcal {T}_{x,b}^{\delta }}\ln \mathbb {E}_x\left[ e^{\int _0^{\tau } (f(X_s)-\lambda _\delta )ds+Mw_\delta (X_{\tau })}\right] , \quad x\in E. \end{aligned}$$
(4.1)

In fact, we set

$$\begin{aligned} \lambda _{\delta }:=\lim _{m\rightarrow \infty } \lambda ^m_\delta ; \end{aligned}$$
(4.2)

note that this constant is well-defined as, from Theorem 3.3, recalling that \(B_m\subset B_{m+1}\), we get \(\lambda ^m_\delta \le \lambda ^{m+1}_\delta \), \(m\in \mathbb {N}\).

First, we state the lower bound for \(\lambda _\delta \).

Lemma 4.1

Let \((w_\delta ,\lambda _\delta )\) be a solution to (4.1). Then, we get \(\lambda _\delta \ge r(f)\).

Proof

The proof follows the lines of the proof of Lemma 2.1 and is omitted for brevity. \(\square \)

Next, we show the existence of a solution to (4.1) under the non-degeneracy assumption \(\lambda _\delta >r(f)\).

Theorem 4.2

Let \(\lambda _\delta \) be given by (4.2) and assume that \(\lambda _\delta >r(f)\). Then, there exists \(w_\delta \in \mathcal {C}_b(E)\) such that (4.1) is satisfied and we get \(\sup _{\xi \in U} w_{\delta }(\xi )=0\).

Proof

We start with some general comments and an outline of the argument. First, note that from Theorem 3.1, for any \(m\in \mathbb {N}\), we get a solution \((w_\delta ^m,\lambda _\delta ^m)\) to (3.2) satisfying \(\sup _{\xi \in U}w^m_\delta (\xi )=0\). Also, from the assumption \(\lambda _\delta >r(f)\) we get \(\lambda ^m_\delta >r(f)\) for \(m\in \mathbb {N}\) sufficiently big (for simplicity, we assume that \(\lambda ^0_\delta >r(f)\)). Thus, using Theorem 3.4, we get that, for any \(m\in \mathbb {N}\), the pair \((w_\delta ^m,\lambda _\delta ^m)\) satisfies (3.1).

Second, to construct a function \(w_\delta \), we use the Arzelà-Ascoli theorem. More specifically, recalling that \(\sup _{\xi \in U}w^m_\delta (\xi )=0\) and using the fact that \(-\Vert c\Vert \le c(x,\xi )\le 0 \), \(x\in E\), \(\xi \in U\), for any \(m\in \mathbb {N}\) and \(x\in E\), we get

$$\begin{aligned} -\Vert c\Vert \le Mw^m_\delta (x)\le 0. \end{aligned}$$

Also, note that, for any \(m\in \mathbb {N}\) and \(x,y\in E\), we have

$$\begin{aligned} |Mw^m_\delta (x)-Mw^m_\delta (y)|\le \sup _{\xi \in U} |c(x,\xi )-c(y,\xi )|. \end{aligned}$$

Consequently, the sequence \((Mw^m_\delta )_{m\in \mathbb {N}}\) is uniformly bounded and equicontinuous. Thus, using the Arzelà-Ascoli theorem combined with a diagonal argument, we may find a subsequence (for brevity still denoted by \((Mw^m_\delta )_{m\in \mathbb {N}}\)) and a map \(\phi _\delta \in \mathcal {C}_b(E)\) such that \(Mw^m_\delta (x)\) converges to \(\phi _\delta (x)\) as \(m\rightarrow \infty \) uniformly in x from any compact set. In fact, using (2.4) from Assumption (A1) and the argument from the first step of the proof of Theorem 4.1 in [20], we get that the convergence is uniform in \(x\in E\). Then, we define

$$\begin{aligned} w_{\delta }(x):=\sup _{\tau \in \mathcal {T}_{x,b}^{\delta }}\ln \mathbb {E}_x\left[ e^{\int _0^{\tau } (f(X_s)-\lambda _\delta )ds+\phi _\delta (X_{\tau })}\right] , \quad x\in E. \end{aligned}$$
(4.3)

To complete the construction, we show that \(w^m_\delta \) converges to \(w_\delta \) uniformly on compact sets. Indeed, in this case we have

$$\begin{aligned} |Mw^m_\delta (x)-Mw_\delta (x)|\le \sup _{\xi \in U}|w^m_\delta (\xi )-w_\delta (\xi )|\rightarrow 0, \quad m\rightarrow \infty , \end{aligned}$$

thus \(\phi _\delta \equiv Mw_\delta \) and from (4.3) we get that (4.1) is satisfied. Also, recalling that from Theorem 3.1 we get \(\sup _{\xi \in U} w^m_\delta (\xi )=0\), \(m\in \mathbb {N}\), we also get \(\sup _{\xi \in U} w_\delta (\xi )=0\).

Finally, to show the convergence, we define the auxiliary functions

$$\begin{aligned}&w_{\delta }^{m,1}(x)&:=\sup _{\tau \in \mathcal {T}_{x,b}^{\delta }}\ln \mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda ^m_\delta )ds+\phi _\delta (X_{\tau \wedge \tau _{B_m}})}\right] , \quad x\in E, \end{aligned}$$
(4.4)
$$\begin{aligned}&w_{\delta }^{m,2}(x)&:=\sup _{\tau \in \mathcal {T}_{x,b}^{\delta }}\ln \mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _\delta )ds+\phi _\delta (X_{\tau \wedge \tau _{B_m}})}\right] , \quad x\in E . \end{aligned}$$
(4.5)

We split the rest of the proof into three steps: (1) proof that \(|w_{\delta }^{m}(x)-w_{\delta }^{m,1}(x)|\rightarrow 0\) as \(m\rightarrow \infty \) uniformly in \(x\in E\); (2) proof that \(|w_{\delta }^{m,1}(x)-w_{\delta }^{m,2}(x)|\rightarrow 0\) as \(m\rightarrow \infty \) uniformly in \(x\in E\); (3) proof that \(|w_{\delta }^{m,2}(x)-w_{\delta }(x)|\rightarrow 0\) as \(m\rightarrow \infty \) uniformly in x from compact sets.

Step 1. We show \(|w_{\delta }^{m}(x)-w_{\delta }^{m,1}(x)|\rightarrow 0\) as \(m\rightarrow \infty \) uniformly in \(x\in E\). Note that, for any \(x\in E\) and \(m\in \mathbb {N}\), we have

$$\begin{aligned} w_\delta ^{m,1}(x)&\le \sup _{\tau \in \mathcal {T}_{x,b}^\delta }\ln \left( \mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _\delta ^m)ds +Mw_\delta ^m(X_{\tau \wedge \tau _{B_m}})}\right] e^{\Vert \phi _\delta -Mw_\delta ^m\Vert }\right) \\&= w_\delta ^m(x)+\Vert \phi _\delta -Mw_\delta ^m\Vert . \end{aligned}$$

Similarly, we get \(w_\delta ^m(x)\le w_\delta ^{m,1}(x)+\Vert \phi _\delta -Mw_\delta ^m\Vert \), thus

$$\begin{aligned} \sup _{x\in E} |w_\delta ^m(x)-w_\delta ^{m,1}(x)|\le \Vert \phi _\delta -Mw_\delta ^m\Vert . \end{aligned}$$

Recalling the fact that \(\phi _\delta \) is a uniform limit of \(Mw_\delta ^m\) as \(m\rightarrow \infty \), we conclude the proof of this step.

Step 2. We show that \(|w_{\delta }^{m,1}(x)-w_{\delta }^{m,2}(x)|\rightarrow 0\) as \(m\rightarrow \infty \) uniformly in \(x\in E\). Recalling that \(\lambda _\delta ^m\uparrow \lambda _\delta \), we get \(w_\delta ^{m,1}(x)\ge w_\delta ^{m,2}(x)\ge -\Vert \phi _\delta \Vert \), \(x\in E\). Thus, using the inequality \(|\ln y-\ln z|\le \frac{1}{\min (y,z)}|y-z|\), \(y,z>0\), we get

$$\begin{aligned} 0\le w_\delta ^{m,1}(x)-w_\delta ^{m,2}(x)\le e^{\Vert \phi _\delta \Vert } (e^{w_\delta ^{m,1}(x)}-e^{w_\delta ^{m,2}(x)}), \quad x\in E. \end{aligned}$$
(4.6)

Then, noting that \(\phi _\delta (\cdot )\le 0\), for any \(m\in \mathbb {N}\) and \(x\in E\), we obtain

$$\begin{aligned} 0 \le e^{w_\delta ^{m,1}(x)}-e^{w_\delta ^{m,2}(x)}&\le \sup _{\tau \in \mathcal {T}_{x,b}^\delta }\left( \mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _\delta ^m)ds +\phi _\delta (X_{\tau \wedge \tau _{B_m}})}\right] \right. \nonumber \\&\quad \left. - \mathbb {E}_x\left[ e^{\int _0^{\tau \wedge \tau _{B_m}} (f(X_s)-\lambda _\delta )ds +\phi _\delta (X_{\tau \wedge \tau _{B_m}})}\right] \right) \nonumber \\&\le \sup _{\tau \in \mathcal {T}_{x,b}^\delta }\mathbb {E}_x\left[ e^{\int _0^{\tau } f(X_s)ds}\left( e^{-\lambda _\delta ^m\tau }- e^{-\lambda _\delta \tau }\right) \right] . \end{aligned}$$
(4.7)

Also, recalling that \(\lambda _\delta ^0\le \lambda _\delta ^m\le \lambda _\delta \), \(m\in \mathbb {N}\), for any \(x\in E\) and \(T\ge 0\), we get

$$\begin{aligned} 0&\le \sup _{\tau \in \mathcal {T}_{x,b}}\mathbb {E}_x\left[ e^{\int _0^{\tau } f(X_s)ds}\left( e^{-\lambda _\delta ^m \tau }- e^{\lambda _\delta \tau }\right) \right] \nonumber \\&\le \sup _{\begin{array}{c} \tau \in \mathcal {T}_{x,b} \end{array}}\mathbb {E}_x\left[ \left( 1_{\{\tau \le T\}}+1_{\{\tau >T\}}\right) e^{\int _0^{\tau } f(X_s)ds}\left( e^{-\lambda _\delta ^m \tau }- e^{-\lambda _\delta \tau }\right) \right] \nonumber \\&\le \sup _{\begin{array}{c} \tau < T\\ \tau \in \mathcal {T}_{x,b} \end{array}}e^{T\Vert f\Vert }\mathbb {E}_x\left[ \left( e^{-\lambda _\delta ^m \tau }- e^{-\lambda _\delta \tau }\right) \right] +\sup _{\begin{array}{c} \tau \ge T\\ \tau \in \mathcal {T}_{x,b} \end{array}}\mathbb {E}_x\left[ e^{\int _0^{\tau } (f(X_s)-\lambda _\delta ^0)ds}\right] . \end{aligned}$$
(4.8)

Recalling \(\lambda _\delta ^0>r(f)\) and using Lemma A.1, for any \(\varepsilon >0\), we may find \(T\ge 0\), such that

$$\begin{aligned} 0\le \sup _{x\in E} \sup _{\begin{array}{c} \tau \ge T\\ \tau \in \mathcal {T}_{x,b} \end{array}}\mathbb {E}_x\left[ e^{\int _0^{\tau } (f(X_s)-\lambda _\delta ^0)ds}\right] \le \varepsilon . \end{aligned}$$

Also, using the inequality \(|e^x-e^y|\le e^{\max (x,y)}|x-y|\), \(x,y\ge 0\), we obtain

$$\begin{aligned}{} & {} \sup _{\tau< T}\mathbb {E}_x\left[ \left( e^{-\lambda _\delta ^m \tau }- e^{-\lambda _\delta \tau }\right) \right] \le \sup _{\tau < T} \mathbb {E}_x\left[ e^{\max (-\lambda _\delta ^m \tau , -\lambda _\delta \tau )}\tau (\lambda _\delta -\lambda _\delta ^m)\right] \nonumber \\{} & {} \quad \le e^{|\lambda _\delta ^m | T} T(\lambda _\delta -\lambda _\delta ^m). \end{aligned}$$
(4.9)

Thus, for fixed \(T\ge 0\), we find \(m\ge 0\), such that \(e^{|\lambda _\delta ^m | T} T(\lambda _\delta -\lambda _\delta ^m)\le \varepsilon \). Hence, recalling (4.6)–(4.8), for any \(x\in E\) and T, m big enough, we get

$$\begin{aligned} 0\le w_\delta ^{m,1}(x)-w_\delta ^{m,2}(x)\le e^{\Vert \phi _\delta \Vert } 2\varepsilon . \end{aligned}$$

Recalling that \(\varepsilon >0\) was arbitrary, we conclude the proof of this step.

Step 3. We show that \(|w_{\delta }^{m,2}(x)-w_{\delta }(x)|\rightarrow 0\) as \(m\rightarrow \infty \) uniformly in x from compact sets. First, we show that \(w_\delta ^{m,2}(x)\le w_\delta (x)\) for any \(m\in \mathbb {N}\) and \(x\in E\). Let \(\varepsilon >0\) and \(\tau _m^\varepsilon \in \mathcal {T}_{x,b}^\delta \) be an \(\varepsilon \)-optimal stopping time for \(w_\delta ^{m,2}(x)\). Then, we get

$$\begin{aligned} w_\delta (x)&\ge \ln \mathbb {E}_x\left[ e^{\int _0^{\tau _m^\varepsilon \wedge \tau _{B_m}} (f(X_s)-\lambda _\delta )ds +\phi _\delta (X_{\tau _m^\varepsilon \wedge \tau _{B_m}})}\right] \ge w_\delta ^{m,2}(x)-\varepsilon . \end{aligned}$$

As \(\varepsilon >0\) was arbitrary, we get \(w_\delta ^{m,2}(x)\le w_\delta (x)\), \(m\in \mathbb {N}\), \(x\in E\). In fact, using a similar argument, for any \(x\in E\), we may show that the map \(m\mapsto w_{\delta }^{m,2}(x)\) is non-decreasing.

Second, let \(\varepsilon >0\) and \(\tau _\varepsilon \in \mathcal {T}_{x,b}^\delta \) be an \(\varepsilon \)-optimal stopping time for \(w_\delta (x)\). Then, we obtain

$$\begin{aligned} 0\le w_\delta (x) - w_\delta ^{m,2}(x)&\le \ln \mathbb {E}_x\left[ e^{\int _0^{\tau _\varepsilon } (f(X_s)-\lambda _\delta )ds +\phi _\delta (X_{\tau _\varepsilon })}\right] +\varepsilon \nonumber \\&\quad - \ln \mathbb {E}_x\left[ e^{\int _0^{\tau _\varepsilon \wedge \tau _{B_m}} (f(X_s)-\lambda _\delta )ds +\phi _\delta (X_{\tau _\varepsilon \wedge \tau _{B_m}})}\right] . \end{aligned}$$
(4.10)

Noting that \(\tau _{B_m}\uparrow +\infty \) as \(m\rightarrow \infty \) and using the quasi left-continuity of X combined with Lemma A.2 and the boundedness of \(\phi _\delta \), we get

$$\begin{aligned} \lim _{m\rightarrow \infty } \mathbb {E}_x\left[ e^{\int _0^{\tau _\varepsilon \wedge \tau _{B_m}} (f(X_s)-\lambda _\delta )ds +\phi _\delta (X_{\tau _\varepsilon \wedge \tau _{B_m}})}\right] = \mathbb {E}_x\left[ e^{\int _0^{\tau _\varepsilon } (f(X_s)-\lambda _\delta )ds +\phi _\delta (X_{\tau _\varepsilon })}\right] . \end{aligned}$$

Thus, using (4.10) and recalling that \(\varepsilon >0\) was arbitrary, we get \(\lim _{m\rightarrow \infty } w_\delta ^{m,2}(x) = w_\delta (x)\). Also, noting that by Propositions A.3 and A.4, the maps \(x\mapsto w_\delta (x)\) and \(x\mapsto w_\delta ^{m,2}(x)\) are continuous, and using the monotonicity of \(m\mapsto w_\delta ^{m,2}(x)\), from Dini’s Theorem we get that \(w_\delta ^{m,2}(x)\) converges to \(w_\delta (x)\) uniformly in x from compact sets, which concludes the proof. \(\square \)

We conclude this section with a verification result related to (4.1).

Theorem 4.3

Let \((w_\delta ,\lambda _\delta )\) be a solution to (4.1) with \(\lambda _\delta >r(f)\). Then, we get

$$\begin{aligned} \lambda _\delta :=\sup _{V\in \mathbb {V}^\delta }\liminf _{n\rightarrow \infty } \frac{1}{n \delta } \ln \mathbb {E}_{(x,V)}\left[ e^{\int _0^{n\delta } f(Y_s)ds+\sum _{i=1}^\infty 1_{\{\tau _i\le n\delta \}}c(Y_{\tau _i^-},\xi _i)}\right] , \end{aligned}$$

where \(\mathbb {V}^\delta \) is a family of impulse control strategies with impulse times on the dyadic time-grid \(\{0,\delta , 2\delta , \ldots \}\).

Proof

The proof follows the lines of the proof of Theorem 2.3 and is omitted for brevity. \(\square \)

5 Existence of a Solution to the Bellman Equation

In this section we construct a solution \((w,\lambda )\) to (2.7), which together with Theorem 2.3 provides a solution to (2.1). The argument uses a dyadic approximation and the results from Sect. 4. More specifically, we fix \(\delta >0\) and consider a family of dyadic time steps \(\delta _k:=\frac{\delta }{2^k}\), \(k\in \mathbb {N}\). First, we specify the value of \(\lambda \). In fact, we define

$$\begin{aligned} \lambda (\delta ):=\liminf _{k\rightarrow \infty }\lambda _{\delta _k}, \end{aligned}$$
(5.1)

where \(\lambda _{\delta _k}\) is a constant given by (4.2), corresponding to \(\delta _k\). In Theorem 5.1, we show that if \(\lambda (\delta )>r(f)\), then there exists a solution to (2.7) with the constant \(\lambda \) given by \(\lambda (\delta )\). Also, in this case \(\lambda \) does not depend on \(\delta \) and the limit inferior could be replaced by the usual limit.

Theorem 5.1

Let \(\delta >0\) and let \(\lambda (\delta )\) be given by (5.1). Assume that \(\lambda (\delta )>r(f)\). Then, there exists \(w\in \mathcal {C}_b(E)\) such that (2.7) is satisfied with \(\lambda =\lambda (\delta )\). Also, \(\lambda (\delta )=\lim _{k\rightarrow \infty } \lambda _{\delta _k}\) and \(\lambda (\delta )\) does not depend on \(\delta >0\), i.e. for any \(\delta _1>0\) and \(\delta _2>0\) such that \(\lambda (\delta _1)>r(f)\) and \(\lambda (\delta _2)>r(f)\), we get \(\lambda (\delta _1)=\lambda (\delta _2)\).

Proof

The argument is partially based on the one used in Theorem 4.2 and thus we discuss only the main points. First, from the assumption \(\lambda (\delta )>r(f)\) we get \(\lambda _{\delta _k}>r(f)\) for sufficiently big \(k\in \mathbb {N}\); to simplify the notation, we assume \(\lambda _{\delta _0}>r(f)\). Hence, using Theorems 4.2, 4.3, and the fact \(\mathbb {V}^{\delta _k}\subset \mathbb {V}^{\delta _{k+1}}\), we inductively show

$$\begin{aligned} \lambda _{\delta _k}=\sup _{V\in \mathbb {V}^{\delta _k}}J(x,V)\le \sup _{V\in \mathbb {V}^{\delta _k}}J(x,V) = \lambda _{\delta _{k+1}}, \quad k\in \mathbb {N}, \, x\in E. \end{aligned}$$

Thus, the sequence \((\lambda _{\delta _k})_{k=k_0}^\infty \) is non-decreasing and, consequently, convergent. Hence, \(\lambda (\delta )=\lim _{k\rightarrow \infty }\lambda _{\delta _k}\). Second, using again Theorem 4.2, for any \(k\in \mathbb {N}\), we find a map \(w_{\delta _k}\in \mathcal {C}_b(E)\) satisfying

$$\begin{aligned} w_{\delta _k}(x)=\sup _{\tau \in \mathcal {T}_{x,b}^{\delta _k}}\ln \mathbb {E}_x\left[ e^{\int _0^{\tau } (f(X_s)-\lambda _{\delta _k} )ds+Mw_{\delta _k}(X_{\tau })}\right] , \quad x\in E \end{aligned}$$

and such that \(\sup _{\xi \in U} w_{\delta _k}(\xi )=0\). Thus, we obtain

$$\begin{aligned} -\Vert c\Vert \le Mw_{\delta _k}(x)\le 0, \quad k\in \mathbb {N},\, x\in E, \end{aligned}$$

and the family \((Mw_{\delta _k})_{k\in \mathbb {N}}\) is uniformly bounded. Also, it is equicontinuous as we have

$$\begin{aligned} |Mw_{\delta _k}(x)-Mw_{\delta _k}(y)|\le \sup _{x\in U} |c(x,\xi )-c(y,\xi )|, \quad x,y\in E. \end{aligned}$$

Thus, using the Arzelà-Ascoli theorem, we may choose a subsequence (for brevity still denoted by \((Mw_{\delta _k})\)), such that \((Mw_{\delta _k})\) converges uniformly on compact sets to some map \(\phi \). In fact, using (2.4) from Assumption (A1) and the argument from the first step of the proof of Theorem 4.1 from [20], we get that \(Mw_{\delta _k}(x)\) converges to \(\phi (x)\) as \(k\rightarrow \infty \) uniformly in \(x\in E\). Next, let us define

$$\begin{aligned} w(x):=\sup _{\tau \in \mathcal {T}_{x,b}}\ln \mathbb {E}_x\left[ e^{\int _0^{\tau } (f(X_s)-\lambda (\delta ))ds+\phi (X_{\tau })}\right] , \quad x\in E. \end{aligned}$$
(5.2)

In the following, we show that \(w_{\delta _k}\) converges to w uniformly in compact sets as \(k\rightarrow \infty \). Then, we get that \(Mw_{\delta _k}\) converges to Mw, hence \(Mw\equiv \phi \) and (2.7) is satisfied.

To show the convergence, we define

$$\begin{aligned} w_{\delta _k}^1(x):=\sup _{\tau \in \mathcal {T}_{x,b}^{\delta _k}}\ln \mathbb {E}_x\left[ e^{\int _0^\tau (f(X_s)-\lambda _{\delta _k})ds+\phi (X_\tau )}\right] , \quad k\in \mathbb {N}, \, x\in E. \end{aligned}$$

In the following, we show that \(|w(x)-w_{\delta _k}^1(x)|\rightarrow 0\) and \(|w_{\delta _k}^1(x)-w_{\delta _k}(x)|\rightarrow 0\) as \(k\rightarrow \infty \) uniformly in x from compact sets. In fact, to show the first convergence, we note that

$$\begin{aligned} w_{\delta _k}^0(x)\le w_{\delta _k}^1(x)\le w_{\delta _k}^2(x), \quad k\in \mathbb {N}, \,x\in E, \end{aligned}$$

where

$$\begin{aligned} w_{\delta _k}^0(x)&:=\sup _{\tau \in \mathcal {T}_{x,b}^{\delta _k}}\ln \mathbb {E}_x\left[ e^{\int _0^\tau (f(X_s)-\lambda (\delta ))ds+\phi (X_\tau )}\right] , \quad k\in \mathbb {N}, \, x\in E,\\ w_{\delta _k}^2(x)&:=\sup _{\tau \in \mathcal {T}_{x,b}}\ln \mathbb {E}_x\left[ e^{\int _0^\tau (f(X_s)-\lambda _{\delta _k})ds+\phi (X_\tau )}\right] , \quad k\in \mathbb {N}, \, x\in E. \end{aligned}$$

Thus, to prove \(|w(x)-w_{\delta _k}^1(x)|\rightarrow 0\) it is enough to show \(|w(x)-w_{\delta _k}^0(x)|\rightarrow 0\) and \(|w(x)-w_{\delta _k}^2(x)|\rightarrow 0\) as \(k\rightarrow \infty \).

For transparency, we split the rest of the proof into three parts: (1) proof that \(|w(x)-w_{\delta _k}^0(x)|\rightarrow 0\) as \(k\rightarrow \infty \) uniformly in x from compact sets; (2) proof that \(|w(x)-w_{\delta _k}^2(x)|\rightarrow 0\) as \(k\rightarrow \infty \) uniformly in \(x\in E\); (3) proof that \(|w_{\delta _k}^1(x)-w_{\delta _k}(x)|\rightarrow 0\) as \(k\rightarrow \infty \) uniformly in \(x\in E\); (4) proof that \(\lambda (\delta )\) does not depend on \(\delta \).

Step 1. We show that \(|w(x)-w_{\delta _k}^0(x)|\rightarrow 0\) as \(k\rightarrow \infty \) as \(k\rightarrow \infty \) uniformly in x from compact sets. First, note that we have \(w_{\delta _k}^0(x)\le w(x)\), \(k\in \mathbb {N}\), \(x\in E\). Next, for any \(x\in E\) and \(\varepsilon >0\), let \(\tau _\varepsilon \in \mathcal {T}_{x,b}\) be an \(\varepsilon \)-optimal stopping time for w(x) and let \(\tau _\varepsilon ^k\) be its \(\mathcal {T}^{\delta _k}_{x,b}\) approximation given by

$$\begin{aligned} \tau _\varepsilon ^k:=\inf \left\{ \tau \in \mathcal {T}^{\delta _k}_{x,b}: \tau \ge \tau _\varepsilon \right\} =\sum _{j=1}^{\infty } 1_{\left\{ \delta \frac{j-1}{2^k}<\tau _\varepsilon \le \delta \frac{j}{2^m}\right\} } \delta \frac{ j}{2^k}. \end{aligned}$$

Then, we get

$$\begin{aligned} 0&\le w(x)-w_{\delta _k}^0(x)\\&\le \mathbb {E}_x\left[ e^{\int _0^{\tau _\varepsilon } (f(X_s)-\lambda (\delta ))ds+\phi (X_{\tau _\varepsilon })}\right] -\mathbb {E}_x\left[ e^{\int _0^{\tau _\varepsilon ^k} (f(X_s)-\lambda (\delta ))ds+\phi (X_{\tau _\varepsilon ^k})}\right] +\varepsilon . \end{aligned}$$

Also, using Proposition A.2 and letting \(k\rightarrow \infty \), we have

$$\begin{aligned} \lim _{k\rightarrow \infty }\mathbb {E}_x\left[ e^{\int _0^{\tau _\varepsilon ^k} (f(X_s)-\lambda (\delta ))ds+\phi (X_{\tau _\varepsilon ^k})}\right] =\mathbb {E}_x\left[ e^{\int _0^{\tau _\varepsilon } (f(X_s)-\lambda (\delta ))ds+\phi (X_{\tau _\varepsilon })}\right] . \end{aligned}$$

Consequently, recalling that \(\varepsilon >0\) was arbitrary, we obtain \(\lim _{k\rightarrow \infty }w_{\delta _k}^0(x)=w(x)\) for any \(x\in E\). Next, noting that \(\mathcal {T}_{x,b}^{\delta _k}\subset \mathcal {T}_{x,b}^{\delta _{k+1}}\), \(k\in \mathbb {N}\), we get \(w_{\delta _k}^0(x)\le w_{\delta _{k+1}}^0(x)\), \(k\in \mathbb {N}\), \(x\in E\). This combined with Propositions A.3, A.4, and Dini’s theorem, we get that the convergence of \(w_{\delta _k}^0\) to w is uniform on compact sets, which concludes the proof of this step.

Step 2. We show that \(|w(x)-w_{\delta _k}^2(x)|\rightarrow 0\) as \(k\rightarrow \infty \) uniformly in \(x\in E\). First, note that \(-\Vert \phi \Vert \le w(x)\le w_{\delta _k}^2(x)\), \(k\in \mathbb {N}\), \(x\in E\). Thus, using the inequality \(|\ln y-\ln z|\le \frac{1}{\min (y,z)}|y-z|\), \(y,z>0\), we get

$$\begin{aligned} 0\le w_{\delta _k}^2(x)-w(x)\le e^{\Vert \phi \Vert }(e^{w_{\delta _k}^2(x)}-e^{w(x)}), \quad k\in \mathbb {N}, \, x\in E. \end{aligned}$$

Also, recalling that \(\phi (\cdot )\le 0\), for any \(k\in \mathbb {N}\) and \(x\in E\), we obtain

$$\begin{aligned} 0\le e^{w_{\delta _k}^2(x)}-e^{w(x)}\le \sup _{\tau \in \mathcal {T}_{x,b}}\mathbb {E}_x\left[ e^{\int _0^\tau f(X_s) ds}\left( e^{-\lambda _{\delta _k}\tau }-e^{-\lambda (\delta )\tau } \right) \right] . \end{aligned}$$

Thus, repeating the argument from the second step of the proof of Theorem 4.2, we get \(w_{\delta _k}^2(x)\rightarrow w(x)\) as \(k\rightarrow \infty \) uniformly in \(x\in E\), which concludes the proof of this step.

Step 3. We show that \(|w_{\delta _k}^1(x)-w_{\delta _k}(x)|\rightarrow 0\) as \(k\rightarrow \infty \) uniformly in \(x\in E\). In fact, recalling that \(\Vert Mw_{\delta _k}-\phi \Vert \rightarrow 0\) as \(k\rightarrow \infty \), the argument follows the lines of the one used in the first step of the proof of Theorem 4.2. This concludes the proof of this step.

Step 4. We show that \(\lambda (\delta )\) does not depend on \(\delta \) as long as \(\lambda (\delta )>r(f)\). More specifically, let \(\delta _1>0\) and \(\delta _2>0\) be such that \(\lambda (\delta _1)>r(f)\) and \(\lambda (\delta _2)>r(f)\). Then, using Steps 1–3, we may construct \(w^{\delta _1}\in \mathcal {C}_b(E)\) and \(w^{\delta _2}\in \mathcal {C}_b(E)\) such that the pairs \((w^{\delta _1},\lambda (\delta _1))\) and \((w^{\delta _2},\lambda (\delta _2))\) satisfy (2.7). Then, using Theorem 2.3, for any \(x\in E\), we get

$$\begin{aligned} \lambda (\delta _1)=\sup _{V\in \mathbb {V}}J(x,V)=\lambda (\delta _2), \end{aligned}$$

which concludes the proof. \(\square \)

Remark 5.2

By the inspection of the proof we get that the statement of Theorem 5.1 holds true if we replace the dyadic sequence of time steps \(\delta _k=\frac{\delta }{2^k}\), \(k\in \mathbb {N}\), by any sequence \((\delta _k)\) converging to zero, as long as we have \(\mathcal {T}_{x,b}^{\delta _k}\subset \mathcal {T}_{x,b}^{\delta _{k+1}}\), \(x\in E\), \(k\in \mathbb {N}\). Note that this condition guarantees the monotonic convergence of \(\lambda _{\delta _k}\) and \(w^0_{\delta _k}\).