1 Introduction

The utility maximization is a kind of stochastic control problems. The dynamic programming approach is often applied to the optimal value function and the so-called HJB equation is derived (see the books [25, 34] for the stochastic control and its applications). Since the HJB equation is a fully nonlinear PDE, the closed-form classical solution cannot be found except for some simple cases: a Black-Scholes complete market model with particular utility functions, see [6, 7]. For constrained market models it has to use numerical methods to solve the HJB equations. The standard approach to solve HJB equation by finite difference schemes is to discretize the derivatives in HJB equation and to solve the resulting finite dimensional control problem. The nonlinear discretized equations are often solved using policy iteration schemes (see e.g., [1, 2, 10,11,12,13,14,15, 20, 21, 26, 27, 30, 31]). Among them, the work by [20, 21] and [2] outlines the theory and implementation of the schemes for solving the coupled HJB equations arising in the American options under regime switching models. No detailed convergence proofs are given therein.

In this paper we propose a different way to solve the problems of terminal wealth utility maximization under regime switching models. The HJB system for the problems is composed by d coupled HJB equations, where d is the number of regime states. Fu et al. [16] introduce a functional operator to generate a sequence of value functions and show that the optimal value function is the limit of this sequence. To get the value functions in the sequence, it needs to solve d decoupled HJB equations in each iterative step. Thus the coupled HJB equations are separated by d single HJB equations and henceforth it seems to be much simpler to solve although the iterations are involved. We study the iterative FDMs with policy iterations for solving the sequence of decoupled HJB equations in [16] and prove the convergence. We use several examples to show that solving the decoupled HJB is more efficient than solving the coupled ones.

The regime switching model allows parameters of asset price dynamics to depend on a finite state Markov chain process. It provides good flexibility for characterizing macro market uncertainties while preserves analytic tractability for underlying asset price dynamics. Hamilton [17] introduces a regime switching model for nonstationary time series and business cycles. Hardy [18] applies a two-regime model to provide a good fit to monthly stock market returns. There has been active research in portfolio optimization with regime switching models. Zhang et al. [35] and Yin et al. [33] study the trading rules in a regime switching market. Zhou and Yin [36] investigate the mean-variance portfolio optimization in regime switching model. Canakog̈lu and Özekici [9] discuss the HARA utility maximization in a regime switching model. Honda [19], Sass and Haussmann [29], and Rieder and Bäuerle [28] solve portfolio optimization problems with partial information and regime switching drift processes. Bäuerle and Rieder [5] and Fu et al. [16] show that the value function satisfies the HJB system of fully coupled nonlinear PDEs and prove the verification theorem. For a power or logarithmic utility function, the HJB equations can be reduced to a system of linear ODEs which are then solved with matrix exponentials. For general utility functions, it seems not possible to solve the system of HJB equations analytically. Ma et al. [23] develop the dual control monte-carlo methods to compute the tight bounds of value function in regime switching utility maximization, but it is not possible to guarantee the convergence in theory and the computation of the lower bound is rather time-consuming. The iterative FDMs with iteration policy developed in this paper is proved to be convergent and numerical comparison is made to show that it is much faster than computing the lower bound using the approach in [23].

The remaining parts of the paper are arranged as follows. In Sect. 2, we introduce the utility maximization under regime switching models and the HJB system and analyze the iterative FDMs with policy iterations for the sequence of decoupled HJB equations. In Sect. 3, we carry out a variety of numerical examples to test the convergence and compare efficiency of the proposed algorithms with the existing methods. Conclusions are given in the final section. The standard FDMs with policy iterations for the coupled HJB equations are given in the appendix.

2 Discretization of the Decoupled HJB Equations

Consider a fixed time horizon \( [0,T]\). Let \((\Omega ,{\mathcal {F}},P)\) be a complete probability space, W a standard brownian motion, \(\varvec{\alpha }\) a continuous time finite state observable Markov Chain process (MCP), which are independent of each other, and let \(\{{\mathcal {F}}_t\}\) be the natural filtration generated by W and \(\varvec{\alpha }\) completed with all P-null sets.

We identify the state space of \(\{\varvec{\alpha _t}\}\) as a finite set of unit vectors \({\mathbb {E}}:=\{\varvec{e}_1,\varvec{e}_2,\ldots ,\varvec{e}_d\}\) where \(\varvec{e}_i \in {\mathbb {R}}^d\) is a column vectors with one in the i-th position and zeros elsewhere, \(j=1,\ldots ,d\). Denote by \(\varvec{Q}=(q_{ij})_{d \times d}\) the generator of the Markov Chain \(\{\varvec{\alpha }_t\}\) with \(q_{ij}\ge 0\) for \(i \ne j\) and \( \displaystyle {\sum _{j=1}^{d}}q_{ij}=0\) for each \(j\in {\mathbb {D}}:=\{1,\ldots ,d\}\). The MCP \(\varvec{\alpha }\) has a semi-martingale representation.

$$\begin{aligned} \varvec{\alpha }_t=\varvec{\alpha }_0+\int _{0}^{t} \varvec{Q}^{ \prime }\varvec{\alpha }_v dv+\varvec{M}_{t},\quad 0\le t \le T, \end{aligned}$$

where \(\varvec{Q}^{ \prime }\) is the transpose of \(\varvec{Q}\), \(\varvec{M}\) is a purely discontinuous square integrable Martingale with initial value zero. Assume the financial market consists of one risk-free bond and one risky stock. The bond and stock price processes B and S are assumed to follow the stochastic differential equations (SDE)

$$\begin{aligned} dB_t=r_{t}B_{t}dt,\quad dS_{t}=S_{t}(\mu _{t}dt+\sigma _{t}dW_{t}),\quad 0\le t \le T, \end{aligned}$$

where \(r_{t}=\varvec{r}\varvec{\alpha }_t\), \(\mu _t=\varvec{\mu \alpha }_t\), \(\sigma _{t}=\varvec{\sigma \alpha }_t\) and \(\varvec{r}=(r_1,\ldots ,r_d)\) is a vector of risk-free interest rates with \(r_i\) being the rate in regime i, and \(\varvec{\mu }=(\mu _1,\ldots ,\mu _d)\) and \(\varvec{\sigma }=(\sigma _{1},\ldots ,\sigma _{d})\) are vectors of return and volatility rates of the risky asset. Assume all rates are positive constants. Denote by \(\varvec{\theta }:=(\theta _{1},\ldots ,\theta _{d}) \) the vector of market prices of risk with \(\theta _{i}=\frac{\mu _i-r_i}{\sigma _i}\) for \(i\in {\mathbb {D}}\).

Let X be the wealth process of a portfolio comprising the bond B and the stock S. The wealth process X satisfies the SDE:

$$\begin{aligned} dX_t=X_t \Big (r_t dt+\pi _t \sigma _{t}\left( \theta _{t} dt+dW_{t}\right) \Big ),\quad 0\le t \le T, \end{aligned}$$

where \(\pi _{t}\) is a progressively measurable control process and represents the proportion of wealth \(X_t\) invested in risky asset \(S_t\) and \(\theta _{t}=\varvec{\theta }\varvec{\alpha }_{t}\) is the market price of risk at time t.

The utility maximization problem is defined by:

$$\begin{aligned} \sup _{\pi } E\left[ U(X_T)\right] , \end{aligned}$$
(1)

where U is a utility function that is continuous, increasing and concave on \([0,\infty ]\). Stochastic control is a standard method that to solve problem (1). To do so, we define the value functions

$$\begin{aligned} {\widetilde{V}}(t,x,j):=\sup _{\pi \in \Pi _{t}} E_{t,x,j}[U(X_T)],\quad j\in {\mathbb {D}}, \end{aligned}$$

where \(E_{t,x,j}\) is the conditional expectation operator given \(X_{t}=x\), \(\varvec{\alpha }_{t}=\varvec{e}_j\) for \(j\in {\mathbb {D}}\) and \(\Pi _{t}:=\{\pi _s,\,s\in [t,T]\}\) is the set of all admissible control strategies over [t, T].

It is proved by [16] that for a continuous, strictly increasing and concave utility function U, the optimal value functions \({\widetilde{V}}(t,x,j)\), for \(j\in {\mathbb {D}}\), satisfy the following system of HJB equations,

$$\begin{aligned}&0=\sup _{\pi ^{(j)}\in \Pi _{t}}\left[ {\widetilde{V}}_{t}(t,x,j)+\frac{1}{2}\sigma _{j}^{2}\left( \pi ^{(j)}\right) ^{2}x^{2}\cdot {\widetilde{V}}_{xx}(t,x,j)+\left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] \right. \nonumber \\&\qquad \quad \left. x\cdot {\widetilde{V}}_{x}(t,x,j) -\,q_{j}{\widetilde{V}}(t,x,j)+\displaystyle \sum _{{\ell =1,\ell \ne j}}^{d} q_{j\ell }{\widetilde{V}}\left( {t},{x},{\ell }\right) \right] , \end{aligned}$$
(2)

on \([0,T]\times (0,+\infty )\) and the terminal and boundary conditions are given by

$$\begin{aligned}&{\widetilde{V}}(T,x,j)=U(x),\; x\in [0,+\infty ), \end{aligned}$$
(3)
$$\begin{aligned}&{\widetilde{V}}(t,0,j)=0,\; t\in [0,T], \end{aligned}$$
(4)
$$\begin{aligned}&{\widetilde{V}}(t,x_{\max },j)={\widetilde{\phi }}(t,x_{\max },j),\; t\in [0,T], \end{aligned}$$
(5)

where \(q_{j}:=\displaystyle \sum _{\ell =1,\ell \ne j}^{d} q_{j\ell }\), and the boundary condition (5) will be specified for concrete problems in the following sections. The verification theorem is also given by [16]. Moreover, [16] define a functional operator \(\mathfrak {R}\),

$$\begin{aligned}&{\widetilde{V}}^{(m+1)}(t,x,j)=\mathfrak {R}{\widetilde{V}}^{(m)}(t,x,j)\nonumber \\&\quad =e^{q_jt}\cdot \sup _{\pi ^{(j)}\in \Pi _{t}} E_{t,x,j} \Big [ \int _{t}^{T} e^{-q_{j}s}\sum _{\ell =1,{\ell \ne j}}^{d} q_{j\ell }{\widetilde{V}}^{(m)}(s,X_{\pi ^{(j)}}^{(j)}(s),\ell )ds\nonumber \\&\qquad + e^{-q_{j}T}U(X_{\pi ^{(j)}}^{(j)}(T))\Big ], \end{aligned}$$
(6)

and claim that the sequences \({\widetilde{V}}^{(m)}(t,x,j)\) converge to the value function \({\widetilde{V}}(t,x,j)\) as m tends to \(\infty \). The function \({\widetilde{V}}^{(0)}(t,x,j)\) is computed by

$$\begin{aligned} {\widetilde{V}}^{(0)}(t,x,j)=E_{t,x,j}\Big [U \Big (x\cdot \exp {(\int _{t}^{T}\langle \varvec{r}, \varvec{\alpha }(s)\rangle ds)} \Big )\Big ]. \end{aligned}$$
(7)

Using dynamic programming principle and variable transformation \(\tau =T-t\), \({\widetilde{V}}^{(m+1)}\left( {t},{x},{j}\right) ={\widetilde{V}}^{(m+1)}\left( T-\tau ,x,j\right) \equiv V^{(m+1)}\left( \tau ,x,j\right) \), Eq. (6) leads to the following HJB equations for \(j\in {\mathbb {D}}\)

$$\begin{aligned} V_{\tau }^{(m+1)}(\tau ,x,j)= & {} \sup _{\pi ^{(j)}\in \Pi _{\tau }}\Big [\frac{1}{2}\sigma _{j}^{2}\left( \pi ^{(j)}\right) ^{2}x^{2}\cdot V_{xx}^{(m+1)}(\tau ,x,j)+\left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] x\nonumber \\&\quad \cdot \, V_{x}^{(m+1)}(\tau ,x,j)-q_{j}V^{(m+1)}(\tau ,x,j)+\sum _{\ell =1,\ell \ne j}^{d} q_{j\ell }V^{(m)}\left( {\tau },{x},{\ell }\right) \Big ],\nonumber \\ \end{aligned}$$
(8)

on \([0,T]\times (0,+\infty )\) with terminal and boundary conditions

$$\begin{aligned}&V^{(m+1)}(0,x,j)=U(x), \quad x\in [0,+\infty ), \end{aligned}$$
(9)
$$\begin{aligned}&V^{(m+1)}(\tau ,0,j)=0, \quad \tau \in [0,T], \end{aligned}$$
(10)
$$\begin{aligned}&V^{(m+1)}(\tau ,x_{\max },j)=\phi (\tau ,x_{\max },j), \quad \tau \in [0,T], \end{aligned}$$
(11)

where \({\widetilde{\phi }}(t,x_{\max },j)={\widetilde{\phi }}(T-\tau ,x_{\max },j)\equiv \phi (\tau ,x_{\max },j)\). This treatment reduces a system of fully coupled HJB equations (2) to a sequence of decoupled HJB equations (8).

We mainly study the iterative FDMs with policy iterations for the system of decoupled HJB equations (8). A grid is constructed consisting of a set of \(M+1\) nodes \(\left\{ {x_{0},\ldots ,x_{M}}\right\} \) with \(x_{0}=0\),   \(x_{M}=x_{\max }\),   \(\Delta x=\frac{x_{\max }}{M}\), following a sequence of N time steps \(\left\{ {\tau _{0},\ldots ,\tau _{N}}\right\} \) with \(\Delta \tau =\frac{T}{N}\), \(\tau _n=n \Delta \tau \). Let \(V_{i}^{(m+1),n}(j)\) be the approximation to \(V^{(m+1)}(\tau _n,x_i,j)\). Equation (8) can be discretized by a standard finite difference method to give

$$\begin{aligned}&\frac{V_{i}^{(m+1),n+1}(j)-V_{i}^{(m+1),n}(j)}{\Delta \tau }\nonumber \\&\quad =\sup _{\pi ^{(j)}\in \Pi _{\tau }}\Big [\frac{1}{2}\sigma _{j}^{2}\left( \pi ^{(j)}\right) ^{2}x_{i}^{2} \frac{V_{i+1}^{(m+1),n+1}(j)-2V_{i}^{(m+1),n+1}(j)+V_{i-1}^{(m+1),n+1}(j)}{ \Delta x^2}\nonumber \\&\qquad +\,\xi \left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] x_{i}\cdot \frac{V_{i+1}^{(m+1),n+1}(j) -V_{i}^{(m+1),n+1}(j)}{\Delta x}\nonumber \\&\qquad +\,(1-\xi )\left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] x_{i}\cdot \frac{V_{i}^{(m+1),n+1}(j) -V_{i-1}^{(m+1),n+1}(j)}{\Delta x}\nonumber \\&\qquad -\,q_{j}V_{i}^{(m+1),n+1}(j)+\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }V_{i}^{(m),n+1}(\ell )\Big ], \end{aligned}$$
(12)

with \(V_0^{(m),n}(j)=0\) and \(V_{M}^{(m),n}(j)=\phi (\tau _{n},x_{M},j)\). For the convenience of analysis, (12) is re-written as

$$\begin{aligned}&\frac{V_{i}^{(m+1),n+1}(j)-V_{i}^{(m+1),n}(j)}{\Delta \tau } \nonumber \\= & {} \Big [\left( -\alpha _{i}^{n+1}(\pi _{n+1}^{(j)}) -\beta _{i}^{n+1}(\pi _{n+1}^{(j)})-q_j\right) V_{i}^{(m+1),n+1}(j) + \alpha _{i}^{n+1}(\pi _{n+1}^{(j)}) V_{i-1}^{(m+1),n+1}(j)\nonumber \\&\quad +\, \beta _{i}^{n+1}(\pi _{n+1}^{(j)}) V_{i+1}^{(m+1),n+1}(j)\Big ]+\sum _{\ell =1,\ell \ne {j}}^{d} q_{j\ell }V_{i}^{(m),n+1}(\ell ), \end{aligned}$$
(13)

where

$$\begin{aligned}&\pi _{n+1}^{(j)}\in \hbox {arg}\sup _{\pi ^{(j)}\in \Pi _{\tau }}\Big [\alpha _{i}^{n+1} (\pi ^{(j)})V_{i-1}^{(m+1),n+1}(j)+\beta _{i}^{n+1} (\pi ^{(j)})V_{i+1}^{(m+1),n+1}(j)\\&\quad +\,\big (-\alpha _{i}^{n+1}(\pi ^{(j)})-\beta _{i}^{n+1} (\pi ^{(j)})-q_j\big )V_{i}^{(m+1),n+1}(j)\Big ], \end{aligned}$$

and

$$\begin{aligned}&\alpha _{i}^{n+1}(\pi _{n+1}^{(j)})=\frac{\sigma _{j}^{2}\big (\pi _{n+1}^{(j)}\big )^{2} x_{i}^{2}}{2\Delta x^2}-\frac{(1-\xi )\left[ \pi _{n+1}^{(j)}(\mu _j-r_j)+r_j\right] x_{i}}{\Delta x},\\&\beta _{i}^{n+1}(\pi _{n+1}^{(j)})=\frac{\sigma _{j}^{2}\big (\pi _{n+1}^{(j)}\big )^{2}x_{i}^{2}}{2\Delta x^2}+\frac{\xi \left[ \pi _{n+1}^{(j)}(\mu _j-r_j)+r_j\right] x_{i}}{\Delta x}, \end{aligned}$$

where at each node, \(\xi \in \{0,1\}\) is chosen to ensure that \(\alpha _{i}^{n+1}(\pi _{n+1}^{(j)})\) and \(\beta _{i}^{n+1}(\pi _{n+1}^{(j)})\) are positive. For ease of analysis, we can also write the Eq. (13) coupled with boundary conditions (10) and (11) into the matrix form. Let

$$\begin{aligned} {\mathbf {V}}^{(m+1),n+1}(j)=\left[ V_{0}^{(m+1),n+1}(j),\ldots ,V_{M}^{(m+1),n+1}(j)\right] '. \end{aligned}$$

Define matrix operator \(A^{(m+1)}(\pi _{n+1}^{(j)})\) by

$$\begin{aligned}&\left[ A^{(m+1)}(\pi _{n+1}^{(j)}){\mathbf {V}}^{(m+1),n+1}(j)\right] _{i+1}\nonumber \\&\quad =\Big [\big (-\alpha _{i}^{n+1}(\pi _{n+1}^{(j)})-\beta _{i}^{n+1} (\pi _{n+1}^{(j)})-q_j\big )V_{i}^{(m+1),n+1}(j)+\alpha _{i}^{n+1} (\pi _{n+1}^{(j)})V_{i-1}^{(m+1),n+1}(j)\nonumber \\&\quad +\,\beta _{i}^{n+1}(\pi _{n+1}^{(j)})V_{i+1}^{(m+1),n+1}(j)\Big ],\quad i=1,\ldots ,M-1. \end{aligned}$$
(14)

Then (13) can be written as

$$\begin{aligned} \left[ {\mathbf {I}}-\Delta \tau A^{(m+1)}(\pi _{n+1}^{(j)})\right] {\mathbf {V}}^{(m+1),n+1}(j) ={\mathbf {V}}^{(m+1),n}(j)+ \phi ^{n+1}(j)-\phi ^{n}(j) +{\mathbf {D}}^{(m),n+1}(j), \end{aligned}$$
(15)

where

$$\begin{aligned} \begin{aligned}&{\mathbf {D}}^{(m),n+1}(j)=\Big [0,\sum _{\ell =1,\ell \ne {j}}^{d} q_{j\ell }V_{1}^{(m),n+1}(\ell ),\ldots ,\sum _{\ell =1,\ell \ne {j}}^{d} q_{j\ell }V_{M-1}^{(m),n+1}(\ell ),0\Big ]',\\&\phi ^{n+1}(j)=\left[ 0,\ldots ,0,\phi _{M}^{n+1}(j)\right] ',\quad \phi _{M}^{n+1}(j):=\phi (\tau _{n+1},x_{M},j). \end{aligned} \end{aligned}$$

From [3, 4, 13], it is known that the stability, consistency and monotonicity of the discretization can ensure the convergence to the viscosity solution. So we will analyze the stability, consistency and monotonicity of (12) or equivalent form (13) or its matrix form (15).

Lemma 2.1

(Stability of the iterative FDMs) If boundary function \(\phi \) in (11) is bounded, then the fully implicit iterative FDMs (12) are stable,

$$\begin{aligned} \Vert {\mathbf {V}}^{(m+1),n+1}\Vert _{\infty }\le \max {\left\{ \Vert {\mathbf {V}}^{(m+1),0}\Vert _{\infty },C_{1},C_{2}\right\} }, \end{aligned}$$

where \(C_{1}\equiv \displaystyle {\max _{i,j,n}}\Big \vert \sum _{\ell =1,\ell \ne {j}}^{d} V_{i}^{(m),n+1}(\ell )\Big \vert \), \(C_{2}\equiv \displaystyle {\max _{j,n}}\left| \phi _{M}^{n+1}(j)\right| \).

Proof

From (13), for \(i=1,\ldots ,M-1\), we have

$$\begin{aligned}&V_{i}^{(m+1),n+1}(j)-V_{i}^{(m+1),n}(j)\nonumber \\&\quad =\Delta \tau \big [\big (-\alpha _{i}^{n+1}(\pi _{n+1}^{(j)}) -\beta _{i}^{n+1}(\pi _{n+1}^{(j)})-q_j\big )V_{i}^{(m+1),n+1}(j)\nonumber \\&\qquad + \alpha _{i}^{n+1}(\pi _{n+1}^{(j)})V_{i-1}^{(m+1),n+1}(j)\nonumber \\&\qquad +\beta _{i}^{n+1}(\pi _{n+1}^{(j)})V_{i+1}^{(m+1),n+1}(j)\big ]+\Delta \tau \sum _{\ell =1,\ell \ne {j}}^{d} q_{j\ell }V_{i}^{(m),n+1}(\ell ), \end{aligned}$$
(16)

and

$$\begin{aligned} V_{0}^{(m+1),n+1}(j)=0,\;\; V_{M}^{(m+1),n+1}(j)=\phi _{M}^{n+1}(j). \end{aligned}$$

From (16), we derive that

$$\begin{aligned}&\vert V_{i}^{(m+1),n+1}(j)\vert \cdot \left( 1+\Delta \tau \big (\alpha _{i}^{n+1}(\pi _{n+1}^{(j)}) +\beta _{i}^{n+1}(\pi _{n+1}^{(j)})+q_j\big )\right) \nonumber \\\le & {} \Vert {\mathbf {V}}^{(m+1),n}\Vert _\infty + \Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty \Delta \tau \big (\alpha _{i}^{n+1}(\pi _{n+1}^{(j)}) +\beta _{i}^{n+1}(\pi _{n+1}^{(j)})\big )\nonumber \\&+\,\Delta \tau \Big \vert \sum _{\ell =1,\ell \ne {j}}^{d} q_{j\ell }V_{i}^{(m),n+1}(\ell )\Big \vert . \end{aligned}$$
(17)

Due to \(\Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty =\displaystyle {\max _{i,j}}\Big \vert V_{i}^{(m+1),n+1}(j)\Big \vert \), there must exist \(i_*\) and \(j_*\), such that

\(\Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty =\Big \vert V_{i_*}^{(m+1),n+1}(j_*)\Big \vert \). If \(i_*\in \{i= 1,\ldots ,M-1\}\), then inserting \(i_*\), \(j_*\) into (17) gives that

$$\begin{aligned} \begin{aligned} \Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty \cdot \left( 1+\Delta \tau q_{j_*}\right)&\le \Vert {\mathbf {V}}^{(m+1),n}\Vert _\infty +\Delta \tau q_{j_*} \max _{i,j} \Big \vert \sum _{\ell =1,\ell \ne {j}}^{d}V_{i}^{(m),n+1}(\ell )\Big \vert . \end{aligned} \end{aligned}$$

So we have

$$\begin{aligned} \Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty \le \max \Big \{\Vert {\mathbf {V}}^{(m+1),n}\Vert _\infty , \max _{i,j} \Big \vert \sum _{\ell =1,\ell \ne {j}}^{d}V_{i}^{(m),n+1}(\ell )\Big \vert \Big \}. \end{aligned}$$
(18)

If \(i_*=M\), then \(\Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty =\displaystyle {\max _{j}}\vert \phi _{M}^{n+1}(j)\vert \). So we have

$$\begin{aligned} \Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty \le \displaystyle {\max _{j,n}}\vert \phi _{M}^{n+1}(j)\vert . \end{aligned}$$
(19)

Combining (18) and (19), we obtain

$$\begin{aligned} \Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty \le \max \Big \{\Vert {\mathbf {V}}^{(m+1),n}\Vert _\infty ,\max _{i,j} \Big \vert \sum _{\ell =1,\ell \ne j}^{d}V_{i}^{(m),n+1}(\ell )\Big \vert ,\max _{j,n}\vert \phi _{M}^{n+1}(j)\vert \Big \}. \end{aligned}$$

Let \(C_{1}\equiv \displaystyle {\max _{i,j,n}}\Big \vert \displaystyle {\sum _{\ell =1,\ell \ne {j}}^{d}} V_{i}^{(m),n+1}(\ell )\Big \vert \), \(C_{2}\equiv \displaystyle {\max _{j,n}}\vert \phi _{M}^{n+1}(j)\vert \). Then we obtain

$$\begin{aligned} \Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty \le \max \left\{ \Vert {\mathbf {V}}^{(m+1),n}\Vert _\infty ,C_{1},C_{2}\right\} . \end{aligned}$$
(20)

Iteratively using (20) gives that

$$\begin{aligned} \Vert {\mathbf {V}}^{(m+1),n+1}\Vert _\infty \le \max \left\{ \Vert {\mathbf {V}}^{(m+1),0}\Vert _\infty ,C_{1},C_{2}\right\} . \end{aligned}$$

\(\square \)

To proceed the analysis, it is convenient to denote (12) as

$$\begin{aligned}&G_{j}\Big (V_{i}^{(m+1),n+1}(j),V_{i-1}^{(m+1),n+1}(j),V_{i+1}^{(m+1),n+1}(j), V_{i}^{(m+1),n}(j),\nonumber \\&\qquad -\, \sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }V_{i}^{(m),n+1}(\ell )\Big )=0, \end{aligned}$$
(21)

where \(G_{j}\) is defined by the left-hand side minus the right-hand side of (12), and denote (8) as

$$\begin{aligned} F_{j}\Big (V_{xx}^{(m+1)}(j),V_{x}^{(m+1)}(j),V_{\tau }^{(m+1)}(j), V^{(m+1)}(j),-\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }V^{(m)}(\ell ),x,\tau \Big )=0, \end{aligned}$$
(22)

where \(F_{j}\) is defined by the left-hand side minus the right-hand side of (8) and \(V^{(m+1)}(j)\) denotes a function \(V^{(m+1)}(j)=V^{(m+1)}(\tau ,x,j)\), \(j\in {\mathbb {D}}\) is the current regime state. Next we give the definitions of the upper and lower semi-continuous envelopes of function \(F_{j}\).

Definition 2.1

The upper and lower semi-continuous envelopes of function \(F_{j}\) are defined respectively by

$$\begin{aligned}&{\overline{F}}_{j}\equiv \limsup _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x\\ {\widetilde{\tau }} \in B(\tau ,\rho )\\ {\widetilde{x}} \in B(x,h) \end{array}}\\&F_{j}\Big (V_{{\widetilde{x}}{\widetilde{x}}}^{(m+1)}(j), V_{{\widetilde{x}}}^{(m+1)}(j),V_{{\widetilde{\tau }}}^{(m+1)}(j),V^{(m+1)}(j), -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }V^{(m)}(\ell ),{\widetilde{x}},{\widetilde{\tau }}\Big ) \end{aligned}$$

and

$$\begin{aligned}&{\underline{F}}_{j}\equiv \liminf _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x\\ {\widetilde{\tau }} \in B(\tau ,\rho )\\ {\widetilde{x}} \in B(x,h) \end{array}} \\&\quad F_{j}\Big (V_{{\widetilde{x}}{\widetilde{x}}}^{(m+1)}(j), V_{{\widetilde{x}}}^{(m+1)}(j),V_{{\widetilde{\tau }}}^{(m+1)}(j),V^{(m+1)}(j), -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }V^{(m)}(\ell ),{\widetilde{x}},{\widetilde{\tau }}\Big ), \end{aligned}$$

where \(B(\cdot ,\circ )\) denotes the neighborhood with center \(\cdot \) and size \(\circ \).

We now give the definitions of the iteration-based viscosity sub-solution, the iteration-based viscosity super-solution and the iteration-based viscosity solution of (22) as follows.

Definition 2.2

Let \(V^{(m+1)}: {\overline{\Omega }}\rightarrow {\mathbb {R}}\) be locally bounded function.

  1. (i)

    If for all \(\varphi ^{(m+1)}\in C^{1,2}({\overline{\Omega }})\) and \(({\overline{\tau }},{\overline{x}})\in {\overline{\Omega }}\) such that \({\overline{V}}^{(m+1)}-\varphi ^{(m+1)}\) has a local maximum at \(({\overline{\tau }},{\overline{x}})\), we have

    $$\begin{aligned}&{\underline{F}}_{j}\Big (\varphi _{xx}^{(m+1)}({\overline{\tau }},{\overline{x}},j),\varphi _{x}^{(m+1)} ({\overline{\tau }},{\overline{x}},j),\varphi _{\tau }^{(m+1)}({\overline{\tau }},{\overline{x}},j), {\overline{V}}^{(m+1)}({\overline{\tau }},{\overline{x}},j),\nonumber \\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }{\overline{V}}^{(m)}({\overline{\tau }},{\overline{x}},\ell ),{\overline{x}},{\overline{\tau }}\Big )\le 0,\;\;\hbox {for}\;m\ge 0, \end{aligned}$$

    where \({\overline{V}}^{(0)}(\tau ,x,\ell )={\widetilde{V}}^{(0)}(T-\tau ,x,\ell )\) in (7), then \(V^{(m+1)}\) is called the iteration-based viscosity sub-solution of (22).

  2. (ii)

    If for all \(\varphi ^{(m+1)}\in C^{1,2}({\overline{\Omega }})\) and \(({\underline{\tau }},{\underline{x}})\in {\underline{\Omega }}\) such that \({\underline{V}}^{(m+1)}-\varphi ^{(m+1)}\) has a local minimum at \(({\underline{\tau }},{\underline{x}})\), we have

    $$\begin{aligned}&{\overline{F}}_{j}\Big (\varphi _{xx}^{(m+1)}({\underline{\tau }},{\underline{x}},j),\varphi _{x}^{(m+1)} ({\underline{\tau }},{\underline{x}},j),\varphi _{\tau }^{(m+1)}({\underline{\tau }},{\underline{x}},j), {\underline{V}}^{(m+1)}({\underline{\tau }},{\underline{x}},j),\\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }{\underline{V}}^{(m)}({\underline{\tau }},{\underline{x}},\ell ),{\underline{x}}, {\underline{\tau }}\Big )\ge 0,\;\; \hbox {for}\;m\ge 0, \end{aligned}$$

    where \({\underline{V}}^{(0)}(\tau ,x,\ell )={\widetilde{V}}^{(0)}(T-\tau ,x,\ell )\) in (7), then \(V^{(m+1)}\) is called the iteration-based viscosity super-solution of (22).

  3. (iii)

    If it is both a sub-solution and super-solution of (22), then we call that \(V^{(m+1)}\) is the iteration-based viscosity solution of (22).

Lemma 2.2

(Consistency of the iterative FDMs) The implicit iterative FDMs (12) are consistent, i.e., for \(\varphi ^{(m+1)}\in C^{1,2}\left( [0,T]\times [0,x_{\max }]\right) \) and \(\psi ^{(m)}\in C\left( [0,T]\times [0,x_{\max }]\right) \), it holds true that

$$\begin{aligned}&\liminf _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x \\ \rho \rightarrow 0 \\ h\rightarrow 0 \end{array}}G_{j}\Big (\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j),\varphi ^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}}-h,j),\varphi ^{(m+1)}({\widetilde{\tau }}, {\widetilde{x}}+h,j),\\&\qquad \qquad \qquad \varphi ^{(m+1)}({\widetilde{\tau }}-\rho ,{\widetilde{x}},j), -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},\ell )\Big ) \\\ge & {} {\underline{F}}_{j}\Big (\varphi _{xx}^{(m+1)}(\tau ,x,j),\varphi _{x}^{(m+1)}(\tau ,x,j), \varphi _{\tau }^{(m+1)}(\tau ,x,j),\varphi ^{(m+1)}(\tau ,x,j),\\&\qquad \qquad \qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}(\tau ,x,\ell ),x,\tau \Big ), \end{aligned}$$

and

$$\begin{aligned}&\limsup _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x \\ \rho \rightarrow 0 \\ h\rightarrow 0 \end{array}}G_{j}\Big (\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j), \varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}}-h,j),\varphi ^{(m+1)}({\widetilde{\tau }}, {\widetilde{x}}+h,j),\\&\qquad \qquad \qquad \varphi ^{(m+1)}({\widetilde{\tau }}-\rho ,{\widetilde{x}},j), -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},\ell )\Big )\\\le & {} {\overline{F}}_{j}\Big (\varphi _{xx}^{(m+1)}(\tau ,x,j),\varphi _{x}^{(m+1)}(\tau ,x,j), \varphi _{\tau }^{(m+1)}(\tau ,x,j),\varphi ^{(m+1)}(\tau ,x,j),\nonumber \\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}(\tau ,x,\ell ),x,\tau \Big ). \end{aligned}$$

Proof

For any stencil \(\{{\widetilde{x}}-h,\;{\widetilde{x}},\;{\widetilde{x}}+h\}\times \{{\widetilde{\tau }}-\rho ,{\widetilde{\tau }}\}\), we have that

$$\begin{aligned}&\Big | G_{j}\Big (\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j),\varphi ^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}}-h,j), \varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}}+h,j),\varphi ^{(m+1)} ({\widetilde{\tau }}-\rho ,{\widetilde{x}},j), \nonumber \\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d}q_{j\ell }\psi ^{(m)} ({\widetilde{\tau }},{\widetilde{x}},\ell )\Big )\nonumber \\&-F_{j} \Big (\varphi _{{\widetilde{x}}{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j),\varphi _{{\widetilde{x}}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) ,\varphi _{{\widetilde{\tau }}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j), \varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j),\nonumber \\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},\ell ),{\widetilde{x}},{\widetilde{\tau }}\Big )\Big | \nonumber \\= & {} \Big |\frac{\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) -\varphi ^{(m+1)}({\widetilde{\tau }}-\rho ,{\widetilde{x}},j)}{\rho }-\sup _{\pi ^{(j)}\in \Pi _{{\widetilde{\tau }}}} B(\pi ^{(j)})\nonumber \\&\qquad -\varphi _{{\widetilde{\tau }}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) +\sup _{\pi ^{(j)}\in \Pi _{{\widetilde{\tau }}}}C(\pi ^{(j)})\Big | \nonumber \\\le & {} \Big |\frac{\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) -\varphi ^{(m+1)}({\widetilde{\tau }}-\rho ,{\widetilde{x}},j)}{\rho } -\varphi _{{\widetilde{\tau }}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) \Big |\nonumber \\&+ \sup _{\pi ^{(j)}\in \Pi _{{\widetilde{\tau }}}}\Big | C(\pi ^{(j)})-B(\pi ^{(j)})\Big |, \end{aligned}$$
(23)

where

$$\begin{aligned} B(\pi ^{(j)})= & {} \frac{1}{2}\sigma _{j}^{2}\big (\pi ^{(j)}\big )^{2}{\widetilde{x}}^{2} \frac{\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}}+h,j) -2\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j)+\varphi ^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}}-h,j)}{ h^2}\\&+\,\xi \left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] {\widetilde{x}}\frac{\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}}+h,j) -\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j)}{h}\\&+\,(1-\xi )\left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] {\widetilde{x}} \frac{\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) -\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}}-h,j)}{h}\\&-\,q_{j}\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j)+\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},\ell ),\\ C(\pi ^{(j)})= & {} \frac{1}{2}\sigma _{j}^{2}\big (\pi ^{(j)}\big )^{2}{{\widetilde{x}}}^{2}\cdot \varphi _{{\widetilde{x}}{\widetilde{x}}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) +\left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] {\widetilde{x}}\varphi _{{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j) \\&-\,q_{j}\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j)+\sum _{\ell =1,\ell \ne j}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},\ell ). \end{aligned}$$

We expand \(\varphi ^{(m+1)}\) at node \(({\widetilde{\tau }},{\widetilde{x}},j)\) with the Taylor series for (23) to give that

$$\begin{aligned}&\Big | \frac{\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) -\varphi ^{(m+1)}({\widetilde{\tau }}-\rho ,{\widetilde{x}},j)}{\rho } -\varphi _{{\widetilde{\tau }}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) \Big | \nonumber \\&+ \sup _{\pi ^{(j)}\in \Pi _{{\widetilde{\tau }}}}\Big | C(\pi ^{(j)})-B(\pi ^{(j)})\Big | \nonumber \\&\quad =\Big |-\frac{1}{2}\rho \varphi _{{\widetilde{\tau }}{\widetilde{\tau }}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) +O(\rho ^2)\Big |+\Big |-\frac{1}{2}\sigma _{j}^{2}\big (\pi ^{(j)}\big )^{2}{{\widetilde{x}}}^{2}\cdot O(h) \nonumber \\&\qquad +\,\xi \left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] {\widetilde{x}}\cdot \Big (-\frac{1}{2}{h}\varphi _{{\widetilde{x}}{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j)+O(h^2)\Big )\nonumber \\&+\,(1-\xi )\left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] {\widetilde{x}}\cdot \Big (\frac{1}{2}{h}\varphi _{{\widetilde{x}}{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j)+O(h^2)\Big )\Big |\nonumber \\= & {} \left| O\left( \rho \right) +O\left( h\right) \right| . \end{aligned}$$
(24)

Therefore, combining (23) with (24) gives that

$$\begin{aligned}&\Big | G_{j}\Big (\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j),\varphi ^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}}-h,j), \varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}}+h,j),\varphi ^{(m+1)} ({\widetilde{\tau }}-\rho ,{\widetilde{x}},j), \nonumber \\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d}q_{j\ell }\psi ^{(m)} ({\widetilde{\tau }},{\widetilde{x}},\ell )\Big )\nonumber \\&-F_{j} \Big (\varphi _{{\widetilde{x}}{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j),\varphi _{{\widetilde{x}}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j) ,\varphi _{{\widetilde{\tau }}}^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j), \varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j),\nonumber \\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},\ell ),{\widetilde{x}},{\widetilde{\tau }}\Big )\Big | \le \left| O\left( \rho \right) +O\left( h\right) \right| . \end{aligned}$$
(25)

Consequently, it follows from (25) that

$$\begin{aligned}&\liminf _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x \\ \rho \rightarrow 0 \\ h\rightarrow 0 \end{array}}G_{j}\Big (\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j), \varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}}-h,j),\varphi ^{(m+1)}({\widetilde{\tau }}, {\widetilde{x}}+h,j),\varphi ^{(m+1)}\\&\qquad \qquad ({\widetilde{\tau }}-\rho ,{\widetilde{x}},j),-\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},\ell )\Big ) \\\ge & {} \liminf _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x \\ \rho \rightarrow 0 \\ h\rightarrow 0 \end{array}}F_{j}\Big (\varphi _{{\widetilde{x}}{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j),\varphi _{{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j),\varphi _{{\widetilde{\tau }}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j),\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j),\\&\qquad \qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},j),{\widetilde{x}},{\widetilde{\tau }}\Big )\\= & {} {\underline{F}}_{j}\Big (\varphi _{xx}^{(m+1)}(\tau ,x,j),\varphi _{x}^{(m+1)}(\tau ,x,j), \varphi _{\tau }^{(m+1)}(\tau ,x,j),\varphi ^{(m+1)}(\tau ,x,j), \\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)},x,\tau \Big ) \end{aligned}$$

and

$$\begin{aligned}&\limsup _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x \\ \rho \rightarrow 0 \\ h\rightarrow 0 \end{array}}G_{j} \Big (\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j),\varphi ^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}}-h,j),\varphi ^{(m+1)}({\widetilde{\tau }}, {\widetilde{x}}+h,j),\varphi ^{(m+1)}\\&\qquad ({\widetilde{\tau }}-\rho ,{\widetilde{x}},j), -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},\ell )\Big ) \\\le & {} \limsup _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x \\ \rho \rightarrow 0 \\ h\rightarrow 0 \end{array}}F_{j}\Big (\varphi _{{\widetilde{x}}{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j),\varphi _{{\widetilde{x}}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j),\varphi _{{\widetilde{\tau }}}^{(m+1)} ({\widetilde{\tau }},{\widetilde{x}},j),\varphi ^{(m+1)}({\widetilde{\tau }},{\widetilde{x}},j),\\&\qquad \qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)}({\widetilde{\tau }},{\widetilde{x}},j),{\widetilde{x}},{\widetilde{\tau }}\Big )\\= & {} {\overline{F}}_{j}\Big (\varphi _{xx}^{(m+1)}(\tau ,x,j), \varphi _{x}^{(m+1)}(\tau ,x,j),\varphi _{\tau }^{(m+1)}(\tau ,x,j),\varphi ^{(m+1)}(\tau ,x,j), \\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }\psi ^{(m)},x,\tau \Big ). \end{aligned}$$

Thus the proof is complete. \(\square \)

Lemma 2.3

(Monotonicity of the iterative FDMs) If boundary function \(\phi \) in (11) is bounded, then the implicit iterative FDMs (12) are monotone in the sense that for any \(\rho _1,\,\rho _2,\,\rho _3,\,\rho _4 \ge 0\), it holds true that

$$\begin{aligned}&G_{j}\Big (V_{i}^{(m+1),n+1}(j),V_{i-1}^{(m+1),n+1}(j)+\rho _1,V_{i+1}^{(m+1),n+1}(j) +\rho _2,V_{i}^{(m+1),n}(j)+\rho _3,\\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }(V_i^{(m),n+1}(\ell )+\rho _4) \Big )\\\le & {} G_{j}\Big (V_{i}^{(m+1),n+1}(j),V_{i-1}^{(m+1),n+1}(j),V_{i+1}^{(m+1),n+1}(j),V_{i}^{(m+1),n}(j),\\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }V_i^{(m),n+1}(\ell )\Big ). \end{aligned}$$

Proof

For any \(\rho _1,\,\rho _2,\,\rho _3,\,\rho _4 \ge 0\), using the definition of \(G_{j}\) in (21), \(q_{j\ell }>0\) for \(j,\, \ell \in {\mathbb {D}}\) and \(j\ne \ell \), and the following inequalities

$$\begin{aligned} \sup _{x}{X(x)}-\sup _{x}{Y(x)}\le \sup _{x}\Big (X(x)-Y(x)\Big ), \end{aligned}$$

we derive that

$$\begin{aligned}&G_{j}\Big (V_{i}^{(m+1),n+1}(j),V_{i-1}^{(m+1),n+1}(j)+\rho _1,V_{i+1}^{(m+1),n+1}(j) +\rho _2,V_{i}^{(m+1),n}(j)+\rho _3,\\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }(V_i^{(m),n+1}(\ell )+\rho _4) \Big )\\&-\,G_{j}\Big (V_{i}^{(m+1),n+1}(j),V_{i-1}^{(m+1),n+1}(j), V_{i+1}^{(m+1),n+1}(j),V_{i}^{(m+1),n}(j),\\&\qquad -\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }V_i^{(m),n+1}(\ell )\Big )\\\le & {} -\frac{\rho _3}{\Delta \tau }+\sup _{\pi ^{(j)}\in \Pi _{\tau }}\left( -q_{j}\rho _4-\alpha _i^{n+1}(\pi _{n+1}^{(j)})\rho _1- \beta _{i}^{n+1}(\pi _{n+1}^{(j)})\rho _2\right) \le 0. \end{aligned}$$

\(\square \)

Lemma 2.4

(Comparison principle) Let \({\overline{V}}\) (resp. \({\underline{V}}\)) be a upper-semi-continuous viscosity sub-solution (resp. lower-semi-continuous viscosity super-solution) with polynomial growth condition to (8). If boundary function \(\phi \) in (11) is bounded and \({\overline{V}}(T,.)\le {\underline{V}}(T,.)\) on \([0,+\infty )\). Then \({\overline{V}}\le {\underline{V}}\) on \([0,T]\times [0,+\infty )\).

Proof

Since both \( b(x,\pi ^{j})=\left[ \pi ^{(j)}(\mu _j-r_j)+r_j\right] x\) and \(a(x,\pi ^{j})=\sigma _{j}\pi ^{(j)}x\) satisfy the Lipschitz condition in x and \(f(\tau ,x,\pi ^{j})=\sum _{\ell =1,\ell \ne j}^{d} q_{j\ell }V^{(m)}\left( {\tau },{x},{\ell }\right) \) is uniformly continuous in \((\tau ,x)\), the proof follows from [25, Theorem 4.4.5]. \(\square \)

From Lemmas 2.1, 2.2, 2.3, we know that (12) or (15) is a consistent, stable, monotone discretization. In [3, 4, 13, 20, 21, 26, 27], they all mention that a consistent, stable, monotone discretization converges to the viscosity solution. In this paper to prove the convergence of the iterative FDMs (12) for solving (8), we must specially deal with the operator iterations from m-th step to \((m+1)\)-th step. The result is presented in the following Theorem 2.1.

Theorem 2.1

(Convergence of the iterative FDMs) Assumed that the original HJB equation (8) satisfies the conditions for Lemma 2.4 and discretization (12) satisfies all the conditions for Lemmas 2.1, 2.2, 2.3. Let \(V^{(m+1),h,\rho }\) denote the continuous form of (12) with \(h=\Delta x\) and \(\rho =\Delta \tau \). Then \(V^{(m+1),h,\rho }\) converges to the unique viscosity solution \(V^{(m+1)}\) of the nonlinear PDE (8), when \(\rho \rightarrow 0\) and \(h\rightarrow 0\).

Proof

If \(m=0\), \(V^{(0)}(\tau ,x,\ell )={\widetilde{V}}^{(0)}(T-\tau ,x,\ell )\) in (7) is equal to the exact value. Using Lemmas 2.1, 2.2, 2.3, 2.4 and following the lines in [4], we can prove that the solution \(V^{(1),h,\rho }\) of (12) converges to the unique viscosity solution \(V^{(1)}\) of the nonlinear PDE (8) as \(\rho \rightarrow 0\) and \(h\rightarrow 0\). Without loss of generality, we assume that the solution \(V^{(m),h,\rho }\) of (12) converges to the unique viscosity solution \(V^{(m)}\) of the nonlinear PDE (8) as \(\rho \rightarrow 0\) and \(h\rightarrow 0\). To complete the proof of this theorem by methods of induction, we only need to prove that the theorem holds true for \(m+1\). Let

$$\begin{aligned} {\overline{u}}^{(m+1)}(\tau ,x,j)\equiv & {} \limsup _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x\\ h \rightarrow 0\\ \rho \rightarrow 0 \end{array}}V^{(m+1),h,\rho }\left( {\widetilde{\tau }},{\widetilde{x}},j\right) \\ \hbox {and}\; {\underline{u}}^{(m+1)}(\tau ,x,j)\equiv & {} \liminf _{\begin{array}{c} {\widetilde{\tau }}\rightarrow \tau \\ {\widetilde{x}}\rightarrow x\\ h \rightarrow 0\\ \rho \rightarrow 0 \end{array}}V^{(m+1),h,\rho }\left( {\widetilde{\tau }},{\widetilde{x}},j\right) \end{aligned}$$

where \(h=\Delta x\) and \(\rho =\Delta \tau \) are the spatial and temporal mesh sizes.

Next, we prove that \({\overline{u}}^{(m+1)}\) is the sub-solution of Eq. (8). To this end, let\(({\overline{\tau }},{\overline{x}})\) be a local maximum point of \({\overline{u}}^{(m+1)}(\tau ,x,j)-\varphi ^{(m+1)}(\tau ,x,j)\) for some \(\varphi ^{(m+1)}\in C^{1,2}([0,T]\times [0,x_{\max }])\). By definition, we can find a neighbourhood \(\Theta \) in \([0,T]\times [0,x_{\max }]\) with center \((\tau ,x)\), whose closure is compact and on which \(({\overline{\tau }},{\overline{x}})\) is a global maximum point of \({\overline{u}}^{(m+1)}(\tau ,x,j)-\varphi ^{(m+1)}(\tau ,x,j)\). Without loss of generality, we may assume the maximum is strict, \({\overline{u}}^{(m+1)}({\overline{\tau }},{\overline{x}},j)=\varphi ^{(m+1)}({\overline{\tau }},{\overline{x}},j)\) and \(\varphi ^{(m+1)} \ge \sup _{h,\rho } \Vert V^{(m+1),h,\rho }\Vert _{\infty }\) outside \(\Theta \). This can be asserted by Lemma 2.1 which indicates that \(\Vert V^{(m+1),h,\rho }\Vert _{\infty }\le C\), where C is a positive constant that is independent of h and \(\rho \). So, we have that

$$\begin{aligned} {\overline{u}}^{(m+1)}(\tau ,x,j)-\varphi ^{(m+1)}(\tau ,x,j) \le {\overline{u}}^{(m+1)}({\overline{\tau }},{\overline{x}},j)-\varphi ^{(m+1)}({\overline{\tau }},{\overline{x}},j)=0, \end{aligned}$$

where \((\tau ,x)\in [0,T]\times [0,x_{\max }]\). There exist sequences \((\tau ^{k}, x^{k})\in [0,T]\times [0,x_{\max }]\), \(h_{k},\,\rho _{k}\) such that \((\tau ^{k}, x^{k})\) is the maximum point of \(V^{(m+1),h_{k},\rho _{k}}(\tau ,x,j)-\varphi ^{(m+1)}(\tau ,x,j)\), and as \(k\rightarrow \infty \), \(h_{k}\rightarrow 0,\,\rho _{k}\rightarrow 0\), \((\tau ^{k}, x^{k})\rightarrow ({\overline{\tau }},{\overline{x}})\), \(V^{(m+1),h_{k},\rho _{k}}(\tau ^{k},x^{k},j)\rightarrow {\overline{u}}^{(m+1)}({\overline{\tau }},{\overline{x}},j)\). Let

$$\begin{aligned} \xi _{k}\equiv V^{(m+1),h_{k},\rho _{k}}(\tau ^{k},x^{k},j)-\varphi ^{(m+1)}(\tau ^{k},x^{k},j). \end{aligned}$$

Then \(\xi _{k}\rightarrow 0\), as \(k\rightarrow \infty \), and since \((\tau ^{k}, x^{k})\) is the maximum point of \(V^{(m+1),h_{k},\rho _{k}}(\tau ,x,j)-\varphi ^{(m+1)}(\tau ,x,j)\), we have

$$\begin{aligned} V^{(m+1),h_{k},\rho _{k}}(\tau ,x,j)\le \varphi ^{(m+1)}(\tau ,x,j)+\xi _{k},\quad \hbox {for all}\; (\tau ,x)\in [0,T]\times [0,x_{\max }]. \end{aligned}$$
(26)

Since \(V^{(m),h_{k},\rho _{k}}(\tau ,x,j)\) converges to the viscosity solution of the nonlinear PDE (8) as \(\rho \rightarrow 0\) and \(h\rightarrow 0\), which is the assumption for the methods of induction, we have that

$$\begin{aligned} {\underline{u}}^{(m)}(\tau ,x,j)\le V^{(m),h_{k},\rho _{k}}(\tau ,x,j)\le {\overline{u}}^{(m)}(\tau ,x,j). \end{aligned}$$
(27)

Therefore, using the monotonicity of \(G_{j}\) in Lemma 2.3 gives that

$$\begin{aligned} 0= & {} G_{j} \Big (V^{(m+1),h_{k},\rho _{k}}(\tau ^{k},x^{k},j),V^{(m+1),h_{k},\rho _{k}} (\tau ^{k},x^{k}-h_{k},j),V^{(m+1),h_{k},\rho _{k}}(\tau ^{k},x^{k}+h_{k},j),\\&\qquad V^{(m+1),h_{k},\rho _{k}}(\tau ^{k}-\rho _{k},x^{k},j),-\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }V^{(m),h_{k},\rho _{k}}(\tau ^{k},x^{k},\ell )\Big ) \\\ge & {} G_{j}\Big (\varphi ^{(m+1)}(\tau ^{k},x^{k},j)+\xi _{k},\varphi ^{(m+1)} (\tau ^{k},x^{k}-h_{k},j)+\xi _{k},\varphi ^{(m+1)}(\tau ^{k},x^{k}+h_{k},j)+\xi _{k},\\&\qquad \varphi ^{(m+1)}(\tau ^{k}-\rho _{k},x^{k},j)+\xi _{k},-\sum _{\ell =1,{\ell }\ne {j}}^{d} q_{j\ell }{\overline{u}}^{(m)}(\tau ^{k},x^{k},\ell )\Big ). \end{aligned}$$

It then follows from the consistency of \(G_{j}\) in Lemma 2.2 that

$$\begin{aligned} 0\ge & {} \liminf _{\begin{array}{c} k \rightarrow \infty \end{array}}G_{j}\Big (\varphi ^{(m+1)}(\tau ^{k},x^{k},j)+\xi _{k},\varphi ^{(m+1)} (\tau ^{k},x^{k}-h_{k},j)+\xi _{k},\varphi ^{(m+1)} (\tau ^{k},x^{k}+h_{k},j)\\&\qquad +\, \xi _{k},\varphi ^{(m+1)}(\tau ^{k}-\rho _{k},x^{k},j)+\xi _{k},-\sum _{\ell =1, {\ell }\ne {j}}^{d}q_{j\ell }{\overline{u}}^{(m)}(\tau ^{k} ,x^{k},\ell )\Big )\\\ge & {} {\underline{F}}_{j}\Big (\varphi _{xx}^{(m+1)}({\overline{\tau }},{\overline{x}},j), \varphi _{x}^{(m+1)}({\overline{\tau }},{\overline{x}},j), \varphi _{\tau }^{(m+1)}({\overline{\tau }},{\overline{x}},j),\varphi ^{(m+1)} ({\overline{\tau }},{\overline{x}},j),\\&\qquad -\, \sum _{\ell =1,{\ell }\ne {j}}^{d}q_{j\ell }{\overline{u}}^{(m)} ({\overline{\tau }},{\overline{x}},\ell ),{\overline{x}},{\overline{\tau }}\Big )\\= & {} {\underline{F}}_{j}\Big (\varphi _{xx}^{(m+1)}({\overline{\tau }},{\overline{x}},j), \varphi _{x}^{(m+1)}({\overline{\tau }},{\overline{x}},j), \varphi _{\tau }^{(m+1)}({\overline{\tau }},{\overline{x}},j),{\overline{u}}^{(m+1)} ({\overline{\tau }},{\overline{x}},j),\\&\qquad -\, \sum _{\ell =1,{\ell }\ne {j}}^{d}q_{j\ell }{\overline{u}}^{(m)} ({\overline{\tau }},{\overline{x}},\ell ),{\overline{x}},{\overline{\tau }}\Big ). \end{aligned}$$

So \({\overline{u}}^{(m+1)}\) is the iteration-based sub-solution of Eq. (8). Analogously, it can be proved that \({\underline{u}}^{(m+1)}\) is the iteration-based super-solution of Eq. (8). Then it follows from the comparison principle (see Lemma 2.4) that

$$\begin{aligned} {\overline{u}}^{(m+1)}({\overline{\tau }},{\overline{x}},j)\le {\underline{u}}^{(m+1)}({\overline{\tau }},{\overline{x}},j). \end{aligned}$$

Furthermore the opposite inequality is obviously true from the definitions of \({\overline{u}}^{(m+1)}\) and \({\underline{u}}^{(m+1)}\). Therefore, we have

$$\begin{aligned} {\overline{u}}^{(m+1)}({\overline{\tau }},{\overline{x}},j) = {\underline{u}}^{(m+1)}({\overline{\tau }},{\overline{x}},j). \end{aligned}$$

This implies that the solution \(V^{(m+1),h,\rho }\) of (12) converges to the unique iteration-based viscosity solution \(V^{(m+1)}\) of the nonlinear PDE (8) as \(\rho \rightarrow 0\) and \(h\rightarrow 0\). \(\square \)

To implement the iterative FDM scheme (12), we need the following algorithm of iteration policy.

figure a

Theorem 2.2

(Convergence of the algorithm of iteration policy) If boundary function \(\phi \) in (11) is bounded, then the sequences \((\hat{\mathbf {V}}(j))^{k}\) in Algorithm 1 converge monotonically to the unique solution of (13) or (15) for any initial iteration value \({(\hat{{\mathbf {V}}})^{0}}\) as \(k\rightarrow \infty \).

Proof

We will first prove that this algorithm is convergent by showing that the sequences \((\hat{\mathbf {V}}(j))^{k}\) for \(k\ge 1\) are non-decreasing and bounded. Subtracting the equations for steps k and \(k+1\) on line 6 in Algorithm 1 leads to that

$$\begin{aligned}&\left[ {\mathbf {I}}-\Delta \tau A^{(m+1)}((\pi _{n+1}^{(j)})^{k})\right] \left[ (\hat{\mathbf {V}}(j))^{k+1} -(\hat{\mathbf {V}}(j))^{k}\right] \\= & {} \Delta \tau \Big [A^{(m+1)}((\pi _{n+1}^{(j)})^{k})-A^{(m+1)}((\pi _{n+1}^{(j)})^{k-1})\Big ] (\hat{\mathbf {V}}(j))^{k}. \end{aligned}$$

From Algorithm 1 (line 7), we know that

$$\begin{aligned} A^{(m+1)}((\pi _{n+1}^{(j)})^{k})(\hat{\mathbf {V}}(j))^{k} =\sup _{\pi ^{(j)}\in \Pi _{\tau }}\left[ A^{(m+1)}(\pi ^{(j)})(\hat{\mathbf {V}}(j))^{k}\right] . \end{aligned}$$

Therefore,

$$\begin{aligned} \left[ A^{(m+1)}((\pi _{n+1}^{(j)})^{k})-A^{(m+1)} ((\pi _{n+1}^{(j)})^{k-1})\right] (\hat{\mathbf {V}}(j))^{k}\ge 0. \end{aligned}$$

From (14) and \(\alpha _{i}^{n+1}((\pi _{n+1}^{(j)})^{k})\), \(\beta _{i}^{n+1}((\pi _{n+1}^{(j)})^{k})\) are non-negative, we know that

\(\left[ {\mathbf {I}}-\Delta \tau A^{(m+1)}((\pi _{n+1}^{(j)})^{k})\right] \) has positive diagonals, non-positive off-diagonals, and is diagonally dominant. So it is an M-matrix. Therefore, we have

$$\begin{aligned} (\hat{\mathbf {V}}(j))^{k+1}-(\hat{\mathbf {V}}(j))^{k}\ge 0,\quad k\ge 1, \end{aligned}$$
(28)

i.e., the sequences \((\hat{\mathbf {V}}(j))^{k}\) for \(k\ge 1\) are non-decreasing. Now we prove that the sequences are bounded. To this end, let

$$\begin{aligned} {\mathbf {b}}(j)={\mathbf {V}}^{(m+1),n}(j)+\Delta \tau {\mathbf {D}}^{(m),n+1}(j) +\varvec{\phi }^{n+1}(j)-\varvec{\phi }^{n}(j). \end{aligned}$$

Since \({\mathbf {V}}^{(m+1),n}(j)\), \({\mathbf {D}}^{(m),n+1}(j)\) and \(\varvec{\phi }^{n+1}(j)-\varvec{\phi }^{n}(j)\) are bounded with infinity norm, we know that \({\mathbf {b}}(j)\) is bounded. Using the notation of \({\mathbf {b}}(j)\), the equation on line 6 in Algorithm 1 can be written as

$$\begin{aligned}&\left[ 1+\Delta \tau \big (\alpha _{i}^{n+1}((\pi _{n+1}^{(j)})^{k}) +\beta _{i}^{n+1}((\pi _{n+1}^{(j)})^{k})+q_j\big )\right] (\hat{V}_{i}(j))^{k+1}\\= & {} \Delta \tau \big (\alpha _{i}^{n+1}((\pi _{n+1}^{(j)})^{k}) (\hat{V}_{i-1}(j))^{k+1}+\beta _{i}^{n+1}((\pi _{n+1}^{(j)})^{k})(\hat{V}_{i+1}(j))^{k+1}\big ) +b_{i}(j), \end{aligned}$$

where \(b_{i}(j)\) denotes the i-th component of vector \({\mathbf {b}}(j)\). Now let \({\mathcal {V}}_{\max }\equiv \displaystyle {\max _{i,j}}(\hat{V}_{i}(j))^{k+1}\), \({\mathcal {B}}_{\max }\equiv \displaystyle {\max _{i,j}}(b_i(j))\). Since all the coefficients, \(\alpha _{i}^{n+1},\, \beta _{i}^{n+1},\, q_j\), are positive, we derive that

$$\begin{aligned}&\left[ 1+\Delta \tau \big (\alpha _{i}^{n+1}((\pi _{n+1}^{(j)})^k) +\beta _{i}^{n+1}((\pi _{n+1}^{(j)})^k)+q_j\big )\right] (\hat{V}_{i}(j))^{k+1}\\\le & {} \Delta \tau \big (\alpha _{i}^{n+1}((\pi _{n+1}^{(j)})^k)+\beta _{i}^{n+1}((\pi _{n+1}^{(j)})^k)+ q_{j}\big ){\mathcal {V}}_{\max } + {\mathcal {B}}_{\max }. \end{aligned}$$

Let \({\underline{i}},\;{\underline{j}}\) be the indices such that \({\mathcal {V}}_{\max }=\displaystyle {\max _{{\underline{i}},{\underline{j}}}} (\hat{V}_{{\underline{i}}}({\underline{j}}))^{k+1}\). Then we derive that

$$\begin{aligned}&\left[ 1+\Delta \tau \big (\alpha _{{\underline{i}}}^{n+1}((\pi _{n+1}^{({\underline{j}})})^k) +\beta _{{\underline{i}}}^{n+1}((\pi _{n+1}^{({\underline{j}})})^k) +q_{{\underline{j}}}\big )\right] {\mathcal {V}}_{\max }\\\le & {} \Delta \tau \big (\alpha _{{\underline{i}}}^{n+1}((\pi _{n+1}^{({\underline{j}})})^k) +\beta _{{\underline{i}}}^{n+1}((\pi _{n+1}^{({\underline{j}})})^k)+ q_{{\underline{j}}}\big ){\mathcal {V}}_{\max } + {\mathcal {B}}_{\max }. \end{aligned}$$

This gives that \({\mathcal {V}}_{\max } \le {\mathcal {B}}_{\max }\). Therefore \(\hat{\mathbf {V}}^{k+1}\) is bounded from above. Consequently, the non-decreasing sequences \( \hat{\mathbf {V}}^{k+1}\) in Algorithm 1 are convergent.

Now we prove that the solution of Algorithm 1 is unique. To this end, let \(\hat{\mathbf {V}}_{1}\) and \(\hat{\mathbf {V}}_{2}\) be the two solutions of Algorithm 1, i.e.,

$$\begin{aligned} \left[ {\mathbf {I}}-\Delta \tau A^{(m+1)}\big (\pi _{n+1,1}^{(j)}\big )\right] \hat{\mathbf {V}}_{1}= & {} {\mathbf {V}}^{(m+1),n}+\Delta \tau {\mathbf {D}}^{(m),n+1}(j)+\varvec{\phi }^{n+1}-\varvec{\phi }^n, \end{aligned}$$
(29)
$$\begin{aligned} \left[ {\mathbf {I}}-\Delta \tau A^{(m+1)}\big (\pi _{n+1,2}^{(j)}\big )\right] \hat{\mathbf {V}}_{2}= & {} {\mathbf {V}}^{(m+1),n}+\Delta \tau {\mathbf {D}}^{(m),n+1}(j)+\varvec{\phi }^{n+1}-\varvec{\phi }^n, \end{aligned}$$
(30)

where

$$\begin{aligned} \pi _{n+1,1}^{(j)}\in \displaystyle {\hbox {arg}\sup _{\pi ^{(j)}\in \Pi _{\tau }}} \left[ A^{(m+1)}(\pi ^{(j)})\hat{\mathbf {V}}_{1}\right] ,\quad \pi _{n+1,2}^{(j)}\in \displaystyle {\hbox {arg}\sup _{\pi ^{(j)}\in \Pi _{\tau }}} \left[ A^{(m+1)}(\pi ^{(j)})\hat{\mathbf {V}}_{2}\right] . \end{aligned}$$

Subtracting Eq. (30) from Eq. (29) gives

$$\begin{aligned} \left[ {\mathbf {I}}-\Delta \tau A^{(m+1)}(\pi _{n+1,2}^{(j)})\right] \left[ \hat{\mathbf {V}}_{1}-\hat{\mathbf {V}}_{2}\right] = \Delta \tau \Big [A^{(m+1)}(\pi _{n+1,1}^{(j)})-A^{(m+1)}(\pi _{n+1,2}^{(j)})\Big ] \hat{\mathbf {V}}_{1}. \end{aligned}$$

Since \(A^{(m+1)}(\pi _{n+1,1}^{(j)})\hat{\mathbf {V}}_{1}=\displaystyle {\sup _{\pi ^{(j)}}}\left[ A^{(m+1)}(\pi ^{(j)})\hat{\mathbf {V}}_{1}\right] \), we have

$$\begin{aligned} \Big [A^{(m+1)}(\pi _{n+1,1}^{(j)}) -A^{(m+1)}(\pi _{n+1,2}^{(j)})\Big ]\hat{\mathbf {V}}_{1}\ge 0. \end{aligned}$$

Since \(\left[ {\mathbf {I}}-\Delta \tau A^{(m+1)}(\pi _{n+1}^{(j)}(2))\right] \) is an M-matrix, we have \(\hat{\mathbf {V}}_{1}\ge \hat{\mathbf {V}}_{2}\). In the same manner, we can prove \(\hat{\mathbf {V}}_{2}\ge \hat{\mathbf {V}}_{1}\). Therefore, we obtain that \(\hat{\mathbf {V}}_{1}= \hat{\mathbf {V}}_{2}\). \(\square \)

Now we are ready to present the convergence results for the whole approach namely iterative FDMs with iteration policy for the original HJB equation (2).

Theorem 2.3

(Convergence of iterative FDMs with iteration policy) If boundary function \(\phi \) in (11) is bounded, then the non-linear iteration solution \( (V_{i}^{(m),n+1}(j))^{k}\) converges to the unique solution of (34), i.e., \((V_{i}^{(m),n+1}(j))^{k}\) converges to \(V(\tau _{n+1},x_{i},j)\), as \(m\rightarrow +\infty \), \(k \rightarrow +\infty \), \(\Delta t\rightarrow 0\) and \(\Delta x\rightarrow 0\).

Proof

We write

$$\begin{aligned}&(V_{i}^{(m),n+1}(j))^{k}-V(\tau _{n+1},x_i,j)\nonumber \\&\quad =(V_{i}^{(m),n+1}(j))^{k}-V_{i}^{(m),n+1}(j)+V_{i}^{(m),n+1}(j)-V^{(m)}(\tau _{n+1},x_{i},j) \nonumber \\&\qquad +\,V^{(m)}(\tau _{n+1},x_{i},j)-V(\tau _{n+1},x_i,j)\nonumber \\&\quad =\hbox {I}_1+\hbox {I}_2+\hbox {I}_3, \end{aligned}$$
(31)

where

$$\begin{aligned} \hbox {I}_1= & {} (V_{i}^{(m),n+1}(j))^{k}-V_{i}^{(m),n+1}(j),\\ \hbox {I}_2= & {} V_{i}^{(m),n+1}(j)-V^{(m)}(\tau _{n+1},x_{i},j), \\ \hbox {I}_3= & {} V^{(m)}(\tau _{n+1},x_{i},j)-V(\tau _{n+1},x_i,j). \end{aligned}$$

We know from Theorem 2.2 that \((V_{i}^{(m),n+1}(j))^{k}\) converges to \(V_{i}^{(m),n+1}(j)\) as \(k\rightarrow +\infty \), i.e.,

$$\begin{aligned} \hbox {I}_1=(V_{i}^{(m),n+1}(j))^{k}-V_{i}^{(m),n+1}(j)\rightarrow 0. \end{aligned}$$

And from Theorem 2.1, we know that \(V_{i}^{(m),n+1}(j)\) converges to \(V^{(m)}(\tau _{n+1},x_{i},j)\), as \(\Delta t\rightarrow 0\) and \(\Delta x\rightarrow 0\), i.e.,

$$\begin{aligned} \hbox {I}_2=V_{i}^{(m),n+1}(j)-V^{(m)}(\tau _{n+1},x_{i},j)\rightarrow 0. \end{aligned}$$

Reference [16] state that as \(m\rightarrow +\infty \),

$$\begin{aligned} \hbox {I}_3=V^{(m)}(\tau _{n+1},x_{i},j)-V(\tau _{n+1},x_i,j)\rightarrow 0. \end{aligned}$$

Therefore, from (31), we obtain that \((V_{i}^{(m),n+1}(j))^{k}\) converges to \(V(\tau _{n+1},x_{i},j)\), as \(m \rightarrow +\infty \) and \(k \rightarrow +\infty \). \(\square \)

Remark 2.1

For nonuniform grids \(\left\{ {x_{0},\ldots ,x_{M}}\right\} \) with \(x_{0}=0\), \(x_{M}=x_{\max }\), the finite difference scheme is modified by replacing \(\alpha _{i}^{n+1}\) and \(\beta _{i}^{n+1}\) respectively by

$$\begin{aligned}&\pi _{n+1}^{(j)}\in \text {arg}\sup _{\pi ^{(j)}\in \Pi _{\tau }}\Big [\alpha _{i}^{n+1} (\pi ^{(j)})V_{i-1}^{(m+1),n+1}(j)+\beta _{i}^{n+1} (\pi ^{(j)})V_{i+1}^{(m+1),n+1}(j)\\&\quad +\,\big (-\alpha _{i}^{n+1}(\pi ^{(j)})-\beta _{i}^{n+1} (\pi ^{(j)})-q_j\big )V_{i}^{(m+1),n+1}(j)\Big ], \end{aligned}$$

and

$$\begin{aligned}&\alpha _{i}^{n+1}(\pi _{n+1}^{(j)})=\frac{\sigma _{j}^{2}\big (\pi _{n+1}^{(j)}\big )^{2} x_{i}^{2}}{(x_{i+1}-x_{i-1})(x_{i}-x_{i-1})}-\frac{(1-\xi )\left[ \pi _{n+1}^{(j)} (\mu _j-r_j)+r_j\right] x_{i}}{x_i-x_{i-1}},\\&\beta _{i}^{n+1}(\pi _{n+1}^{(j)})=\frac{\sigma _{j}^{2}\big (\pi _{n+1}^{(j)}\big )^{2} x_{i}^{2}}{(x_{i+1}-x_{i-1})(x_{i+1}-x_{i})}+\frac{\xi \left[ \pi _{n+1}^{(j)}(\mu _j-r_j) +r_j\right] x_{i}}{x_{i+1}-x_i}. \end{aligned}$$

Adapting \(\xi =0\;\text {or}\; 1\), it can ensure that \(\alpha _{i}^{n+1}(\pi _{n+1}^{(j)})\) and \(\beta _{i}^{n+1}(\pi _{n+1}^{(j)})\) are positive. The stability, consistency and monotonicity discussions and theorems can be adapted to the nonuniform grids.

3 Numerical Examples

In this section, we solve several examples using the iterative FDMs with policy iterations for the sequence of decoupled HJB equations and the coupled ones for power, non-HARA and Yaari utility functions. The iterative FDMs with policy iterations for the coupled HJB equations stem from [20], which solve the American option pricing under regime switching. But the scheme has to be modified for the utility maximization, since the policy for the utility maximization is different from that for American options. For convenience to the readers, we provide the scheme in the appendix. Moreover the boundary conditions to the HJB system are constructed.

Example 3.1

We consider two-state Markov chain process (MCP) with generating matrix

$$\begin{aligned} {\mathbf {Q}} = \left( \begin{array}{ccc} -1/3 &{}\quad 1/3 \\ 1/2 &{}\quad -1/2 \end{array} \right) . \end{aligned}$$

The riskless interest rates, return and volatility rates of risky asset are given by,

$$\begin{aligned} r=(0.05,0.01),\; \mu =(0.13,0.07),\; \sigma =(0.20,0.30). \end{aligned}$$

The power utility function is \(U(x)=\frac{x^{\frac{1}{2}}}{1/2}\). The initial wealth at time \(t=0\) is \(x=1\) and the investment period \(T=1\).

The boundary conditions (5) are constructed by

$$\begin{aligned} {\widetilde{V}}(t,x_{\max },j)= & {} E\Big [\exp {\big (\int _{0}^{T}<\varvec{r}, \varvec{\alpha }_{t}>dt\big )}\Big ]\cdot U(x_{\max }),\nonumber \\= & {} <\exp {[(Q-\text {diag}(\varvec{r}))(T-t)]\cdot \varvec{1}},\;\varvec{e_j}> \cdot U(x_{\max }),\quad t\in [0,T], \end{aligned}$$
(32)

where the second equality is calculated by [8]. The construction of the boundary condition is motivated by that we allocate all of the wealth measured by the utility to the risk-free bond over [t, T]. To verify the boundary condition (32) is correct, we compare it with the exact ones

$$\begin{aligned} {\widetilde{V}}(t,x_{\max },j)=a(t,j)\frac{x_{\max }^p}{p}, \quad t\in [0,T], \end{aligned}$$
(33)

where the expression of a(t, j) is given by [16].

In Table 1, FDM-D-HJB denotes the iterative FDMs with policy iterations for solving the decoupled HJB and FDM-C-HJB for the coupled ones, \(N,\; M\) are the number of time and space mesh. The benchmark value is calculated by explicit formula given by (see [16]). The benchmark values are 2.19913 and 2.08313 respectively for the current regime state being 1 and 2. The numerics in Table 1 show that the approach is convergent and FDM-D-HJB takes much less time than FDM-C-HJB.

Figures 1 and 2 test the convergence rates of FDM-C-HJB and FDM-D-HJB for space and time, respectively. The absolute value of the slope for the log-scale plots is just the convergence rate. Figures 1 and 2 respectively show that the convergence rate for space is about 2 and for time approximately 1.

Table 1 Numerical results for Example 3.1 (power utility) for two-state regime switching
Full size table
Fig. 1
figure 1

The log-scale plots of the errors for FDM-D-HJB or FDM-C-HJB with fixed number of time meshes \(N=10{,}240\)

Full size image
Fig. 2
figure 2

The log-scale plots of the errors for FDM-D-HJB or FDM-C-HJB with fixed number of space meshes \(M=256\)

Full size image

Example 3.2

For comparison, we use example in [23], which considers the non-HARA utility function

$$\begin{aligned} U(x)=\frac{1}{3}H(x)^{-3}+H(x)^{-1}+xH(x). \end{aligned}$$

for \(x>0\), where \(H(x)=\sqrt{2}(-1+\sqrt{1+4x})^{-1/2}\) and 3-state MCP with generating matrix

$$\begin{aligned} {\mathbf {Q}} = \left( \begin{array}{ccc} -1.5a &{}\quad a &{}\quad 0.5a\\ b &{}\quad -2b &{}\quad b\\ 0.5c &{}\quad c &{}\quad -1.5c \end{array} \right) \end{aligned}$$

where \(a,\,b,\,c\) are positive constants. The riskless interest rates, return and volatility rates of risky asset are given by \(r=(0.06,0.04,0.01)\), \(\mu =(0.20,0.12,0.07)\), \(\sigma =(0.25,0.20,0.30)\). The boundary condition is given by (32).

In Table 2, we observe that the value computed by the FDM-D-HJB and FDM-C-HJB is between the lower and upper bound, which shows that the computation is correct. Also we see that the FDM-D-HJB is more efficient than the FDM-C-HJB.

Table 2 Numerical results for Example 3.2 (non-HARA utility) with random choices of generator matrix Q for three-state regime switching
Full size table

Example 3.3

We consider the Yaari utility function: \(U(x)=\min (x,H)\) with \(H=2\). Since the second derivative of the Yaari utility function is 0, which leads to a degenerate equation, we shall use the smoothing technique which is similar to [32]. Define the approximate utility function

$$\begin{aligned} U_\varepsilon (x)=\min \left( \lim _{\varepsilon \rightarrow 0}\big (\frac{-4\varepsilon }{x_{\max }^2}x^2+\frac{4\varepsilon }{x_{\max }}x\big )+x, H\right) . \end{aligned}$$

Note that, when \(\varepsilon \rightarrow 0\), \(U_\varepsilon (x)\rightarrow U(x)\). Consider 2-state Markov Chain process with generating matrix,

$$\begin{aligned} {\mathbf {Q}} = \left( \begin{array}{ccc} -a &{}\quad a \\ b &{}\quad -b \end{array} \right) \end{aligned}$$

where a, b are positive constants. The riskless interest rates, return and volatility rates of risky asset are given by,

$$\begin{aligned} r=(0.05,0.01),\;\mu =(0.13,0.07),\;\sigma =(0.20,0.30). \end{aligned}$$

The initial wealth at time \(t=0\) is \(x=1\), the investment period \(T=1\), the boundary condition given by (32) and the smoothing parameter \(\varepsilon =10^{-6}\). Since the threshold H of the Yaari utility function is given by 2, it is reasonable to take \(x_{\max }=2\).

In Table 3, the number of time meshes is taken as \(N=4000\) and the number of space uniform meshes \(M=200\). When \(\tau \) is close to 0, the value function is close to the utility function whose first-order derivative is discontinuous and therefore more meshes points are needed to improve the accuracy. Based on this observation, the nonuniform meshes are designed as follows: The number of space uniform meshes is taken as \(M=200\) for \(\tau \in [0,T/2]\) and \(M=100\) for \(\tau \in [T/2,T]\). The numerics in Table 3 show that for the Yaari utility, the values fall between the lower and upper bounds and still the FDM-D-HJB takes less time than FDM-C-HJB. Moreover, the average computational time with nonuniform meshes is about \(52\%\) of that with uniform meshes for FDM-D-HJB and \(57\%\) for FDM-C-HJB under almost the same accuracy. This shows that the nonuniform meshes can be used to improve the accuracy of the algorithm.

Table 3 Numerical results for Example  3.3 (Yaari utility) with random choices of generator matrix Q for two-state regime switching
Full size table

4 Conclusions

In this paper, we extend the finite difference methods (FDMs) with policy iterations to the HJB system arising from regime switching utility maximization problems. The coupled HJB equations and the sequence of decoupled HJB equations derived by [16] are solved respectively by the standard and iterative FDMs with policy iterations. Numerical examples for power, non-HARA, and Yaari utilities are conducted to exhibit the accuracy and efficiency of the approach and show that solving the sequence of decoupled HJB equations is more efficient than the coupled one. The convergence of the approach is proved and some new techniques (e.g., introducing of the iteration-based viscosity solution) are used to overcome the difficulties caused by the errors from the iterative FDMs for solving of the sequences of HJB equations and the policy iterations. In the future it will be interesting to study the numerical methods for the high-dimensional HJB equations arising in the utility maximization based on multiple stochastic factors. To avoid the curse of the dimensionality, it worth to study the radial basis function methods and the kernel-based methods which are proposed by [22] and [24] for solving the linear partial differential equations arising in option pricing.