Approximation Schemes for Mixed Optimal Stopping and Control Problems with Nonlinear Expectations and Jumps

Nonlinear expectations as in (1.2), and hence HJBVIs as in (1.3), arise naturally in financial mathematics, for instance as models for American options in a market with constrained portfolios [23], from recursive utility optimization problems [9], from dynamic risk measures [30], and from robust pricing and risk measures under probability model uncertainty [31]. In Sect. 5, we describe in more detail the situation of an ambiguity-averse investor who chooses the optimal wealth allocation and liquidation time of an asset whose price process has infinite activity jumps. This problem can be modelled naturally within the setup described above, and therefore leads directly to an HJBVI of the type (1.3) with all its key features present (i.e., singular non-local term, nonlinear driver, control optimisation, obstacle term). As it is usually difficult to obtain analytic solutions of HJBVIs, it is necessary to design efficient and robust numerical methods for solving these fully nonlinear PIDEs, of which we give a worked-out example in Sect. 5.

We also remark that, to the best of our knowledge, even for the case with linear expectations (i.e. f in the special form from above), there is no published numerical scheme covering the generality of (1.3). However, there is a vast literature on monotone approximations for local HJB equations (see, e.g., [2, 11, 13] and references therein) and a number of works covering specific extensions. For instance, monotone finite-difference quadrature schemes are proposed in References [5, 6, 7, 10] for nonlocal HJB equations. We refer the reader also to Reference [14] for penalty approximations to nonlocal variational inequalities and to Reference [33] for an application of policy iteration together with penalization to solve HJB obstacle problems. Probabilistic methods for solving HJB equations (without jumps and optimal stopping) can be found, for example, in Reference [26].

All the aforementioned PDE methods solve (1.3) (with \(f(\alpha ,\mathbf{x},y,z,k)\equiv f(\alpha ,x)-ry\)) by the standard “discretize, then optimize” approach, where one discretizes the operators in (1.3), and solves the resulting nonlinear discretized equations using policy iteration, or more generally semi-smooth Newton methods [21, 28].

However, this standard approach cannot be easily extended to nonlinear f which is only assumed to be Lipschitz (and generally is not semi-smooth [28]), which prevents a direct application of Newton-like solvers (see Reference [34] for a special case of the discrete optimization problem with \(f(\alpha ,\mathbf{x},y,z,k)\equiv f(\alpha ,x,y)\) differentiable and concave in y, and \(\mathbf{A }\) finite).

Moreover, at each step of policy iteration, one needs to identify the global optimal policy for each computational node. The nonlinear driver f and other PDE coefficients may have sufficiently complicate nonlinearities in the control variable such that the only way to construct a convergent algorithm is to discretize the admissible control set, and perform exhaustive search to determine the optimal policy at each node.

Another approach to solve (1.3) uses piecewise constant policy time stepping (PCPT) as in Reference [29]. It is based explicitly on a discrete approximation of the admissible set by a finite set, say with J elements, and then defines a piecewise decoupled system of PDEs corresponding to these J (constant) controls. The information from the different solutions is assembled at the end of each timestep by taking the pointwise maximum.

As a specific scheme for these semi-linear PDEs, we propose an implicit Euler time discretization, monotone (semi-Lagrangian) approximations for local diffusions, a monotone quadrature-based scheme for the nonlocal terms, and the Lax-Friedrichs scheme for the nonlinearity in the gradient. The different solutions may be defined on different discretization grids by possibly high order monotonicity preserving interpolations. This approach not only avoids policy iteration, but also allows for an easier construction of convergent monotone schemes and an efficient parallel implementation of the individual semi-linear PDEs. Note that it is essential to obtain a monotone discretization, since it is well-known that non-monotone schemes may fail to converge or even converge to false “solutions” [13]. By Godunov’s Theorem [20], in general, one can expect a monotone scheme to be at most first-order accurate.

In this section, we formulate the mixed optimal stopping and control problem with nonlinear expectation and introduce the connection between such problems and HJBVIs, which is crucial for the subsequent developments. We start with some useful notation that is needed frequently in the rest of this work.

For any given \(t \in [0,T]\), we define the t-translated Brownian motion \(W^t:=(W_s-W_t)_{s \ge t}\) and the t-translated Poisson random measure \(N^t:=N(]t,s],\cdot )_{s \ge t}\). We denote by \({\tilde{N}}^t(dt,de)=N^t(dt,de)-\nu (de)dt\) the compensated process of \(N^t\), and by \(\mathbb {F}^t=\{\mathcal {F}^t_s\}_{s\in [t,T]}\) be the filtration generated by \(W^t\) and \(N^t\) augmented by the P-null sets.

Furthermore, we introduce several spaces: \(L^2_\nu \) is the space of Borel functions \(l:E\rightarrow {{\mathbb {R}}}\) with \(\Vert l\Vert _\nu ^2:=\int _E|l(e)|^2\,\nu (de)<\infty \); \(\mathbb {H}^2_t\) (resp. \(\mathbb {H}^2_{t,\nu }\)) is the space of \({{\mathbb {R}}}^{d}\)-valued (resp. real-valued) \(\mathbb {F}^t\)-predictable processes \((\pi _s)\) (resp. \((l_s(\cdot )\)) with \(\mathbb {E}\int _t^T |\pi _s|^2\,ds<\infty \) (resp. \(\mathbb {E}\int _t^T \Vert l_s\Vert _{L^2_\nu }^2\,ds<\infty \)); \(\mathcal {S}^2_t\) is the space of real-valued \(\mathbb {F}^t\)-adapted càdlàg processes \((\psi _s)\) with \(\mathbb {E}[\sup _{t \le s \le T} \psi _s^2]<\infty \).

For notational convenience, in the sequel, we will write \({\hat{f}}\) as f, and denote by \(\psi ^\alpha \) a generic function \(\psi \) with control-dependence.

We emphasize that since the matrix \(\sigma ^\alpha (\sigma ^\alpha )^T\) is only assumed to be nonnegative definite, both the diffusion coefficient \(\sigma ^\alpha (\sigma ^\alpha )^T\) and the jump intensity \(\eta \) of (2.5) are allowed to vanish at some points. Consequently, there is no Laplacian smoothing from the second-order differential operator nor fractional Laplacian smoothing from the nonlocal operator to this degenerate equation (2.5). Therefore, in general, this HJBVI will not admit classical solutions, and we shall interpret the equation in the following viscosity sense based on semi-continuous envelopes of the equation [1, 29].

Under Assumption 2.1, the HJBVI (2.5) is well-posed in the class of bounded continuous functions (see References [16, 17]). The unique viscosity solution of (2.5) (after a change of time variable) can be further characterized as the optimal value function (2.4) of the mixed control problem. In other words, to obtain the optimal value function for all initial times t and initial states x, it is equivalent to design effective numerical schemes to solve (2.5).

Moreover, a strong comparison principle holds for the HJBVI (2.5), the proof of which is similar to that in Reference [16] (without controls) and hence omitted. In particular, if U is a bounded viscosity subsolution and V is a bounded viscosity supersolution to (2.5) with \(U(0,\cdot )\le V(0,\cdot )\), we have \(U(\mathbf{x})\le V(\mathbf{x})\) for all \(\mathbf{x}\in {\bar{\mathcal {Q}}}_T\).

For notational simplicity, we label our approximations only by h and assume in the sequel that \(\Delta t\) is a given function of h with \(\Delta t\rightarrow 0\) as \(h\rightarrow 0\).

We then turn to discretizing the local terms. We introduce the modified drift by \({\tilde{b}}^\alpha (x)=b^\alpha (x)+b^\alpha _{\varepsilon }(x)\), and write the modified diffusion \({\tilde{\sigma }}^\alpha =({\tilde{\sigma }}^\alpha _1,\ldots , {\tilde{\sigma }}^\alpha _d)\), where \({\tilde{\sigma }}_l^\alpha \), \(l=1,\ldots , d\) is the l-th column of \({\tilde{\sigma }}^\alpha \) defined in (3.1).

We end this section with a remark about the implementation of the implicit scheme (3.18). To avoid solving linear systems with the dense matrices resulting from the discretization of the nonlocal operators, we write the solution to (3.18) as the fixed point of a sparse contraction mapping T, such that sufficient accuracy is achieved in practice by a few fixed point iterations.

Moreover, since in practice we can evaluate the discrete nonlocal operators \(K^{\alpha ,1}_{\varepsilon ,h}U^{n}_{j}\) and \(B^{\alpha }_{\varepsilon ,h}U^{n}_{j}\) at all grid points in \(O(N\log N)\) operations using a FFT (see e.g. [14]), and in each iteration a sparse linear system is solved (in one dimension it is tridiagonal), the total complexity of the implicit scheme is still close to linear.

It is also worth pointing out that even if we chose explicit approximations for the nonlocal operators, due to the nonlinearity of f, one still has to perform iterations to solve the resulting nonlinear equations. Because we only assume Lipschitz continuity but no higher regularity of f, we adopt the Picard iteration to solve for \(U^{n}\).

In this section, we establish the convergence of the numerical approximations in Sect. 3 to the viscosity solution of the HJBVI (2.5). To simplify the notation, we will occasionally drop the terms \(\{K^\alpha u\}_{\alpha \in \mathbf{A }}\) and \(\{B^\alpha u\}_{\alpha \in \mathbf{A }}\) in (3.3), (3.5) and (3.6), and simply denote them by \(F(\mathbf{x},u,Du,D^2u)=0\) in the sequel.

In this section, we present several numerical experiments to analyse the effectiveness of the numerical scheme proposed in Sect. 3. We shall investigate the convergence of numerical solutions with respect to the switching cost, timestep, and mesh size, and show that a relatively coarse discretization of the admissible control set already leads to an accurate approximation.

The performance of this investment is evaluated under a particular nonlinear expectation, called the recursive utility process (see e.g. [9]), which is associated with a BSDE (with Lipschitz continuous drivers). It generalizes the standard additive utilities by including a dependence on the future utility (corresponding to the future wealth). Roughly speaking, the recursive utility depends on the future utility through the dependance of the driver f on y, and can also depend on the “variability” or “volatility” of future utility through the dependance of f on z and k.

Now we are ready to discuss the selection of the discretization parameters in detail. The density of the tempered stable measure \(\nu \) enables us to improve the estimates in Lemma 4.10 to \(\sum _{m\not =0}\kappa ^{\alpha ,n}_{h,m,i}\le \log (\varepsilon )\), and hence choosing \(\varepsilon =h\) and \(\Delta t=O(h)\) leads us to a consistent approximation to the switching system (3.6). Moreover, choosing \(\theta =\frac{1}{40}\) and \(\Delta t=\frac{h}{15}\) ensures the numerical flux is stable and the contraction constant of T in (3.19) is less than \(\frac{1}{10}\).

We then proceed to analyze the effect of the control discretization. We pick the same parameters as those in Table 1, except that \(b=0.25\), which is chosen such that it is now possible that the optimal control is attained in the interior of (0, 1) (as seen from a similar argument as earlier). Computations are performed using Matlab R2016b on a 3.30GHz Intel Xeon ES-2667 16-Core processor with 256 GB RAM to enable parallelization. Table 3 illustrates the numerical results for different control meshes (\(J=1/\delta +1\)) with a fixed mesh size \(h=0.005\) and switching cost \(c=1/2560\), and also compares the runtime with or without parallelization.

We can clearly observe from line (a) second order convergence of the numerical solutions, and a relatively coarse control mesh has already yielded an accurate approximation with a negligible control discretization error.

Next, we discuss lines (b)–(f) which analyse the algorithm’s parallel efficiency. Hereby, the implicit finite difference scheme for individual components of the switching system (i.e., (3.9), for different j) is solved independently on different processors, while the maximisation step (3.8) requires communication between processors.

The total execution time with and without parallelization are presented in line (b) and (c), respectively, which indicate a significant reduction of computational times. Moreover, by subtracting the communication time among clusters, as shown in line (d), from the total runtime, we can obtain the actual time spent on executing the numerical scheme (line (e)). The speed-up rate of the parallelization is shown in line (f), which grows with the number of controls, and remains stable at the number of cores. Therefore, together with parallelization, piecewise constant timestepping enables us to achieve a high accuracy in the control discretization without significantly increasing the computational time, which is an advantage over policy iterations, which do not parallelise naturally.

The numerical value function and the corresponded feedback control strategy with \(J=21\) are presented in Figure 1, in which the white area represents the region where the obstacle is active, and otherwise the colour indicates the value of the optimal control, as shown in the panel on the right. The approximation to the optimal control pair \((\tau ,\alpha )\) was found from the numerical solution as follows (see also (3.8) and Remark 3), noting that in our tests \(x_{j,i}=x_{k,i}\) for all j, k, and therefore no interpolation is needed:

$$\begin{aligned} i_n^*\in & {} \mathrm{argmax}_{k}{U}^{n}_{k,i}, \\ \theta _i^n= & {} \left\{ \begin{array}{rl} 0 &{} \max _{k}{U}^{n}_{k,i} > \xi (t_n,x_{1,i}), \\ 1 &{} \max _{k}{U}^{n}_{k,i} \le \xi (t_n,x_{1,i}), \end{array} \right. \end{aligned}$$

where \(\alpha _i^n = \alpha _{i_n^*}\) is an approximation to the optimal policy and \(\{(t_n,x_{1,i}): \theta _i^n=1\}\) is an approximation to the stopping region.

Open image in new window

This paper provides a PDE approximation scheme for the value function of a mixed stochastic control/optimal stopping problem with nonlinear expectations and infinite activity jumps, which is the unique viscosity solution of a nonlocal HJB variational inequality. The approach that we have adopted is based on piecewise constant policy time stepping (PCPT), which reduces the problem to a system of semi-linear PDEs, and a monotone approximation scheme. We prove the convergence of the numerical scheme and illustrate the theoretical results with some numerical examples in the case of a recursive utility maximisation problem.

To the best of our knowledge, this is the first paper which proposes a numerical approximation for a control problem in such a generality. Natural next steps would be to derive theoretical results on the convergence rate and to extend this approach to the case of Hamilton–Jacobi–Bellman–Isaac equations obtained in Reference [8].