Quasi-Regression Monte-Carlo Scheme for Semi-Linear PDEs and BSDEs with Large Scale Parallelization on GPUs

Abstract

In this article we design a novel quasi-regression Monte Carlo algorithm in order to approximate the solution of discrete time backward stochastic differential equations, and we analyze the convergence of the proposed method. The algorithm also approximates the solution to the related semi-linear parabolic partial differential equation obtained through the well known Feynman–Kac representation. For the sake of enriching the algorithm with high order convergence a weighted approximation of the solution is computed and appropriate conditions on the parameters of the method are inferred. With the challenge of tackling problems in high dimensions we propose suitable projections of the solution and efficient parallelizations of the algorithm taking advantage of powerful many core processors such as graphics processing units.

Appendix A: Technical Estimates on Student’s t-Distribution

In this appendix, estimates on the CDF of the Student’s t-distribution, its inverse and the derivatives of the inverse of the CDF are computed. As we have been doing in the article, we use the same notation C for any generic constants, we will keep the same C from line to line, in order the alleviate the computations, although its value changes. In this appendix $\nu _l$ stands for Student’s t-distribution density (2.11) with parameter $\mu$, $F_{\nu _{l}}$ its marginal CDF and $F^{-1}_{\nu }$ its inverse.

Lemma 11

(Estimates on $F_{\nu _{l}}$ and $F^{-1}_{\nu _{l}}$) Let$x\in {\mathbb {R}}$, $u\in (0,1)$. The following estimates hold

$$\begin{aligned} \begin{aligned} F_{\nu _{l}}(x)&\sim _{x\rightarrow -\infty } \dfrac{c_\mu }{\mu |x|^{\mu } }, \\ 1-F_{\nu _{l}}(x)&\sim _{x\rightarrow +\infty } \dfrac{c_\mu }{\mu |x|^{\mu }} , \\ F^{-1}_{\nu _{l}}(u)&\sim _{u\rightarrow 0^+} -{\tilde{c}}_\mu u^{-1/\mu } , \\ F^{-1}_{\nu _{l}}(u)&\sim _{u\rightarrow 1^-} {\tilde{c}}_\mu (1-u)^{-1/\mu }, \end{aligned} \end{aligned}$$

(A.1)

with${\tilde{c}}_\mu = \left( \dfrac{c_\mu }{\mu }\right) ^{1/\mu }.$

Proof

In view of (2.11) we know that $\nu _{l}(x_l)\sim _{x_l\rightarrow \pm \infty }\dfrac{c_\mu }{|x_l|^{\mu +1}}$. Besides,

$$\begin{aligned} F_{\nu _{l}}(x) = \int _{-\infty }^x \nu _l(x_l)\mathrm{d}x_l \sim _{x\rightarrow -\infty } \int _{-\infty }^x \dfrac{c_\mu }{|x_l|^{\mu +1}} \mathrm{d}x_l = \dfrac{c_\mu }{\mu |x|^{\mu } }. \end{aligned}$$

The distribution $\nu _l$ being symmetric, we have $F_{\nu _{l}}(x)+F_{\nu _{l}}(-x)=1$ and $F^{-1}_{\nu _{l}}(u)=-F^{-1}_{\nu _{l}}(1-u)$. Therefore, the estimate $1- F_{\nu _{l}}(x)$ for $x\rightarrow +\infty$ follows from the case $x\rightarrow -\infty$.

In order to compute $F^{-1}_{\nu _{l}}$ we use that $F_{\nu _{l}}(F^{-1}_{\nu _{l}}(u))=u$:

If $u\rightarrow 0^+$, $F^{-1}_{\nu _{l}}(u)\rightarrow -\infty$, $u = F_{\nu _{l}}(F^{-1}_{\nu _{l}}(u)) \sim _{u\rightarrow 0^+} \dfrac{c_\mu }{\mu |F^{-1}_{\nu _{l}}(u)|^\mu }$. Therefore,

$$\begin{aligned}&|F^{-1}_{\nu _{l}}(u)|^\mu \sim _{u\rightarrow 0^+} \dfrac{c_\mu }{\mu u},\\&|F^{-1}_{\nu _{l}}(u)| \underset{u\rightarrow 0^+}{=} -F^{-1}_{\nu _{l}}(u) \sim _{u\rightarrow 0^+}\left( \dfrac{c_\mu }{\mu u}\right) ^{1/\mu }, \end{aligned}$$

which is the advertised result. The case $u\rightarrow 1^-$ follows from the case for $u\rightarrow 0^+$ using again the symmetry of the distribution $\nu _l$. $\square$

Lemma 12

(Estimates on derivatives of $\frac{1}{(1+|\mathbf {x}|^2)^{q/2}}$) Let$\mathbf {x}\in {\mathbb {R}}^d$, $q\in {\mathbb {R}}^+$and$n\in {\mathbb {N}}$. The following upper bound holds

$$\begin{aligned} \left| \partial ^n_{x_l} \dfrac{1}{(1+|\mathbf {x}|^2)^{q/2}}\right| \le C( 1+|\mathbf {x}|^2)^{-q/2-n/2}. \end{aligned}$$

(A.2)

Proof

We only have to consider the case $n\ge 1$. In order to compute $\partial ^n_{x_l} \dfrac{1}{(1+|\mathbf {x}|^2)^{q/2}}$ we use the following Faà di Bruno’s formula (as long as the partial derivative is only with respect to one variable we can extend in the following way the one-dimensional Faà di Bruno’s formula (2.34)):

$$\begin{aligned} \begin{aligned} \partial _{x_l}^n f(g(\mathbf {x}))&= \sum _{m=1}^n \frac{1}{m!}\mathrm{{d}}_{u}^m f(u)\big |_{u=g(\mathbf {x})} \sum _{\mathbf {j}\in J_{m,n}} \dfrac{n!}{j_1!j_2!\cdots j_m!} \prod _{i=1}^m \partial _{x_l}^{j_i} g(\mathbf {x}),\\ J_{m,n}&=\{\mathbf {j}=(j_1,\ldots ,j_m)\in {\mathbb {N}}^m_+:j_1+\ldots +j_m = n\}. \end{aligned} \end{aligned}$$

(A.3)

Here, consider $f(u)=u^{-q/2}$ and $g(\mathbf {x})=1+|\mathbf {x}|^2$: $\mathrm{d}^m_u f(u) = C_{m,q} u^{-q/2-m}$ and $\partial _{x_l}^{j} g(\mathbf {x})=2 x_l{\mathbf {1}}_{j=1}+2{\mathbf {1}}_{j=2}$. Observe that in the Faà di Bruno’s formula (A.3), the sum over $\mathbf {j}\in J_{m,n}$ will be made only for $j_i=1$ with $m_1:=\#\{j_i:\mathbf {j}\in J_{m,n}, j_i=1\}$ terms, and for $j_i=2$ with $m_2:=\#\{j_i:\mathbf {j}\in J_{m,n}, j_i=2\}$ terms. Owing to this property and by definition of $J_{m,n}$, we have $m_1+2m_2=n$ and $m_1+m_2=m$, which results in $m_1=2m-n$ and $m_2=n-m$. Invoking the form of the derivatives $\partial _{x_l}^{j_i} g(\mathbf {x})$, it readily follows that

$$\begin{aligned} \prod _{i=1}^m \partial _{x_l}^{j_i} g(\mathbf {x})=2^m x^{m_1}_l. \end{aligned}$$

All in all, we deduce

$$\begin{aligned}&\left| \partial ^n_{x_l} \dfrac{1}{(1+|\mathbf {x}|^2)^{q/2}}\right| \le C \sum _{m= \lceil {n/2\rceil }}^n ( 1+|\mathbf {x}|^2)^{-q/2-m} (1+|x_l|)^{2m-n}\\&\quad \le C \sum _{m= \lceil {n/2}\rceil }^n ( 1+|\mathbf {x}|^2)^{-q/2-m} (1+|\mathbf {x}|^2)^{m-n/2}\\&\quad \le C( 1+|\mathbf {x}|^2)^{-q/2-n/2}. \end{aligned}$$

$\square$

Lemma 13

(Estimates on derivatives of $\frac{1}{(1+|\mathbf {x}|^2)^{q/2}}$) Let$\mathbf {x}\in {\mathbb {R}}^d$, $q\in {\mathbb {R}}^+$, $\mathbf {n} = (n_1,\ldots ,n_d) \in {\mathbb {N}}_+^d$. The following upper bound holds

$$\begin{aligned} \left| {\partial ^{\mathbf {n}}_x} \dfrac{1}{(1+|\mathbf {x}|^2)^{q/2}}\right| \le C( 1+|\mathbf {x}|^2)^{-q/2-\overline{\mathbf {n}}/2}. \end{aligned}$$

(A.4)

Proof

We only have to consider the case $\overline{\mathbf {n}}\ge 1$. For such $\mathbf {n}$, we use the following multivariate Faà di Bruno’s formula derived from [15, Corollary 2.10]

$$\begin{aligned} \begin{aligned} {\partial ^{\mathbf {n}}_x} f(g(\mathbf {x}))&= \sum _{m=1}^{\overline{\mathbf {n}}} \mathrm{{d}}_{u}^m f(u)\big |_{u=g(\mathbf {x})} \sum _{J\in \mathbf {J}_{m,\mathbf {n}}} C_{m,J} \prod _{i=1}^m {\partial ^{\mathbf {j}_i}_{x}} g(\mathbf {x}),\\ \mathbf {J}_{m,\mathbf {n}}&=\left\{ J= \begin{pmatrix} \mathbf {j}_1 \\ \vdots \\ \mathbf {j}_m \end{pmatrix} \in {\mathbb {N}}^m \times {\mathbb {N}}^d : \mathbf {j}_1+\ldots +\mathbf {j}_m = \mathbf {n}, \mathbf {j}_1,\ldots ,\mathbf {j}_m \ne \mathbf {0}\right\} , \end{aligned} \end{aligned}$$

(A.5)

where the positive constants $C_{m,J}$ depend on m and the matrix J under consideration.

Note that non coordinate-wise derivatives of $g(\mathbf {x})=1+|\mathbf {x}|^2$ are all zero. Besides, coordinate-wise derivatives of g are $\partial _{x_l}^{j} g(\mathbf {x})=2 x_l{\mathbf {1}}_{j=1}+2{\mathbf {1}}_{j=2}$. For a given J, let $a_{l} \in {\mathbb {N}}\, \forall l=1,\ldots ,d$ denote the number of vectors in J with 1 in the l-th coordinate and zero anywhere else. It holds that

$$\begin{aligned} &\prod _{i=1}^m \partial ^{\mathbf {j}_i}_x g(\mathbf {x})= {\mathbf {1}}_{2m\ge \overline{\mathbf {n}}}C x_1^{a_{1}}\cdots x_d^{a_{d}} \text{ with } a_{1}+\ldots +a_{d} = 2m-\overline{\mathbf {n}},\\ &{\left| \prod _{i=1}^m \partial ^{\mathbf {j}_i}_x g(\mathbf {x}) \right| } \le C (1+|\mathbf {x}|^2)^{a_{1}/2}\cdots (1+|\mathbf {x}|^2)^{a_{d}/2} {\mathbf {1}}_{\sum _{l=1}^d a_{l} = 2m-\overline{\mathbf {n}}}\\ &\qquad\qquad\quad\le C (1+|\mathbf {x}|^2)^{m-\overline{\mathbf {n}}/2} \end{aligned}$$

for $2m\ge \overline{\mathbf {n}}$. All in all, we deduce

$$\begin{aligned}&\left| {\partial ^{\mathbf {n}}_x} \dfrac{1}{(1+|\mathbf {x}|^2)^{q/2}}\right| \\&\quad \le C \sum _{m= \lceil {\overline{\mathbf {n}}/2}\rceil }^{\overline{\mathbf {n}}} ( 1+|\mathbf {x}|^2)^{-q/2-m} (1+|\mathbf {x}|^2)^{m-\overline{\mathbf {n}}/2}\\&\quad \le C( 1+|\mathbf {x}|^2)^{-q/2-\overline{\mathbf {n}}/2}. \end{aligned}$$

$\square$

Lemma 14

(Estimates on derivatives of Student’s t-distribution) Let$n\in {\mathbb {N}}$. The following upper bound holds

$$\begin{aligned} |\mathrm{d}_x^{n}\nu _l(x)| \le C(1+x^2)^{-\frac{\mu +1}{2}-\frac{n}{2}}. \end{aligned}$$

(A.6)

Proof

This result follows readily using the previous estimate (A.2). $\square$

Lemma 15

(Estimates on derivatives of $F^{-1}_{\nu _{l}}$) Let$n\in {\mathbb {N}}$. The following upper bounds hold

$$\begin{aligned} \left| \mathrm{d}^{n}_{u}F^{-1}_{\nu _{l}}(u) \right|&\le _{u\rightarrow 0^+} C u^{-\frac{\mu n +1}{\mu }}, \\ \left| \mathrm{d}^{n}_{u}F^{-1}_{\nu _{l}}(u) \right|&\le _{u\rightarrow 1^-} C (1-u)^{-\frac{\mu n +1}{\mu }}, \end{aligned}$$

(A.7)

Cis a constant depending on$\mu$andn.

Proof

Using the combinatorial formula for higher derivatives of inverses [40] and the fact that for $b\in {\mathbb {N}}$, $b\ge 1$, $\mathrm{d}^b_{x}F_{\nu _{l}}(x) = \mathrm{d}_x^{b-1}\nu _l(x)$,

$$\begin{aligned}&\mathrm{d}^{n+1}_{u}F^{-1}_{\nu _{l}}(u) = \sum _{k=0}^n \dfrac{(-1)^k}{k!} [\nu _l(x)|_{x=F^{-1}_{\nu _{l}}(u)}]^{-n-k-1} \\&\quad \times \sum _{\begin{array}{c} b_1+\cdots +b_{k} = n+k \\ b_i\ge 2 \end{array}} \dfrac{(n+k)!}{b_1!b_2!\cdots b_k!} \prod _{i=1}^{k} \mathrm{d}_x^{b_i-1}\nu _l(x)|_{x=F^{-1}_{\nu _{l}}(u)}. \end{aligned}$$

Therefore, using (A.6),

$$\begin{aligned}&|\mathrm{d}^{n+1}_{u}F^{-1}_{\nu _{l}}(u)| \le \sum _{k=0}^n C \left[ \left( 1+\left( F^{-1}_{\nu _{l}}(u)\right) ^2\right) ^{-\frac{\mu +1}{2}} \right] ^{-n-k-1} \\&\qquad \times \sum _{\begin{array}{c} b_1+\cdots +b_{k} = n+k \\ b_i\ge 2 \end{array}} \quad \prod _{i=1}^{k} \left( 1+\left( F^{-1}_{\nu _{l}}(u)\right) ^2\right) ^{-\frac{\mu +1}{2}-\frac{b_i-1}{2}} \\&\quad \le \sum _{k=0}^n C \left( 1+\left( F^{-1}_{\nu _{l}}(u)\right) ^2\right) ^{\frac{\mu +1}{2}(n+k+1)-\frac{k(\mu +1)+n}{2}}\\&\quad \le C \left( 1+\left( F^{-1}_{\nu _{l}}(u)\right) ^2\right) ^{\frac{\mu (n+1)+1}{2}}. \end{aligned}$$

Finally, combining the previous upper bound with (A.1) yields

$$\begin{aligned} \left| \mathrm{d}^{n}_{u}F^{-1}_{\nu _{l}}(u) \right|&\le _{u\rightarrow 0^+} C \left| F^{-1}_{\nu _{l}}(u) \right| ^{\mu n+1} \le _{u\rightarrow 0^+} C u^{-\frac{\mu n +1}{\mu }}, \\ \left| \mathrm{d}^{n}_{u}F^{-1}_{\nu _{l}}(u) \right|&\le _{u\rightarrow 1^-} C (1-u)^{-\frac{\mu n +1}{\mu }}, \end{aligned}$$

where the constant C may change value along computations. $\square$