Multi-index Monte Carlo: when sparsity meets sampling

Abstract

We propose and analyze a novel multi-index Monte Carlo (MIMC) method for weak approximation of stochastic models that are described in terms of differential equations either driven by random measures or with random coefficients. The MIMC method is both a stochastic version of the combination technique introduced by Zenger, Griebel and collaborators and an extension of the multilevel Monte Carlo (MLMC) method first described by Heinrich and Giles. Inspired by Giles’s seminal work, we use in MIMC high-order mixed differences instead of using first-order differences as in MLMC to reduce the variance of the hierarchical differences dramatically. This in turn yields new and improved complexity results, which are natural generalizations of Giles’s MLMC analysis and which increase the domain of the problem parameters for which we achieve the optimal convergence, ${\mathcal {O}}(\mathrm {TOL}^{-2}).$ Moreover, in MIMC, the rate of increase of required memory with respect to $\mathrm {TOL}$ is independent of the number of directions up to a logarithmic term which allows far more accurate solutions to be calculated for higher dimensions than what is possible when using MLMC. We motivate the setting of MIMC by first focusing on a simple full tensor index set. We then propose a systematic construction of optimal sets of indices for MIMC based on properly defined profits that in turn depend on the average cost per sample and the corresponding weak error and variance. Under standard assumptions on the convergence rates of the weak error, variance and work per sample, the optimal index set turns out to be the total degree type. In some cases, using optimal index sets, MIMC achieves a better rate for the computational complexity than the corresponding rate when using full tensor index sets. We also show the asymptotic normality of the statistical error in the resulting MIMC estimator and justify in this way our error estimate, which allows both the required accuracy and the confidence level in our computational results to be prescribed. Finally, we include numerical experiments involving a partial differential equation posed in three spatial dimensions and with random coefficients to substantiate the analysis and illustrate the corresponding computational savings of MIMC.

Appendices

Appendix A: Asymptotic normality of the MIMC estimator

Lemma 5.1

(Asymptotic Normality of the MIMC Estimator). Consider the MIMC estimator introduced in (2), ${\mathcal {A}}$, based on a set of multi indices, ${\mathcal {I}}(\mathrm {TOL})$, and given by

$$\begin{aligned} {\mathcal {A}} = \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}} \sum _{m=1}^{M_{{{{\varvec{\alpha }}}}}} \frac{{ \Delta \fancyscript{S} }_{{{{{\varvec{\alpha }}}}}}(\omega _{{{{{\varvec{\alpha }}}}}, m})}{M_{{{{\varvec{\alpha }}}}}}. \end{aligned}$$

Assume that for $1\le i\le d$ there exists $0<L_i(\mathrm {TOL})$ such that

$$\begin{aligned} {\mathcal {I}}(\mathrm {TOL}) \subset \{{{{{\varvec{\alpha }}}}}\in \mathbb {N}^d : \alpha _i \le L_i(\mathrm {TOL}),\quad \text { for } 1\le i\le d\}. \end{aligned}$$

(54)

Denote ${Y_{{{{\varvec{\alpha }}}}}= |{ \Delta \fancyscript{S} }_{{{{\varvec{\alpha }}}}}- {\mathrm {E}\left[ { \Delta \fancyscript{S} }_{{{{\varvec{\alpha }}}}}\right] }|}$ and assume that the following inequalities

$$\begin{aligned} Q_S \prod _{i=1}^d \exp (-\alpha _i s_i)&\le {\mathrm {E}\left[ Y_{{{{{\varvec{\alpha }}}}}}^2\right] },&\end{aligned}$$

(55a)

$$\begin{aligned} {\mathrm {E}\left[ Y_{{{{{\varvec{\alpha }}}}}}^{2+\rho }\right] }&\le Q_R \prod _{i=1}^d \exp (-\alpha _i r_i),&\end{aligned}$$

(55b)

hold for strictly positive constants $\rho , \{s_i, r_i\}_{i=1}^d, Q_S$ and $Q_R$. Choose the number of samples on each level, $M_{{{{\varvec{\alpha }}}}}(\mathrm {TOL})$, to satisfy, for strictly positive sequences $\{\tilde{s}_i\}_{i=1}^d$ and $\{H_{{{\varvec{\tau }}}}\}_{{{{\varvec{\tau }}}}\in {\mathcal {I}}(\mathrm {TOL})}$ and for all ${{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})$,

$$\begin{aligned} M_{{{{\varvec{\alpha }}}}}\ge \mathrm {TOL}^{-2} \,C_M \left( \prod _{i=1}^d \exp (-\alpha _i\tilde{s}_i) \right) H_{{{{\varvec{\alpha }}}}}^{-1} \left( \sum _{{{{\varvec{\tau }}}}\in {\mathcal {I}}(\mathrm {TOL})} H_{{{\varvec{\tau }}}}\right) . \end{aligned}$$

(56)

Denote, for all $1 \le i \le d,$

$$\begin{aligned} p_i = (\rho /2) \tilde{s}_i - r_i + (1+\rho /2) s_i \end{aligned}$$

(57)

and choose $0 < c_i$ such that whenever $0 < p_i$, the inequality $c_i < \rho /p_i$ holds. Finally, if we take the quantities $L_i(\mathrm {TOL})$ in (54) to be

$$\begin{aligned} L_i(\mathrm {TOL}) = c_i \log (\mathrm {TOL}^{-1}) + { o( \log (\mathrm {TOL}^{-1}) )} ,\quad \text { for all }\,\,1 \le i \le d, \end{aligned}$$

then we have

$$\begin{aligned} \lim _{\mathrm {TOL}\downarrow 0} {\mathrm {P}\left[ \frac{{\mathcal {A}}- {\mathrm {E}\left[ {\mathcal {A}}\right] }}{\sqrt{{\mathrm {Var}\left[ {\mathcal {A}}\right] }}} \le z\right] } = \Phi \left( z \right) , \end{aligned}$$

where $\Phi (z)$ is the normal cumulative distribution function of a standard normal random variable.

Proof

We prove this theorem by ensuring that the Lindeberg condition [11, Lindeberg–Feller Theorem, p. 114] (also restated in [10, Theorem A.1]) is satisfied. The condition becomes in this case

$$\begin{aligned} \lim _{\mathrm {TOL}\downarrow 0} \underbrace{\frac{1}{{\mathrm {Var}\left[ {\mathcal {A}}\right] }} \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} \sum _{m=1}^{M_{{{{\varvec{\alpha }}}}}} {\mathrm {E}\left[ \frac{Y_{{{{\varvec{\alpha }}}}}^2}{M_{{{{\varvec{\alpha }}}}}^2} \mathbf {1}_{\frac{Y_{{{{\varvec{\alpha }}}}}}{M_{{{{\varvec{\alpha }}}}}} > \epsilon \sqrt{{\mathrm {Var}\left[ {\mathcal {A}}\right] }}}\right] }}_{= F} = 0, \end{aligned}$$

for all $\epsilon > 0$. Below we make repeated use of the following identity for non-negative sequences ${\{a_{{{{\varvec{\alpha }}}}}\}}$ and ${\{b_{{{{\varvec{\alpha }}}}}\}}$ and $q \ge 0$:

$$\begin{aligned} \sum _{{{{{\varvec{\alpha }}}}}}{a_{{{{\varvec{\alpha }}}}}^q b_{{{{\varvec{\alpha }}}}}} \le \left( \sum _{{{{\varvec{\alpha }}}}}a_{{{{\varvec{\alpha }}}}}\right) ^q \sum _{{{{\varvec{\alpha }}}}}b_{{{{\varvec{\alpha }}}}}. \end{aligned}$$

(58)

First, we use the Markov inequality to bound

$$\begin{aligned} F&= \frac{1}{{\mathrm {Var}\left[ {\mathcal {A}}\right] }} \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} \sum _{m=1}^{M_{{{{\varvec{\alpha }}}}}} {\mathrm {E}\left[ \frac{Y_{{{{\varvec{\alpha }}}}}^2}{M_{{{{\varvec{\alpha }}}}}^2} \mathbf {1}_{Y_{{{{\varvec{\alpha }}}}}> \epsilon \sqrt{{\mathrm {Var}\left[ {\mathcal {A}}\right] }} M_{{{{\varvec{\alpha }}}}}}\right] } \\&\le \frac{\epsilon ^{-\rho }}{{\mathrm {Var}\left[ {\mathcal {A}}\right] }^{1+\rho /2}} \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} M_{{{{\varvec{\alpha }}}}}^{-1-\rho } {\mathrm {E}\left[ Y_{{{{\varvec{\alpha }}}}}^{2+\rho }\right] }. \end{aligned}$$

Using (58) and substituting for the variance ${\mathrm {Var}\left[ {\mathcal {A}}\right] }$ where we denote ${\mathrm {Var}\left[ { \Delta \fancyscript{S} }_{{{{\varvec{\alpha }}}}}\right] } = {\mathrm {E}\left[ ( \Delta \mathcal S_{{{{\varvec{\alpha }}}}}- {\mathrm {E}\left[ \Delta \mathcal S_{{{{\varvec{\alpha }}}}}\right] } )^{2}\right] }$ by $V_{{{{\varvec{\alpha }}}}}$, we find

$$\begin{aligned} F&\le \frac{\epsilon ^{-\rho } ( \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} M_{{{{\varvec{\alpha }}}}}^{-1} V_{{{{\varvec{\alpha }}}}})^{1+\rho /2}}{( \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} V_{{{{\varvec{\alpha }}}}}M_{{{{\varvec{\alpha }}}}}^{-1} )^{1+\rho /2}} \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} V_{{{{\varvec{\alpha }}}}}^{-1-\rho /2} M_{{{{\varvec{\alpha }}}}}^{-\rho /2} {\mathrm {E}\left[ Y_{{{{\varvec{\alpha }}}}}^{2+\rho }\right] } \\&= \epsilon ^{-\rho } \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} V_{{{{\varvec{\alpha }}}}}^{-1-{\rho }/{2}} M_{{{{\varvec{\alpha }}}}}^{-{\rho }/{2}} {\mathrm {E}\left[ Y_{{{{\varvec{\alpha }}}}}^{2+\rho }\right] }. \end{aligned}$$

Using the lower bound in (56) on the number of samples, $M_{{{{\varvec{\alpha }}}}}$, and (58), again yields

$$\begin{aligned} F&\le C_M^{-\rho /2} \epsilon ^{-\rho } \mathrm {TOL}^\rho \left( \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} V_{{{{\varvec{\alpha }}}}}^{-1-{\rho }/{2}} \left( \prod _{i=1}^d \exp \left( \frac{\rho \alpha _i\tilde{s}_i}{2}\right) \right) H_{{{{\varvec{\alpha }}}}}^{\rho /2} {\mathrm {E}\left[ Y_{{{{\varvec{\alpha }}}}}^{2+\rho }\right] } \right) \\&\qquad \qquad \quad \left( \sum _{{{{\varvec{\tau }}}}\in {\mathcal {I}}(\mathrm {TOL})} H_{{{\varvec{\tau }}}}\right) ^{-\rho /2} \\&\le C_M^{-\rho /2} \epsilon ^{-\rho } \mathrm {TOL}^\rho \left( \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} V_{{{{\varvec{\alpha }}}}}^{-1-{\rho }/{2}}\left( \prod _{i=1}^d \exp \left( \frac{\rho \alpha _i\tilde{s}_i}{2}\right) \right) {\mathrm {E}\left[ Y_{{{{\varvec{\alpha }}}}}^{2+\rho }\right] } \right) . \end{aligned}$$

Finally, using the bounds (55a) and (55b),

$$\begin{aligned} F&\le \underbrace{C_M^{-\rho /2} \epsilon ^{-\rho } Q_S^{-1-{\rho }/{2}} Q_R}_{= C_F} \mathrm {TOL}^\rho \left( \sum _{{{{{\varvec{\alpha }}}}}\in {\mathcal {I}}(\mathrm {TOL})} \left( \prod _{i=1}^d \exp \left( p_i \alpha _i \right) \right) \right) . \end{aligned}$$

Next, define three sets of dimension indices:

$$\begin{aligned} \hat{I}_1= & {} \{1\le i \le d: p_i < 0\}, \\ \hat{I}_2= & {} \{1\le i \le d: p_i = 0\}, \\ \hat{I}_3= & {} \{1\le i \le d: p_i > 0\}. \end{aligned}$$

Then, using (54) yields

$$\begin{aligned} F\le & {} C_F \mathrm {TOL}^\rho \prod _{i=1}^d \left( \sum _{\alpha _i=0}^{L_i} \exp \left( p_i \alpha _i \right) \right) \\\le & {} C_F \mathrm {TOL}^\rho \prod _{i \in \hat{I}_1} \frac{1}{1-\exp (p_i)} \prod _{i \in \hat{I}_2} L_i \prod _{i \in \hat{I}_3} \frac{1 - \exp (p_i (L_i+1))}{1-\exp (p_i)}. \end{aligned}$$

To conclude, observe that if $|\hat{I}_3| = 0$, then $\lim _{\mathrm {TOL}\downarrow 0} F= 0$ for any choice of $L_i\ge 0,\,1\le i\le d$. Similarly, if $|\hat{I}_3| > 0$, since we assumed that $c_i p_i < \rho$ holds for all $i \in \hat{I}_3$, then $\lim _{\mathrm {TOL}\downarrow 0} F= 0$. $\square$

Remark

The lower bound on the number of samples per index (56) mirrors choice (9), the latter being the optimal number of samples satisfying constraint (7). Specifically, $H_{{{{\varvec{\alpha }}}}}= \sqrt{V_{{{{\varvec{\alpha }}}}}W_{{{{\varvec{\alpha }}}}}}$ and $\tilde{s}_i = s_i$. Furthermore, notice that the previous Lemma bounds the growth of L from above, while Theorems 2.1 and 2.2 bound the value of L from below to satisfy the bias accuracy constraint.

Appendix B: Integrating an exponential over a simplex

Lemma 6.1

The following identity holds for any $L > 0$ and $a \in \mathbb {R}$:

$$\begin{aligned} { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \le L \}}} \exp (a |{{\varvec{x}}}|) d {{\varvec{x}}}= & {} (-a)^{-d} \left( 1 - \exp (La) \sum _{j=0}^{d-1} \frac{(-La)^j}{j!} \right) \nonumber \\= & {} \frac{1}{(d-1)!} \int _{0}^{L} \exp (at) t^{d-1}\, d t. \end{aligned}$$

(59)

Proof

$$\begin{aligned} { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \le L \}}} \exp (a |{{\varvec{x}}}|) d {{\varvec{x}}} = L^d { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \le 1 \}}} \exp (aL |{{\varvec{x}}}|) d {{\varvec{x}}}. \end{aligned}$$

Then, we prove, by induction on d and for $b = aL$, the following identity:

$$\begin{aligned} { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \le 1 \}}} \exp (b |{{\varvec{x}}}|) d {{{\varvec{x}}}} =(-b)^{-d} \left( 1 - \exp (b) \sum _{j=0}^{d-1} \frac{(-b)^j}{j!} \right) . \end{aligned}$$

First, for $d=1$, we have

$$\begin{aligned} \int _0^1 \exp (b x) d x = \frac{\exp (b) - 1}{b}. \end{aligned}$$

Next, assuming that the identity is true for $d-1$, we prove it for d. Indeed, we have

$$\begin{aligned}&{ \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \le 1 \}}} \exp (b |{{\varvec{x}}}|) d {{\varvec{x}}}\\&\quad = \int _0^1 \exp (b y) \left( { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^{d-1} \,:\, |{{\varvec{x}}}| \le 1-y \}}} \exp (b |{{\varvec{x}}}|) d {{\varvec{x}}} \right) d y \\&\quad = \int _0^1 \exp (b y) \left( 1-y \right) ^{d-1} \left( { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^{d-1} \,:\, |{{\varvec{x}}}| \le 1 \}}} \exp ((1-y) b |{{\varvec{x}}}|) d {{\varvec{x}}} \right) d y \\&\quad = \int _0^1 \exp (b y) \frac{\left( 1-y \right) ^{d-1}}{(-(1-y)b)^{d-1}} \left( 1 - \exp ((1-y)b) \sum _{j=0}^{d-2} \frac{(-(1-y)b)^j}{j!} \right) d y \\&\quad = \int _0^1 \left[ \frac{\exp (b y)}{(-b)^{d-1}} - \frac{\exp (b)}{(-b)^{d-1}} \sum _{j=0}^{d-2} \frac{(-(1-y)b)^j}{j!} \right] d y \\&\quad = \frac{\left( -1 \right) ^{d-1}}{b^d} \left( \exp (b) -1 \right) - \frac{(-1)^{d-1} \exp (b)}{b^{d-1}} \sum _{j=0}^{d-2} \frac{(-b)^j}{(j+1)!} \\&\quad = \frac{\left( -1 \right) ^{d}}{b^d} - \frac{\left( -1 \right) ^{d}}{b^d} \exp (b) - \frac{(-1)^{d} \exp (b)}{b^{d}} \sum _{j=1}^{d-1} \frac{(-b)^j}{(j)!} \\&\quad = (-b)^{-d} \left( 1 - \exp (b) \sum _{j=0}^{d-1} \frac{(-b)^j}{j!} \right) . \end{aligned}$$

Finally, the second equality in (59) follows by repeatedly integrating by parts. $\square$

Lemma 6.2

For $a \in \mathbb {R}^d$, assume $A = \max _{i=1,2\ldots d} a_i > 0$ and denote

$$\begin{aligned} \mathfrak {a}_1&= \#\left\{ i = 1,2,\ldots d \,:\, a_i = A\right\} ,\quad&\mathfrak {a}_2&=d-\mathfrak {a}_1. \end{aligned}$$

Then, for any $L > 0$, there exists an $\epsilon > 0$ satisfying

$$\begin{aligned} \epsilon \le A - \max \left( 0, \max _{\begin{array}{c} i=1,2\ldots d \\ a_i < A \end{array}} a_i \right) , \end{aligned}$$

such that the following inequality holds:

$$\begin{aligned} { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \le L \}}} \exp ({{\varvec{a}}} \cdot {{\varvec{x}}}) d {{\varvec{x}}} \le \mathfrak {C_W}({{\varvec{a}}}) \exp \left( A L\right) L^{\mathfrak {a}_1 -1}. \end{aligned}$$

(60)

Here, the constant $\mathfrak {C_W}({{\varvec{a}}})$ is given by

$$\begin{aligned} \mathfrak {C_W}({{\varvec{a}}}) = {\left\{ \begin{array}{ll} \frac{1}{A(d-1)!} &{} \text { if }\,\,{\mathfrak {a}}_1 = d \\ \frac{4}{\epsilon (2A - \epsilon )} \frac{\exp (1-\mathfrak a_2 )}{(\mathfrak a_1-1)! (\mathfrak a_2-1)!} \left( \frac{2 (\mathfrak a_2 -1)}{\epsilon } \right) ^{\mathfrak {a}_2 -1} &{} \text {otherwise} \end{array}\right. } \end{aligned}$$

(61)

Proof

First, note that ${{\varvec{a}}} = A {{\varvec{1}}}$ for some scalar $A > 0$ and ${{\varvec{1}}} = (1,1,\ldots , 1)$ if and only if $\mathfrak {a}_1 = d$. Then Lemma 6.1 immediately gives

$$\begin{aligned} { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^{d} \,:\, |{{\varvec{x}}}| \le L \}}} \exp ( {{\varvec{a}}} \cdot {{\varvec{x}}}) d {{\varvec{x}}} \le \frac{L^{d-1} \exp (A L)}{A (d-1)!}. \end{aligned}$$

Otherwise, recall that

$$\begin{aligned} x^j \le \left( \frac{j}{b}\right) ^j \exp (-j) \exp (bx) \end{aligned}$$

(62)

holds for any $x>0, b>0$ and $j\in \mathbb {N}$. Then, using Lemma 6.1 and (62), we can write

$$\begin{aligned}&{ \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^{d} \,:\, |{{\varvec{x}}}| \le L \}}} \exp ( {{{\varvec{a}}}} \cdot {{\varvec{x}}}) d {{\varvec{x}}} \\&\quad \le { \int _{ \{ {{\varvec{x}}}_2 \in \mathbb {R}_{+}^{\mathfrak {a}_2 } \,:\, |{{\varvec{x}}}_2| \le L \}}} \exp \left( \left( A - \epsilon \right) |{{\varvec{x}}}_2|\right) \left( { \int _{ \{ {{\varvec{x}}}_1 \in \mathbb {R}_{+}^{\mathfrak {a}_1 } \,:\, |{{\varvec{x}}}_1| \le L - |{{\varvec{x}}}_2| \}}} \exp (A |{{\varvec{x}}}_1|) d {{\varvec{x}}}_1 \right) d {{\varvec{x}}}_2 \\&\quad = \frac{1}{(\mathfrak {a}_1-1) !} { \int _{ \{ {{\varvec{x}}}_2 \in \mathbb {R}_{+}^{\mathfrak {a}_2 } \,:\, |{{\varvec{x}}}_2| \le L \}}} \exp \left( (A - \epsilon ) |{{\varvec{x}}}_2|\right) \left( \int _0^{L - |{{\varvec{x}}}_2|} \exp (A t) t^{\mathfrak {a}_1 -1} d t \right) d {{\varvec{x}}}_2 \\&\quad = \frac{1}{(\mathfrak {a}_1-1) !} \int _0^{L} \exp (A t) t^{\mathfrak {a}_1 -1} \left( { \int _{ \{ {{\varvec{x}}}_2 \in \mathbb {R}_{+}^{\mathfrak {a}_2 } \,:\, |{{\varvec{x}}}_2| \le L -t \}}} \exp \left( (A - \epsilon ) |{{\varvec{x}}}_2|\right) d {{\varvec{x}}}_2 \right) d t\\&\quad = \frac{1}{(\mathfrak {a}_1-1) !(\mathfrak {a}_2 -1)!} \int _0^{L} \exp (A t) t^{\mathfrak {a}_1 -1} \left( \int _0^{L-t} \exp \left( ( A - \epsilon ) z\right) z^{\mathfrak {a}_2 -1} d z \right) d t\\&\quad \le {\mathfrak {C}} \int _0^{L} \exp (A t) t^{\mathfrak {a}_1 -1} \left( \int _0^{L-t} \exp \left( \frac{z(2A - \epsilon ) }{2} \right) d z \right) d t,\\&\qquad \quad \text {where}\quad {\mathfrak {C}} = \frac{\exp (1-\mathfrak {a}_2 )}{(\mathfrak {a}_1-1)! (\mathfrak {a}_2-1)!} \left( \frac{2 (\mathfrak {a}_2 -1)}{\epsilon } \right) ^{\mathfrak {a}_2 -1}, \end{aligned}$$

continuing

$$\begin{aligned}&{ \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^{d} \,:\, |{{\varvec{x}}}| \le L \}}} \exp ( {{{\varvec{a}}}} \cdot {{\varvec{x}}}) d {{\varvec{x}}} \\&\quad \le {\mathfrak {C}} \exp \left( \frac{L(2 A - \epsilon ) }{2} \right) \frac{2}{2A - \epsilon } \int _0^{L} t^{\mathfrak {a}_1 -1} \exp \left( \frac{\epsilon t}{2} \right) d t \\&\quad \le {\mathfrak {C}} \exp \left( \frac{L(2 A - \epsilon ) }{2} \right) \frac{2 L^{\mathfrak {a}_1 -1}}{2 A -\epsilon } \int _0^{L} \exp \left( \frac{\epsilon t}{2} \right) d t \\&\quad \le \frac{4{\mathfrak {C}}}{\epsilon (2A - \epsilon )} \exp \left( A L\right) L^{\mathfrak {a}_1 -1}. \end{aligned}$$

$\square$

Lemma 6.3

The following inequality holds for any $L \ge 1$ and ${{\varvec{a}}} \in \mathbb {R}_{+}^d$:

$$\begin{aligned} { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| > L \}}} \exp (-{{\varvec{a}}} \cdot {{\varvec{x}}}) d {{\varvec{x}}} \le \mathfrak {C_B}({{\varvec{a}}}) \exp \left( -A L\right) L^{\mathfrak {a}_1 -1}, \end{aligned}$$

where

$$\begin{aligned} \mathfrak {C_B}({{\varvec{a}}}) = {\left\{ \begin{array}{ll} \sum _{j=0}^{d-1} \frac{A^{j-d}}{j!} &{} \quad \text { if }\,\,{\mathfrak {a}}_1 = d \\ {\left( A + \epsilon \right) }^{-\mathfrak {a}_2} \sum _{j=0}^{\mathfrak {a}_1-1} \frac{A^{j-\mathfrak {a}_1}}{j!} + \frac{ 2\sum _{j=0}^{\mathfrak {a}_2-1} \exp (-j) \left( \frac{2 j}{\epsilon } \right) ^{j} \frac{{\left( A + \epsilon \right) }^{j-\mathfrak {a}_2}}{j!}}{ (\mathfrak {a}_1-1)! \epsilon } &{}\quad \text {otherwise} \end{array}\right. }\nonumber \\ \end{aligned}$$

(63)

and

$$\begin{aligned} A&= \min _{i=1,2\ldots d} a_i, \quad \epsilon = \min _{\begin{array}{c} i=1,2\ldots d \\ a_i > A \end{array}} a_i - A, \\ \mathfrak {a}_1&= \#\left\{ i = 1,2,\ldots d \,:\, a_i = A\right\} . \\ \mathfrak {a}_2&=d-\mathfrak {a}_1. \end{aligned}$$

Proof

First, note that ${{\varvec{a}}} = A {{\varvec{1}}}$ for some scalar $A > 0$ and ${{\varvec{1}}} = (1,1,\ldots , 1)$ if and only if $\mathfrak {a}_1 = d$. Then Lemma 6.1 immediately gives:

$$\begin{aligned}&{ \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^{d} \,:\, |{{\varvec{x}}}| > L \}}} \exp (- A |{{\varvec{x}}}|) d {{\varvec{x}}} \\&\quad = \int _{{{\varvec{x}}} \in \mathbb {R}_{+}^{d} } \exp ( -A |{{\varvec{x}}}|) d {{\varvec{x}}} - { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^{d} \,:\, |{{\varvec{x}}}| \le L \}}} \exp ( -A |{{\varvec{x}}}|) d {{\varvec{x}}} \\&\quad = A^{-d} - A^{-d} \left( 1 - \exp (-AL) \sum _{j=0}^{d-1} \frac{(AL)^j}{j!} \right) \\&\quad \le \exp (-AL) L^{d-1} \sum _{j=0}^{d-1} \frac{A^{j-d}}{j!} \\ \end{aligned}$$

Otherwise, without loss of generality, assume that $a_i \le a_j$ for all $1\le i \le j\le d$. Then, again using Lemma 6.1, we can write

$$\begin{aligned}&{ \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \ge L \}}} \exp \left( -\sum _{i=1}^d a_i x_i \right) \, d {{\varvec{x}}} \\&\quad \le { \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \ge L \}}} \exp \left( -A \sum _{i=1}^{\mathfrak {a}_1} x_i -{\left( A + \epsilon \right) } \sum _{i=\mathfrak {a}_1+1}^d x_i\right) \, d {{\varvec{x}}} \\&\quad = \left[ \int _{{{\varvec{x}}} \in \mathbb {R}_{+}^d} \exp \left( - A \sum _{i=1}^{\mathfrak {a}_1} x_i -{\left( A + \epsilon \right) } \sum _{i=\mathfrak {a}_1+1}^d x_i\right) d {{\varvec{x}}} \right. \\&\left. \qquad -{ \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \le L \}}} \exp \left( -A \sum _{i=1}^{\mathfrak {a}_1} x_i -{\left( A + \epsilon \right) } \sum _{i=\mathfrak {a}_1+1}^d x_i \right) \, d {{\varvec{x}}} \right] , \end{aligned}$$

where

$$\begin{aligned} \int _{{{\varvec{x}}} \in \mathbb {R}_{+}^d} \exp \left( - A \sum _{i=1}^{\mathfrak {a}_1} x_i -(A + \epsilon ) \sum _{i=\mathfrak {a}_1+1}^d x_i\right) d {{\varvec{x}}} = A ^{-\mathfrak {a}_1} {\left( A + \epsilon \right) }^{-\mathfrak {a}_2}. \end{aligned}$$

Now consider

$$\begin{aligned}&{ \int _{ \{ {{\varvec{x}}} \in \mathbb {R}_{+}^d \,:\, |{{\varvec{x}}}| \le L \}}} \exp \left( - A \sum _{i=1}^{\mathfrak {a}_1} x_i -{\left( A + \epsilon \right) } \sum _{i=\mathfrak {a}_1+1}^d x_i \right) \, d {{\varvec{x}}}\\&\quad ={ \int _{ \{ {{\varvec{x}}}_2 \in \mathbb {R}_{+}^{\mathfrak {a}_2} \,:\, |{{\varvec{x}}}_2| \le L \}}} \exp \left( - {\left( A + \epsilon \right) } |{{\varvec{x}}}_2| \right) \\&\qquad \quad \times \left( { \int _{ \{ {{\varvec{x}}}_1 \in \mathbb {R}_{+}^{\mathfrak {a}_1} \,:\, |{{\varvec{x}}}_1| \le L - |{{\varvec{x}}}_2| \}}} \exp \left( - A |{{\varvec{x}}}_1| \right) \, d {{\varvec{x}_1}}\right) \, d {{\varvec{x}_2}}\\&\quad = \frac{1}{(\mathfrak {a}_1-1)!} { \int _{ \{ {{\varvec{x}}}_2 \in \mathbb {R}_{+}^{\mathfrak {a}_2} \,:\, |{{\varvec{x}}}_2| \le L \}}} \exp \left( -{\left( A + \epsilon \right) } |{{\varvec{x}}}_2| \right) \left( \int _0^{L-|{{\varvec{x}}}_2|} \exp (-A t) t^{\mathfrak {a}_1-1} d t \right) d {{\varvec{x}_2}}\\&\quad = \frac{1}{(\mathfrak {a}_1-1)!} \int _0^{L} \exp (-A t) t^{\mathfrak {a}_1-1} \left( { \int _{ \{ {{\varvec{x}}}_2 \in \mathbb {R}_{+}^{\mathfrak {a}_2} \,:\, |{{\varvec{x}}}_2| \le L-t \}}} \exp \left( - {\left( A + \epsilon \right) } |{{\varvec{x}}}_2| \right) d {{\varvec{x}_2}} \right) d t\\&\quad = \frac{1}{(\mathfrak {a}_1-1)! (\mathfrak {a}_2-1)!} \int _0^{L} \exp (-A t) t^{\mathfrak {a}_1-1} \left( \int _0^{L-t} \exp (-{\left( A + \epsilon \right) } z) z^{\mathfrak {a}_2-1}\, d z \right) d t \\&\quad = \frac{{\left( A + \epsilon \right) }^{-\mathfrak {a}_2}}{(\mathfrak {a}_1-1)!} \int _0^{L} \exp (-A t) t^{\mathfrak {a}_1-1}\\&\quad \qquad \times \left( 1 -\exp (-{\left( A + \epsilon \right) } (L-t)) \sum _{j=0}^{\mathfrak {a}_2-1} \frac{({\left( A + \epsilon \right) } (L-t))^j}{j!} \right) d t \\&\quad = A ^{-\mathfrak {a}_1} {\left( A + \epsilon \right) }^{-\mathfrak {a}_2} - A ^{-\mathfrak {a}_1} {\left( A + \epsilon \right) }^{-\mathfrak {a}_2}\left( \exp (-A L) \sum _{j=0}^{\mathfrak {a}_1-1} \frac{(A L)^j}{j!} \right) \\&\qquad - \frac{{\left( A + \epsilon \right) }^{-\mathfrak {a}_2}}{(\mathfrak {a}_1-1)!} \int _0^{L} \exp (-A t) t^{\mathfrak {a}_1-1}\\&\quad \qquad \times \left( \exp (-{\left( A + \epsilon \right) } (L-t)) \sum _{j=0}^{\mathfrak {a}_2-1} \frac{({\left( A + \epsilon \right) } (L-t))^j}{j!} \right) d t. \end{aligned}$$

Here, we can bound

$$\begin{aligned}&A ^{-\mathfrak {a}_1} {\left( A + \epsilon \right) }^{-\mathfrak {a}_2}\left( \exp (-A L) \sum _{j=0}^{\mathfrak {a}_1-1} \frac{(A L)^j}{j!} \right) \\&\quad \le A ^{-\mathfrak {a}_1} {\left( A + \epsilon \right) }^{-\mathfrak {a}_2}\exp (-A L) L^{\mathfrak {a}_1 -1} \sum _{j=0}^{\mathfrak {a}_1-1} \frac{A ^j}{j!}. \end{aligned}$$

Recall that ${\epsilon } > 0$ and bound, using (62) for $(L-t)^j$ with $b = {\epsilon }/{2}$,

$$\begin{aligned}&\frac{{\left( A + \epsilon \right) }^{-\mathfrak {a}_2}}{(\mathfrak {a}_1-1)!} \int _0^{L} \exp (-A t) t^{\mathfrak {a}_1-1}\left( \exp (-{\left( A + \epsilon \right) } (L-t)) \sum _{j=0}^{\mathfrak {a}_2-1} \frac{({\left( A + \epsilon \right) } (L-t))^j}{j!} \right) d t \\&\quad \le \frac{{\left( A + \epsilon \right) }^{-\mathfrak {a}_2}}{(\mathfrak {a}_1-1)!} \left( \sum _{j=0}^{\mathfrak {a}_2-1} \exp (-j) \left( \frac{2 j}{ \epsilon } \right) ^{j} \frac{{\left( A + \epsilon \right) }^j}{j!} \right) \exp \left( -L\left( \frac{ 2 A + \epsilon }{2} \right) \right) \int _0^{L}\\&\qquad \exp \left( \frac{\epsilon t }{2} \right) t^{\mathfrak {a}_1-1} d t \\&\quad \le \frac{{\left( A + \epsilon \right) }^{-\mathfrak {a}_2}}{(\mathfrak {a}_1-1)!} \left( \frac{2}{\epsilon } \right) \left( \sum _{j=0}^{\mathfrak {a}_2-1} \exp (-j) \left( \frac{2 j}{ \epsilon } \right) ^{j} \frac{{\left( A + \epsilon \right) }^j}{j!} \right) \exp \left( -A L\right) L^{\mathfrak {a}_1-1}. \end{aligned}$$

$\square$

Appendix C: List of definitions

In this section, for easier reference, we list definitions of notation that is used in multiple pages or sections throughout the current work