An accept-reject algorithm for the positive multivariate normal distribution

Abstract

The need to simulate from a positive multivariate normal distribution arises in several settings, specifically in Bayesian analysis. A variety of algorithms can be used to sample from this distribution, but most of these algorithms involve Gibbs sampling. Since the sample is generated from a Markov chain, the user has to account for the fact that sequential draws in the sample depend on one another and that the sample generated only follows a positive multivariate normal distribution asymptotically. The user would not have to account for such issues if the sample generated was i.i.d. In this paper, an accept-reject algorithm is introduced in which variates from a positive multivariate normal distribution are proposed from a multivariate skew-normal distribution. This new algorithm generates an i.i.d. sample and is shown, under certain conditions, to be very efficient.

Appendix

1.1 The perfect sampling algorithm of Phillipe and Robert

Perfect sampling involves coupling two Markov chains, one which begins at $\mathbf{x}^\mathrm{min}$, where

$$\begin{aligned} \mathbf{x}^\mathrm{min} = \mathrm{arg}_{ \mathbf{x}: \mathbf{x} \in \mathbb{R }_+^d} \min \left( \exp \left\{ -{\frac{1}{2}} \left( \mathbf{x} - \mu \right)^T \Sigma ^{-1} \left( \mathbf{x} - \mu \right) \right\} \right), \end{aligned}$$

and another which begins at $\mathbf{x}^\mathrm{max}$, where

$$\begin{aligned} \mathbf{x}^\mathrm{max} = \mathrm{arg}_{ \mathbf{x}: \mathbf{x} \in \mathbb{R }_+^d} \max \left( \exp \left\{ -{\frac{1}{2}} \left( \mathbf{x} - \mu \right)^T \Sigma ^{-1} \left( \mathbf{x} - \mu \right) \right\} \right). \end{aligned}$$

Once these two chains coalesce, an exact (or perfect) sample is generated.

While PR’s perfect sampling algorithm for the positive multivariate normal distribution is theoretically appealing, there are some computational issue associated with it which any practitioner should be aware of. The first issue involves the location of $\mathbf{x}^\mathrm{min}$, the value of $\mathbf{x}$ at which the density of $\mathbf{X}$ achieves its lowest value. Mira et al. (2001) make it clear that for a perfect sampling algorithm such as this to truly generate an exact sample, one of the chains must begin at $\mathbf{x}^\mathrm{min}$. For a positive multivariate normal distribution, $\mathbf{x}^\mathrm{min} = \left( + \infty , +\infty , \ldots , +\infty \right)^T$. Starting a Markov chain at this point, however, is computationally infeasible. Phillipe and Robert get around this problem by setting $\mathbf{x}^\mathrm{min}$ to an extreme positive value, but it is never made clear how this extreme value should vary with the parameters of the positive multivariate normal distribution. Setting $\mathbf{x}^\mathrm{min}$ to such a value also results in a sample that is not truly exact. Their algorithm also involves an accept-reject step, and it is not clear how the acceptance probability of this step varies with the parameters of the positive multivariate normal distribution.

1.2 Correction to Gupta et al. (2004)

Gupta et al. (2004) state the following theorem

If

$$\begin{aligned} \left( \begin{array}{c} \mathbf{W} \\ \mathbf{Z} \end{array} \right) \sim MVN_{2d} \left[ \left( \begin{array}{c} \xi \\ \beta \end{array} \right), \left( \begin{array}{cc} \mathbf{I} + \mathbf{R} \Omega \mathbf{R}^T&\mathbf{R} \Omega \\ \Omega \mathbf{R}^T&\Omega \end{array} \right) \right] \end{aligned}$$

then $\mathbf{Y} = \mathbf{Z}|(\mathbf{W} > \mathbf{0}) \sim f_\mathbf{Y} \left(\mathbf{y}; \beta , \xi , \mathbf{R}, \Omega \right)$ where

$$\begin{aligned} f_\mathbf{Y} \left( \mathbf{y}; \beta , \xi , \mathbf{R}, \Omega \right) = {\frac{1}{\Phi _d \left( \mathbf{0}; -\xi , \mathbf{I} + \mathbf{R} \Omega \mathbf{R}^T \right)}} \phi _d \left( \mathbf{y}; \beta , \Omega \right) \Phi _d \left( \mathbf{R} \left( \mathbf{y} - \beta \right) - \xi , \mathbf{I} \right). \end{aligned}$$

The density $f_\mathbf{Y}(\mathbf{y}; \beta , \xi , \mathbf{R}, \Omega )$ is incorrect. The correct density is

$$\begin{aligned} f_\mathbf{Y} \left( \mathbf{y}; \beta , \xi , \mathbf{R}, \Omega \right) = {\frac{1}{\Phi _d \left( \mathbf{0}; -\xi , \mathbf{I} + \mathbf{R} \Omega \mathbf{R}^T \right)}} \phi _d \left( \mathbf{y}; \beta , \Omega \right) \Phi _d \left( \mathbf{R} \left( \beta - \mathbf{y} \right) - \xi , \mathbf{I} \right). \end{aligned}$$

Proof

First observe that

$$\begin{aligned}&f_\mathbf{Y} \left( \mathbf{y}; \beta , \xi , \mathbf{R}, \Omega \right) = f_{\mathbf{Z}| \mathbf{W}} \left( \mathbf{z} | \mathbf{w} > \mathbf{0} \right) = {\frac{ f_\mathbf{Z} \left( \mathbf{z} \right) \mathbb{P } \left( \mathbf{W} > \mathbf{0}|\mathbf{z} \right)}{ \mathbb{P } \left( \mathbf{W} > \mathbf{0} \right)}}&\quad = {\frac{\phi _d \left( z; \beta , \Omega \right) \mathbb{P } \left( -\mathbf{W} < \mathbf{0}|\mathbf{z} \right)}{ \mathbb{P } \left( -\mathbf{W} < \mathbf{0} \right)}}. \end{aligned}$$

Since $\mathbf{W}| \left( \mathbf{Z} = \mathbf{z} \right) \sim MVN_d \left( \xi + \mathbf{R} \left( \mathbf{z} - \beta \right), \mathbf{I} \right)$, we get

$$\begin{aligned} f_{\mathbf{Z}|\mathbf{W}} \left( \mathbf{z}|\mathbf{w} > \mathbf{0} \right)&= {\frac{\phi _d \left( \mathbf{z}; \beta , \Omega \right) \Phi _d \left( \mathbf{0}; \mathbf{R} \left( \beta - \mathbf{z} \right) - \xi , \mathbf{I} \right)}{ \Phi _d \left( \mathbf{0}; -\xi , \mathbf{I} + \mathbf{R} \Omega \mathbf{R}^{\prime } \right)}} \\&\Longrightarrow f_\mathbf{Y} \left( \mathbf{y}; \beta , \xi , \mathbf{R}, \Omega \right) = {\frac{\phi _d \left( \mathbf{y}; \beta , \Omega \right) \Phi _d \left( \mathbf{0}; \mathbf{R} \left( \beta - \mathbf{y} \right) - \xi , \mathbf{I} \right)}{ \Phi _d \left( \mathbf{0}; -\xi , \mathbf{I} + \mathbf{R} \Omega \mathbf{R}^{\prime } \right)}} \end{aligned}$$

$\square$

1.3 Proof of Lemma 1

By setting $\beta = \mu + \alpha$ for some $\alpha \in \mathbb{R }^d, m \left( \mathbf{y}; \beta , \Omega \right)$ becomes

$$\begin{aligned} m \left( \mathbf{y}; \beta , \Omega \right)&= \left( \mathbf{y} - \mu \right)^T \Sigma ^{-1} \left( \mathbf{y} - \mu \right) - \left[ \mathbf{y} - \left( \mu + \alpha \right) \right]^T \Omega ^{-1} \left[ \mathbf{y} - \left( \mu + \alpha \right) \right] \\&= \left( \mathbf{y} - \mu \right)^T \Sigma ^{-1} \left( \mathbf{y} - \mu \right) - \left( \mathbf{y} - \mu \right)^T \Omega ^{-1} \left( \mathbf{y} - \mu \right) + 2 \alpha ^T \Omega ^{-1}&\quad \left( \mathbf{y} - \mu \right) - \alpha ^T \Omega ^{-1} \alpha \end{aligned}$$

Observe that at $\mathbf{y} = \mu , m \left( \mathbf{y}; \beta , \Omega \right) = -\alpha ^T \Omega ^{-1} \alpha$ which is negative for all $\alpha \ne \mathbf{0}$ since $\Omega ^{-1}$ is positive definite. Since $m \left( \mathbf{y}; \beta , \Omega \right) \ge 0~\forall ~\mathbf{y} \in \mathbb{R }^d, \alpha$ has to equal $\mathbf{0}$, which implies $\beta = \mu$ $\square$

1.4 Proof of Lemma 2

This is a proof by contradiction. If $m \left( \mathbf{y}; \beta , \Omega \right) \ge 0~\forall ~\mathbf{y} \in \mathbb{R }^d$, then with algebra it follows that

$$\begin{aligned} \exp \left\{ -{\frac{1}{2}} \left( \mathbf{y} - \mu \right)^T \Sigma ^{-1} \left( \mathbf{y} - \mu \right) \right\} \le \exp \left\{ -{\frac{1}{2}} \left( \mathbf{y} - \beta \right)^T \Omega ^{-1} \left( \mathbf{y} - \beta \right) \right\} ~~\forall ~\mathbf{y} \in \mathbb{R }^d. \end{aligned}$$

Now assume that $|\Omega | < |\Sigma |$. If this is true, it follows that $\phi _d \left( \mathbf{y}; \mu , \Sigma \right) < \phi _d \left( \mathbf{y}; \beta , \Omega \right)~~ \forall ~\mathbf{y} \in \mathbb{R }^d$, which is impossible since both functions integrate to 1 over $\mathbb{R }^d$ $\square$

1.5 Proof of Theorem 1

By restricting $m \left( \mathbf{y}; \beta , \Omega \right) \ge 0~~\forall ~\mathbf{y} \in \mathbb{R }^d$, it follows from Lemma 1 that $\beta = \mu$, and from Lemma 2 that $\Omega$ is some matrix such that $|\Omega | \ge |\Sigma |$. Since $\sup _{\mathbf{y} \in \mathbb{R }^d} \left\{ h \left( \mathbf{y}; \beta , \Omega \right) \right\}$ increases with $|\Omega |$, $\Omega$ should be some matrix such that its determinant is as close to $|\Sigma |$ as possible, and such that $m \left( \mathbf{y}; \beta , \Omega \right) \ge 0~\forall ~\mathbf{y} \in \mathbb{R }^d$. A value of $\Omega$ that has a determinant greater equal to $|\Sigma |$ yet still satisfies the restriction placed on $m \left( \mathbf{y}; \beta , \Omega \right)$ is $\Omega = \Sigma$. With $\Omega = \Sigma$, $\inf _{\beta , \Omega : m \left( \mathbf{y}; \beta , \Omega \right) > 0 } \left[ \sup _{ \mathbf{y} \in \mathbb{R }^d} \left\{ h \left( \mathbf{y}; \beta , \Omega \right) \right\} \right] = 1$. $\square$

1.6 Proof of Theorem 2

First observe that

$$\begin{aligned}&\sup _{\mathbf{y} \in \mathbb{R }_+^d} \left\{ t \left( \mathbf{y}; \beta ^*, \xi , \mathbf{R}, \Omega ^*, \mathcal{A }, l \right) \right\} = \sup _{\mathbf{y} \in \mathbb{R }_+^d} \left\{ {\frac{ \mathbb{P } \left( \mathbf{W} \in \mathcal{A } \right)}{ \mathbb{P } \left( \left. \mathbf{W} \in \mathcal{A } \right| \left( \mathbf{Z} = \mathbf{y} \right) \right)}} \right\} \nonumber \\&\quad = \sup _{\mathbf{y} \in \mathbb{R }_+^d} \left\{ {\frac{ \mathbb{P } \left( \mathbf{e}^T \mathbf{W} > 0 \right) }{ \mathbb{P } \left( \left. \mathbf{e}^T \mathbf{W} > 0 \right| \left( \mathbf{Z} = \mathbf{y} \right) \right) }} \right\} \nonumber \\&\quad = \sup _{\mathbf{y} \in \mathbb{R }_+^d} \left\{ {\frac{ \Phi _1 \left( 0; - \mathbf{e}^T \xi , 1 + r^2 \mathbf{e}^T \mathrm{Cov} \left( \mathbf{Z}^* \right) \mathbf{e} \right)}{ \Phi _1 \left(0; -\mathbf{e}^T \xi - r \mathbf{e}^T \mathbf{H} \mathbf{y} + r \mathbf{e}^T \mathbf{H} \beta ^*, 1 \right)}} \right\} \nonumber \\&\quad = {\frac{ \Phi _1 \left( 0; - \mathbf{e}^T \xi , 1 + r^2 \mathbf{e}^T \mathrm{Cov} \left( \mathbf{Z}^* \right) \mathbf{e} \right)}{ \inf _{\mathbf{y} \in \mathbb{R }_+^d} \left\{ \Phi _1 \left(0; -\mathbf{e}^T \xi - r \mathbf{e}^T \mathbf{H} \mathbf{y} + r \mathbf{e}^T \mathbf{H} \beta ^*, 1 \right) \right\} }} \end{aligned}$$

(15)

The second to last equality in (15) follows since $\mathbf{W} \sim MVN_{l^*} \left( \xi , \mathbf{I} + r^2 \mathrm{Cov} \left( \mathbf{Z}^* \right) \right)$, and

$$\begin{aligned} \mathbf{W} | \left( \mathbf{Z} = \mathbf{y} \right) \sim MVN_{l^*} \left( \xi + r \mathbf{H} \left( \mathbf{y} - \beta ^* \right), \mathbf{I} \right). \end{aligned}$$

Since $r > 0$, $\min ( \mathbf{e} ) < 0$, and all elements of $\mathbf{H}$ are equal to 0 or 1, the vector $-r \mathbf{e}^T \mathbf{H}$ has at least one positive element. With this being true, it follows that $-r \mathbf{e}^T \mathbf{H} \mathbf{y}$ spans over all values of $\mathbb{R }$ when $\mathbf{y} \in \mathbb{R }_+^d$. It follows that $\inf _{\mathbf{y} \in \mathbb{R }_+^d} \left\{ \Phi _1 \left(0; -\mathbf{e}^T \xi - r \mathbf{e}^T \mathbf{H} \mathbf{y} + r \mathbf{e}^T \mathbf{H} \mu , 1 \right) \right\} = 0,$ making

$$\begin{aligned} \sup _{\mathbf{y} \in \mathbb{R }_+^d}\left\{ t \left( \mathbf{y}; \beta ^*, \xi , \mathbf{R}, \Omega ^*, \mathcal{A }, l \right) \right\} = + \infty . \end{aligned}$$

$\square$

1.7 Proof of Theorem 3

First observe that

$$\begin{aligned}&\sup _{\mathbf{y} \in \mathbb{R }_+^d} \left\{ t \left( \mathbf{y}; \beta ^*, \xi , \mathbf{R}, \Omega ^*, \mathcal{A }^*, l^* \right) \right\} = \sup _{\mathbf{y} \in \mathbb{R }_+^d} \left\{ {\frac{ p_\mathrm{SAR} \left( \mu , \Sigma \right)^{-1} \mathbb{P } \left( \mathbf{W} \in \mathcal{A }^* \right)}{ \mathbb{P } \left( \left. \mathbf{W} \in \mathcal{A } \right| \left( \mathbf{Z} = \mathbf{y} \right) \right)}} \right\} \nonumber \\&\quad = \sup _{ \mathbf{y} \in \mathbb{R }_+^d} \left\{ {\frac{ p_\mathrm{SAR} \left( \mu , \Sigma \right)^{-1}\mathbb{P } \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbf{W} > 0 \right)}{ \mathbb{P } \left( \left. \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbf{W} > 0 \right| \left( \mathbf{Z} = \mathbf{y} \right) \right) }} \right\} \nonumber \\&\quad = \sup _{ \mathbf{y} \in \mathbb{R }_+^d} \left\{ {\frac{ p_\mathrm{SAR} \left( \mu , \Sigma \right)^{-1} \Phi _1 \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \xi \left( 1 + r^2 \omega ^{\mathbf{Z}^*} \right)^{-{\frac{1}{2}}}; 0, 1 \right)}{ \Phi _1 \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \xi - r \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right) + r \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbf{H} \mathbf{y}; 0, 1 \right)}} \right\} \nonumber \\&\quad = {\frac{ p_\mathrm{SAR} \left( \mu , \Sigma \right)^{-1} \Phi _1 \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \xi \left( 1 + r^2 \omega ^{\mathbf{Z}^*} \right)^{-{\frac{1}{2}}}; 0, 1 \right)}{ \inf _{ \mathbf{y} \in \mathbb{R }_+^d} \left\{ \Phi _1 \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \xi - r \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right) + r \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbf{y}; 0,1 \right) \right\} }} \nonumber \\&\quad = {\frac{ \Phi _1 \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \xi \left( 1 + r^2 \omega ^{\mathbf{Z}^*} \right)^{-{\frac{1}{2}}}; 0, 1 \right)}{ p_\mathrm{SAR} \left( \mu , \Sigma \right) \Phi _1 \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \xi - r \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right); 0, 1 \right) }} \end{aligned}$$

(16)

We have to select values of $r$ and $\xi$ that make $q$ as small as possible and $s$ as large as possible, where

$$\begin{aligned} q = \left. \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \xi \right. \left( 1 + r^2 \omega ^{\mathbf{Z}^*} \right)^{\frac{1}{2}}~~\mathrm{and}~~s = \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \xi - r \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right). \end{aligned}$$

With lots of algebra, we get that $r$ satisfies

$$\begin{aligned} r = {\frac{ -s \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right) - q \left[ \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right) \right)^2 + \omega ^{\mathbf{Z}^*} \left( s^2 - q^2 \right) \right]^{\frac{1}{2}}}{ \left[ \left( \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right) \right)^2 - q^2 \omega ^{\mathbf{Z}^*} \right]}}. \end{aligned}$$

(17)

If $q > \left. \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right) \right. \sqrt{ \omega ^{\mathbf{Z}^*}}$, and $s > |q|$, then $r$ is real and positive. The value of $s$ can thus be arbitrarily larger than $|q|$, making the denominator of (16) arbitrarily close to $p_\mathrm{SAR} (\mu , \Sigma )$, and $q$ can be arbitrarily close (yet larger) than $\left. \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right) \right. \sqrt{ \omega ^{ \mathbf{Z}^*}}$ making the numerator of (16) arbitrarily close to ${ \Phi _1 \left( \left. \left( \mathbf{h}^{\mathbf{Z}^*} \right)^T \mathbb{E } \left( \mathbf{Z}^* \right) \right. \sqrt{ \omega ^{ \mathbf{Z}^*}}; 0, 1 \right) = \Phi _1 \left( g(\mu , \Sigma ); 0,1 \right)}.$ $\square$

1.8 Proof of Corollary 1

To prove the corollary, the following three results are necessary.

Result 1

Let $\alpha \in (0,1)$. For a negative value of $\kappa _1$, $\Phi _1 \left( \left( \kappa _1 + \epsilon _1 \right); 0, 1 \right) - \Phi _1 \left( \kappa _1; 0, 1 \right) = u_1 \left( \delta _1 \right)$ where $u_1 \left( \delta _1 \right) = o \left( \delta _1^{1 - \alpha } \right)$ as $\delta _1 \searrow 0$.

Proof

For a negative value of $\kappa _1$,

$$\begin{aligned} \Phi _1 \left( \kappa _1; 0, 1 \right)&= \displaystyle \int \limits _{-\infty }^{\kappa _1} {\frac{1}{ \sqrt{2 \pi }}} e^{-{\frac{t^2}{2}}} dt =.5 - \displaystyle \int \limits _0^{|\kappa _1|} {\frac{1}{\sqrt{2 \pi }}} e^{-{\frac{t^2}{2}}} dt =.5 - {\frac{1}{2}} {\frac{2}{\sqrt{\pi }}} \displaystyle \int \limits _0^{\frac{|\kappa _1|}{\sqrt{2}}} e^{-{z^2}} dz \\&= .5 - \mathrm{erf} \left( {\frac{ |\kappa _1|}{2}} \right) \left( {\frac{1}{2}} \right) \end{aligned}$$

With the Taylor expansion of the erf function, we get

$$\begin{aligned}&\Phi _1\left( \kappa _1; 0, 1 \right)=.5 - {\frac{1}{\sqrt{\pi }}} \sum _{n=0}^{\infty } {\frac{ \left( -1 \right)^n \left( | \kappa _1 | / \sqrt{2} \right)^{2n+1}}{ n! (2n+1)}}\\&\Phi _1\left( \kappa _1 ; 0, 1\right) =.5 - {\frac{1}{\sqrt{\pi }}} \left( {\frac{ |\kappa _1|}{\sqrt{2}}} - {\frac{ |\kappa _1|^3}{2^{1.5} \cdot 3}} + {\frac{ |\kappa _1|^5}{2^{2.5} \cdot 10}} - \cdots \right)\\&\Longrightarrow \phi _1 \left((\kappa _1 + \delta _1); 0, 1 \right) - \Phi _1 \left( \kappa _1; 0, 1 \right) \\&\quad = {\frac{1}{\sqrt{\pi }}} \left( {\frac{ | \kappa _1|}{2^{.5}}} - \frac{|\kappa _1+\delta _{1}|}{2^{.5}} - {\frac{ |\kappa _1|^3}{2^{1.5} \cdot 3}} + {\frac{ | \kappa _1 + \delta _1 |^3}{2^{1.5} \cdot 3}} + {\frac{ | \kappa _1 |^5}{ 2^{2.5} \cdot 10}} - {\frac{ | \kappa _1 + \delta _1|^5}{2^{2.5} \cdot 10}} + \cdots \right) \\&\quad ={\frac{1}{\sqrt{\pi }}} \left( {\frac{ |\kappa _1| - | \kappa _1 + \delta _1|}{2^{.5}}} - {\frac{ ( | \kappa _1 |^3 - | \kappa _1 + \delta _1|^3)}{2^{1.5} \cdot 3}}+ {\frac{ ( | \kappa _1|^5 - | \kappa _1 + \delta _1|^5)}{2^{2.5} \cdot 10}} - \cdots \right) \\&\quad \le {\frac{1}{\sqrt{\pi }}} \left( |\delta _1| + ( | \kappa _1 |^3 - | \kappa _1 + \delta _1|^3) + ( | \kappa _1|^5 - | \kappa _1 + \delta _1|^5) + \cdots \right). \end{aligned}$$

Now observe that $|\kappa _1|^j - |\kappa _1 + \delta _1|^j \!=\! |\kappa _1|^j - \left( | \kappa _1| - \delta _1 \right)^j \!=\! - \sum _{k=1}^j \genfrac(){0.0pt}{}{j}{k} | \kappa _1|^{j-k} (-\delta _1)^k$, in which case

$$\begin{aligned} \left| \Phi _1 \left( \left( \kappa _1 + \delta _1 \right); 0, 1 \right) - \Phi _1 \left( \kappa _1; 0, 1 \right) \right|&\le {\frac{1}{\sqrt{\pi }}} \left( {\delta _1} + \sum _{k=1}^3 \genfrac(){0.0pt}{}{3}{k} | \kappa _1|^{3-k} |\delta _1|^k \right.\\&\left.+ \sum _{k=1}^5 \genfrac(){0.0pt}{}{5}{k} | \kappa _1|^{5-k} | \delta _1|^k + \cdots \right) \\ \Longrightarrow {\frac{ \left| \Phi _1 \left( \left( \kappa _1 + \delta _1 \right); 0, 1 \right) - \Phi \left( \kappa _1; 0, 1 \right) \right| }{ \delta _1^{1-\alpha }}}&\le {\frac{1}{\sqrt{\pi }}} \left( \delta _1^{\alpha } + \sum _{k=1}^3 \genfrac(){0.0pt}{}{3}{k} | \kappa _1|^{3-k} |\delta _1|^{k+\alpha -1} \right.\\&\left.+ \sum _{k=1}^5 \genfrac(){0.0pt}{}{5}{k} | \kappa _1|^{5-k} | \delta _1|^{k+\alpha -1} + \cdots \right), \end{aligned}$$

and the value of the right goes to 0 as $\delta _1 \searrow 0$. Thus, $\Phi _1 \left( \kappa _1 + \delta _1; 0, 1 \right) - \Phi _1 \left( \kappa _1; 0, 1 \right) = u_1 \left( \delta _1 \right)$ where $u_1 = o \left( \delta _1^{1-\alpha } \right)$ as $\delta _1 \searrow 0$. $\square$

Result 2

Let $\gamma \in (0,1)$. $\Phi _1 \left( \left( \kappa _2 + \delta _2 \right); 0, 1 \right) - 1 = u_2 \left( \delta _2 \right)$ where $u_2 \left( \delta _2 \right) = o \left( e^{-\delta _2 \gamma } \right)$ as $\delta _2 \longrightarrow + \infty$.

Proof

First observe that

$$\begin{aligned} \left| \Phi _1 \left( \left( \kappa _2 + \delta _2 \right); 0, 1 \right) - 1 \right| = \displaystyle \int \limits _{\kappa _2 + \delta _2}^{+\infty } {\frac{1}{\sqrt{2 \pi }}} e^{-{\frac{t^2}{2}}} dt. \end{aligned}$$

As $\delta _2$ gets larger, eventually $\kappa _2 + \delta _2$ is greater than 2, in which case

$$\begin{aligned} \int \limits _{\kappa _2 + \delta _2}^{\infty } {\frac{1}{\sqrt{2 \pi }}} e^{-{\frac{t^2}{2}}} dt&\le \displaystyle \int \limits _{\kappa _2 + \delta _2}^{\infty } {\frac{1}{\sqrt{2 \pi }}} e^{-t}dt \\ \Longrightarrow \left| \Phi _1 \left( \left( \kappa _2 + \delta _2 \right); 0, 1 \right) - 1 \right|&\le \left. -{\frac{1}{\sqrt{2 \pi }}} e^{-t} \right|_{\kappa _2 + \delta _2}^{+\infty } = {\frac{1}{\sqrt{ 2 \pi }}} e^{-(\kappa _2 + \delta _2)} \\ {\frac{ \left| \Phi _1 \left( \left( \kappa _2 + \delta _2 \right); 0, 1 \right) - 1 \right|}{ e^{-\delta _2 \gamma }}}&\le {\frac{1}{\sqrt{2 \pi }}} e^{-\kappa _2} e^{-\delta _2 + \delta _2 \gamma } = {\frac{1}{\sqrt{2 \pi }}} e^{-\kappa _2} e^{-\delta _2 \left( 1 - \gamma \right)}, \end{aligned}$$

and, when $\gamma \in (0,1)$, the right side of the last equation goes to 0 as $\delta _2 \longrightarrow + \infty$. $\square$

Result 3

Let $\alpha \in (0,1)$ and $\gamma \in (0,1)$. Then ${\frac{ \Phi _1 \left( \left( \kappa _1 + \delta _1 \right); 0, 1 \right)}{ \Phi _1 \left( \left( \kappa _2 + \delta _2 \right); 0, 1 \right)}} - \Phi _1 \left( \kappa _1; 0, 1 \right) = v_1 \left( \delta _1 \right) + v_2 \left( \delta _2 \right)$, where $v_1 \left( \delta _1 \right) = o \left( \delta _1^{1 - \alpha } \right)$ as $\delta _1 \searrow 0$. and $v_2 \left( \delta _2 \right) = o \left( e^{-{\delta _2 \gamma }} \right)$ as $\delta _2 \longrightarrow \infty$.

Proof

From Results 1 and 2, we can say that

$$\begin{aligned} {\frac{ \Phi _1 \left( \left( \kappa _1 + \delta _1 \right); 0, 1 \right)}{ \Phi _1 \left( \left( \kappa _2 + \delta _2 \right); 0, 1 \right)}} - \Phi _1 \left( \kappa _1; 0, 1 \right) = {\frac{ \Phi _1 \left( \kappa _1; 0, 1 \right) + u_1 \left( \delta _1 \right)}{ 1 + u_2 \left( \delta _2 \right) }} - \Phi _1 \left( \kappa _1; 0, 1 \right), \end{aligned}$$

where $u_1 \left( \delta _1 \right) = o \left( \delta _1^{1 - \alpha } \right)$ as $\delta _1 \searrow 0$, and $u_2 \left( \delta _2 \right) = o \left( e^{-{\delta _2 \gamma }} \right)$ as $\delta _2 \longrightarrow \infty$. Note that

$$\begin{aligned} {\frac{ \Phi _1 \left( \kappa _1; 0, 1 \right) + u_1 \left( \delta _1 \right)}{ 1 + u_2 \left( \delta _2 \right) }} - \Phi _1 \left( \kappa _1; 0, 1 \right)&= {\frac{ u_1 \left( \delta _1 \right) - \Phi _1 \left( \kappa _1; 0, 1 \right) u_2 \left( \delta _2 \right)}{ 1 + u_2 \left( \delta _2 \right)}}\\&\le 2 u_1 \left( \delta _1 \right) + 2 \Phi \left( \kappa _1; 0, 1 \right) u_2 \left( \delta _2 \right). \end{aligned}$$

The last inequality holds because $1 + u_2 \left( \delta _2 \right) = \Phi _1 \left( \kappa _2 + \delta _2; 0, 1 \right) \ge .5$ when $\delta _2$ is sufficiently large. This implies that

$$\begin{aligned} \left| {\frac{ \Phi _1 \left( \kappa _1 + \delta _1; 0, 1 \right)}{ \Phi _1 \left( \kappa _2 + \delta _2; 0, 1 \right)}} - \Phi _1 \left( \kappa _1; 0, 1 \right) \right| \le v_1 \left( \delta _1 \right) + v_2 \left( \delta _2 \right) \end{aligned}$$

where $v_1(\delta _1) = 2 u_1 \left( \delta _1 \right)$ and $v_2 \left( \delta _2 \right) = -2 \Phi _1 \left( \kappa _1; 0, 1 \right) u_2 \left( \delta _2 \right)$, and since $v_1 \left( \delta _1 \right)$ and $v_2 \left( \delta _2 \right)$ are multiples of $u_1 \left( \delta _1 \right)$ and $u_2 \left( \delta _2 \right)$, equivalent asymptotic results hold. $\square$

Proof of Corollary

From the steps given in the Proof of Theorem 3, it is clear that by setting $\mathbf{R} = r \mathbf{H}$, where $r = b \left( \delta _1, \delta _2 \right)$ and $\xi = \chi \left( \delta _1, \delta _2 \right)$, that

$$\begin{aligned}&\sup _{ \mathbf{y} \in \mathbb{R }_+^d} \left\{ t \left( \mathbf{y}; \beta ^*, \xi , \mathbf{R}, \Omega ^*, \mathcal{A }^*, l^* \right) \right\}&\quad = \left. \Phi _1 \left( g \left( \mu , \Sigma \right) + \delta _1; 0, 1 \right) \right. \left( p_\mathrm{SAR} \left( \mu , \Sigma \right) \Phi _1 \left( \left| g \left( \mu , \Sigma \right) \right| + \delta _2; 0, 1 \right) \right), \end{aligned}$$

and from Result 3 above, it follows that

$$\begin{aligned}&\left. \Phi _1 \left( g \left( \mu , \Sigma \right) + \delta _1; 0, 1 \right) \right. \left( p_\mathrm{SAR} \left( \mu , \Sigma \right) \Phi _1 \left( \left| g \left( \mu , \Sigma \right) \right|+ \delta _2; 0, 1 \right) \right)\\&\quad - \left. \Phi _1 \left( g \left( \mu , \Sigma \right); 0, 1 \right) \right. p_\mathrm{SAR} \left( \mu , \Sigma \right) = f_1 \left( \delta _1 \right) + f_2 \left( \delta _2 \right) \end{aligned}$$

where $f_1 \left( \delta _1 \right) = o \left( \delta _1^{1 - \alpha } \right)$ with $\alpha \in (0,1)$ as $\delta _1 \searrow 0$, and $f_2 \left( \delta _2 \right) = 0 \left( e^{-\delta _2 \gamma } \right)$ with $\gamma \in (0,1)$ as $\delta _2 \longrightarrow \infty$. $\square$

1.9 Website with R Code

The website http://carstenbotts.com/wp-content/uploads/2011/10/TruncNormalSampler21.txt contains the R function trunc_normal_smplr.