An improved exact sampling algorithm for the standard normal distribution

Abstract

In 2016, Karney proposed an exact sampling algorithm for the standard normal distribution. In this paper, we study the computational complexity of this algorithm under the random deviate model. Specifically, Karney’s algorithm requires the access to an infinite sequence of independently and uniformly random deviates over the range (0, 1). We give a theoretical estimate of the expected number of uniform deviates used by this algorithm until it completes, and present an improved algorithm with lower uniform deviate consumption. The experimental results also shows that our improved algorithm has better performance than Karney’s algorithm.

Appendix

1.1 The Proof of Proposition 1

Proof

Let $p_1=1/\sqrt{e}$ and $p_0=1-p_1$. Assume that $k\ge 0$ is an integer which is generated in step 1. Then, the expected number of Bernoulli random values from ${\mathcal {B}}_{p_1}$ used by step 1 conforms to a geometric distribution. Its expected value is

$$\begin{aligned} 1+p_1+p_1^2+\ldots =\sum _{k=0}^\infty p_1^k=1/p_0. \end{aligned}$$

In step 2 for a given $k\ge 0$, the expected number of Bernoulli random values used by step 2 is

$$\begin{aligned} \sum _{j=1}^{k(k-1)-1}(p_1^{j-1}p_0)j+p_1^{k(k-1)-1}k(k-1)=\sum _{j=0}^{k(k-1)-1}p_1^j=\frac{1-p_1^{k(k-1)}}{p_0} \end{aligned}$$

In particular, if $k=0$ or $k=1$, then k is directly accepted in step 2. Then, on average, the expected number of Bernoulli random values used by step 2 is

$$\begin{aligned} \sum _{k=2}^\infty (p_1^kp_0)\frac{1-p_1^{k(k-1)}}{p_0}=\sum _{k=2}^\infty (p_1^k-p_1^{k^2})=\sum _{k=0}^\infty (p_1^k-p_1^{k^2})=1/p_0-G, \end{aligned}$$

where $G=\sum _{k=0}^\infty p_1^{k^2}$. Therefore, the expected number of Bernoulli random values from ${\mathcal {B}}_{p_1}$ used by Algorithm 1 for executing both steps 1 and 2 can be given by

$$\begin{aligned} 1/p_0+1/p_0-G=2/p_0-G\approx 3.32967. \end{aligned}$$

Furthermore, the probability of Algorithm 1 not going back to step 1 in step 2 is $\sum _{k=0}^\infty p_1^{k^2}p_0\approx 0.689875$. So, the expected number of Bernoulli random values from ${\mathcal {B}}_{p_1}$ used by Algorithm 1 for successfully generating an integer from ${\mathcal {D}}_{{\mathbb {Z}}^+,1}$ is about $3.32967/0.689875\approx 4.82649$. $\square$

1.2 The Proof of Proposition 3

Lemma 1

For a given $k\ge 1$, the probability that Algorithm 3 restarts⁵n times can be given by

$$\begin{aligned} \left( \frac{x}{m}+\frac{2k}{m}\right) ^n\left( \frac{x^n}{n!}\right) , \end{aligned}$$

where $m=2k+2$ and n is a positive integer.

Proof

(of Lemma 1) For a given $k\ge 1$, there are two cases in which the algorithm goes to step 2: (1) $z<y$ and $f=1$ with probability $x\cdot (2k/m)$; (2) $z<y$, $f=0$ and $r<x$ with probability $x\cdot (x/m)$. Thus, the probability that the algorithm goes to step 2 is equal to

$$\begin{aligned} x\frac{x}{m}+x\cdot \frac{2k}{m}=x\left( \frac{x}{m}+\frac{2k}{m}\right) . \end{aligned}$$

After going back to step 2 one time, the probability that the algorithm goes to step 2 once again is equal to

$$\begin{aligned} \left( x\left( \frac{x}{m}+\frac{2k}{m}\right) \right) \frac{x}{2}\cdot \frac{x}{m}+\left( x\left( \frac{x}{m}+\frac{2k}{m}\right) \right) \frac{x}{2}\cdot \frac{2k}{m}=\frac{x^2}{2}\left( \frac{x}{m}+\frac{2k}{m}\right) ^2. \end{aligned}$$

Generally, the probability that the algorithm restarts n times ($n\ge 1$) can be given by

$$\begin{aligned} \left( \frac{x}{m}+\frac{2k}{m}\right) ^n\left( \frac{x^n}{n!}\right) , \end{aligned}$$

where ${x^n}/{n!}$ is the probability that a set of uniform deviates $z_1, z_2, \ldots z_n$ over the range (0, 1) satisfy $x>z_1>z_2>\cdots >z_n$. The proof can be completed by using the mathematical induction on n. $\square$

Proof of Proposition 3

We can see that Algorithm 3 needs

$$\begin{aligned} 1+1+\left( \frac{x}{m}\right) \cdot 1=2+\left( \frac{x}{m}\right) \end{aligned}$$

uniform deviates on average every time it goes back to step 2, where the first deviate is for step 2, the second one is used by the random selector C(m), and the third one is possibly required when $f=0$ in Algorithm 3. By Lemma 1, the probability of restarting $n-1$ times ($n\ge 1$) is

$$\begin{aligned} \left( \frac{x}{m}+\frac{2k}{m}\right) ^{n-1}\left( \frac{x^{n-1}}{(n-1)!}\right) . \end{aligned}$$

By the binomial theorem, for a given $n\ge 1$, Algorithm 3 uses

$$\begin{aligned} \sum _{i=0}^{n-1}\left( \begin{array}{c} n-1\\ i\end{array}\right) \left( \frac{x}{m}\right) ^i\left( \frac{2k}{m}\right) ^{n-1-i}(2n-2+i) \end{aligned}$$

uniform deviates on average if it restarts $n-1$ times.⁶ For a given $k\ge 1$, there are three cases in which the algorithm goes to step 6 and returns the result before it goes to step 2: (1) $z>y$ with probability $(1-x)$ at a cost of one new uniform deviate; (2) $f=-1$ with probability x(1/m) at a cost of two new uniform deviates; (3) $f=0$ and $r>x$ with probability $x(1/m)(1-x)$ at a cost of three new uniform deviates. Then, after restarting $n-1$ times ($n\ge 1$), there are three cases in which the algorithm goes to step 6 and returns the result before it goes back to step 2 once again.

• (1) $z>y$ with probability

  $$\begin{aligned} \left( \frac{x}{m}+\frac{2k}{m}\right) ^{n-1}\left( \frac{x^{n-1}}{(n-1)!}-\frac{x^n}{n!}\right) , \end{aligned}$$

  and at a cost of

  $$\begin{aligned} \sum _{n=1}^{\infty }\sum _{i=0}^{n-1}\left( \begin{array}{c}n-1\\ i\end{array}\right) \left( \frac{x}{m}\right) ^i\left( \frac{2k}{m}\right) ^{n-1-i}(2n-1+i) \left( \frac{x^{n-1}}{(n-1)!}-\frac{x^n}{n!}\right) \end{aligned}$$

  (1)

  uniform deviates, where ${x^{n-1}}/{(n-1)!}-{x^n}/{n!}$ is the probability that the length of the longest decreasing sequence $x>z_1>z_2>\ldots >z_n$ is n.

• (2) $f=-1$ with probability

  $$\begin{aligned} \left( \frac{x}{m}+\frac{2k}{m}\right) ^{n-1}\left( \frac{x^n}{n!}\right) \left( \frac{1}{m}\right) , \end{aligned}$$

  and at a cost of

  $$\begin{aligned} \sum _{n=1}^{\infty }\sum _{i=0}^{n-1}\left( \begin{array}{c}n-1\\ i\end{array}\right) \left( \frac{x}{m}\right) ^i\left( \frac{2k}{m}\right) ^{n-1-i}(2n+i) \left( \frac{x^n}{n!}\right) \left( \frac{1}{m}\right) \end{aligned}$$

  (2)

  uniform deviates.

• (3) $f=0$ and $r>x$ with probability

  $$\begin{aligned} \left( \frac{x}{m}+\frac{2k}{m}\right) ^{n-1}\left( \frac{x^n}{n!}\right) \left( \frac{1}{m}\right) (1-x), \end{aligned}$$

  and at a cost of

  $$\begin{aligned} \sum _{n=1}^{\infty }\sum _{i=0}^{n-1}\left( \begin{array}{c}n-1\\ i\end{array}\right) \left( \frac{x}{m}\right) ^i\left( \frac{2k}{m}\right) ^{n-1-i}(2n+1+i) \left( \frac{x^n}{n!}\right) \left( \frac{1}{m}\right) (1-x) \end{aligned}$$

  (3)

  uniform deviates. Therefore, for a given $k\ge 1$, by combining the above three cases, i.e., summing Eqs. (1)–(3), we can obtain the expected number of uniform deviates used by Algorithm 3, which can be reduced to a function of x:

  $$\begin{aligned} \frac{(4k+x+3)\cdot \tau _k(x)-2k-3}{2k+x}, \end{aligned}$$

  where $\tau _k(x)=\exp (x\frac{2k+x}{2k+2})$. $\square$

1.3 The Proof of Proposition 4

Proof

If $k=0$, after restarting $n-1$ times ($n\ge 1$), there are three cases in which Algorithm 3 goes to step 6 and returns the result before it goes back to step 2 again.

• (1) $f=-1$ with probability

  $$\begin{aligned} \left( \frac{1}{2}\right) ^n\left( \frac{x^{n-1}}{(n-1)!}\right) x^{n-1}, \end{aligned}$$

  and at a cost of

  $$\begin{aligned} \sum _{n=1}^{\infty }\left( \frac{1}{2}\right) ^n\left( \frac{x^{n-1}}{(n-1)!}\right) x^{n-1}(2n-2) \end{aligned}$$

  (4)

  uniform deviates.

• (2) $z>y$ with probability

  $$\begin{aligned} \left( \frac{1}{2}\right) ^nx^{n-1}\left( \frac{x^{n-1}}{(n-1)!}-\frac{x^n}{n!}\right) , \end{aligned}$$

  and at a cost of

  $$\begin{aligned} \sum _{n=1}^{\infty }\left( \frac{1}{2}\right) ^nx^{n-1}\left( \frac{x^{n-1}}{(n-1)!}-\frac{x^n}{n!}\right) (2n-1) \end{aligned}$$

  (5)

  uniform deviates.

• (3) $f=0$ and $r>x$ with probability

  $$\begin{aligned} \left( \frac{1}{2}\right) ^nx^{n-1}\left( \frac{x^n}{n!}\right) (1-x), \end{aligned}$$

  and at a cost of

  $$\begin{aligned} \sum _{n=1}^{\infty }\left( \frac{1}{2}\right) ^nx^{n-1}\left( \frac{x^n}{n!}\right) (1-x)(2n) \end{aligned}$$

  (6)

  uniform deviates. Therefore, when $k=0$, by summing Eqs. (4)–(6), we have the expected number of uniform deviates used by Algorithm 3, which can be reduced to a function of x:

  $$\begin{aligned} \frac{(x+2)\cdot \tau _0(x)-2}{2x}, \end{aligned}$$

  where $\tau _0(x)=\exp (\frac{x^2}{2})$. $\square$