Accelerating MCMC by Rare Intermittent Resets

Abstract

We propose a scheme for accelerating Markov Chain Monte Carlo by introducing random resets that become increasingly rare in a precise sense. We show that this still leads to the desired asymptotic average and establish an associated concentration bound. We show by numerical experiments that this scheme can be used to advantage in order to accelerate convergence by a judicious choice of the resetting mechanism.

V. S. Borkar—Work of this author was supported in part by a S. S. Bhatnagar Fellowship from the Council of Scientific and Industrial Research, Government of India.

S. Chaudhuri—Now with the Dept. of EECS, Uni. of California, Berkeley, Cory Hall, Berkeley 94720, CA.

A Appendix

Here we provide the proofs of Theorem 1 and 2 in Sect. 2.

Proof of Theorem 1:

By the martingale law of large numbers, we have

$$\zeta _n(j) := \frac{1}{n}\sum _{m=1}^n(I\{X_m=j\} - \sum _iq_{m-1}(j|i)I\{X_{m-1} = i\}) \rightarrow 0$$

a.s. $\forall \ j \in S$. Combining this with (1), we have

$$\frac{1}{n}\sum _{m=1}^n(I\{X_m=j\} - \sum _ip(j|i)I\{X_{m-1} = i\}) \rightarrow 0$$

a.s. Hence any limit point $\nu ^*$ of $\{\nu _n\}$ as $n\rightarrow \infty$ satisfies

$$\nu ^*(j) = \sum _i\nu ^*(i)p(j|i) \ \forall \ j \in S.$$

This implies (2).    $\square$

Proof of Theorem 2:

By (3), we have,

$$\begin{aligned} \Vert \nu _n - \pi \Vert \le C\Vert \nu _n - \nu _nP\Vert . \end{aligned}$$

(4)

Note that

Bounding each term, we get

$$ |\nu _n(j) - \sum _ip(j|i)\nu _n(i)| \le |\zeta _n(j)| + \frac{1}{n} + \frac{\eta (n)}{n}.$$

Hence

$$\begin{aligned} \Vert \nu _n - \nu _nP\Vert \le \Vert \zeta _n\Vert + \sqrt{s}\left( \frac{\eta (n) + 1}{n}\right) , \end{aligned}$$

and

$$\Vert \nu _n - \pi \Vert \le C\left[ \Vert \zeta _n\Vert + \sqrt{s}\left( \frac{\eta (n) + 1}{n}\right) \right] .$$

For $j \in S$ and $\bar{x} := [x_0, \cdots , x_n]$, let

$$f(\bar{x}) := \frac{1}{n}\sum _{m=1}^n(I\{x_m = j\} - \sum _ip(j|i)I\{x_{m-1} = i\}).$$

Defining $\bar{y}$ analogously, note that $f(\bar{x}) - f(\bar{y}) \le \frac{2}{n}\sum _{m=1}^n I\{ x_i \ne y_i\}$. By Corollary 2.10 of [20], we then have, for any $\delta > 0$,

$$P\left( |\zeta _n(j)| < \frac{\delta }{C\sqrt{s}}\right) \ \ge \ 1 - 2e^{-\frac{\delta ^2n}{2sC^2\tau _{min}}}.$$

This proves the claim.    $\square$