Accelerating sequential Monte Carlo with surrogate likelihoods

Abstract

Delayed-acceptance is a technique for reducing computational effort for Bayesian models with expensive likelihoods. Using a delayed-acceptance kernel for Markov chain Monte Carlo can reduce the number of expensive likelihoods evaluations required to approximate a posterior expectation. Delayed-acceptance uses a surrogate, or approximate, likelihood to avoid evaluation of the expensive likelihood when possible. Within the sequential Monte Carlo framework, we utilise the history of the sampler to adaptively tune the surrogate likelihood to yield better approximations of the expensive likelihood and use a surrogate first annealing schedule to further increase computational efficiency. Moreover, we propose a framework for optimising computation time whilst avoiding particle degeneracy, which encapsulates existing strategies in the literature. Overall, we develop a novel algorithm for computationally efficient SMC with expensive likelihood functions. The method is applied to static Bayesian models, which we demonstrate on toy and real examples.

A Appendix

1.1 A.1 One-move diversification

An alternative to ESJD diversification can be found in a simple method from South et al. (2019) for choosing the number of MCMC runs—the basis of which is from Drovandi and Pettitt (2011). In this regime, the number of cycles, k, is chosen so that each particle moves at least once in k iterations. A move occurs when a proposal is accepted using an MH kernel. We will refer to this criterion as one-move diversification. For a fixed scaling parameter h, one-move diversification uses the MH acceptance rates to determine the average number of MCMC cycles required for at least one proposal per particle to be accepted. More generally, one could require a higher minimum number of moves, but for simplicity we just consider the case of at least one move.

This section will consider a single mutation step of the SMC algorithm, consisting of multiple cycles of the MCMC kernel, indexed by $s \in \{1,2, \ldots , k\}$. Assuming the probability of moving (or acceptance, $\alpha ^{(1)}$) is equal across cycles, the average probability (across the tempered posterior distribution) that at least one is accepted in a sequence of k cycles, $\alpha ^{(k)}$, is

$$\begin{aligned} \alpha ^{(k)} = 1 - \left( 1-\alpha ^{(1)}\right) ^{k} \end{aligned}$$

(23)

for $k \in \{1,2,\ldots \}$. A pilot mutation step can be used to estimate the average acceptance rate across the particles in a single step, $\widehat{\alpha }^{(1)}$. We can then find k such that $\alpha ^{(k)} \ge p_{\min }$ for some threshold $0< p_{\min } < 1$. The formula to choose the total number of iterations, k, is

$$\begin{aligned} k = \left\lceil \frac{\log (1 - p_{\min })}{\log \left( 1-\widehat{\alpha }^{(1)}\right) } \right\rceil \end{aligned}$$

(24)

where $\widehat{\alpha }^{(1)}$ is the estimated acceptance rate from the pilot run of the MH kernel.

To frame this in the context of optimising computation time, note that the underlying criterion is to ensure a sufficient number of mutation steps are taken so that the probability of at least one move is greater than $p_{\min }$ for a given particle.

If we denote a move by $\Vert \varvec{\theta }_{s} - \varvec{\theta }_{s-1}\Vert _{0}$, where $\Vert \cdot \Vert _{0}$ is the zero “norm”, the corresponding diversification criterion can be expressed with

$$\begin{aligned} D(k, {\varvec{\phi }}) = \mathsf {P}\left( \sum _{s=1}^{k} \left\| \varvec{\theta }_{s} - \varvec{\theta }_{s-1} \right\| _{0} \ge 1 \right) \quad \text {and} \quad d= p_{\min } \end{aligned}$$

(25)

where the probability is taken with respect to the acceptance rates of the Metropolis-Hastings steps. Of course, this expression for $D(k, {\varvec{\phi }})$ is a more general version of (23) and coincides if we assume the probability of acceptance is equal across particle locations and MCMC iterations, s. We emphasise the norm notation to draw a comparison to the jumping distance diversification in Sect. 4.1. That is, we can write $P(k, {\varvec{\phi }})$ in (8) as

$$\begin{aligned} P(k, {\varvec{\phi }}) = \mathsf {P}\left( \mathsf {E}\left[ \sum _{s=1}^{k} \left\| \varvec{\theta }_{s} - \varvec{\theta }_{s-1} \right\| ^{2}_{\Sigma } \right] \ge d \right) \end{aligned}$$

where the expectation is with respect to the random acceptance over k cycles of the MH kernel. Written in this way, $P(k, {\varvec{\phi }})$ elicits an interesting comparison to (25); it is a change of “norm” when moving between one-move and jumping distance diversification.

Now we wish to use one-move criterion to select the tuning parameters. If we use different proposal kernel tuning parameters for particular subsets of particles, the acceptance rate will be a function of those parameters, so we write $\alpha ^{(k)}$ as $\alpha ^{(k)}({\varvec{\phi }})$. The optimisation stated in (3) can be simplified as stated in Proposition 3.

Proposition 3

Assume the cost function is $C(k, {\varvec{\phi }}) = k \times L_{F}$, approximating the cost of a standard Metropolis-Hastings step, and $D(k,{\varvec{\phi }}) = \alpha ^{(k)}({\varvec{\phi }})$. The latter also corresponds to (25) assuming a uniform acceptance rate across the support of $\varvec{\theta }$. Then the general problem in (3) is equivalent to

$$\begin{aligned} \mathop {\hbox {arg min}}\limits _{{\varvec{\phi }} \in \Phi }~ \frac{\log (1 - p_{\min })}{\log \left( 1-\alpha ^{(1)}({\varvec{\phi }})\right) } \end{aligned}$$

(26)

where the general diversification threshold, d, has been replaced by the probability $p_{\min }$.

Proposition 3 is the solution to choosing the best tuning parameters with the one-move criterion and MH-cost. It closely connects to the original decision for k without tuning parameters (24). A proof of Proposition 3 is in Appendix A.4.

In general, we expect the tuning criterion in Proposition 3 to perform poorly. This can be demonstrated by a simple, but highly applicable, example. If the tuning parameter is the step size for an MH mutation, i.e. ${\varvec{\phi }} = [h]$, then we would expect the acceptance probability, $\alpha ^{(1)}({\varvec{\phi }})$, to be monotone decreasing in h. Hence, the minimisation in (26) will prefer the minimum step size possible, which will ensure at least one move with the minimal computation cost. In other words, the diversification criterion in (25) is only concerned with the probability of at least one move, not the quality of this move.

Due to the aforementioned shortcoming, a diversification criterion that also measures the quality of the mutation is desirable. For this reason, we focus on the ESJD as a criterion in the main text.

Table 4 Median (80% interval) multiplicative improvement of computation time relative to MH-SMC (using median tuning method) for simulation study

1.2 A.2 Proof of Proposition 1

Let $\mathsf {D}_m = \left\{ (k,{\varvec{\phi }}) \in \mathbb {Z}^{+} \times \Phi :D(k,{\varvec{\phi }}) \ge d\right\}$ with

$$\begin{aligned} D(k,{\varvec{\phi }}) = \mathrm {median}\left\{ \sum _{s=1}^{k} J_{s}({\varvec{\phi }}) \right\} . \end{aligned}$$

Using the multivariate Jensen inequality for medians in (Merkle (2010), Theorem 5.2), we have that

$$\begin{aligned} \sum _{s=1}^{k}\mathrm {median}\left\{ J_{s}({\varvec{\phi }}) \right\} \le \mathrm {median}\left\{ \sum _{s=1}^{k} J_{s}({\varvec{\phi }}) \right\} \end{aligned}$$

and assuming the jumping distances are iid, for a given ${\phi }$, we further reduce this to

$$\begin{aligned} k \times \mathrm {median}\left\{ J_{1}({\varvec{\phi }}) \right\} \le \mathrm {median}\left\{ \sum _{s=1}^{k} J_{s}({\varvec{\phi }}) \right\} . \end{aligned}$$

(27)

We can define the set $\tilde{\mathsf {D}}_m$ as

$$\begin{aligned} \tilde{\mathsf {D}}_m= & {} \left\{ (k,{\varvec{\phi }}) \in \mathbb {Z}^{+} \times \Phi :\tilde{D}(k,{\varvec{\phi }}) \ge d\right\} \\&\tilde{D}(k, {\varvec{\phi }}) =k \times \mathrm {median}\left\{ J_{1}({\varvec{\phi }}) \right\} \end{aligned}$$

then we see from (27) that $\tilde{\mathsf {D}}_m \subseteq \mathsf {D}_m$.

1.3 A.3 Proof of Proposition 2

Under the MH cost function, $C(k, {\varvec{\phi }}) = k \times L_{F}$, and approximate ESJD diversification criterion,

$$\begin{aligned} \tilde{\mathsf {D}}_m =&\left\{ (k,{\varvec{\phi }}) \in \mathbb {Z}^{+} \times \Phi : \tilde{D}(k,{\varvec{\phi }}) \ge d\right\} \\ \text {where }&\tilde{D}(k,{\varvec{\phi }}) = k \times \mathrm {median}\left\{ J_{1}({\varvec{\phi }}) \right\} , \end{aligned}$$

the inequality for the diversification criterion can be rearranged into

$$\begin{aligned} k \ge \frac{d}{\mathrm {median}\left\{ J_{1}({\varvec{\phi }}) \right\} }. \end{aligned}$$

Under this restriction, note that

$$\begin{aligned} C(k, {\varvec{\phi }}) \ge L_{F} \times \frac{d}{\mathrm {median}\left\{ J_{1}({\varvec{\phi }}) \right\} }. \end{aligned}$$

so under these conditions, the general problem in (3) is equivalent to

$$\begin{aligned} \mathop {\hbox {arg min}}\limits _{{\varvec{\phi }} \in \Phi }~ \left( \mathrm {median}\left\{ J_{1}({\varvec{\phi }}) \right\} \right) ^{-1} \equiv \mathop {\hbox {arg max}}\limits _{\varvec{\phi } \in \Phi }~ \mathrm {median}\left\{ J_{1}({\varvec{\phi }}) \right\} . \end{aligned}$$

1.4 A.4 Proof of Proposition 3

Under the MH cost function, $C(k, {\phi }) = k \times L_{F}$, and one-move diversification criterion,

$$\begin{aligned} \mathsf {D} =&\left\{ (k,{\varvec{\phi }}) \in \mathbb {Z}^{+} \times \Phi : \alpha ^{(k)}({\varvec{\phi }}) \ge p_{\min }\right\} \\ \text {where }&\alpha ^{(k)}({\varvec{\phi }}) = 1 - (1-\alpha ^{(1)}({\varvec{\phi }}))^{k}, \end{aligned}$$

the inequality for the diversification criterion can be rearranged into

$$\begin{aligned} k \ge \frac{\log (1 - p_{\min })}{\log (1-\alpha ^{(1)}({\varvec{\phi }}))}. \end{aligned}$$

Under this restriction, note that

$$\begin{aligned} C(k, {\varvec{\phi }}) \ge L_{F} \times \frac{\log (1 - p_{\min })}{\log (1-\alpha ^{(1)}({\varvec{\phi }}))}. \end{aligned}$$

so under these conditions, the general problem in (3) is equivalent to

$$\begin{aligned} \mathop {\hbox {arg min}}\limits _{{\varvec{\phi }} \in \Phi }~ \frac{\log (1 - p_{\min })}{\log (1-\alpha ^{(1)}({\varvec{\phi }}))}. \end{aligned}$$

1.5 A.5 Additional figures and tables from simulation

Fig. 3

Example of posterior $\varvec{\beta }$ densities from three SMC algorithms (example 1)

Fig. 4

Example of posterior $\varvec{\beta }$ densities from three SMC algorithms (example 2)

Fig. 5

Example of posterior $\varvec{\beta }$ densities from three SMC algorithms (example 3)

Fig. 6

Example of posterior $\varvec{\beta }$ densities from three SMC algorithms (example 4)

Fig. 7

Posteriors of $\phi _{1}$ from 10 replicates of the four SMC algorithms. The SMC algorithms using only the surrogate likelihood (approx) are annealed to $\gamma _{T} \in \{0.01,1\}$ and $\gamma _{T} = 0.01$. The latter of which is the initial particle set for the surrogate first annealing procedure

Fig. 8

Posteriors of $\theta _{1}$ from 10 replicates of the four SMC algorithms. The SMC algorithms using only the surrogate likelihood (approx) are annealed to $\gamma _{T} \in \{0.01,1\}$ and $\gamma _{T} = 0.01$. The latter of which is the initial particle set for the surrogate first annealing procedure

1.6 A.6 Additional figures from ARFIMA model example

Fig. 9

Posteriors of d from 10 replicates of the four SMC algorithms. The SMC algorithms using only the surrogate likelihood (approx) are annealed to $\gamma _{T} \in \{0.01,1\}$ and $\gamma _{T} = 0.01$. The latter of which is the initial particle set for the surrogate first annealing procedure

Fig. 10

Posteriors of $\sigma ^{2}$ from 10 replicates of the four SMC algorithms. The SMC algorithms using only the surrogate likelihood (approx) are annealed to $\gamma _{T} \in \{0.01,1\}$ and $\gamma _{T} = 0.01$. The latter of which is the initial particle set for the surrogate first annealing procedure