Delayed acceptance particle MCMC for exact inference in stochastic kinetic models

Abstract

Recently-proposed particle MCMC methods provide a flexible way of performing Bayesian inference for parameters governing stochastic kinetic models defined as Markov (jump) processes (MJPs). Each iteration of the scheme requires an estimate of the marginal likelihood calculated from the output of a sequential Monte Carlo scheme (also known as a particle filter). Consequently, the method can be extremely computationally intensive. We therefore aim to avoid most instances of the expensive likelihood calculation through use of a fast approximation. We consider two approximations: the chemical Langevin equation diffusion approximation (CLE) and the linear noise approximation (LNA). Either an estimate of the marginal likelihood under the CLE, or the tractable marginal likelihood under the LNA can be used to calculate a first step acceptance probability. Only if a proposal is accepted under the approximation do we then run a sequential Monte Carlo scheme to compute an estimate of the marginal likelihood under the true MJP and construct a second stage acceptance probability that permits exact (simulation based) inference for the MJP. We therefore avoid expensive calculations for proposals that are likely to be rejected. We illustrate the method by considering inference for parameters governing a Lotka–Volterra system, a model of gene expression and a simple epidemic process.

Appendix

Recall that $\mathbf {x}=\{x_{t}\,|\, 1\le t \le T\}$ denotes values of the latent MJP and $\mathbf {y}=\{y_{t}\,|\, t=1,2,\ldots ,T\}$ denotes the collection of (noisy) observations on the MJP at discrete times. In addition, we define $\mathbf {x}_{t}=\{x_{s}\,|\, t-1<s\le t\}$ and $\mathbf {y}_{t}=\{y_{s}\,|\, s=1,2,\ldots , t\}$.

1.1 PMMH scheme

The PMMH scheme has the following algorithmic form.

• 1. Initialisation, $i=0$,

  • (a) set $c^{(0)}$ arbitrarily and

  • (b) run an SMC scheme targeting $p(\mathbf {x}|\mathbf {y},c^{(0)})$, and let $\widehat{p}(\mathbf {y}|c^{(0)})$ denote the marginal likelihood estimate

• 2. For iteration $i\ge 1$,

  • (a) sample $c^{*}\sim q(\cdot | c^{(i-1)})$,

  • (b) run an SMC scheme targeting $p(\mathbf {x}|\mathbf {y},c^{*})$, and let $\widehat{p}(\mathbf {y}|c^{*})$ denote the marginal likelihood estimate,

  • (c) with probability min$\{1,A\}$ where

    $$\begin{aligned} A=\frac{\widehat{p}(\mathbf {y}|c^{*}) p(c^{*})}{\widehat{p}(\mathbf {y}|c^{(i-1)}) p(c^{(i-1)})} \times \frac{q(c^{(i-1)} | c^{*})}{q(c^{*} | c^{(i-1)})} \end{aligned}$$

    accept a move to $c^{*}$ otherwise store the current values

Note that the PMMH scheme can be used to sample the joint posterior $p(c,\mathbf {x}|\mathbf {y})$. Essentially, a proposal mechanism of the form $q(c^{*}|c)\widehat{p}(\mathbf {x}^{*}|\mathbf {y},c^{*})$, where $\widehat{p}(\mathbf {x}^{*}|\mathbf {y},c^{*})$ is an SMC approximation of $p(\mathbf {x}^{*}|\mathbf {y},c^{*})$, is used. The resulting MH acceptance ratio is as above. Full details of the PMMH scheme including a proof establishing that the method leaves the target $p(c,\mathbf {x}|\mathbf {y})$ invariant can be found in Andrieu et al. (2010).

1.2 SMC scheme

A sequential Monte Carlo estimate of the marginal likelihood $p(\mathbf {y}|c)$ under the MJP can be constructed using (for example) the bootstrap filter of Gordon et al. (1993). Algorithmically, we perform the following sequence of steps.

• 1. Initialisation.

  • (a) Generate a sample of size $N$, $\{x_{1}^{1},\ldots ,x_{1}^{N}\}$ from the initial density $p(x_{1})$.

  • (b) Assign each $x_{1}^{i}$ a (normalised) weight given by

    $$\begin{aligned} w_{1}^{i}=\frac{w_{1}^{*i}}{\sum _{i=1}^{N}w_{1}^{*i}}, \quad \text {where}\quad w_{1}^{*i}=p(y_{1}|x_{1}^{i},c)\,. \end{aligned}$$

  • (c) Construct and store the currently available estimate of marginal likelihood,

    $$\begin{aligned} \widehat{p}({y}_{1}|c) = \frac{1}{N}\sum _{i=1}^{N} w_{1}^{*i}\,. \end{aligned}$$

  • (d) Resample $N$ times with replacement from $\{x_{1}^{1},\ldots ,x_{1}^{N}\}$ with probabilities given by $\{w_{1}^{1},\ldots ,w_{1}^{N}\}$.

• 2. For times $t=1,2,\ldots ,T-1$,

  • (a) For $i=1,\ldots ,N$: draw $\mathbf {X}_{t+1}^{i}\sim p\big (\mathbf {x}_{t+1}|{x}_{t}^{i},c\big )$ using the Gillespie algorithm.

  • (b) Assign each $\mathbf {x}_{t+1}^{i}$ a (normalised) weight given by

    $$\begin{aligned} w_{t+1}^{i}=\frac{w_{t+1}^{*i}}{\sum _{i=1}^{N}w_{t+1}^{*i}}, \quad \text {where}\quad w_{t+1}^{*i}=p(y_{t+1}|x_{t+1}^{i},c)\, . \end{aligned}$$

  • (c) Construct and store the currently available estimate of marginal likelihood,

    $$\begin{aligned} \widehat{p}(\mathbf {y}_{t+1}|c)&= \widehat{p}(\mathbf {y}_{t}|c)\widehat{p}(y_{t+1}|\mathbf {y}_{t},c)\\&=\widehat{p}(\mathbf {y}_{t}|c)\frac{1}{N}\sum _{i=1}^{N} w_{t+1}^{*i}\,. \end{aligned}$$

  • (d) Resample $N$ times with replacement from $\{\mathbf {x}_{t+1}^{1},\ldots ,\mathbf {x}_{t+1}^{N}\}$ with probabilities given by $\{w_{t+1}^{1},\ldots ,w_{t+1}^{N}\}$.

1.3 Marginal likelihood under the linear noise approximation

Assume an observation regime of the form

$$\begin{aligned} Y_{t}=G'X_{t}+\varepsilon _{t}\,,\qquad \varepsilon _{t}\sim \text {N}\left( 0,\varSigma \right) \end{aligned}$$

where $G$ is a constant matrix of dimension $u\times p$ and $\varepsilon _{t}$ is a length-$p$ Gaussian random vector.

Now suppose that $X_{1}\sim N(a,C)$ a priori. The marginal likelihood under the LNA, $p_{a}(\mathbf {y}|c)$ can be obtained as follows.

• 1. Initialisation. Compute

  $$\begin{aligned} p_{a}(y_{1}|c)=\phi \left( y_{1}\,;\, G'a\,,\,G'CG+\varSigma \right) \end{aligned}$$

  where $\phi (\cdot \,;\,a\,,\,C)$ denotes the Gaussian density with mean vector $a$ and variance matrix $C$. The posterior at time $t=1$ is therefore $X_{1}|y_{1}\sim N(a_{1},C_{1})$ where

  $$\begin{aligned} a_{1}&= a+CG\left( G'CG+\varSigma \right) ^{-1}\left( y_{1}-G'a\right) \\ C_{1}&= C-CG\left( G'CG+\varSigma \right) ^{-1}G'C\,. \end{aligned}$$

• 2. For times $t=1,2,\ldots ,T-1$,

  • (a) Prior at $t+1$. Initialise the LNA with $z_{t}=a_{t}$, $m_{t}=0$ and $V_{t}=C_{t}$. Note that this implies $m_{s}=0$ for all $s>t$. Therefore, integrate the ODEs (6) and (10) forward to $t+1$ to obtain $z_{t+1}$ and $V_{t+1}$. Hence

    $$\begin{aligned} X_{t+1}|\mathbf {y}_{t}\sim N(z_{t+1},V_{t+1})\,. \end{aligned}$$

  • (b) One step forecast. Using the observation equation, we have that

    $$\begin{aligned} Y_{t+1}|\mathbf {y}_{t}\sim N\left( G'z_{t+1},G'V_{t+1}G+\varSigma \right) . \end{aligned}$$

    Compute

    $$\begin{aligned} p_{a}(\mathbf {y}_{t+1}|c)&=p_{a}(\mathbf {y}_{t}|c)p_{a}(y_{t+1}|\mathbf {y}_{t},c)\\&=p_{a}(\mathbf {y}_{t}|c)\,\phi \left( y_{t+1}\,;\, G'z_{t+1}\,,\,G'V_{t+1}G+\varSigma \right) . \end{aligned}$$

  • (c) Posterior at $t+1$. Combining the distributions in (a) and (b) gives the joint distribution of $X_{t+1}$ and $Y_{t+1}$ (conditional on $\mathbf {y}_{t}$ and $c$) as

    $$\begin{aligned} \left( \begin{array}{c} X_{t+1} \\ Y_{t+1} \end{array}\right) \sim N\left\{ \left( \begin{array}{c} z_{t+1} \\ G'z_{t+1} \end{array} \right) \,,\, \left( \begin{array}{cc} V_{t+1} &{} V_{t+1}G \\ G'V_{t+1} &{} G'V_{t+1}G+\varSigma \end{array} \right) \right\} \end{aligned}$$

    and therefore $X_{t+1}|\mathbf {y}_{t+1}\sim N(a_{t+1},C_{t+1})$ where

    $$\begin{aligned} a_{t+1}&= z_{t+1}+V_{t+1}G\left( G'V_{t+1}G+\varSigma \right) ^{-1}\left( y_{t+1}-G'z_{t+1}\right) \\ C_{t+1}&= V_{t+1}-V_{t+1}G\left( G'V_{t+1}G+\varSigma \right) ^{-1}G'V_{t+1}\,. \end{aligned}$$