# Efficient sampling of conditioned Markov jump processes

## Abstract

We consider the task of generating draws from a Markov jump process (MJP) between two time points at which the process is known. Resulting draws are typically termed *bridges*, and the generation of such bridges plays a key role in simulation-based inference algorithms for MJPs. The problem is challenging due to the intractability of the conditioned process, necessitating the use of computationally intensive methods such as weighted resampling or Markov chain Monte Carlo. An efficient implementation of such schemes requires an approximation of the intractable conditioned hazard/propensity function that is both cheap and accurate. In this paper, we review some existing approaches to this problem before outlining our novel contribution. Essentially, we leverage the tractability of a Gaussian approximation of the MJP and suggest a computationally efficient implementation of the resulting conditioned hazard approximation. We compare and contrast our approach with existing methods using three examples.

## Keywords

Markov jump process Conditioned hazard Chemical Langevin equation Linear noise approximation## 1 Introduction

Markov jump processes (MJPs) can be used to model a wide range of discrete-valued, continuous-time processes. Our focus here is on the MJP representation of a reaction network, which has been ubiquitously applied in areas such as epidemiology (Fuchs 2013; Lin and Ludkovski 2013; McKinley et al. 2014), population ecology (Matis et al. 2007; Boys et al. 2008) and systems biology (Wilkinson 2009, 2018; Sherlock et al. 2014). Whilst exact, forward simulation of this class of MJP is straightforward (Gillespie 1977), the reverse problem of performing fully Bayesian inference for the parameters governing the MJP given partial and/or noisy observations is made challenging by the intractability of the observed data likelihood. Simulation-based approaches to inference typically involve “filling in” event times and types between the observation times. A key repeated step in many inference mechanisms starts with a sample of possible states at one observation time and, for each element of the sample, creates a trajectory starting with the sample value and ending at the time of the next observation with a value that is consistent with the next observation. The resulting conditioned samples are typically referred to as *bridges*, and ideally, the bridge should be a draw from the exact distribution of the path given the initial condition and the observation. However, except for a few simple cases, exact simulation of MJP bridges is infeasible, necessitating approximate bridge constructs that can be used as a proposal mechanism inside a weighted resampling and/or Markov chain Monte Carlo (MCMC) scheme.

The focus of this paper is the development of an approximate bridge construct that is both accurate and computationally efficient. Our contribution can be applied in a generic observation regime that allows for discrete, partial and noisy measurements of the MJP, and is particularly effective compared to competitors in the most difficult regime where the observations are sparse in time and the observation variance is small. Many bridge constructs have been proposed for partially observed stochastic differential equations [SDEs, e.g. Delyon and Hu (2006), Bladt and Sørensen (2014), Bladt et al. (2016), Schauer et al. (2017) and Whitaker et al. (2017)], but the literature on bridges for MJPs is relatively sparse. Recent progress involves an approximation of the instantaneous rate or hazard function governing the conditioned process. For example, Boys et al. (2008) linearly interpolate the hazard between observation times but require full and error-free observation of the system of interest. Fearnhead (2008) recognises that the conditioned hazard requires the intractable transition probability mass function of the MJP. This is then directly approximated by substituting the transition density associated with the coarsest possible discretisation of a spatially continuous approximation of the MJP, the chemical Langevin equation (Gillespie 2000). Golightly and Wilkinson (2015) derive a conditioned hazard by approximating the expected number of events between observations, given the observations themselves. Unfortunately, the latter two approaches typically perform poorly when the behaviour of the conditioned process is nonlinear.

We take the approach of Fearnhead (2008) as a starting point and replace the intractable MJP transition probability with the transition density governing the linear noise approximation (LNA) (Kurtz 1970; Elf and Ehrenberg 2003; Komorowski et al. 2009; Schnoerr et al. 2017). Whilst the LNA has been used as an inferential model (see e.g. Ruttor and Opper (2009) and Ruttor et al. (2010) for a maximum likelihood approach and Stathopoulos and Girolami (2013) and Fearnhead et al. (2014) for an MCMC approach), we believe that this is the first attempt to use the LNA to develop a bridge construct for simulation of conditioned MJPs. We find that the LNA offers superior accuracy over a single step of the CLE (which must be discretised in practice), at the expense of computational efficiency. Notably, the LNA solution requires, for each event time in each trajectory, integrating forwards until the next event time a system of ordinary differential equations (ODEs) whose dimension is quadratic in the number of MJP components. We therefore leverage the linear Gaussian structure of the LNA to derive a bridge construct that only requires a single full integration of the LNA ODEs, irrespective of the number of transition events on each bridge or the number of bridges required. We compare the resulting novel construct to several existing approaches using three examples of increasing complexity. In the final, real data application, we demonstrate use of the construct within a pseudo-marginal Metropolis–Hastings scheme, for performing fully Bayesian inference for the parameters governing an epidemic model.

The remainder of this paper is organised as follows. In Sect. 2, we define a Markov jump process as a probabilistic description of a reaction network. We consider the task of sampling conditioned jump processes in Sect. 3 and review two existing approaches. Our novel contribution is presented in Sect. 4 and illustrated in Sect. 5. Conclusions are drawn in Sect. 6.

## 2 Reaction networks

*u*species \(\mathcal {X}_1, \mathcal {X}_2, \ldots ,\mathcal {X}_u\) and

*v*reactions \(\mathcal {R}_1,\mathcal {R}_2,\ldots ,\mathcal {R}_v\) such that reaction \(\mathcal {R}_i\) is written as

*t*, and let \(X_t\) be the

*u*-vector \(X_t = (X_{1,t},X_{2,t}, \ldots , X_{u,t})'\). The effect of a particular reaction is to change the system state \(X_t\) abruptly and discretely. Hence, if the

*i*th reaction occurs at time

*t*, the new state becomes

*i*th column of the \(u\times v\) stoichiometry matrix

*S*. The time evolution of \(X_t\) is therefore most naturally described by a continuous-time, discrete-valued Markov process defined in the following section.

### 2.1 Markov jump processes

*t*is

*i*th reaction occurs by time

*t*. The process \(R_{i,t}\) is a counting process with intensity \(h_i(x_t)\), known in this setting as the reaction hazard, which depends on the current state of the system \(x_t\). Explicitly, we have that

*Gillespie’s direct method*(Gillespie 1977), given by Algorithm 1.

## 3 Sampling conditioned MJPs

Denote by \(\varvec{X}=\{X_{s}\,|\, 0< s \le T\}\) the MJP sample path over the interval (0, *T*]. Complete information on an observed sample path \(\varvec{x}\) corresponds to all reaction times and types. To this end, let \(n_{r}\) denote the total number of reaction events; reaction times (assumed to be in increasing order) and types are denoted by \((t_{i},\nu _{i})\), \(i=1,\ldots ,n_{r}\), \(\nu _{i}\in \{1,\ldots ,v\}\), and we take \(t_{0}=0\) and \(t_{n_{r}+1}=T\).

*T*subject to Gaussian error, giving a single observation \(y_{T}\) on the random variable

*d*vector,

*P*is a constant matrix of dimension \(u\times d\), and \(\varepsilon _{T}\) is a length-

*d*Gaussian random vector. The role of the matrix

*P*is to provide a flexible set-up allowing for various observation scenarios. For example, taking

*P*to be the \(u\times u\) identity matrix corresponds to the case of observing all components of \(X_t\) (subject to error). We denote the density linking \(Y_{T}\) and \(X_{T}\) as \(p(y_{T}|x_{T})\).

*t*, so that sampling events might not be straightforward; however, the time dependence is sufficiently small that it can be ignored and the resulting bridge mechanism, which has a constant rate between events, still leads to efficient proposals.

### 3.1 Weighted resampling

*t*is ignored so that both \(h_0\) and \(\tilde{h}_0\) are piece-wise constant (between reaction events). Hence, in practice, we evaluate the weight using

*N*from \(p(x_0)\) in which case (4) can be used to estimate \(p(y_T)\).

It remains for us to find a suitable form of \(\tilde{h}(x_{t}|y_{T})\). In what follows, we review two existing methods before presenting a novel, alternative approach. Comparisons are made in Sect. 5.

### 3.2 Golightly and Wilkinson approach

*t*and let \(\varDelta R_{t}\) denote the number of reaction events over the time \(T-t=\varDelta t\). Golightly and Wilkinson (2015) approximate \(\varDelta R_{t}\) by assuming a constant reaction hazard over the whole non-infinitesimal time interval, \(\varDelta t\). A Gaussian approximation to the corresponding Poisson distribution then gives

### 3.3 Fearnhead approach

*T*] and suppose that we have simulated as far as time \(t\in [0,T]\). For reaction \(\mathcal {R}_i\), let \(x'=x_{t}+S^{i}\). Recall that \(S^{i}\) denotes the

*i*th column of the stoichiometry matrix so that \(x'\) is the state of the MJP after a single occurrence of \(\mathcal {R}_i\). The conditioned hazard of \(\mathcal {R}_i\) satisfies

*u*-vector of standard Brownian motion and \(\sqrt{S{\text {diag}}\{h(X_t)\}S'}\) is a \(u\times u\) matrix

*B*such that \(BB'=S{\text {diag}}\{h(X_t)\}S'\). Since the CLE can rarely be solved analytically, it is common to work with a discretisation such as the Euler–Maruyama discretisation:

*Z*is a standard multivariate Gaussian random variable. Combining (8) with the observation model (1) gives an approximate conditioned hazard as

## 4 Improved constructs

We take (6) as a starting point and replace \(p(y_{T}|X_{t}=x')\) and \(p(y_{T}|X_{t}=x_t)\) using the linear noise approximation (LNA) (Kurtz 1970; Elf and Ehrenberg 2003; Komorowski et al. 2009; Schnoerr et al. 2017). We first describe the LNA and then consider two constructions for bridges from a known initial condition, \(x_0\), to a potentially noisy observation \(Y_T\), based on different implementations of the LNA. The first is expected to be more accurate as the approximate hazard is recalculated after every event by re-integrating a set of ODEs from the event time to the observation time both from the current value and once for each possible next reaction. The second is more computationally efficient as the recalculation is based on a single, initial integration of a set of ODEs from time 0 to time *T*.

### 4.1 Linear noise approximation

*i*,

*j*)th element \((F_t)_{i,j}=\partial \alpha _i(z_t)/\partial z_{j,t}\). The SDE in (12) can be solved by first defining the \(u\times u\) fundamental matrix \(G_t\) as the solution of

*u*-vector of zeros) and \(V_0=0_{u\times u}\) (the \(u\times u\) zero matrix). Solving (11) and (15) gives the approximating distribution of \(X_t\) as

### 4.2 LNA bridge with restart

*t*,

*T*] to give output denoted by \(z_{T|t}\) and \(V_{T|t}\). Similarly, the initial conditions are denoted \(z_{t|t}=x_t\) and \(V_{t|t}=0_{u\times u}\). We refer to use of the LNA in this way as the LNA with restart (LNAR). The approximation to \(p(y_{T}|X_{t}=x_t)\) is given by

*t*approaches

*T*), the conditioned hazard in (6) must be calculated for \(x_t\) and for each \(x'\) obtained after the

*v*possible transitions of the process. Consequently, the ODE system given by (11) and (15) must be solved at each event time for each of the \(v+1\) possible states. Since the LNA ODEs are rarely tractable (necessitating the use of a numerical solver), this approach is likely to be prohibitively expensive, computationally. In the next section, we outline a novel strategy for reducing the cost associated with integrating the LNA ODE system, that only requires one full integration.

### 4.3 LNA bridge without restart

Consider the solution of the ODE system given by (11), (13) and (14) over the interval (0, *T*] with respective initial conditions \(Z_0=x_0\), \(G_0=I_u\) and \(\psi _0=0_{u\times u}\). Although in practice a numerical solver must be used, we assume that the solution can be obtained over a sufficiently fine time grid to allow reasonable approximation to the ODE solution at an arbitrary time \(t\in (0,T]\), denoted by \(z_t\), \(G_t\) and \(\psi _t\).

*t*,

*T*] with initial conditions \(G_{t|t}=I_{u}\) and \(\psi _{t|t}=0_{u\times u}\). Crucially, the ODE system satisfied by \(z_t\) is not re-integrated (and hence the residual term at time

*t*is \(\tilde{M}_t=x_t-z_t\)). Moreover, \(G_{T|t}\) and \(\psi _{T|t}\) can be obtained without further integration. We have that

*T*] is required, giving a computationally efficient construct. The conditioned hazard takes the form

The accuracy of \(p_{\text {lna}}\) (and therefore the accuracy of the resulting conditioned hazard) is likely to depend on *T*, the length of the inter-observation period over which a realisation of the conditioned process is required. For example, the residual process \(\tilde{M}_t\) will approximate the true (intractable) residual process increasingly poorly if \(z_t\) and \(X_t\) diverge significantly as *t* increases. We investigate the effect of inter-observation time in the next section.

## 5 Applications

In order to examine the empirical performance of the methods proposed in Sect. 4, we consider three examples of increasing complexity. These are a simple (and tractable) death model, the stochastic Lotka–Volterra model examined by Boys et al. (2008) among others and a susceptible-infected-removed (SIR) epidemic model. For the last of these, we use the best-performing LNA-based construct to drive a pseudo-marginal Metropolis–Hastings (PMMH) scheme to perform fully Bayesian inference for the rate constants *c*. Using real data consisting of susceptibles and infectives during the well-studied Eyam plague (Raggett 1982), we compare bridge-based PMMH with a standard implementation (using blind forward simulation) and a recently proposed scheme based on the alive particle filter (Drovandi et al. 2016). All algorithms are coded in R and were run on a desktop computer with an Intel Core i7-4770 processor at 3.40 GHz.

### 5.1 Death model

*t*.

Death model

Blind | CH | F-CLE | F-LNAR | F-LNA | |
---|---|---|---|---|---|

\(X_T=x_{T,(50)}\) | |||||

\(T=0.5\) | 3026, \(8.9\times 10^{-2}\) | 4142, \(2.8\times 10^{-2}\) | 3782, \(4.4\times 10^{-2}\) | 3852, \(4.1\times 10^{-2}\) | 3751, \(4.5\times 10^{-2}\) |

\(T=1\) | 2806, \(8.8\times 10^{-2}\) | 3528, \(4.8\times 10^{-2}\) | 3594, \(4.5\times 10^{-2}\) | 3856, \(3.3\times 10^{-2}\) | 3648, \(4.3\times 10^{-2}\) |

\(T=2\) | 1540, \(8.2\times 10^{-2}\) | 1161, \(3.9\times 10^{-1}\) | 3564, \(4.7\times 10^{-2}\) | 3966, \(3.0\times 10^{-2}\) | 3900, \(3.3\times 10^{-2}\) |

\(X_T=x_{T,(1)}\) | |||||

\(T=0.5\) | 247, \(1.1\times 10^{-1}\) | 3969, \(1.8\times 10^{-3}\) | 3228, \(2.5\times 10^{-3}\) | 3278, \(2.5\times 10^{-3}\) | 3107, \(2.9\times 10^{-3}\) |

\(T=1\) | 339, \(1.1\times 10^{-1}\) | 3194, \(3.8\times 10^{-3}\) | 2106, \(9.3\times 10^{-3}\) | 3515, \(2.9\times 10^{-3}\) | 3281, \(3.6\times 10^{-3}\) |

\(T=2\) | 73, \(3.1\times 10^{-2}\) | 135, \(1.9\times 10^{-1}\) | 1015, \(2.1\times 10^{-2}\) | 3275, \(2.6\times 10^{-3}\) | 2894, \(3.7\times 10^{-3}\) |

\(X_T=x_{T,(99)}\) | |||||

\(T=0.5\) | 646, \(1.0\times 10^{-1}\) | 4316, \(2.3\times 10^{-3}\) | 3926, \(3.8\times 10^{-3}\) | 4042, \(3.5\times 10^{-3}\) | 3995, \(3.7\times 10^{-3}\) |

\(T=1\) | 436, \(9.9\times 10^{-2}\) | 3901, \(2.6\times 10^{-3}\) | 3806, \(2.8\times 10^{-3}\) | 4017, \(2.3\times 10^{-3}\) | 3938, \(2.5\times 10^{-3}\) |

\(T=2\) | 223, \(5.2\times 10^{-2}\) | 1660, \(2.1\times 10^{-2}\) | 3660, \(3.7\times 10^{-3}\) | 4067, \(2.3\times 10^{-3}\) | 3862, \(3.0\times 10^{-3}\) |

*GW*, F-CLE, F-LNAR and F-LNA. Each was run \(m=5000\) times with \(N=10\) samples to give a set of 5000 estimates of the transition probability \(\pi (x_{t}|x_{0})\), and we denote this set by \(\widehat{\pi }^{1:m}(x_{t}|x_{0})\). To compare the algorithms, we report the effective sample size

*T*increases, due to the linear form being unable to adequately describe the exponential like decay exhibited by the true conditioned process. Whilst the F-CLE approach performs well when \(x_T=x_{T,(50)}\) and \(x_T=x_{T,(99)}\), it is unable to match the performance of the LNA-based methods across all scenarios. The effect of not restarting the LNA (i.e. by re-integrating the LNA ODEs after each value of the jump process is generated) appears to be minimal here, with both F-LNAR and F-LNA giving comparable ESS and ReMSE values.

### 5.2 Lotka–Volterra

*t*. It is then straightforward to obtain the Euler-Maruyama approximation of the CLE, for use in the conditioned hazard described in Sect. 3.3 and given by (9).

Lotka–Volterra model

\(T=1\) | \(T=2\) | \(T=3\) | \(T=4\) | |
---|---|---|---|---|

\(y_{T,(1)}\) | (53.34, 27.99) | (75.83, 22.59) | (109.51, 20.90) | (157.34, 23.65) |

\(y_{T,(50)}\) | (73.25, 58.43) | (108.69, 39.92) | (162.03, 41.23) | (238.62, 49.89) |

\(y_{T,(99)}\) | (95.33, 58.43) | (147.28, 58.26) | (225.77, 64.19) | (337.65, 83.79) |

Figure 1 compares summaries (mean plus and minus two standard deviations) of each competing bridge process with the same summaries of the true conditioned process (obtained via simulation), for the extreme case of \(T=4\) and \(y_T=y_{T,(99)}\). Plainly, the blind forward simulation approach and CLE-based Fearnhead approach (F-CLE) are unable to match the dynamics of the true conditioned process. Moreover, we found that these bridges gave very small effective sample sizes for \(T\ge 2\) and we therefore omit these results from the following analysis.

*T*increases, as it is unable to match the nonlinear dynamics of the true conditioned process. In contrast, although more computationally expensive, F-LNAR and F-LNA maintain high ESS values as

*T*is increased. Consequently, in terms of ESS per second, CH is outperformed by F-LNAR for \(T\ge 3\) and F-LNA for \(T\ge 2\). Due to not having to restart the LNA ODEs after each simulated value of the jump process, F-LNA is around an order of magnitude faster than F-LNAR in terms of CPU time, with the difference increasing as

*T*is increased. Given then the comparable ESS values obtained for F-LNAR and F-LNA, we see that in terms of ESS/s, F-LNA outperforms F-LNAR by at least an order of magnitude in all cases and outperforms CH by 1-2 orders of magnitude when \(T=4\).

Lotka–Volterra model

\(x_0'\) | \(\sigma \) | \(T=1\) | \(T=2\) | \(T=3\) | \(T=4\) |
---|---|---|---|---|---|

(10, 10) | 1 | (15.80, 7.68) | (25.46, 5.94) | (41.17, 4.72) | (67.11, 3.92) |

(25, 25) | 2.5 | (38.67, 20.04) | (60.72, 16.71) | (96.09, 14.92) | (152.50, 14.87) |

(50, 50) | 5 | (73.25, 58.43) | (108.69, 39.92) | (162.03, 41.23) | (238.62, 49.89) |

The LNA is known to break down as an inferential model in situations involving low counts of the MJP components (Schnoerr et al. 2017). Therefore, to investigate the performance of the use of the LNA in constructing an approximate conditioned hazard in low-count scenarios, we additionally considered an initial condition with \(x_{1,0}=x_{2,0}\in \{10,25,50\}\) and took \(y_T\) as the median of \(Y_T|X_0=x_0\) for \(T\in \{1,2,3,4\}\). To fix the relative effect of the measurement error, we took \(\sigma =1\) for the case \(x_0=(10,10)'\) and scaled \(\sigma \) in proportion to the components of \(x_0\) for the remaining scenarios. The resulting values of \(y_T\) can be found in Table 3. We report results based on weighted resampling using \(N=5000\) and F-LNA in Fig. 3. We see that when the initial condition is decreased from \(x_0=(50,50)'\) to \(x_0=(10,10)'\), ESS decreases by a factor of around 1.6 (4906 vs 2998) when \(T=1\) and 2.5 (4562 vs 1853) when \(T=4\). Nevertheless, computational cost decreases as \(x_0\) decreases (and in turn, the expected number of reaction events in the observation window decreases). Hence, there is little difference in overall efficiency (ESS/s) across the three scenarios.

### 5.3 SIR model

#### 5.3.1 Model and data

Eyam plague data

Time (months) | ||||||||
---|---|---|---|---|---|---|---|---|

0 | 0.5 | 1 | 1.5 | 2 | 2.5 | 3 | 4 | |

Susceptibles | 254 | 235 | 201 | 153 | 121 | 110 | 97 | 83 |

Infectives | 7 | 14 | 22 | 29 | 20 | 8 | 8 | 0 |

#### 5.3.2 Pseudo-marginal Metropolis–Hastings

*p*(

*c*) to the rate constants

*c*, Bayesian inference may proceed via the marginal parameter posterior

*c*, we have the unbiased estimator

*p*(

*c*,

*u*). Now, running an MH scheme with a proposal density of the form \(q(c^*|c)p(u^*|c^*)\) gives the acceptance probability

*N*to balance mixing performance and computational cost can found in Doucet et al. (2015) and Sherlock et al. (2015). The variance of the log posterior (denoted \(\sigma ^{2}_{N}\), computed with

*N*samples) at a central value of

*c*(e.g. the estimated posterior median) should be around 2. In what follows, we use a random walk on \(\log c\) as the parameter proposal. The innovation variance is taken to be the marginal posterior variance of \(\log c\) estimated from a pilot run and further scaled to give an acceptance rate of around 0.2–0.3. We followed Ho et al. (2018) by adopting independent \(N(0,100^2)\) priors for \(\log c_i\), \(i=1,2\).

Although we do not pursue it here, the case of nonzero measurement error is easily accommodated by iteratively running Algorithm 2 in full, for each observation time \(t_i\), \(i=1,\ldots ,7\). At time \(t_i\), \(y_T\) is replaced by \(y_{t_{i+1}}\) and \(x_0\) is replaced by \(x_{t_i}^j\). At time \(t_1\), \(x_0\) can be replaced by a draw from a prior density \(p(x_{t_1})\) placed on the unobserved initial value. The product (across time) of the average unnormalised weight can be shown to give an unbiased estimator of the observed data likelihood (Del Moral 2004; Pitt et al. 2012). We refer the reader to Golightly and Wilkinson (2015) and the references therein for further details of the resulting Metropolis–Hastings scheme.

#### 5.3.3 Results

We ran PMMH using the observed data likelihood estimator based on (23), with trajectories drawn using either forward simulation or the Fearnhead approach based on the LNA (without restart). We designate the former as “Blind” and the latter as “F-LNA”. Additionally, we ran PMMH using the observed data likelihood estimator based on (24). We designate this scheme as “Alive”.

*c*is fixed at the estimated posterior mean. We note the nonlinear behaviour of the conditioned process over this time interval, with similar nonlinear dynamics observed for other intervals (not reported). Table 5 summarises the computational and statistical performance of the competing inference schemes. We measure statistical efficiency by calculating minimum (over each parameter chain) effective sample size per second (mESS/s). As is appropriate for MCMC output, we use

*k*and \(n_{\text {iters}}\) is the number of iterations in the main monitoring run. Inspection of Table 5 reveals that although use of the alive particle filter only requires \(N=8\) (compared to \(N=5000\) and \(N=100\) for Blind and F-LNA, respectively), it exhibits the largest CPU time. We found that for parameter values in the tails of the posterior, Alive would often require many thousands of forward simulations to obtain \(N=8\) matches. Consequently, Alive is outperformed by Blind by a factor of 2 in terms of overall efficiency. Use of the LNA-driven bridge (without restart) gives a further improvement over Blind of a factor of 2.

## 6 Discussion

Performing efficient sampling of a Markov jump process (MJP) between a known value and a potentially partial or noisy observation is a key requirement of simulation-based approaches to parameter inference. Generating end-point conditioned trajectories, known as bridges, is challenging due to the intractability of the probability function governing the conditioned process. Approximating the hazard function associated with the conditioned process (that is, the conditioned hazard), and correcting draws obtained via this hazard function using weighted resampling or Markov chain Monte Carlo offers a viable solution to the problem. Recent approaches in this direction (Fearnhead 2008; Golightly and Wilkinson 2015) give approximate hazard functions that utilise a Gaussian approximation of the MJP. For example, Golightly and Wilkinson (2015) approximate the number of reactions between observation times as Gaussian. Fearnhead (2008) recognises that the conditioned hazard can be written in terms of the intractable transition probability associated with the MJP. The transition probability is replaced with a Gaussian transition density obtained from the Euler–Maruyama approximation of the chemical Langevin equation. In both approaches, the remaining time until the next observation is treated as a single discretisation. Consequently, the accuracy of the resulting bridges deteriorates as the inter-observation time increases.

SIR model

Algorithm | | CPU (s) | mESS | mESS/s | Rel. |
---|---|---|---|---|---|

Alive | 8 | 126,697 | 737 | 0.0058 | 1 |

Blind | 5000 | 68,177 | 863 | 0.0127 | 2.2 |

F-LNA | 100 | 25,752 | 644 | 0.0250 | 4.3 |

When the dimension of the statespace is finite, then the transition probability from a known state at time 0 to a known state at time *T* can be calculated exactly and efficiently via the action of a matrix exponential on a vector (e.g. Sidje and Stewart 1999), giving the likelihood directly; alternatively, the uniformisation method of Rao and Teh (2013) may be used for Bayesian inference. The recent article Georgoulas et al. (2017) extends the standard finite-statespace matrix-exponential method to an infinite statespace pseudo-marginal MCMC algorithm which uses random truncation (e.g. Glynn and Rhee 2014) to produce a realisation from an unbiased estimator of the likelihood when the observations are exact. In contrast to the algorithms which we have investigated, which simulate paths for the process and whose performance improves as the observation noise increases, any extension to the algorithm of Georgoulas et al. (2017) that allows for observation error would reduce the efficiency of the algorithm. This suggests the possibility that for small enough observation noise an extension to the algorithm in Georgoulas et al. (2017) might be more efficient than our non-restarting bridge. Investigations into the relative efficiencies of such algorithms are ongoing.

This article has focused on bridges from a known initial condition. When the initial condition is unknown, such as typically arises in a particle filter-based analysis, a sample from the distribution of the initial state, \(\{x_0^{1},\ldots ,x_0^N\}\), is available and a separate bridge to the observation is required from each element of the sample. In this case, two different implementations of the LNA bridge without restarting are possible. In the first implementation, trajectories \(\varvec{X}^i|x_0^i,y_T\) are generated using one full integration of (11), (13) and (14) over (0, *T*] *for each*\(x_0^i\). That is, each trajectory has (11) initialised at \(x_0^i\). In the second implementation, (11), (13) and (14) are integrated *just once*, irrespective of the number of required trajectories. This can be achieved by initialising (11) at some plausible value e.g. \(E(X_0)\). Although the second implementation will be more computationally efficient than the first, some loss of accuracy is expected, especially when the uncertainty in \(X_0\) is large. A single integral, however, may well be adequate in the cases which are the focus of this article: where the observation noise is small. Investigating the efficiency of the bridge construct in this scenario, as well as in multi-scale settings (see e.g. Thomas et al. 2014) where some reactions regularly occur more frequently than others, remains the subject of ongoing research.

## Notes

## References

- Bailey, N.T.J.: The Elements of Stochastic Processes with Applications to the Natural Sciences. Wiley, New York (1964)zbMATHGoogle Scholar
- Bladt, M., Sørensen, M.: Simple simulation of diffusion bridges with application to likelihood inference for diffusions. Bernoulli
**20**, 645–675 (2014)MathSciNetzbMATHGoogle Scholar - Bladt, M., Finch, S., Sørensen, M.: Simulation of multivariate diffusion bridges. J. R. Stat. Soc. Ser. B Stat. Methodol.
**78**, 343–369 (2016)MathSciNetGoogle Scholar - Boys, R.J., Wilkinson, D.J., Kirkwood, T.B.L.: Bayesian inference for a discretely observed stochastic kinetic model. Stat. Comput.
**18**, 125–135 (2008)MathSciNetGoogle Scholar - Brémaud, P.: Point Processes and Queues: Martingale Dynamics. Springer, New York (1981)zbMATHGoogle Scholar
- Cox, J.C., Ingersoll, J.E., Ross, S.A.: A theory of the term structure of interest rates. Econometrica
**53**, 385–407 (1985)MathSciNetzbMATHGoogle Scholar - Del Moral, P.: Feynman–Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Springer, New York (2004)zbMATHGoogle Scholar
- Del Moral, P., Jasra, A., Lee, A., Yau, C., Zhang, X.: The alive particle filter and its use in particle Markov chain Monte Carlo. Stoch. Anal. Appl.
**33**, 943–974 (2015)MathSciNetzbMATHGoogle Scholar - Delyon, B., Hu, Y.: Simulation of conditioned diffusion and application to parameter estimation. Stoch. Anal. Appl.
**116**, 1660–1675 (2006)MathSciNetzbMATHGoogle Scholar - Doucet, A., Pitt, M.K., Kohn, R.: Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator. Biometrika
**102**, 295–313 (2015)MathSciNetzbMATHGoogle Scholar - Drovandi, C.C., McCutchan, R.: Alive SMC\(^2\): Bayesian model selction for low-count time series models with intractable likelihoods. Biometrics
**72**, 344–353 (2016)MathSciNetzbMATHGoogle Scholar - Drovandi, C.C., Pettitt, A.N., McCutchan, R.: Exact and approximate Bayesian inference for low count time series models with intractable likelihoods. Bayesian Anal.
**11**, 325–352 (2016)MathSciNetzbMATHGoogle Scholar - Elf, J., Ehrenberg, M.: Fast evolution of fluctuations in biochemical networks with the linear noise approximation. Genome Res.
**13**(11), 2475–2484 (2003)Google Scholar - Fearnhead, P.: Computational methods for complex stochastic systems: a review of some alternatives to MCMC. Stat. Comput.
**18**, 151–171 (2008)MathSciNetGoogle Scholar - Fearnhead, P., Giagos, V., Sherlock, C.: Inference for reaction networks using the linear noise approximation. Biometrics
**70**, 457–466 (2014)MathSciNetzbMATHGoogle Scholar - Fuchs, C.: Inference for Diffusion Processes with Applications in Life Sciences. Springer, Heidelberg (2013)zbMATHGoogle Scholar
- Georgoulas, A., Hillston, J., Sanguinetti, G.: Unbiased Bayesian inference for population Markov jump processes via random truncations. Stat. Comput.
**27**, 991–1002 (2017)MathSciNetzbMATHGoogle Scholar - Gillespie, D.T.: Exact stochastic simulation of coupled chemical reactions. J. Phys. Chem.
**81**, 2340–2361 (1977)Google Scholar - Gillespie, D.T.: A rigorous derivation of the chemical master equation. Physica A
**188**, 404–425 (1992)Google Scholar - Gillespie, D.T.: The chemical Langevin equation. J. Chem. Phys.
**113**(1), 297–306 (2000)Google Scholar - Glynn, P.W., Rhee, C.-H.: Exact estimation for Markov chain equilibrium expectations. J. Appl. Probab.
**51A**, 377–389 (2014)MathSciNetzbMATHGoogle Scholar - Golightly, A., Kypraios, T.: Efficient SMC\(^2\) schemes for stochastic kinetic models. Stat. Comput. (2017). https://doi.org/10.1007/s11222-017-9789-8
- Golightly, A., Wilkinson, D.J.: Bayesian inference for Markov jump processes with informative observations. SAGMB
**14**(2), 169–188 (2015)MathSciNetzbMATHGoogle Scholar - Ho, L.S.T., Xu, J., Crawford, F.W., Minin, V.N., Suchard, M.A.: Birth/birth-death processes and their computable transition probabilities with biological applications. J. Math. Biol.
**76**(4), 911–944 (2018)MathSciNetzbMATHGoogle Scholar - Komorowski, M., Finkenstadt, B., Harper, C., Rand, D.: Bayesian inference of biochemical kinetic parameters using the linear noise approximation. BMC Bioinform.
**10**(1), 343 (2009)Google Scholar - Kurtz, T.G.: Solutions of ordinary differential equations as limits of pure jump Markov processes. J. Appl. Probab.
**7**, 49–58 (1970)MathSciNetzbMATHGoogle Scholar - Kurtz, T.G.: The relationship between stochastic and deterministic models for chemical reactions. J. Chem. Phys.
**57**, 2976–2978 (1972)Google Scholar - Lin, J., Ludkovski, M.: Sequential Bayesian inference in hidden Markov stochastic kinetic models with application to detection and response to seasonal epidemics. Stat. Comput.
**24**, 1047–1062 (2013)MathSciNetzbMATHGoogle Scholar - Matis, J.H., Kiffe, T.R., Matis, T.I., Stevenson, D.E.: Stochastic modeling of aphid population growth with nonlinear power-law dynamics. Math. Biosci.
**208**, 469–494 (2007)MathSciNetzbMATHGoogle Scholar - McKinley, T.J., Ross, J.V., Deardon, R., Cook, A.R.: Simulation-based Bayesian inference for epidemic models. Comput. Stat. Data Anal.
**71**, 434–447 (2014)MathSciNetzbMATHGoogle Scholar - Petzold, L.: Automatic selection of methods for solving stiff and non-stiff systems of ordinary differential equations. SIAM J. Sci. Stat. Comput.
**4**(1), 136–148 (1983)zbMATHGoogle Scholar - Pitt, M.K., dos Santos Silva, R., Giordani, P., Kohn, R.: On some properties of Markov chain Monte Carlo simulation methods based on the particle filter. J. Econom.
**171**(2), 134–151 (2012)MathSciNetzbMATHGoogle Scholar - Raggett, G.: A stochastic model of the Eyam plague. J. Appl. Stat.
**9**, 212–225 (1982)zbMATHGoogle Scholar - Rao, V., Teh, Y.W.: Fast MCMC sampling for Markov jump processes and extensions. J. Mach. Learn. Res.
**14**, 3295–3320 (2013)MathSciNetzbMATHGoogle Scholar - Ruttor, A., Opper, M.: Efficient statistical inference for stochastic reaction processes. Phys. Rev. Lett.
**103**, 230601 (2009)Google Scholar - Ruttor, A., Sanguinetti, G., Opper, M.: Approximate inference for stochastic reaction networks. In: Lawrence, N.D., Girolami, M., Rattray, M., Sanguinetti, G. (eds.) Learning and Inference in Computational Systems Biology, pp. 277–296. The MIT press, Cambridge (2010)Google Scholar
- Schauer, M., van der Meulen, F., van Zanten, H.: Guided proposals for simulating multi-dimensional diffusion bridges. Bernoulli
**23**, 2917–2950 (2017)MathSciNetzbMATHGoogle Scholar - Schnoerr, D., Sanguinetti, G., Grima, R.: Approximation and inference methods for stochastic biochemical kinetics—a tutorial review. J. Phys. A
**50**, 093001 (2017)MathSciNetzbMATHGoogle Scholar - Sherlock, C., Golightly, A., Gillespie, C.S.: Bayesian inference for hybrid discrete-continuous systems biology models. Inverse Probl.
**30**, 114005 (2014)zbMATHGoogle Scholar - Sherlock, C., Thiery, A., Roberts, G.O., Rosenthal, J.S.: On the effciency of pseudo-marginal random walk Metropolis algorithms. Ann. Stat.
**43**(1), 238–275 (2015)zbMATHGoogle Scholar - Sidje, R.B., Stewart, W.J.: A numerical study of large sparse matrix exponentials arising in Markov chains. Comput. Stat. Data Anal.
**29**(3), 345–368 (1999)zbMATHGoogle Scholar - Stathopoulos, V., Girolami, M.A.: Markov chain Monte Carlo inference for Markov jump processes via the linear noise approximation. Philos. Trans. R. Soc. A
**371**, 20110541 (2013)MathSciNetzbMATHGoogle Scholar - Thomas, P., Popovic, N., Grima, R.: Phenotypic switching in gene regulatory networks. PNAS
**111**, 6994–6999 (2014)Google Scholar - Whitaker, G.A., Golightly, A., Boys, R.J., Sherlock, C.: Improved bridge constructs for stochastic differential equations. Stat. Comput.
**27**, 885–900 (2017)MathSciNetzbMATHGoogle Scholar - Wilkinson, D.J.: Stochastic modelling for quantitative description of heterogeneous biological systems. Nat. Rev. Genet.
**10**, 122–133 (2009)Google Scholar - Wilkinson, D.J.: Stochastic Modelling for Systems Biology, 3rd edn. Chapman & Hall/CRC Press, Boca Raton (2018)zbMATHGoogle Scholar

## Copyright information

**OpenAccess**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.