Stability of noisy Metropolis–Hastings

Pseudo-marginal Markov chain Monte Carlo methods for sampling from intractable distributions have gained recent interest and have been theoretically studied in considerable depth. Their main appeal is that they are exact, in the sense that they target marginally the correct invariant distribution. However, the pseudo-marginal Markov chain can exhibit poor mixing and slow convergence towards its target. As an alternative, a subtly different Markov chain can be simulated, where better mixing is possible but the exactness property is sacrificed. This is the noisy algorithm, initially conceptualised as Monte Carlo within Metropolis, which has also been studied but to a lesser extent. The present article provides a further characterisation of the noisy algorithm, with a focus on fundamental stability properties like positive recurrence and geometric ergodicity. Sufficient conditions for inheriting geometric ergodicity from a standard Metropolis–Hastings chain are given, as well as convergence of the invariant distribution towards the true target distribution. Electronic supplementary material The online version of this article (doi:10.1007/s11222-015-9604-3) contains supplementary material, which is available to authorized users.


Intractable target densities and the pseudo-marginal algorithm
Suppose our aim is to simulate from an intractable probability distribution π for some random variable X , which takes values in a measurable space (X , B(X )). In addition, let π have a density π(x) with respect to some reference measure μ(dx), e.g. the counting or the Lebesgue measure. By intractable we mean that an analytical expression for the density π(x) is not available and so implementation of a Markov chain Monte Carlo (MCMC) method targeting π is not straightforward. One possible solution to this problem is to target a different distribution on the extended space (X × W, B(X ) × B(W)), which admits π as marginal distribution. The pseudomarginal algorithm (Beaumont 2003;Andrieu and Roberts 2009) falls into this category since it is a Metropolis-Hastings (MH) algorithm targeting a distributionπ N , associated to the random vector (X, W ) defined on the product space (X × W, B(X ) × B(W)) where W ⊆ R + 0 := [0, ∞). It is given bȳ where Q x,N (x,N )∈X ×N + is a family of probability distributions on (W, B(W)) satisfying for each (x, N ) ∈ X × N E Q x,N W x,N ≡ 1, for W x,N ∼ Q x,N (·). (2) Throughout this article, we restrict our attention to the case where for each x ∈ X , W x,N is Q x,N -a.s. strictly positive, for reasons that will become clear. The random variables W x,N x,N are commonly referred as the weights. Formalising this algorithm using (1) and (2) was introduced by Andrieu and Vihola (2015), and "exactness" follows immediately:π admits π as a marginal. Given a proposal kernel q : X × B(X ) → [0, 1], the respective proposal of the pseudo-marginal is given bȳ q N (x, w; dy, du) := q(x, dy)Q y,N (du), and, consequently, the acceptance probability can be expressed as α N (x, w; y, u) := min 1, π(dy)uq(y, dx) π(dx)wq (x, dy) .
The pseudo-marginal algorithm defines a time-homogeneous Markov chain, with transition kernelP N on the measurable space (X × W, B(X ) × B(W)). A single draw from P N (x, w; ·, ·) is presented in Algorithm 1. (3): return (Y, U ), otherwise:
Due to its exactness and straightforward implementation in many settings, the pseudo-marginal has gained recent interest and has been theoretically studied in some depth, see e.g. Andrieu and Roberts (2009), Vihola (2014, 2015), Doucet et al. (2015), Girolami et al. (2013), Maire et al. (2014) and Sherlock et al. (2015). These studies typically compare the pseudo-marginal Markov chain with a "marginal" Markov chain, arising in the case where all the weights are almost surely equal to 1, and (3) is then the standard Metropolis-Hastings acceptance probability associated with the target density π and the proposal q.

Examples of pseudo-marginal algorithms
A common source of intractability for π occurs when a latent variable Z on (Z , B(Z )) is used to model observed data, as in hidden Markov models (HMMs) or mixture models. Although the density π(x) cannot be computed, it can be approximated via importance sampling, using an appropriate auxiliary distribution, say ν x . Here, appropriate means π x ν x , where π x denotes the conditional distribution of Z given X = x. Therefore, for this setting, the weights are given by where Z (k) x k∈{1,..., N } i.i.d. ∼ ν x (·), which motivates the following generic form when using averages of unbiased estimators where W (k) It is clear that (4) describes only a special case of (2). Nevertheless, we will pay special attention to the former throughout the article. For similar settings to (4) see Andrieu and Roberts (2009).
Since (2) is more general, it allows W x,N to be any random variable with expectation 1. Sequential Monte Carlo (SMC) methods involve the simulation of a system of some number of particles, and provide unbiased estimates of likelihoods associated with HMMs (see Del Moral 2004, Proposition 7.4.1 or Pitt et al. 2012 irrespective of the size of the particle system. Consider the model given by Fig. 1. The random variables {X t } T t=0 form a time-homogeneous Markov chain with transition f θ (·|x t−1 ) that depends on a set of parameters θ . The observed random variables {Y t } T t=1 are conditionally independent given the unobserved {X t } T t=1 and are distributed according to g θ (·|x t ), which also may depend on θ . The likelihood function for θ is given by l(θ ; y 1 , . . . , y T ) where E f θ denotes expectation w.r.t. the θ -dependent law of {X t } T t=1 , and we assume for simplicity that the initial value X 0 = x 0 is known. If we denote byl N (θ ; y 1 , . . . , y T ) the unbiased SMC estimator of l(θ ; y 1 , . . . , y T ) based on N particles, we can then define W θ,N :=l N (θ ; y 1 , . . . , y T ) l(θ ; y 1 , . . . , y T ) , and (2) is satisfied but (4) is not. The resulting pseudomarginal algorithm is developed and discussed in detail in Andrieu et al. (2010), where it and related algorithms are referred to as particle MCMC methods.

The noisy algorithm
Although the pseudo-marginal has the desirable property of exactness, it can suffer from "sticky" behaviour, exhibiting Fig. 1 Hidden Markov model poor mixing and slow convergence towards the target distribution (Andrieu and Roberts 2009;Lee and Łatuszyński 2014). The cause for this is well-known to be related with the value of the ratio between W y,N and W x,N at a particular iteration. Heuristically, when the value of the current weight (w in (3)) is large, proposed moves can have a low probability of acceptance. As a consequence, the resulting chain can get "stuck" and may not move after a considerable number of iterations.
In order to overcome this issue, a subtly different algorithm is performed in some practical problems (see, e.g., McKinley et al. 2014). The basic idea is to refresh, independently from the past, the value of the current weight at every iteration. The ratio of the weights between W y,N and W x,N still plays an important role in this alternative algorithm, but here refreshing W x,N at every iteration can improve mixing and the rate of convergence. This alternative algorithm is commonly known as Monte Carlo within Metropolis (MCWM), as in O'Neill et al. (2000), Beaumont (2003) or Andrieu and Roberts (2009), since typically the weights are Monte Carlo estimates as in (4). From this point onwards it will be referred as the noisy MH algorithm or simply the noisy algorithm to emphasize that our main assumption is (2). Due to independence from previous iterations while sampling W x,N and W y,N , the noisy algorithm also defines a time-homogeneous Markov chain with transition kernelP N , but on the measurable space (X , B(X )). A single draw fromP N (x, ·) is presented in Algorithm 2, and it is clear that we restrict our attention to strictly positive weights because the algorithm is not welldefined when both W y,N and W x,N are equal to 0.
Even though these algorithms differ only slightly, the related chains have very different properties. In Algorithm 2, the value w is generated at every iteration whereas in Algorithm 1, it is treated as an input. As a consequence, Algorithm 1 produces a chain on (X × W, B(X ) × B(W)) contrasting with a chain from Algorithm 2 taking values on (X , B(X )). However, the noisy chain is not invariant under π and it is not reversible in general. Moreover, it may not even have an invariant distribution as shown by some examples in Sect. 2. From O'Neill et al. (2000) and Fernández-Villaverde and Rubio-Ramírez (2007), it is evident that the implementation of the noisy algorithm goes back even before the appearance of the pseudo-marginal, the latter initially conceptualised as Grouped Independence Metropolis-Hastings (GIMH) in Beaumont (2003). Theoretical properties, however, of the noisy algorithm have mainly been studied in tandem with the pseudo-marginal by Beaumont (2003), Andrieu and Roberts (2009) and more recently by Alquier et al. (2014).
The noisy chain generated by Algorithm 2 can be seen as a perturbed version of an idealised Markov chain where the weights W x,N x,N are all equal to one. Perturbed Markov chains have been investigated in , e.g., Roberts et al. (1998), Breyer et al. (2001, Shardlow and Stuart (2000), Mitrophanov (2005), Ferré et al. (2013). More recently Pillai and Smith (2014) and Rudolf and Schweizer (2015) study such chains using the notion of Wasserstein distance. We focus on total variation distance, a particular case of the Wasserstein distance. The relationship between our work and these latter papers is pointed out in subsequent remarks.

Objectives of the article
The objectives of this article can be illustrated using a simple example. Let N (·|μ, σ 2 ) denote a univariate Gaussian distribution with mean μ and variance σ 2 and π(·) = N (·|0, 1) be a standard normal distribution. Let the weights W x,N be as in (4) with Q x (·) = log N · − 1 2 σ 2 , σ 2 and σ 2 := 5, where log N (·|μ, σ 2 ) denotes a log-normal distribution of parameters μ and σ 2 . In addition, let the proposal q be random walk given by q(x, ·) = N (·|x, 4). For this example, Fig. 2 shows the estimated densities using the noisy chain for different values of N . It appears that the noisy chain has an invariant distribution, and as N increases it seems to approach the desired target π . Our objectives here are to answer the following types of questions about the noisy algorithm in general: 1. Does an invariant distribution exist, at least for N large enough? 2. Does the noisy Markov chain behave like the marginal chain for sufficiently large N ? 3. Does the invariant distribution, if it exists, converge to π as N increases?
We will see that the answer to the first two questions is negative in general. However, all three questions can be answered positively when the marginal chain is geometrically ergodic and the distributions of the weights satisfy additional assumptions.

Marginal chains and geometric ergodicity
In order to formalise our analysis, let P denote the Markov transition kernel of a standard MH chain on (X , B(X )), targeting π with proposal q. We will refer to this chain and this algorithm using the term marginal (as in Andrieu and Roberts 2009;Andrieu and Vihola 2015), which is the idealised version for which the noisy chain and corresponding algorithm are simple approximations. Therefore where α is the MH acceptance probability and ρ is the rejection probability, given by α(x, y) := min 1, π(dy)q(y, dx) π(dx)q(x, dy) and Similarly, for the transition kernelP N of the noisy chain, moves are proposed according to q but are accepted usinḡ α N (as in (3)) instead of α, once values for W x,N and W y,N are sampled. In order to distinguish the acceptance probabilities between the noisy and the pseudo-marginal processes, despite being the same after sampling values for the weights, defineα Hereα N is the expectation of a randomised acceptance probability, which permits defining the transition kernel of the noisy chain bỹ whereρ N is the noisy rejection probability given bỹ As briefly noted before, the noisy kernelP N is just a perturbed version of P involving a ratio of weights in the noisy acceptance probabilityα N . When such weights are identically one, i.e. Q x,N ({1}) = 1, the noisy chain reduces to the marginal chain, whereas the pseudo-marginal becomes the marginal chain with an extra component always equal to 1. So far, the terms slow convergence and "sticky" behaviour have been used in a relative vague sense. A powerful characterisation of the behaviour of a Markov chain is provided by geometric ergodicity, defined below. Geometrically ergodic Markov chains have a limiting invariant probability distribution, which they converge towards geometrically fast in total variation (Meyn and Tweedie 2009). For any Markov kernel K : X × B(X ) → [0, 1], let K n be the n-step transition kernel, which is given by Definition 1.1 (Geometric ergodicity) A ϕ-irreducible and aperiodic Markov chain := (Φ i ) i≥0 on a measurable space (X , B(X )), with transition kernel P and invariant distribution π , is geometrically ergodic if there exists a finite function V ≥ 1 and constants τ < 1, R < ∞ such that Here, · T V denotes the total variation norm given by where μ is any signed measure.
Geometric ergodicity does not necessarily provide fast convergence in an absolute sense. For instance, consider cases where τ , or R, from Definition 1.1 are extremely close to one, or very large respectively. Then the decay of the total variation distance, though geometric, is not particularly fast (see Roberts and Rosenthal 2004 for some examples). Nevertheless, geometric ergodicity is a useful tool when analysing non-reversible Markov chains as will become apparent in the noisy chain case. Moreover, in practice one is often interested in estimating E π [ f (X )] for some function f : X → R, which is done by using ergodic averages of the form In this case, geometric ergodicity is a desirable property since it can guarantee the existence of a central limit theorem (CLT) for e n,m ( f ), see Chan and Geyer (1994) and Rosenthal (1997, 2004) for a more general review. Also, its importance is related with the construction of consistent estimators of the corresponding asymptotic variance in the CLT, as in Flegal and Jones (2010). As noted in Andrieu and Roberts (2009), if the weights W x,N x,N are not essentially bounded then the pseudomarginal chain cannot be geometrically ergodic; in such cases the "stickiness" may be more evident. In addition, under mild assumptions (in particular, thatP N has a left spectral gap), from Andrieu and Vihola (2015, Proposition 10) and Lee and Łatuszyński (2014), a sufficient but not necessary condition ensuring the pseudo-marginal inherits geometric ergodicity from the marginal, is that the weights are uniformly bounded. This certainly imposes a tight restriction in many practical problems.
The analyses in Roberts (2009) andAlquier et al. (2014) mainly study the noisy algorithm in the case where the marginal Markov chain is uniformly ergodic, i.e. when it satisfies (8) with sup x∈X V (x) < ∞. However, there are many Metropolis-Hastings Markov chains for statistical estimation that cannot be uniformly ergodic, e.g. random walk Metropolis chains when π is not compactly supported. Our focus is therefore on inheritance of geometric ergodicity by the noisy chain, complementing existing results for the pseudo-marginal chain.

Outline of the paper
In Sect. 2, some simple examples are presented for which the noisy chain is positive recurrent, so it has an invariant probability distribution. This is perhaps the weakest stability property that one would expect a Monte Carlo Markov chain to have. However, other fairly surprising examples are presented for which the noisy Markov chain is transient even though the marginal and pseudo-marginal chains are geometrically ergodic. Section 3 is dedicated to inheritance of geometric ergodicity from the marginal chain, where two different sets of sufficient conditions are given and are further analysed in the context of arithmetic averages given by (4). Once geometric ergodicity is attained, it guarantees the existence of an invariant distributionπ N for the noisy chain. Under the same sets of conditions, we show in Sect. 4 thatπ N and π can be made arbitrarily close in total variation as N increases. Moreover, explicit rates of convergence are possible to obtain in principle, when the weights arise from an arithmetic average setting as in (4).

Homogeneous weights with a random walk proposal
Assume a log-concave target distribution π on the positive integers, whose density with respect to the counting measure is given by where h : N + → R is a convex function. In addition, let the proposal distribution be a symmetric random walk on the integers, i.e.
From Mengersen and Tweedie (1996), it can be seen that the marginal chain is geometrically ergodic. Now, assume the distribution of the weights W m,N m,N is homogeneous with respect to the state space, meaning In addition, assume W N > 0 Q N -a.s., then for m ≥ 2 For this particular class of weights and using the fact that h is convex, the noisy chain is geometrically ergodic, implying the existence of an invariant probability distribution. (9). In addition, let the distribution of the weights be homogeneous as in (10). Then, the chain generated by the noisy kernelP N is geometrically ergodic.

Proposition 2.1 Consider a log-concave target density on the positive integers and a proposal density as in
It is worth noting that the distribution of the weights, though homogeneous with respect to the state space, can be taken arbitrarily, as long as the weights are positive. Homogeneity ensures that the distribution of the ratio of such weights is not concentrated near 0, due to its symmetry around one, i.e. for z > 0 In contrast, when the support of the distribution Q N is unbounded, the corresponding pseudo-marginal chain cannot be geometrically ergodic.

Particle MCMC
More complex examples arise when using particle MCMC methods, for which noisy versions can also be performed. They may prove to be useful in some inference problems. Consider again the hidden Markov model given by Fig. 1. As before, set X 0 = x 0 and let Therefore, once a prior distribution for θ is specified, p(·) say, the aim is to conduct Bayesian inference on the posterior distribution In this particular setting, the posterior distribution is tractable. This will allows us to compare the results obtained from the exact and noisy versions, both relying on the SMC estimatorl N (θ ; y 1 , . . . , y T ) of the likelihood. Using a uniform prior for the parameters and a random walk proposal, Fig. 3 shows the run and autocorrelation function (acf) for the autoregressive parameter a of the marginal chain. Similarly, Fig. 4 shows the corresponding run and acf for both the pseudo-marginal and the noisy chain when N = 250. Plots for the other parameters and different values of N can be found in Online Appendix 2. It is noticeable how the pseudomarginal gets "stuck", resulting in a lower acceptance than the marginal and noisy chains. In addition, the acf of the noisy chain seems to decay faster than that of the pseudo-marginal chain.
Finally, Figs. 5 and 6 show the estimated posterior densities for the parameters when N = 250 and N = 750, respectively. There, the trade-off between the pseudo-marginal and the noisy algorithm is noticeable. For lower values of N , the pseudo-marginal will require more iterations due to the slow mixing, whereas the noisy converges faster towards an unknown noisy invariant distribution. By increasing N , the mixing in the pseudo-marginal improves and the noisy invariant approaches the true posterior. Plots for other values of N can also be found in Online Appendix 2.

Transient noisy chain with homogeneous weights
In contrast with example in Sect. 2.1, this one shows that the noisy algorithm can produce a transient chain even in simple settings. Let π be a geometric distribution on the positive integers, whose density with respect to the counting measure is given by In addition, assume the proposal distribution is a simple random walk on the integers, i.e.
Consider N = 1 and as in Sect. 2.1, let the distribution of weights be homogeneous and given by (13) where Ber(s) denotes a Bernoulli random variable of parameter s ∈ (0, 1). There exists a relationship between s, b and ε that guarantees the expectation of the weights is identically one. The following proposition, proven in Appendix 1 by taking θ > 1/2, shows that the resulting noisy chain can be transient for certain values of b, and θ . (11) and a proposal density as in (12). In addition, let the weights when N = 1 be given by (13). Then, for some b, ε and θ the chain generated by the noisy kernelP N =1 is transient.

Proposition 2.2 Consider a geometric target density as in
In contrast, since the weights are uniformly bounded by b, the pseudo-marginal chain inherits geometric ergodicity for any θ , b and . The left plot in Fig. 7 shows an example. We will discuss the behaviour of this example as N increases in Sect. 3.4 .

Transient noisy chain with non-homogeneous weights
One could argue that the transient behaviour of the previous example is related to the large value of θ in the proposal distribution. However, as shown here, for any value of θ ∈ (0, 1) one can construct weights satisfying (2) for which the noisy chain is transient. With the same assumptions as in the example in Sect. 2.3, except that now the distribution of weights is not homogeneous but given by the noisy chain will be transient for b large enough. The proof can be found in Appendix 1. (11) and a proposal density as in (12). In addition, let the weights when N = 1 be given by (14). Then, for any θ ∈ (0, 1) there exists some b > 1 such that the chain generated by the noisy kernelP N =1 is transient.

Proposition 2.3 Consider a geometric target density as in
The reason for this becomes apparent when looking at the behaviour of the ratios of weights. Even though ε m → 0 as m → ∞, the non-monotonic behaviour of the sequence implies Hence, the ratio of the weights can become arbitrarily large or arbitrarily close to zero with a non-negligible probability. This allows the algorithm to accept moves to the right more often, if m is large enough. Once again, the pseudomarginal chain inherits the geometrically ergodic property from the marginal. See the central and right plots of Fig. 7 for two examples using different proposals. Again, we will come back to this example in Sect. 3.4, where we look at the behaviour of the associated noisy chain as N increases.

Inheritance of ergodic properties
The inheritance of various ergodic properties of the marginal chain by pseudo-marginal Markov chains has been established using techniques that are powerful but suitable only for reversible Markov chains (see, e.g. Andrieu and Vihola 2015). Since the noisy Markov chains treated here can be non-reversible, a suitable tool for establishing geometric ergodicity is the use of Foster-Lyapunov functions, via geometric drift towards a small set.
Definition 3.1 (Small set) Let P be the transition kernel of a Markov chain . A subset C ⊆ X is small if there exists a positive integer n 0 , ε > 0 and a probability measure ν(·) on (X , B(X )) such that the following minorisation condition holds The following theorem, which is immediate from combining Roberts and Rosenthal (1997 Theorem 3.1 Suppose that Φ is a φ-irreducible and aperiodic Markov chain with transition kernel P and invariant distribution π . Then, the following statements are equivalent: (i) There exists a small set C, constants λ < 1 and b < ∞, and a function V ≥ 1 finite for some x 0 ∈ X satisfying the geometric drift condition (ii) The chain is π -a.e. geometrically ergodic, meaning that for π -a.e. x ∈ X it satisfies (8) for some V ≥ 1 (which can be taken as in (i)) and constants τ < 1, R < ∞.
From this point onwards, it is assumed that the marginal and noisy chains are φ-irreducible and aperiodic. In addition, for many of the following results, it is required that (P1) The marginal chain is geometrically ergodic, implying its kernel P satisfies the geometric drift condition in (16) for some constants λ < 1 and b < ∞, some function V ≥ 1 and a small set C ⊆ X .

Conditions involving a negative moment
From the examples of the previous section, it is clear that the weights play a fundamental role in the behaviour of the noisy chain. The following theorem states that the noisy chain will inherit geometric ergodicity from the marginal under some conditions on the weights involving a strengthened version of the Law of Large Numbers and convergence of negative moments.
The above result is obtained by controlling the dissimilarity of the marginal and noisy kernels. This is done by looking at the corresponding rejection and acceptance probabilities. The proofs of the following lemmas appear in Appendix 1.
Lemma 3.1 For any δ > 0 Lemma 3.2 Let ρ(x) andρ N (x) be the rejection probabilities as defined in (5) and (7) respectively. Then, for any δ > 0 Lemma 3.3 Let α(x, y) andα N (x, y) be the acceptance probabilities as defined in (5) and (6) respectively. Then, Notice that (W1) and (W2) allow control on the bounds in the above lemmas. While Lemma 3.2 provides a bound for the difference of the rejection probabilities, Lemma 3.3 gives one for the ratio of the acceptance probabilities. The proof of Theorem 3.2 is now presented.
Proof of Theorem 3.2 Since the marginal chain P is geometrically ergodic, it satisfies the geometric drift condition in (16) for some λ < 1, b < ∞, some function V ≥ 1 and a small set C ⊆ X . Now, using the above lemmas By (W1) and (W2), for any ε, δ > 0 there exists N 0 ∈ N + such that Taking δ = ε 2 and ε ∈ 0, 1−λ 1+λ , the noisy chainP N also satisfies a geometric drift condition for the same function V and small set C, completing the proof.
Remark 3.1 In fact, (W1) and (W2) together guarantee for any δ > 0 which is the crucial assumption in Pillai and Smith (2014, Lemma 3.6) for obtaining a similar drift condition.

Conditions on the proposal distribution
In this subsection a different bound for the acceptance probabilities is provided, which allows dropping assumption (W2) but imposes a different one on the proposal q instead.
(P1 * ) (P1) holds and for the same drift function V in (P1) there exists K < ∞ such that the proposal kernel q satisfies Theorem 3.3 Assume (P1*) and (W1). Then, there exists N 0 ∈ N + such that for all N ≥ N 0 , the noisy chain with transition kernelP N is geometrically ergodic.
In order to prove Theorem 3.3 the following lemma is required. Its proof can be found in Appendix 1. In contrast with Lemma 3.3, this lemma provides a bound for the additive difference of the noisy and marginal acceptance probabilities. (5) and (6), respectively. Then, for any η > 0

Lemma 3.4 Let α(x, y) andα N (x, y) be the acceptance probabilities as defined in
Proof of Theorem 3.3 Using Lemmas 3.2 and 3.4 with η = δ whenever N ≥ N 0 . This implies and using (P1*) Taking δ = ε 2 and ε ∈ 0, 1−λ 1+K , the noisy chainP N also satisfies a geometric drift condition for the same function V and small set C, completing the proof.
Remark 3.2 By itself, (W1) implies for any δ > 0 but it needs to be paired with (P1*) to obtain the desired result. These assumptions are comparable to those in Pillai and Smith (2014, Lemma 3.6), taking f constant therein. Additionally, (W1) and (P1*) imply the required conditions on E and λ in Rudolf and Schweizer (2015, Corollary 31), where a similar result is proved in terms of V -uniform ergodicity.
In general, assumption (P1*) may be difficult to verify as one must identify a particular function V , but it is easily satisfied when restricting to log-Lipschitz targets and when using a random walk proposal of the form where · denotes the usual Euclidean distance. To see this the following assumption is required, which is a particular case of (P1) and is satisfied under some extra technical conditions (see, e.g., Roberts and Tweedie 1996).
(P1 * * ) X ⊆ R d . The target π is log-Lipschitz, meaning that for some L > 0 (P1) holds taking the drift function V = π −s , for any s ∈ (0, 1). The proposal q is a random walk as in (17) satisfying for some a > 0.
See Appendix 1 for a proof of the following proposition.

Conditions for arithmetic averages
In the particular setting where the weights are given by (4), sufficient conditions on these can be obtained to ensure geometric ergodicity is inherited by the noisy chain. For the simple case where the weights are homogeneous with respect to the state space (W1) is automatically satisfied. In order to attain (W2), the existence of a negative moment for a single weight is required. See Appendix 1 for a proof of the following result. (4).

Proposition 3.2 Assume weights as in
For homogeneous weights, (18) implies (W2). When the weights are not homogeneous, stronger conditions are needed for (W1) and (W2) to be satisfied. An appropriate first assumption is that the weights are uniformly integrable.
The second condition imposes an additional assumption on the distribution of the weights {W x } x near 0.
Then, there exists N 0 ∈ N + such that for all N ≥ N 0 , the noisy chain with transition kernelP N is geometrically ergodic.
The proof of Proposition 3.3 follows the statement of Lemma 3.5, whose proof can be found in Appendix 1. This lemma allows us to characterise the distribution of W x,N near 0 assuming (W4) and also provides conditions for the existence and convergence of negative moments.
(i) Suppose Z is a positive random variable, and assume that for z ∈ (0, γ ) Then, is a collection of positive and independent random variables, and assume that for each i ∈ {1, . . . , N } and z ∈ (0, γ ) Then, for z ∈ (0, γ )

(iii) Let the weights be as in (4). If for some N
(iv) Assume (W1) and let g : R + → R be a function that is continuous at 1 and bounded on the interval [γ, ∞). Then To prove (ii), assume (W4) and by part (ii) of Lemma 3.5, for w ∈ (0, γ ) Take p > 1 and define N 0 := p β + 1, then using part (i) of Hence, by Hölder's inequality and applying part (iii) of Lemma 3.5, for N ≥ N 0 Since γ < 1 and by (W1) Now, for fixed γ ∈ (0, 1) the function g(x) = x −1 is bounded and continuous on [γ, ∞), implying by part (iv) of Lemma 3.5 Finally, using (19) and (20) lim N →∞ sup x∈X E W −1 x,N − 1 = 0, and by the triangle inequality the result follows.

Remarks on results
Equipped Despite conditions (W1) and (W2) guaranteeing the inheritance of geometric ergodicity for the noisy chain, they are not necessary. Consider a modification of the example in Sect. 2.3, where the weights are given by where b m > 1 and ε m ∈ (0, 1] for all m ≥ 1. Again, there exists a relationship between the variables b m , ε m and s m for ensuring the expectation of the weights is equal to one. Let Bin (N , s) denote a binomial distribution of parameters N ∈ N + and s ∈ (0, 1). Then, in the arithmetic average context, W m,N becomes where b m > 1 and ε m ∈ (0, 1] for all m ≥ 1. For particular choices of the sequences {b m } m∈N + and {ε m } m∈N + , the resulting noisy chain can be geometrically ergodic for all N ≥ 1, even though neither (W1) nor (W2) hold.

Proposition 3.4 Consider a geometric target density as in
(11) and a proposal density as in (12). In addition, let the weights be as in (21)  Then, the chain generated by the noisy kernelP N is geometrically ergodic for any N ∈ N + .
Finally, in many of the previous examples, increasing the value of N seems to improve the ergodic properties of the noisy chain. However, the geometric ergodicity property is not always inherited, no matter how large N is taken. The following proposition shows an example rather similar to Proposition 3.4, but in which the ratio ε m−1 ε m does not converge as m → ∞.
Proposition 3.5 Consider a geometric target density as in (11) and a proposal density as in (12). In addition, let the weights be as in (21)  Then, the chain generated by the noisy kernelP N is transient for any N ∈ N + .

Convergence of the noisy invariant distribution
So far the only concern has been whether the noisy chain inherits the geometric ergodicity property from the marginal chain. As an immediate consequence, geometric ergodicity guarantees the existence of an invariant probability distributionπ N forP N , provided N is large enough. In addition, using the same conditions from Sect. 3, we can characterise and in some cases quantify the convergence in total variation ofπ N towards the desired target π , as N → ∞.

Convergence in total variation
The following definition, taken from Roberts et al. (1998), characterises a class of kernels satisfying a geometric drift condition as in (16) for the same V , C, λ and b.
Definition 4.1 (Simultaneous geometric ergodicity) A class of Markov chain kernels {P k } k∈K is simultaneously geometrically ergodic if there exists a class of probability measures {ν k } k∈K , a measurable set C ⊆ X , a real valued measurable function V ≥ 1, a positive integer n 0 and positive constants ε, λ, b such that for each k ∈ K: (i) C is small for P k , with P n 0 k (x, ·) ≥ εν k (·) for all x ∈ C; (ii) the chain P k satisfies the geometric drift condition in (16) with drift function V , small set C and constants λ and b.
Provided N is large, the noisy kernels {P N +k } k≥0 together with the marginal P will be simultaneous geometrically ergodic. This will allow the use of coupling arguments for ensuringπ N and π get arbitrarily close in total variation. The main additional assumption is (P2) For some ε > 0, some probability measure ν(·) on (X , B(X )) and some subset C ⊆ X , the marginal acceptance probability α and the proposal kernel q satisfy α(x, y)q(x, dy) ≥ εν(dy), for x ∈ C.
Remark 4.1 (P2) ensures the marginal chain satisfies the minorisation condition in (15), purely attained by the sub-kernel α(x, y)q(x, dy). This occurs under fairly mild assumptions (see, e.g., Roberts and Tweedie 1996, Theorem 2.2). (i) there exists N 0 ∈ N + such that the class of kernels P,P N 0 ,P N 0 +1 , . . . is simultaneously geometrically ergodic; Part (iii) of the above theorem is mainly a consequence of Roberts et al. (1998, Theorem 9) when parts (i) and (ii) hold. Indeed, by the triangle inequality, Provided N ≥ N 0 , the first two terms in (22) can be made arbitrarily small by increasing n. In addition, due to the simultaneous geometrically ergodic property, the first term in (22) is uniformly controlled regardless the value of N . Finally, using an inductive argument, part (ii) implies that for all x ∈ X and all n ∈ N + lim N →∞ P n N (x, ·) − P n (x, ·) T V = 0.
Proof of Theorem 4.1 From the proofs of Theorems 3.2 and 3.3, there exists N 1 ∈ N + such that the class of kernels P,P N 1 ,P N 1 +1 , . . . satisfies condition (ii) in Definition 4.1 for the same function V , small set C and constants λ N 1 , b N 1 . Respecting (i), for any δ ∈ (0, 1) Then, by Lemma 3.1 Due to (P2), To prove (ii) apply Lemma 3.2 and Lemma 3.4 to get sup A∈B(X ) Finally, taking N → ∞ and by (W1) The result follows since η and δ can be taken arbitrarily small. For (iii), see Theorem 9 in Roberts et al. (1998) for a detailed proof.

Remark 4.2 A Wasserstein distance variant of part (iii) in
Theorem 4.1 has been proved in Rudolf and Schweizer (2015, Corollary 28), in which control of the difference betweenα N and α is still required and can be obtained using (W1).
-IfΦ N n−1 = Φ n−1 , draw independently Φ n ∼ P(y, ·) and Φ N n ∼P N (y, ·). Since and noting an induction argument can be applied to obtain Therefore, using the coupling inequality, the third term in (22) can be bounded by On the other hand, using the simultaneous geometric ergodicity of the kernels and provided N is large enough, the noisy and marginal kernels will each satisfy a geometric drift condition as in (16) with a common drift function V ≥ 1, small set C and constants λ, b. Therefore, by Theorem 3.1, there exist R > 0, and τ < 1 such that Explicit values for R and τ are in principle possible, as done in Rosenthal (1995) and Meyn and Tweedie (1994). For simplicity assume inf x∈X V (x) = 1, then combining (24) and (25) in (22), for all n ∈ N + So, if an analytic expression in terms of N is available for the second term on the right hand side of (26), it will be possible to obtain an explicit rate of convergence forπ N and π .
In any case Finally take N large enough such that log 2Rr(N ) log τ −1 ≥1, and from (26) obtaining the result.
Remark 4.3 A general result bounding the total variation between the law of a Markov chain and a perturbed version is presented in Rudolf and Schweizer (2015, Theorem 21). This is done using the connection between the V -norm distance and the Wasserstein distance introduced in Hairer and Mattingly (2011). With such a result, and considering the same assumptions in Theorem 4.2, one could in principle obtain an explicit value for D in (27).
Moreover, when the weights are expressed in terms of arithmetic averages as in (4), an explicit expression for r (N ) can be obtained whenever there exists a uniformly bounded moment. This is a slightly stronger assumption than (W3).
(W5) There exists k > 0, such that the weights {W x } x satisfy .
If in addition (W5) holds for all k > 0, then for any ε ∈ (0, 1) there will exist D ε > 0 and N 0 ∈ N + such that for all N ≥ N 0 ,

Discussion
In this article, fundamental stability properties of the noisy algorithm have been explored. The noisy Markov kernels considered are perturbed Metropolis-Hastings kernels defined by a collection of state-dependent distributions for non-negative weights all with expectation 1. The general results do not assume a specific form for these weights, which can be simple arithmetic averages or more complex random variables. The former may arise when unbiased importance sampling estimates of a target density are used, while the latter may arise when such densities are estimated unbiasedly using a particle filter. Two different sets of sufficient conditions were provided under which the noisy chain inherits geometric ergodicity from the marginal chain. The first pair of conditions, (W1) and (W2), involve a stronger version of the Law of Large Numbers for the weights and uniform convergence of the first negative moment, respectively. For the second set, (W1) is still required but (W2) can be replaced with (P1*), which imposes a condition on the proposal distribution. These conditions also imply simultaneous geometric ergodicity of a sequence of noisy Markov kernels together with the marginal Markov kernel, which then ensures that the noisy invariant π N converges to π in total variation as N increases. Moreover, an explicit bound for the rate of convergence betweeñ π N and π is possible whenever an explicit bound (that is uniform in x) is available for the convergence betweenP N (x, ·) and P(x, ·).
When weights are arithmetic averages as in (4), specific conditions were given for inheriting geometric ergodicity from the corresponding marginal chain. The uniform integrability condition in (W3) ensures that (W1) is satisfied, whereas (W4) is essential for satisfying (W2). Regarding the noisy invariant distributionπ N , (W5), which is slightly stronger than (W3), leads to an explicit bound on the rate of convergence of this distribution to π .
The noisy algorithm remains undefined when the weights have positive probability of being zero. If both weights were zero one could accept the move, reject the move or keep sampling new weights until one of them is not zero. Each of these lead to different behaviour.
As seen in the examples of Sect. 3.4, the behaviour of the ratio of the weights (at least in the tails of the target) plays an important role in the ergodic properties of the noisy chain. In this context, it seems plausible to obtain geometric noisy chains, even when the marginal is not, if the ratio of the weights decays sufficiently fast to zero in the tails. Another interesting possibility, that may lead to future research, is to relax the condition on the expectation of the weights to be identically one.
Acknowledgments All authors would like to thank the EPSRCfunded Centre for Research in Statistical methodology (EP/D002060/1). The first author was also supported by Consejo Nacional de Ciencia y Tecnología. The third author's research was also supported by the EPSRC Programme Grant i-like (EP/K014463/1).
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecomm ons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Appendix 1: Proofs On state-dependent random walks
The following proposition for state-dependent Markov chains on the positive integers will be useful for addressing some proofs. See Norris (1999) for a proof of parts (i) and (ii), for part (iii) see Callaert and Keilson (1973), which is proved within the birth-death process context.
If k = +∞, it is clear that the limit in (29) diverges, consequently the noisy chain is geometrically ergodic according to Proposition 5.1. If k < ∞, the noisy chain will be geometrically ergodic if or equivalently to Now consider two cases, first if P k −1 < Z < k > 0 then it is clear that which satisfies (30). Finally, if P k −1 < Z < k = 0 then implying from (28) and leading to (30).
Let a m := m j=2 c j , then a sufficient condition for the series ∞ m=2 a m to converge, implying a transient chain according to Proposition 5.1, is l 2 < 1. This is the case for b ≥ 3 + 1−θ θ 3 , since Hence, the resulting noisy chain is transient if b ≥ 3 + 1−θ θ 3 , for any θ ∈ (0, 1).

Section 3
Proof of Lemma 3.1 For any δ > 0 Finally, using Lemma 3.1 Proof of Lemma 3.3 For the first claim apply Jensen's inequality and the fact that Proof of Lemma 3.4 Using the inequality Finally, using the transformation u = z − x, which implies (P1*).
Proof of Proposition 3.2 By properties of the arithmetic and harmonic means which implies, by Jensen's inequality, Then, using Fatou's lemma and the law of large numbers Finally, since Proof of Lemma 3.5 The proof of (i) is motivated by Piegorsch and Casella (1985, Theorem 2.1) and Khuri and Casella (2002, Theorem 3), however the existence of a density function is not assumed here. Since Z − p is positive, For part (ii), since the random variables {Z i } are positive, then for any z > 0 Therefore, for z ∈ (0, γ ) Part (iii) can be seen as a consequence of W x,N and W x,N +1 being convex ordered and g(x) = x − p being a convex function for x > 0 and p ≥ 0, (see, e.g., Andrieu and Vihola 2014). We provide a self-contained proof by defining for j ∈ {1, . . . , N + 1} S ( j) x,N :=  ( j) x,N and since the arithmetic mean is greater than or equal to the geometric mean where Hölder's inequality has been used and the fact that the random variables S ( j) x,N : j ∈ 1, . . . , N + 1 are identically distributed according to Q x,N .
For part (iv), let M γ = sup y∈[γ,∞) |g(y)| and due to continuity at y = 1, for any ε > 0 there exists a δ > 0 such that Therefore, for fixed ε and by (W1) and because ε m → 0, it is clear that (l + δ) m → ∞ as m → ∞. Therefore, l + δ > 1 and since δ can be taken arbitrarily small, it is true that l ≥ 1. Now, for weights as in (21) and using a simple random walk proposal, the noisy acceptance probability can be expressed as Since b m → ∞, then s m → 0 as m → ∞; therefore, any term in (35) and (36), for which j + k = 0, tends to zero as m → ∞. Take δ small enough, such that (C + δ) 2 δ < 1, hence then the convex function f : R + → R + where f (s) = 3s + 2 3+k C k s 1+k N k , is minimised at Applying Theorem 4.2 by taking r (N ) ∝ N 1− 2 2+k and noting log N 1− 2 2+k ≤ log(N ), the result is obtained.