Appendix
Proof of Proposition 1
We prove this result by induction. For \(t = 1\), let
$$\begin{aligned} \pi _1({\mathrm {d}}\theta ) = \pi _0({\mathrm {d}}\theta ) \frac{G_1(\theta )}{\int _\varTheta G_1(\theta ) \pi _0({\mathrm {d}}\theta )} = \pi _0({\mathrm {d}}\theta ) \frac{G_1(\theta )}{(G_1,\pi _0)}. \end{aligned}$$
Since \(G_1 \in B(\varTheta )\) it follows that
$$\begin{aligned} \sup _{\theta \in \varTheta } \left| \frac{G_1(\theta )}{(G_1,\pi _0)}\right| = \frac{\sup _{\theta \in \varTheta } G_1(\theta )}{(G_1,\pi _0)} < \infty \end{aligned}$$
because of Assumption 1. Hence \(\pi _1 \ll \pi _0\) is a proper measure. Assume next, as an induction hypothesis, that \(\pi _{T-1} \ll \pi _0\). Then
$$\begin{aligned} \pi _T({\mathrm {d}}\theta ) = \pi _{T-1}({\mathrm {d}}\theta ) \frac{G_T(\theta )}{(G_T,\pi _{T-1})} \end{aligned}$$
and Assumption 1 implies (again) that
$$\begin{aligned} \frac{\sup _{\theta \in \varTheta } G_T(\theta )}{(G_T,\pi _{T-1})} < \infty , \end{aligned}$$
hence \(\pi _T\) is proper and \(\pi _T \ll \pi _0\). Therefore, the Radon-Nikodym derivative of the final measure \(\pi _T\) w.r.t. the prior \(\pi _0\) is
$$\begin{aligned} \frac{{\mathrm {d}}\pi _T}{{\mathrm {d}}\pi _0}(\theta ) \propto \prod _{t=1}^T G_t(\theta ) = \exp \left( -\sum _{i=1}^n f_i(\theta )\right) . \end{aligned}$$
From here, it readily follows that maximizing this Radon-Nikodym derivative is equivalent to solving problem (1.1). \(\square \)
Proof of Theorem 1
We proceed by an induction argument. At time \(t = 0\), the bound
$$\begin{aligned} \Vert (\varphi ,\pi _0^N) - (\varphi ,\pi _0) \Vert _p \le \frac{c_{0,p} \Vert \varphi \Vert _\infty }{\sqrt{N}} \end{aligned}$$
is a straightforward consequence of the Marcinkiewicz–Zygmund inequality (Shiryaev 1996) because the particles \(\{\theta _0^{(i)}\}_{i=1}^N\) are i.i.d samples from \(\pi _0\).
Assume now that, after iteration \(t-1\), we have a particle set \(\{{\theta }_{t-1}^{(i)}\}_{i=1}^N\) and the empirical measure \(\pi ^N_{t-1}(\text{ d }\theta _{t-1}) = \frac{1}{N} \sum _{i=1}^N \delta _{\theta _{t-1}^{(i)}} (\text{ d }\theta _{t-1})\), which satisfies
$$\begin{aligned} \left\| (\varphi ,\pi _{t-1}) - (\varphi ,\pi _{t-1}^N)\right\| _p \le \frac{c_{t-1,p} \Vert \varphi \Vert _\infty }{\sqrt{N}}. \end{aligned}$$
(A.1)
We first analyze the error in the jittering step. To this end, we construct the jittered random measure
$$\begin{aligned} \hat{\pi }^N_t({\mathrm {d}}\theta ) = \frac{1}{N} \sum _{i=1}^N \delta _{\hat{\theta }_t^{(i)}}({\mathrm {d}}\theta ) \end{aligned}$$
and iterate the triangle inequality to obtain
$$\begin{aligned}&\Vert (\varphi ,\pi _{t-1}) - (\varphi ,\hat{\pi }_t^N)\Vert _p \le \Vert (\varphi ,\pi _{t-1}) - (\varphi ,\pi ^N_{t-1})\Vert _p \nonumber \\&\quad +\Vert (\varphi ,\pi _{t-1}^N) - (\varphi ,\kappa {\pi }_{t-1}^N)\Vert _p \nonumber \\&\quad + \Vert (\varphi ,\kappa \pi _{t-1}^N) - (\varphi ,\hat{\pi }_t^N)\Vert _p, \end{aligned}$$
(A.2)
where
$$\begin{aligned} \kappa \pi _{t-1}^N = \int \kappa (\text{ d }\theta |\theta _{t-1}) \pi _{t-1}^N(\text{ d }\theta _{t-1}) = \frac{1}{N} \sum _{i=1}^N \kappa (\text{ d }\theta |{\theta _{t-1}^{(i)}}). \end{aligned}$$
The first term on the right-hand side (rhs) of (A.2) is bounded by the induction hypothesis (A.1). For the second term, we note that,
$$\begin{aligned}&\left| (\varphi ,\pi _{t-1}^N) - (\varphi ,\kappa {\pi }_{t-1}^N)\right| \nonumber \\&\quad = \left| \frac{1}{N} \sum _{i=1}^N \varphi (\theta _{t-1}^{(i)}) - \frac{1}{N} \sum _{i=1}^N \int \varphi (\theta ) \kappa (\text{ d }\theta |{\theta _{t-1}^{(i)}}) \right| \nonumber \\&\quad = \left| \frac{1}{N} \sum _{i=1}^N \int \left( \varphi (\theta _{t-1}^{(i)}) - \varphi (\theta )\right) \kappa (\text{ d }\theta |{\theta _{t-1}^{(i)}}) \right| \nonumber \\&\quad \le \frac{1}{N} \sum _{i=1}^N \int \left| \varphi (\theta _{t-1}^{(i)}) - \varphi (\theta ) \right| \kappa (\text{ d }\theta |{\theta _{t-1}^{(i)}})\nonumber \\&\quad \le \frac{c_\kappa \Vert \varphi \Vert _\infty }{\sqrt{N}}, \end{aligned}$$
(A.3)
where the last inequality follows from Assumption 2. The upper bound in (A.3) is deterministic, so the inequality readily implies that
$$\begin{aligned} \Vert (\varphi ,\pi _{t-1}^N) - (\varphi ,\kappa \pi _{t-1}^N)\Vert _p \le \frac{c_\kappa \Vert \varphi \Vert _\infty }{\sqrt{N}}. \end{aligned}$$
(A.4)
For the last term on the right-hand side of (A.2), we let \({{\mathcal {F}}}_{t-1}\) be the \(\sigma \)-algebra generated by the random sequence \(\{\theta _{0:t-1}^{(i)},\hat{\theta }_{1:t-1}^{(i)}\}_{i=1}^N\). Let us first note that
$$\begin{aligned} {\mathbb E}\left[ (\varphi ,\hat{\pi }_t) | {{\mathcal {F}}}_{t-1}\right]&= \frac{1}{N} \sum _{i=1}^N {\mathbb E}\left[ \varphi (\hat{\theta }_t^{(i)}) | {{\mathcal {F}}}_{t-1} \right] \\&= \frac{1}{N}\sum _{i=1}^N \int \varphi (\theta ) \kappa ({\mathrm {d}}\theta |\theta _{t-1}^{(i)}) = (\varphi ,\kappa \pi _{t-1}^N). \end{aligned}$$
Therefore, the difference \((\varphi ,\hat{\pi }_t^N) -(\varphi ,\kappa \pi _{t-1}^N)\) takes the form
$$\begin{aligned} (\varphi ,\hat{\pi }_t^N) - (\varphi ,\kappa \pi _{t-1}^N) = \frac{1}{N} \sum _{i=1}^N S^{(i)}, \end{aligned}$$
where \(S^{(i)} = \varphi (\hat{\theta }_t^{(i)}) - {\mathbb E}[\varphi (\hat{\theta }_t^{(i)})|{{\mathcal {F}}}_{t-1}]\), \(i = 1,\ldots ,N\), are zero-mean and conditionally independent random variables, with \(|S^{(i)}| \le 2 \Vert \varphi \Vert _\infty \). Then, we readily obtain the bound
$$\begin{aligned} {\mathbb E}\left[ \left. \left| (\varphi ,\hat{\pi }_t^N) - (\varphi ,\kappa \pi _{t-1}^N)\right| ^p \right| {{\mathcal {F}}}_{t-1}\right]&= \frac{1}{N^p} {\mathbb E}\left[ \left. \left| \sum _{i=1}^N S^{(i)}\right| ^p\right| {{\mathcal {F}}}_{t-1}\right] \nonumber \\&\le \frac{B_{t,p} N^{\frac{p}{2}}\Vert \varphi \Vert _\infty ^p}{N^p}. \end{aligned}$$
(A.5)
where the relation (A.5) follows from the Marcinkiewicz–Zygmund inequality (Shiryaev 1996) and \(B_{t,p} < \infty \) is some constant independent of N. Taking unconditional expectations on both sides of (A.5) and then computing \((\cdot )^\frac{1}{p}\) yields
$$\begin{aligned} \Vert (\varphi ,\hat{\pi }_t^N) - (\varphi ,\kappa \pi _{t-1}^N)\Vert _p \le \frac{\hat{c}_{t,p} \Vert \varphi \Vert _\infty }{\sqrt{N}}. \end{aligned}$$
(A.6)
where \(\hat{c}_{t,p} = B_{t,p}^{\frac{1}{p}}\) is a finite constant independent of N. Therefore, taking together (A.1), (A.4) and (A.6) we have established that
$$\begin{aligned} \Vert (\varphi ,\pi _{t-1}) - (\varphi ,\hat{\pi }_t^N)\Vert _p \le&\frac{c_{1,t,p} \Vert \varphi \Vert _\infty }{\sqrt{N}}, \end{aligned}$$
(A.7)
where \(c_{1,t,p} = c_{t-1,p} + c_\kappa + \hat{c}_{t,p} < \infty \) is a finite constant independent of N.
Next, we have to bound the error after the weighting step. We recall that
$$\begin{aligned} \pi _t(\text{ d }\theta ) = \pi _{t-1}(\text{ d }\theta ) \frac{G_t(\theta )}{(G_t,\pi _{t-1})} \end{aligned}$$
and define
$$\begin{aligned} \tilde{\pi }_t^N(\text{ d }\theta ) = \hat{\pi }^N_{t}(\text{ d }\theta ) \frac{G_t(\theta )}{(G_t,\hat{\pi }^N_{t})} \end{aligned}$$
where \(\tilde{\pi }_t^N\) denotes the weighted measure. We first note that
$$\begin{aligned}&|(\varphi ,\pi _t) - (\varphi ,\tilde{\pi }_t^N)| = \left| \frac{(\varphi G_t, \pi _{t-1})}{(G_t,\pi _{t-1})} - \frac{(\varphi G_t, \hat{\pi }_t^N)}{(G_t,\hat{\pi }_t^N)} \pm \frac{(\varphi G_t, \hat{\pi }^N_t)}{(G_t,\pi _{t-1})} \right| \nonumber \\&\le \frac{\left| (\varphi G_t, \pi _{t-1}) - (\varphi G_t, \hat{\pi }_t^N)\right| + \Vert \varphi \Vert _\infty |(G_t,\hat{\pi }_t^N) - (G_t,\pi _{t-1})|}{(G_t,\pi _{t-1})}. \end{aligned}$$
(A.8)
Using Minkowski’s inequality together with (A.7) and (A.8) yields
$$\begin{aligned} \Vert (\varphi ,\pi _t) - (\varphi ,\tilde{\pi }_t^N)\Vert _p&\le \frac{c_{1,t,p} \Vert \varphi G_t\Vert _\infty + c_{1,t,p} \Vert \varphi \Vert _\infty \Vert G_t\Vert _\infty }{(G_t,\pi _{t-1})\sqrt{N}}, \\&\le \frac{2 c_{1,t,p} \Vert \varphi \Vert _\infty \Vert G_t\Vert _\infty }{(G_t,\pi _{t-1})\sqrt{N}} \end{aligned}$$
where the second inequality follows from \(\Vert \varphi G_t \Vert _\infty \le \Vert \varphi \Vert _\infty \Vert G_t\Vert _\infty \). More concisely, we have
$$\begin{aligned} \Vert (\varphi ,\pi _t) - (\varphi ,\tilde{\pi }_t^N)\Vert _p \le \frac{c_{2,t,p} \Vert \varphi \Vert _\infty }{\sqrt{N}} \end{aligned}$$
(A.9)
where the constant
$$\begin{aligned} c_{2,t,p} = \frac{2 c_{1,t,p} \Vert G_t\Vert _\infty }{(G_t,\pi _{t-1})} < \infty \end{aligned}$$
is independent of N. Note that the assumptions on \((G_t)_{t\ge 1}\) imply that \((G_t,\pi _{t-1}) > 0\).
Finally, we bound the resampling step. Note that the resampling step consists of drawing N i.i.d samples from \(\tilde{\pi }_t^N\), i.e., \(\theta _t^{(i)} \sim \tilde{\pi }_t^N\) i.i.d for \(i = 1, \ldots ,N\), and then constructing
$$\begin{aligned} \pi ^N_t(\text{ d }\theta ) = \frac{1}{N} \sum _{i=1}^N \delta _{\theta _t^{(i)}}(\text{ d }\theta ). \end{aligned}$$
Since samples are i.i.d, as in the base case, we have,
$$\begin{aligned} \Vert (\varphi ,\tilde{\pi }_t^N) - (\varphi ,\pi _t^N)\Vert _p \le \frac{\tilde{c}_p \Vert \varphi \Vert _\infty }{\sqrt{N}}, \end{aligned}$$
(A.10)
for some constant \(\tilde{c}_p < \infty \) independent of N. Now combining (A.9) and (A.10), we have the desired result,
$$\begin{aligned} \Vert (\varphi ,\pi _t) - (\varphi ,\pi _t^N)\Vert _p \le \frac{c_t \Vert \varphi \Vert _\infty }{\sqrt{N}} \end{aligned}$$
where \(c_t = c_{2,t,p} + \tilde{c}_p\) is a finite constant independent of N. \(\square \)
Proof of Corollary 1
From Theorem 1, we obtain
$$\begin{aligned} \Vert (\varphi ,\pi _t) - (\varphi ,\pi _t^N)\Vert _p \le \frac{c_t \Vert \varphi \Vert _\infty }{\sqrt{N}}, \end{aligned}$$
where \(c_t < \infty \) is a constant independent of N. Let us choose \(p\ge 4\) and \(0< \epsilon < 1\). We construct the nonnegative random variable
$$\begin{aligned} U_{t,\epsilon }^{p} = \sum _{N=1}^\infty N^{\frac{p}{2} - 1 - \epsilon } |(\varphi ,\pi _t) - (\varphi ,\pi _t^N)|^p. \end{aligned}$$
and use Fatou’s lemma to obtain
$$\begin{aligned} {\mathbb E}[U_{t,\epsilon }^{p}]&\le \sum _{N=1}^\infty N^{\frac{p}{2} - 1 - \epsilon } {\mathbb E}\left[ \left| (\varphi ,\pi _t) - (\varphi ,\pi ^N_t)\right| ^p \right] ,\nonumber \\&\le c^p \Vert \varphi \Vert _\infty ^p \sum _{N=1}^\infty N^{- 1 - \epsilon } < \infty , \end{aligned}$$
(A.11)
where the second inequality follows from Theorem 1. The relationship (A.11) implies that the r.v. \(U^p_{t,\epsilon }\) is a.s. finite.
Finally, since (trivially) \(N^{\frac{p}{2} - 1 - \epsilon } |(\varphi ,\pi _t) - (\varphi ,\pi _t^N)|^p \le U_{t,\epsilon }^{p}\), we have
$$\begin{aligned} |(\varphi ,\pi _t) - (\varphi ,\pi _t^N)| \le \frac{U_{t,\delta }}{N^{\frac{1}{2} - \delta }}, \end{aligned}$$
(A.12)
where \(\delta = \frac{1 + \epsilon }{p}\) and \(U_{t,\delta } = (U_{t,\epsilon }^{p})^{\frac{1}{p}}\). Since \(p\ge 4\) and \(0< \epsilon < 1\), it follows that \(0< \delta < \frac{1}{2}\). The almost sure convergence follows from (A.12). Taking \(N\rightarrow \infty \) yields
$$\begin{aligned} \lim _{N\rightarrow \infty } |(\varphi ,\pi _t) - (\varphi ,\pi _t^N)| = 0 \quad \quad \text {a.s.} \end{aligned}$$
\(\square \)
Proof of Proposition 2
Recall the assumption
$$\begin{aligned} |F_t(\theta ) - F_t(\theta ')| \le \ell _t \Vert \theta -\theta '\Vert . \end{aligned}$$
We write \(F_t^\star = \min _{\theta \in \varTheta } F_t(\theta )\), which is assumed to be finite, but not necessarily nonnegative. We first prove that \(\exp (-F_t(\theta ))\) is also Lipschitz continuous. Note that we trivially have \(\exp (-F_t(\theta )) \le \exp (-F^\star _t)\) for all \(\theta \) since \(F_t(\theta ) \ge F_t^\star \) for all \(\theta \). Now consider any \((\theta ,\theta ') \in \varTheta \times \varTheta \). We first consider the case where \(F_t(\theta ) \le F_t(\theta ')\). We obtain
$$\begin{aligned} 0 < e^{-F_t(\theta )} - e^{-F_t(\theta ')}&= e^{-F_t(\theta )} \left( 1 - e^{F_t(\theta ) - F_t(\theta ')}\right) ,\nonumber \\&\le e^{-F_t(\theta )} \left( 1 - (1 + F_t(\theta ) - F_t(\theta '))\right) , \end{aligned}$$
(A.13)
where we have used the inequality \(e^a \ge 1 + a\). Therefore, we readily obtain from (A.13)
$$\begin{aligned} 0< e^{-F_t(\theta )} - e^{-F_t(\theta ')}&\le e^{-F_t(\theta )} \left( F_t(\theta ') - F_t(\theta )\right) ,\nonumber \\&\le e^{-F_t^\star } \left( F_t(\theta ') - F_t(\theta )\right) \end{aligned}$$
(A.14)
$$\begin{aligned}&= e^{-F_t^\star } |F_t(\theta ') - F_t(\theta )|, \end{aligned}$$
(A.15)
since \(F_t(\theta ) \le F_t(\theta ')\). Next, assume otherwise, i.e., \(F_t(\theta ) \ge F_t(\theta ')\). In this case, we can also show using the same line of reasoning that
$$\begin{aligned} e^{-F_t(\theta ')} - e^{-F_t(\theta )}&\le e^{-F_t^\star } \left( F_t(\theta ) - F_t(\theta ')\right) \end{aligned}$$
(A.16)
$$\begin{aligned}&= e^{-F_t^\star } |F_t(\theta ') - F_t(\theta )|, \end{aligned}$$
(A.17)
since \(F_t(\theta ) \ge F_t(\theta ')\). Therefore, we can conclude (combining (A.15) and (A.16)) that
$$\begin{aligned} |e^{-F_t(\theta )} - e^{-F_t(\theta ')}| \le e^{-F_t^\star } |F_t(\theta ') - F_t(\theta )| \le e^{-F_t^\star } \ell _t \Vert \theta - \theta '\Vert , \end{aligned}$$
where the last inequality holds because \(F_t\) is Lipschitz. Finally recall that
$$\begin{aligned} \pi _t(\theta ) = \frac{e^{-F_t(\theta )}}{Z_{\pi _t}}, \end{aligned}$$
where we denote \(Z_{\pi _t} = \int _\varTheta e^{-F_t(\theta )}{\mathrm {d}}\theta \). We straightforwardly obtain
$$\begin{aligned} |\pi _t(\theta ) - \pi _t(\theta ')| \le \frac{1}{Z_{\pi _t}} e^{-F_t^\star } \ell _t \Vert \theta - \theta '\Vert . \end{aligned}$$
\(\square \)
Proof of Theorem 2
Using the proof of Theorem 4.2 and Corollary 4.1 in Crisan and Míguez (2014), we obtain
$$\begin{aligned} \sup _{\theta \in \varTheta } | {{\mathsf {p}}}_t^N(\theta ) - \pi _t(\theta )| \le \frac{V_{1,\varepsilon }}{\left\lfloor {N^{\frac{1}{2 (d + 1)}}}\right\rfloor ^{1-\varepsilon }}, \end{aligned}$$
where \(V_{1,\varepsilon }\) is an a.s. finite random variable. Noting that
$$\begin{aligned} \sup _{a \ge 1} \frac{a}{\left\lfloor a \right\rfloor } = 2, \end{aligned}$$
we obtain
$$\begin{aligned} \sup _{\theta \in \varTheta } | {{\mathsf {p}}}_t^N(\theta ) - \pi _t(\theta )| \le \frac{V_\varepsilon }{N^{\frac{1-\varepsilon }{2 (d + 1)}}}, \end{aligned}$$
where \(V_\varepsilon = 2 V_{1,\varepsilon }\) is an almost surely finite random variable. \(\square \)
Proof of Theorem 3
Recall that \(\pi _t(\theta )\) is a probability density w.r.t. the Lebesgue measure. Choose \(\theta _t^\star \in \arg \max _{\theta \in \varTheta } \pi _t(\theta )\) and construct the ball
$$\begin{aligned} B_{t,n}^\star := B\left( \theta _t^\star , \frac{1}{n} \right) \subset \varTheta \end{aligned}$$
where \(n \ge 1\) is a positive integer. We assume, without loss of generality, that \(B_{t,1}^\star \subseteq \varTheta \) and denote
$$\begin{aligned} \pi _t(B_{t,n}^\star ) = \int _{B_{t,n}^\star } \pi _t(\theta ) {\mathrm {d}}\theta \quad \text{ and } \quad \pi _t^N(B_{t,n}^\star ) = \int _{B_{t,n}^\star } \pi _t^N({\mathrm {d}}\theta ). \end{aligned}$$
Also recall that the grid of points generated by the SMC sampler at time t is \(\{ \theta _t^{(i)} \}_{1 \le i \le N} \subset \varTheta \) and the estimate of \(\theta _t^\star \) obtained from the grid is denoted
$$\begin{aligned} \theta _t^{\star ,N} \in \arg \max _{\theta \in \{ \theta _t^{(i)} \}_{1 \le i \le N} } {\textsf {p}}_t^N(\theta ), \end{aligned}$$
(A.18)
where \({\textsf {p}}_t^N(\theta )\) is the kernel density estimator of \(\pi _t\). Our argument to prove Theorem 3 proceeds in two steps:
-
1.
We show that, for any given \(n \ge 1\), one can a.s. find N sufficiently large to ensure that \(\{ \theta _t^{(i)} \}_{1 \le i \le N} \cap B_{t,n}^\star \ne \emptyset \), i.e., that there are points of the grid contained in the ball \(B_{t,n}^\star \). Moreover, we deduce an inequality that relates the radius \(n^{-1}\) of the ball \(B_{t,n}^\star \) with the number of necessary particles N.
-
2.
From the existence of at least one particle \(\theta _t^{(i)}\) inside \(B_{t,n}^\star \) and the assumption that \(\pi _t(\theta )\) is Lipschitz, we deduce bounds for the differences \(|\pi _t(\theta _t^\star )-\pi _t(\theta _t^{(i)})|\) and \(|\pi _t(\theta _t^{\star ,N})-\pi _t(\theta _t^{(i)})|\), and, as a consequence, for the approximation error \(|\pi _t(\theta _t^{\star ,N})-\pi _t(\theta _t^\star )|\).
The ball \(B_{t,n}^\star \) is a.s. non-empty
Since \(\pi _t(\theta )\) is assumed continuous at every \(\theta _t^\star \in \arg \max _{\theta \in \varTheta }\)\( \pi _t(\theta )\), we have \(\pi _t(B_{t,n}^\star )>0\). Therefore, for every \(n<\infty \), Theorem 2 ensures that there exists \(N_n\) (a.s. finite) such that for all \(N \ge N_n\),
$$\begin{aligned} \left| \pi _t^N(B_{t,n}^\star ) - \pi _t(B_{t,n}^\star ) \right|< \frac{ U_{t,\delta } }{ N^{\frac{1}{2}-\delta } } < \frac{ \pi _t(B_{t,n}^\star )}{2}, \end{aligned}$$
(A.19)
where \(U_{t,\delta }\) is an a.s. finite random variable and \(\delta \in (0,\frac{1}{2})\) is an arbitrarily small constant (both independent of N). Moreover, the second inequality in (A.19) implies that
$$\begin{aligned} \pi _t^N(B_{t,n}^\star )> \frac{\pi _t(B_{t,n}^\star )}{2} > 0. \end{aligned}$$
(A.20)
Therefore, for all \(N>N_n\) there exists at least one integer \(i_b \in \{1, \ldots , N\}\) such that \(\theta _t^{(i_b)} \in B_{t,n}^\star \).
To be specific, since \(\pi _t(\theta )\) is a density w.r.t. the Lebesgue measure, we can readily find a lower bound for the integral \(\pi _t(B_{t,n}^\star )\), namely
$$\begin{aligned} \frac{\pi _t(B_{t,n}^\star )}{2}> \frac{1}{2} \text {Leb}\left( B_{t,n}^\star \right) \times \inf _{\theta \in B_{t,n}^\star } \pi _t(\theta ) > c_{t,d} n^{-d} \end{aligned}$$
where \(\text {Leb}(B_{t,n}^\star ) = \frac{\pi ^{\frac{d}{2}}}{\varGamma \left( \frac{d}{2}+1 \right) n^d}\) is the Lebesgue measure of the d-dimensional ball with radius \(n^{-1}\), \(\varGamma (\cdot )\) is Euler’s gamma function and
$$\begin{aligned} c_{t,d} := \frac{\pi ^{\frac{d}{2}}}{2 \varGamma \left( \frac{d}{2}+1 \right) n^d} \times \inf _{\theta \in B_{t,1}^\star } \pi _t(\theta ) > 0. \end{aligned}$$
Therefore, for any given \(n<\infty \), if we choose N such that \(0< \frac{ U_{t,\delta } }{ N^{\frac{1}{2}-\delta } } < c_{t,d}n^{-d} \), i.e.,
$$\begin{aligned} N \ge N_n := \left( \frac{ U_{t,\delta } }{ c_{t,d} } \right) ^{\frac{2}{1-2\delta }} n^{\frac{2d}{1-2\delta }} \end{aligned}$$
(A.21)
then the inequalities (A.19) and (A.20) hold a.s. (note that \(N_n<\infty \) a.s. because \(n<\infty \) and \(U_{t,\delta }<\infty \) a.s.).
Error bounds
Choose \(i_b \in \{1, \ldots , N\}\) such that \(\theta _t^{(i_b)} \in B_{t,n}^\star \). Such index exists a.s. whenever N satisfies the inequality (A.21). Let us recall the construction of the estimate \(\theta _t^{\star ,N}\) from expression (A.18) and denote
$$\begin{aligned} {\hat{\theta }}_t^{\star ,N} \in \arg \max _{\theta \in \varTheta } {\textsf {p}}_t^N(\theta ). \end{aligned}$$
Let \(L_t<\infty \) be the Lipschitz constant of the pdf \(\pi _t(\theta )\). Since \(\theta _t^{(i_b)} \in B_{t,n}^\star \), we readily obtain the upper bound
$$\begin{aligned} \pi _t(\theta _t^\star ) - \pi _t(\theta _t^{(i_b)}) < L_t n^{-1} \end{aligned}$$
and, therefore,
$$\begin{aligned} \pi _t(\theta _t^\star ) - L_t n^{-1} < \pi _t(\theta _t^{(i_b)}). \end{aligned}$$
(A.22)
However, using Theorem 2 we obtain
$$\begin{aligned} \left| \pi _t(\theta _t^{(i_b)}) - {\textsf {p}}_t^N(\theta _t^{(i_b)}) \right| < \frac{ V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} }, \end{aligned}$$
(A.23)
where \(\varepsilon \in (0,1)\) is an arbitrarily small constant and \(V_{t,\varepsilon }\) is an a.s. finite random variable, both independent of N. Combining (A.22) and (A.23) yields
$$\begin{aligned} {\textsf {p}}_t^N(\theta _t^{(i_b)})> \pi _t(\theta _t^{(i_b)}) - \frac{ V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} } > \pi _t(\theta _t^\star ) - L_t n^{-1} - \frac{ V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} } \end{aligned}$$
and, as a consequence,
$$\begin{aligned} {\textsf {p}}_t^N(\theta _t^{\star ,N}) \ge {\textsf {p}}_t^N(\theta _t^{(i_b)}) > \pi _t(\theta _t^\star ) - L_t n^{-1} - \frac{ V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} }. \nonumber \\ \end{aligned}$$
(A.24)
Moreover, using Theorem 2 again, we find that
$$\begin{aligned} \left| \pi _t({\hat{\theta }}_t^{\star ,N}) - {\textsf {p}}_t^N({\hat{\theta }}_t^{\star ,N}) \right| < \frac{ V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} }, \end{aligned}$$
(A.25)
with the same constant \(\varepsilon \in (0,1)\) and a.s. finite random variable \(V_{t,\varepsilon }\) as in (A.24). Since \(\pi _t({\hat{\theta }}_t^{\star ,N}) \le \pi _t(\theta _t^\star )\), the inequality (A.25) implies that
$$\begin{aligned} {\textsf {p}}_t^N({\hat{\theta }}_t^{\star ,N}) < \pi _t(\theta _t^\star ) + \frac{ V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} } \end{aligned}$$
and, since \({\textsf {p}}_t^N(\theta _t^{\star ,N}) \le {\textsf {p}}_t^N({\hat{\theta }}_t^{\star ,N})\), we arrive at
$$\begin{aligned} {\textsf {p}}_t^N(\theta _t^{\star ,N}) < \pi _t(\theta _t^\star ) + \frac{ V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} }. \end{aligned}$$
(A.26)
Taking the inequalities (A.24) and (A.26) together, we readily obtain the uniform bound (for \(\theta \in \varTheta \))
$$\begin{aligned} \left| \pi _t(\theta _t^\star ) - {\textsf {p}}_t^N(\theta _t^{\star ,N}) \right| < \frac{ V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} } + L_t n^{-1} \end{aligned}$$
(A.27)
and a simple triangle inequality then yields
$$\begin{aligned} \left| \pi _t(\theta _t^{\star ,N}) - \pi _t(\theta _t^\star ) \right|\le & {} \left| \pi _t(\theta _t^{\star ,N}) - {\textsf {p}}_t^N(\theta _t^{\star ,N}) \right| \nonumber \\&+ \left| {\textsf {p}}_t^N(\theta _t^{\star ,N}) - \pi _t(\theta _t^\star ) \right| \nonumber \\< & {} \frac{ 2V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} } + L_t n^{-1}, \end{aligned}$$
(A.28)
where the second inequality follows from (A.27) and yet another application of Theorem 2.
The inequality (A.28) holds for any pair of integers (N, n) that satisfies the relationship (A.21). For any given N, sufficiently large for
$$\begin{aligned} n_N := \sup \left\{ m \in \mathbb {N}: m^{-1} > \left( \frac{ U_{t,\delta } }{ c_{t,d} } \right) ^{\frac{1}{d}} \frac{1}{N^{\frac{1 - 2\delta }{2d}}} \right\} \end{aligned}$$
to be well defined, the pair consisting of N and \(n=n_N\) satisfies (A.21), while
$$\begin{aligned} n_N^{-1} \le 2 \left( \frac{ U_{t,\delta } }{ c_{t,d} } \right) ^{\frac{1}{d}} \frac{1}{N^{\frac{1 - 2\delta }{2d}}}. \end{aligned}$$
(A.29)
Hence, if we substitute \(n=n_N\) in the inequality (A.28) and then apply the inequality (A.29) we arrive at
$$\begin{aligned} \left| \pi _t(\theta _t^{\star ,N}) - \pi _t(\theta _t^\star ) \right| < \frac{ 2V_{t,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} } + 2 \left( \frac{ U_{t,\delta } }{ c_{t,d} } \right) ^{\frac{1}{d}} \frac{ L_t }{ N^{\frac{1 - 2\delta }{2d}} }, \end{aligned}$$
(A.30)
where \(V_{t,\varepsilon }\) and \(U_{t,\delta }\) are a.s. finite, and \(L_t\) and \(c_{t,d}\) are finite. The constants \(\varepsilon \in (0,1)\) and \(\delta \in (0,1/2)\) can be chosen arbitrarily small. Hence, if we let \(0<\delta = \varepsilon / 2<\frac{1}{2}\), the r.h.s. of (A.30) can be upper bounded, which results in the bound
$$\begin{aligned} \left| \pi _t(\theta _t^{\star ,N}) - \pi _t(\theta _t^\star ) \right| < \frac{ W_{t,d,\varepsilon } }{ N^{\frac{1-\varepsilon }{2(d+1)}} }, \end{aligned}$$
where
$$\begin{aligned} W_{t,d,\varepsilon } = 2\left[ V_{t,\varepsilon } + \left( \frac{ U_{t,\delta (\varepsilon )} }{ c_{t,d} } \right) ^{\frac{1}{d}} L_t \right] < \infty \quad \text{ a.s. } \end{aligned}$$
Proof of Corollary 2
Recall that
$$\begin{aligned} \Vert f\Vert _\infty = \sup _{\theta \in \varTheta } |f(\theta )| < \infty . \end{aligned}$$
Note that Theorem 3 implies that
$$\begin{aligned} 0 \le e^{-f(\theta ^\star )} - e^{- f(\theta _T^{\star ,N})} \le \frac{W_{T,d,\varepsilon } Z_{\pi _T}}{N^\frac{1}{2(d+1)}}, \end{aligned}$$
(A.31)
where \(Z_{\pi _T}\) is the normalizing constant of \(\pi _T\). Next, we lower bound the left-hand side of (A.31) as
$$\begin{aligned} e^{-f(\theta ^\star )} - e^{- f(\theta _T^{\star ,N})}&= e^{- f(\theta _T^{\star ,N})} \left( e^{f(\theta _T^{\star ,N}) - f(\theta ^\star )} - 1\right) \nonumber \\&\ge e^{-\Vert f\Vert _\infty } (f(\theta _T^{\star ,N}) - f(\theta ^\star )) \end{aligned}$$
(A.32)
where the last inequality follows from the relationships
$$\begin{aligned} e^{-f(\theta _T^{\star ,N})} \ge e^{-\Vert f\Vert _\infty } \end{aligned}$$
(since \(f(\theta _T^{\star ,N}) \le \Vert f\Vert _\infty \)) and \(e^a \ge a + 1\) for \(a \in {\mathbb R}\). Combining (A.31) and (A.32), we obtain
$$\begin{aligned} f(\theta _T^{\star ,N}) - f(\theta ^\star ) \le \frac{\tilde{W}_{T,d,\varepsilon }}{{N^\frac{1}{2(d+1)}}} \end{aligned}$$
where
$$\begin{aligned} \tilde{W}_{T,d,\varepsilon } = Z_{\pi _T} W_{T,d,\varepsilon } e^{\Vert f\Vert _\infty } \end{aligned}$$
is a.s. finite.