Frequentist standard errors of Bayes estimators

Abstract

Frequentist standard errors are a measure of uncertainty of an estimator, and the basis for statistical inferences. Frequestist standard errors can also be derived for Bayes estimators. However, except in special cases, the computation of the standard error of Bayesian estimators requires bootstrapping, which in combination with Markov chain Monte Carlo can be highly time consuming. We discuss an alternative approach for computing frequentist standard errors of Bayesian estimators, including importance sampling. Through several numerical examples we show that our approach can be much more computationally efficient than the standard bootstrap.

Appendix

1.1 Proof of the convergence result of Sect. 3

Here we discuss the convergence of (1). Suppose that $\omega ^{(b)}(\theta )$ and $\theta \omega ^{(b)}(\theta )$ are integrable functions of $\theta$ with respect to the posterior distribution of the original data $\pi (\theta |\varvec{D})$ so that $G_s^{(b)} = \int \theta ^s\omega ^{(b)}(\theta ) \pi (\theta |\varvec{D})d\theta /K_\pi = E_{\pi (\cdot |\varvec{D})}\{\theta ^s\omega ^{(b)}(\theta )\}/K_\pi$ is finite for all b and $s = 0, 1$. Therefore, as $M \rightarrow \infty$, from the ergodic theorem (Jones 2004; Robert and Casella 2005), with probability 1,

$$\begin{aligned}&\frac{1}{M} \sum _{j=1}^{M}\omega ^{(b)}(\theta _j) \rightarrow E_{\pi (\cdot |\varvec{D})}\{\omega ^{(b)}(\theta )\} = K_\pi G_0^{(b)}, \\&\quad \frac{1}{M} \sum _{j=1}^{M}\theta _j\omega ^{(b)}(\theta _j) \rightarrow E_{\pi (\cdot |\varvec{D})}\{\theta \omega ^{(b)}(\theta )\}= K_\pi G_1^{(b)}. \end{aligned}$$

From Remark 1 in Sect. 3, $\omega ^{(b)}(\theta ) = \exp \{\ell ^{(b)}(\theta ) - \ell (\theta )\}$ implies $\omega ^{(b)}(\theta )$ is positive for all $\theta$. Therefore, $\sum _{j=1}^{M}\omega ^{(b)}(\theta _j) > 0$ and $G_0^{(b)} > 0$, and consequently

$$\begin{aligned} \widehat{\theta }^{(b)}_{\mathrm{is}}=\frac{ \sum ^M_{j=1}\theta _j\omega ^{(b)}(\theta _j) }{\sum ^M_{j=1}\omega ^{(b)}(\theta _j)} \rightarrow \frac{G_1^{(b)}}{G_0^{(b)}} = \widehat{\theta }^{(b)} \end{aligned}$$

with probability 1 as $M \rightarrow \infty$.

1.2 Computational complexity of the two approaches for the logistic regression example

^{1 2}