Estimating Marginal Posterior Densities

Abstract

In Bayesian inference, a joint posterior distribution is available through the likelihood function and a prior distribution. One purpose of Bayesian inference is to calculate and display marginal posterior densities because the marginal posterior densities provide complete information about parameters of interest. As shown in Chapter 2, a Markov chain Monte Carlo (MCMC) sampling algorithm, such as the Gibbs sampler or a Metropolis-Hastings algorithm, can be used to draw MCMC samples from the posterior distribution. Chapter 3 also demonstrates how we can easily obtain posterior quantities such as posterior means, posterior standard deviations, and other posterior quantities from MCMC samples. However, when a Bayesian model becomes complicated, it may be difficult to obtain a reliable estimator of a marginal posterior density based on the MCMC sample. A traditional method for estimating marginal posterior densities is kernel density estimation. Since the kernel density estimator is nonparametric, it may not be efficient. On the other hand, the kernel density estimator may not be applicable for some complicated Bayesian models. In the context of Bayesian inference, the joint posterior density is typically known up to a normalizing constant. Using the structure of a posterior density, a number of authors (e.g., Gelfand, Smith, and Lee 1992; Johnson 1992; Chen 1993 and 1994; Chen and Shao 1997c; Chib 1995; Verdinelli and Wasserman 1995) propose parametric marginal posterior density estimators based on the MCMC sample. In this chapter, we present several available Monte Carlo (MC) methods for computing marginal posterior density estimators, and we also discuss how well marginal posterior density estimation works using the Kullback—Leibler (K—L) divergence as a performance measure.