Nonconjugate priors and Metropolis-Hastings algorithms

Abstract

When conjugate or semiconjugate prior distributions are used, the posterior distribution can be approximated with the Monte Carlo method or the Gibbs sampler. In situations where a conjugate prior distribution is unavailable or undesirable, the full conditional distributions of the parameters do not have a standard form and the Gibbs sampler cannot be easily used. In this section we present the Metropolis-Hastings algorithm as a generic method of approximating the posterior distribution corresponding to any combination of prior distribution and sampling model. This section presents the algorithm in the context of two examples: The first involves Poisson regression, which is a type of generalized linear model. The second is a longitudinal regression model in which the observations are correlated over time.