Summary
The Gibbs sampler, being a popular routine amongst Markov chain Monte Carlo sampling methodologies, has revolutionized the application of Monte Carlo methods in statistical computing practice. The performance of the Gibbs sampler relies heavily on the choice of sweep strategy, that is, the means by which the components or blocks of the random vector X of interest are visited and updated. We develop an automated, adaptive algorithm for implementing the optimal sweep strategy as the Gibbs sampler traverses the sample space. The decision rules through which this strategy is chosen are based on convergence properties of the induced chain and precision of statistical inferences drawn from the generated Monte Carlo samples. As part of the development, we analytically derive closed form expressions for the decision criteria of interest and present computationally feasible implementations of the adaptive random scan Gibbs sampler via a Gaussian approximation to the target distribution. We illustrate the results and algorithms presented by using the adaptive random scan Gibbs sampler developed to sample multivariate Gaussian target distributions, and screening test and image data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amit, Y. (1996). Convergence Properties of the Gibbs Sampler for Perturbations of Gaussians.Annals of Statistics 24, 122–140.
Amit, Y. and Grenander, U. (1991). Comparing Sweep Strategies for Stochastic Relaxation.Journal of Multivariate Analysis 37, 197–222.
Bernardo, J. M. and Smith, A. F. M. (1994).Bayesian Theory. Wiley, New York.
Besag, J., Green, P., Higdon, D., and Mengersen, K. (1995). Bayesian Computation and Stochastic Systems.Statistical Science 10, 3–41.
Bradley, R. C. (1986). Basic Properties of Strong Mixing Conditions. InDependence in Probability and Statistics, E. Eberlein and M. S. Taqqu (Eds.). Birkhaüser. Boston.
Gelfand, A. E. and Smith, A. F. M. (1990). Sampling-Based Approaches to Calculating Marginal Densities.Journal of the American Statistical Association 85, 398–409.
Geman, S. and Geman, D. (1984). Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images.IEEE Transactions on Pattern Analysis and Machine Intelligence 6, 721–741.
Jones, P. D., New, M., Parker, D. E., Martin, S., and Rigor, I. G. (1999). Surface air temperature and its changes over the past 150 years.Review of Geophysics 37, 173–199.
Joseph, L., Gyorkos, T. W., and Coupai, L. (1995). Bayesian Estimation of Disease Prevalence and Parameters for Diagnostics Tests in the Absence of a Gold Standard.American Journal of Epidemiology 141, 263–272.
Kagan, A. M., Linnik, Yu. V., Rao, C. (1973).Characterization problems in mathematical statistics, translated from the Russian by B. Ramachandran. Wiley, New York.
Levine, R. A. and Casella, G. (2003). Optimizing Random Scan Gibbs Samplers. Lawrence Livermore National Laboratory, Livermore, CA, UCRL-JC-145679.
Levine, R. A., Yu, Z., Hanley, W. G., and Nitao, J. J. (2003). Implementing Componentwise Hastings Algorithms. Lawrence Livermore National Laboratory, Livermore, CA, UCRL-JC-145678.
Liu, J., Wong, W. H., and Kong, A. (1995). Correlation Structure and Convergence Rate of the Gibbs Sampler with Various Scans.Journal of the Royal Statistical Society, B 57, 157–169.
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. (1953). Equation of State Calculations by Fast Computing Machines.The Journal of Chemical Physics 21, 1087–1092.
Mira, A. (2001). Ordering and Improving Performance of Monte Carlo Markov Chains.Statistical Science 16, 340–350.
Roberts, G. O. and Sahu, S. K. (2001). Approximate Predetermined Convergence Properties of the Gibbs Sampler.Journal of Computational and Graphical Statistics 10, 216–229.
Roberts, G. O. and Sahu, S. K. (1997). Updating Schemes, Correlation Structure, Blocking and Parameterization for the Gibbs Sampler.Journal of the Royal Statistical Society, B 59 291–317.
Sahu, S. K. and Roberts, G. O. (1999). On Convergence of the EM Algorithm and the Gibbs Sampler.Statistics and Computing 9, 55–64.
Sarmanov, O. V. and Zaharov, V. K. (1960). Maximum Coefficients of Multiple Correlation.Doklady Akademii Nauk SSSR 29, 269–271.
Yee, J., Johnson, W. O., and Samaniego, F. J. (2002). Asymptotic Approximations to Posterior Distributions via Conditional Moment Equations.Biometrika 89, 755–767.
Author information
Authors and Affiliations
Additional information
Research by RL and ZY supported in part by a US National Science Foundation FRG grant 0139948 and a grant from Lawrence Livermore National Laboratory, Livermore, California, USA.
Rights and permissions
About this article
Cite this article
Levine, R.A., Yu, Z., Hanley, W.G. et al. Implementing random scan Gibbs samplers. Computational Statistics 20, 177–196 (2005). https://doi.org/10.1007/BF02736129
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF02736129