Skip to main content

Random variate transformations in the Gibbs sampler: issues of efficiency and convergence

Abstract

In the non-conjugate Gibbs sampler, the required sampling from the full conditional densities needs the adoption of black-box sampling methods. Recent suggestions include rejection sampling, adaptive rejection sampling, generalized ratio of uniforms, and the Griddy-Gibbs sampler. This paper describes a general idea based on variate transformations which can be tailored in all the above methods and increase the Gibbs sampler efficiency. Moreover, a simple technique to assess convergence is suggested and illustrative examples are presented.

This is a preview of subscription content, access via your institution.

References

  • Armitage, P. and Berry, G. (1987) Statistical methods in medical research. Blackwell Scientific Publications, Oxford.

    Google Scholar 

  • Besag, J. and Green, P. J. (1993) Spatial statistics and Bayesian computation (with discussion). Journal of the Royal Statistical Society, Series B, 55, 1, 25–37.

    Google Scholar 

  • Carlin, B. P. and Gelfand, A. E. (1991) An iterative Monte Carlo method for nonconjugate Bayesian analysis. Statistics and Computing, 1, 119–128.

    Google Scholar 

  • Deák, I. (1981) An economical method for random number generation and a normal generation. Computing, 27, 113–121.

    Google Scholar 

  • Dellaportas, P. and Smith, A. F. M. (1993) Bayesian inference for generalised linear and proportional hazards models via Gibbs sampling. Applied Statistics, 42, 443–459.

    Google Scholar 

  • Dellaportas, P. and Wright, D. E. (1991) Positive embedded integration in Bayesian analysis. Statistics and Computing, 1, 1–12.

    Google Scholar 

  • Devroye, L. (1982) A note on approximations in random variate generation. Journal of Statistical Computing and Simulation, 14, 149–158.

    Google Scholar 

  • Devroye, L. (1986) Non-uniform random variate generation. Springer-Verlag, New York.

    Google Scholar 

  • Gelfand, A. E. and Smith, A. F. M. (1990) Sampling based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.

    Google Scholar 

  • Gelman, A. and Rubin, D. B. (1992) Inference from iterative simulation using multiple sequences (with discussion). Statistical Science, 7, 457–511.

    Google Scholar 

  • Geman, S. and Geman, D. (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.

    Google Scholar 

  • Gilks, W. R. and Wild, P. (1992) Adaptive rejection sampling for Gibbs sampling. Applied Statistics, 41, 337–348.

    Google Scholar 

  • Gilks, W. R., Clayton, D. G., Spiegelhalter, D. J., Best, N. G., McNeil, A. J., Sharpies, L. D. and Kirby, A. J. (1993) Modelling complexity: applications of Gibbs sampling in medicine (with discussion). Journal of the Royal Statistical Society, Series B, 55, 1, 39–52.

    Google Scholar 

  • Grieve, A. P. (1987) Applications of Bayesian software: two examples. The Statistician, 36, 283–288.

    Google Scholar 

  • Hastings, W. K. (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97–109.

    Google Scholar 

  • Hills, S. E. and Smith, A. F. M. (1992) Parameterization issues in Bayesian inference. In Bayesian statistics, 4, ed. J. M. Bernardo et al., pp. 227–246, Oxford University Press.

  • Kloeck, T. and van Dijk, H. K. (1978) Bayesian estimates for equation system parameters. An application of integration by Monte Carlo. Econometrika, 46, 1–19.

    Google Scholar 

  • Lee, T. M. (1992) Comparison of sampling based approaches for Bayesian computation. PhD Thesis, Department of Statistics, University of Connecticut.

  • Lyness, J. N. (1977) Quid, Quo, Quadrature? In The state of the art in numerical analysis, pp. 535–560, Academic Press, London.

    Google Scholar 

  • Lyness, J. N. and Kaganove, J. J. (1977) A technique for comparing automatic quadrature routines. The Computer Journal, 20, 170–177.

    Google Scholar 

  • Naylor, J. A. and Smith, A. F. M. (1982) Applications of a method for the efficient computation of posterior distributions. Applied Statistics, 31, 214–225.

    Google Scholar 

  • Oh, M. S. and Berger, J. O. (1992) Adaptive importance sampling in Monte Carlo integration. Journal of Statistical Computing and Simulation, 41, 143–168.

    Google Scholar 

  • Rao, C. R. (1973) Linear statistical inference and applications. Wiley, New York.

    Google Scholar 

  • Rice, J. R. (1975) A metalgorithm for adaptive quadrature. Journal of the Association for Computing Machinery, 22, 61–82.

    Google Scholar 

  • Ritter, C. and Tanner, M. A. (1992) Facilitating the Gibbs sampler: the Gibbs stopper and the Griddy-Gibbs sampler. Journal of the American Statistical Association, 87, 861–868.

    Google Scholar 

  • Robinson, I. (1979) A comparison of numerical integration programs. Journal of Computational and Applied Mathematics, 5, 207–223.

    Google Scholar 

  • Smith, A. F. M. and Roberts, G. O. (1993) Bayesian computation via the Gibbs sampler and related Markov Chain Monte Carlo methods (with discussion). Journal of the Royal Statistical Society, Series B, 55, 3–23.

    Google Scholar 

  • Tanner, M. A. (1991) Tools for statistical inference. Springer-Verlag, Berlin.

    Google Scholar 

  • Tanner, M. A. and Wong, W. H. (1987) The calculation of posterior distributions by data augmentation (with discussion). Journal of the American Statistical Association, 82, 528–550.

    Google Scholar 

  • Tierney, L. (1991) Markov Chains for Exploring Posterior Distributions. Technical Report No. 560, School of Statistics, University of Minnesota.

  • Wakefield, J. C., Gelfand, A. E. and Smith, A. F. M. (1991) Efficient generation of random variates via the ratio-of-uniforms method. Statistics and Computing, 1, 129–133.

    Google Scholar 

  • Wakefield, J. C., Smith, A. F. M., Racine-Poon, A. and Gelfand, A. E. (1993) Bayesian analysis of linear and non-linear population models using the Gibbs sampler. Applied Statistics, 43, 201–221.

    Google Scholar 

  • Zeger, S. and Karim, M. R. (1991) Generalised linear models with random effects: a Gibbs sampling approach. Journal of the American Statistical Association, 86, 79–86.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Dellaportas, P. Random variate transformations in the Gibbs sampler: issues of efficiency and convergence. Stat Comput 5, 133–140 (1995). https://doi.org/10.1007/BF00143944

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00143944

Keywords

  • Gibbs sampler
  • adaptive rejection sampling
  • economical method
  • Griddy-Gibbs sampler