Statistics and Computing

, Volume 5, Issue 2, pp 133–140 | Cite as

Random variate transformations in the Gibbs sampler: issues of efficiency and convergence

  • Petros Dellaportas
Papers

Abstract

In the non-conjugate Gibbs sampler, the required sampling from the full conditional densities needs the adoption of black-box sampling methods. Recent suggestions include rejection sampling, adaptive rejection sampling, generalized ratio of uniforms, and the Griddy-Gibbs sampler. This paper describes a general idea based on variate transformations which can be tailored in all the above methods and increase the Gibbs sampler efficiency. Moreover, a simple technique to assess convergence is suggested and illustrative examples are presented.

Keywords

Gibbs sampler adaptive rejection sampling economical method Griddy-Gibbs sampler 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Armitage, P. and Berry, G. (1987) Statistical methods in medical research. Blackwell Scientific Publications, Oxford.Google Scholar
  2. Besag, J. and Green, P. J. (1993) Spatial statistics and Bayesian computation (with discussion). Journal of the Royal Statistical Society, Series B, 55, 1, 25–37.Google Scholar
  3. Carlin, B. P. and Gelfand, A. E. (1991) An iterative Monte Carlo method for nonconjugate Bayesian analysis. Statistics and Computing, 1, 119–128.Google Scholar
  4. Deák, I. (1981) An economical method for random number generation and a normal generation. Computing, 27, 113–121.Google Scholar
  5. Dellaportas, P. and Smith, A. F. M. (1993) Bayesian inference for generalised linear and proportional hazards models via Gibbs sampling. Applied Statistics, 42, 443–459.Google Scholar
  6. Dellaportas, P. and Wright, D. E. (1991) Positive embedded integration in Bayesian analysis. Statistics and Computing, 1, 1–12.Google Scholar
  7. Devroye, L. (1982) A note on approximations in random variate generation. Journal of Statistical Computing and Simulation, 14, 149–158.Google Scholar
  8. Devroye, L. (1986) Non-uniform random variate generation. Springer-Verlag, New York.Google Scholar
  9. Gelfand, A. E. and Smith, A. F. M. (1990) Sampling based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.Google Scholar
  10. Gelman, A. and Rubin, D. B. (1992) Inference from iterative simulation using multiple sequences (with discussion). Statistical Science, 7, 457–511.Google Scholar
  11. Geman, S. and Geman, D. (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.Google Scholar
  12. Gilks, W. R. and Wild, P. (1992) Adaptive rejection sampling for Gibbs sampling. Applied Statistics, 41, 337–348.Google Scholar
  13. Gilks, W. R., Clayton, D. G., Spiegelhalter, D. J., Best, N. G., McNeil, A. J., Sharpies, L. D. and Kirby, A. J. (1993) Modelling complexity: applications of Gibbs sampling in medicine (with discussion). Journal of the Royal Statistical Society, Series B, 55, 1, 39–52.Google Scholar
  14. Grieve, A. P. (1987) Applications of Bayesian software: two examples. The Statistician, 36, 283–288.Google Scholar
  15. Hastings, W. K. (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97–109.Google Scholar
  16. Hills, S. E. and Smith, A. F. M. (1992) Parameterization issues in Bayesian inference. In Bayesian statistics, 4, ed. J. M. Bernardo et al., pp. 227–246, Oxford University Press.Google Scholar
  17. Kloeck, T. and van Dijk, H. K. (1978) Bayesian estimates for equation system parameters. An application of integration by Monte Carlo. Econometrika, 46, 1–19.Google Scholar
  18. Lee, T. M. (1992) Comparison of sampling based approaches for Bayesian computation. PhD Thesis, Department of Statistics, University of Connecticut.Google Scholar
  19. Lyness, J. N. (1977) Quid, Quo, Quadrature? In The state of the art in numerical analysis, pp. 535–560, Academic Press, London.Google Scholar
  20. Lyness, J. N. and Kaganove, J. J. (1977) A technique for comparing automatic quadrature routines. The Computer Journal, 20, 170–177.Google Scholar
  21. Naylor, J. A. and Smith, A. F. M. (1982) Applications of a method for the efficient computation of posterior distributions. Applied Statistics, 31, 214–225.Google Scholar
  22. Oh, M. S. and Berger, J. O. (1992) Adaptive importance sampling in Monte Carlo integration. Journal of Statistical Computing and Simulation, 41, 143–168.Google Scholar
  23. Rao, C. R. (1973) Linear statistical inference and applications. Wiley, New York.Google Scholar
  24. Rice, J. R. (1975) A metalgorithm for adaptive quadrature. Journal of the Association for Computing Machinery, 22, 61–82.Google Scholar
  25. Ritter, C. and Tanner, M. A. (1992) Facilitating the Gibbs sampler: the Gibbs stopper and the Griddy-Gibbs sampler. Journal of the American Statistical Association, 87, 861–868.Google Scholar
  26. Robinson, I. (1979) A comparison of numerical integration programs. Journal of Computational and Applied Mathematics, 5, 207–223.Google Scholar
  27. Smith, A. F. M. and Roberts, G. O. (1993) Bayesian computation via the Gibbs sampler and related Markov Chain Monte Carlo methods (with discussion). Journal of the Royal Statistical Society, Series B, 55, 3–23.Google Scholar
  28. Tanner, M. A. (1991) Tools for statistical inference. Springer-Verlag, Berlin.Google Scholar
  29. Tanner, M. A. and Wong, W. H. (1987) The calculation of posterior distributions by data augmentation (with discussion). Journal of the American Statistical Association, 82, 528–550.Google Scholar
  30. Tierney, L. (1991) Markov Chains for Exploring Posterior Distributions. Technical Report No. 560, School of Statistics, University of Minnesota.Google Scholar
  31. Wakefield, J. C., Gelfand, A. E. and Smith, A. F. M. (1991) Efficient generation of random variates via the ratio-of-uniforms method. Statistics and Computing, 1, 129–133.Google Scholar
  32. Wakefield, J. C., Smith, A. F. M., Racine-Poon, A. and Gelfand, A. E. (1993) Bayesian analysis of linear and non-linear population models using the Gibbs sampler. Applied Statistics, 43, 201–221.Google Scholar
  33. Zeger, S. and Karim, M. R. (1991) Generalised linear models with random effects: a Gibbs sampling approach. Journal of the American Statistical Association, 86, 79–86.Google Scholar

Copyright information

© Chapman & Hall 1995

Authors and Affiliations

  • Petros Dellaportas
    • 1
  1. 1.Department of StatisticsAthens University of EconomicsAthensGreece

Personalised recommendations