Advertisement

Statistics and Computing

, Volume 8, Issue 4, pp 319–335 | Cite as

Convergence assessment techniques for Markov chain Monte Carlo

  • STEPHEN P. BROOKS
  • GARETH O. ROBERTS
Article

Abstract

MCMC methods have effectively revolutionised the field of Bayesian statistics over the past few years. Such methods provide invaluable tools to overcome problems with analytic intractability inherent in adopting the Bayesian approach to statistical modelling.

However, any inference based upon MCMC output relies critically upon the assumption that the Markov chain being simulated has achieved a steady state or “converged”. Many techniques have been developed for trying to determine whether or not a particular Markov chain has converged, and this paper aims to review these methods with an emphasis on the mathematics underpinning these techniques, in an attempt to summarise the current “state-of-play” for convergence assessment techniques and to motivate directions for future research in this area.

MCMC Gibbs Sampler Metropolis Hastings Convergence Rate 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ahmad, R. (1976) On the Multivariate K-Sample Problem and the Generalization of the Kolmogorov-Smirnov Test. Annals of the Institute of Statistical Mathematics 28, 259–265.Google Scholar
  2. Amit, Y. (1991) On Rates of Convergence of Stochastic Relaxation for Gaussian and Non-Gaussian Distributions. Journal of Multivariate Analysis 38, 82–99.Google Scholar
  3. Amit, Y. and Grenander, U. (1991) Comparing Sweep Strategies for Stochastic Relaxation. Journal of Multivariate Analysis 37, 197–222.Google Scholar
  4. Applegate, D., Kannan, R. and Polson, N. G. (1990) Random Polynomial Time Algorithms for Sampling from Joint Distributions. Technical report, Carnegie-Mellon University, Tech. Rep. No 500.Google Scholar
  5. Asmussen, S., Glynn, P. W. and Thorisson, H. (1992) Stationarity Detection in the Initial Transient Problem. ACM Transactions on Modelling and Computer Simulation 2, 130–157.Google Scholar
  6. Brittain, E. H. (1987) P-Values for the Multi-Sample Kolmogorov-Smirnov Test using the Expanded Bonferroni Approximation. Communications in Statistics — Theory and Methods 16(3), 821–835.Google Scholar
  7. Brooks, S. P. (1997) Discussion to Richardson and Green (1997) Journal of the Royal Statistical Society, Series B 59, 774–775Google Scholar
  8. Brooks, S. P. (1998a), Markov Chain Monte Carlo Method and its Application. The Statistician 47, 69–100.Google Scholar
  9. Brooks, S. P. (1998b) MCMC Convergence Diagnosis via Multivariate Bounds on Log-Concave Densities. The Annals of Statistics 26, 398–433.Google Scholar
  10. Brooks, S. P. (1998c) Quantitative Convergence Diagnosis for MCMC via CUSUMS. Statistics and Computing 8, 267–274.Google Scholar
  11. Brooks, S. P., Dellaportas, P. and Roberts, G. O. (1997) A Total Variation Method for Diagnosing Convergence of MCMC Algorithms. Journal of Computational and Graphical Statistics 6, 251–265.Google Scholar
  12. Brooks, S. P. and Gelman, A. (1998) Alternative Methods for Monitoring Convergence of Iterative Simulations. Journal of Computational and Graphical Statistics 7, 434–455.Google Scholar
  13. Brooks, S. P. and Roberts, G. O. (1997) On Quantile Estimation and MCMC Convergence. Biometrika In press.Google Scholar
  14. Chan, K. S. (1993) Asymptotic Behaviour of the Gibbs Sampler. Journal of the American Statistical Association 88, 320–328.Google Scholar
  15. Chorneyko, I. Z. and Zing, L. L. K. (1982) K-Sample Analogues of Two-Sample Kolmogorov-Smirnov Statistics. Selecta Statistica Canadiana 6, 37–62.Google Scholar
  16. Conover, W. J. (1965) Several K-Sample Kolmogorov-Smirnov Tests. Annals of Mathematical Statistics 36, 1019–1026.Google Scholar
  17. Conover, W. J. (1967) A k-Sample Extension of the One-Sided Two-Sample Smirnov Test Statistic. Annals of Mathematical Statistics 38, 1726–1730.Google Scholar
  18. Cowles, M. K. and Carlin, B. P. (1996) Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review. Journal of the American Statistical Association 91, 883–904.Google Scholar
  19. Cowles, M. K. and Rosenthal, J. S. (1998) A Simulation Approach to Convergence Rates for Markov Chain Monte Carlo. Technical report, Harvard School of Public Health.Google Scholar
  20. Diaconis, P. (1988) Group Representations in Probability and Statistics, vol. 11 of Lecture Notes — Monograph Series. Institute of Mathematical Statistics.Google Scholar
  21. Diaconis, P. and Stroock, D. (1991) Geometric Bounds for Eigenvalues of Markov Chains. Annals of Applied Probability 1, 36–61.Google Scholar
  22. Fasano, G. and Franceschini, A. (1987) A Multidimensional Version of the Kolmogorov-Smirnov Test. Monthly Notices of the Royal Astronomical Society 225, 155–170.Google Scholar
  23. Garren, S. and Smith, R. L. (1995) Estimating the Second Largest Eigenvalue of a Markov Transition Matrix. Technical report, University of Cambridge.Google Scholar
  24. Gelfand, A. E. (1992) Discussion of Gelman and Rubin (1992) Statistical Science 7, 486–487.Google Scholar
  25. Gelfand, A. E., Hills, S. E., Racine-Poon, A. and Smith, A. F. M. (1990) Illustration of Bayesian Inference in Normal data Models using Gibbs Sampling. Journal of the American Statistical Association 85, 972–985.Google Scholar
  26. Gelfand, A. E. and Smith, A. F. M. (1990) Sampling Based Approaches to Calculating Marginal Densities. Journal of the American Statistical Association 85, 398–409.Google Scholar
  27. Gelman, A. and Rubin, D. B. (1992) Inference from Iterative Simulation using Multiple Sequences. Statistical Science 7, 457–511.Google Scholar
  28. Geman, S. and Geman, D. (1984) Stochastic Relaxation, Gibbs Distributions and the Bayesian Restoration of Images. IEEE Transactions on pattern analysis and machine intelligence 6, 721–741.Google Scholar
  29. Geweke, J. (1989) Bayesian Inference in Econometric Models using Monte-Carlo Integration. Econometrica 57, 1317–1339.Google Scholar
  30. Geweke, J. (1992) Evaluating the Accuracy of Sampling-Based Approaches to the Calculation of Posterior Moments. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, pp. 169–193, New York: Oxford University Press.Google Scholar
  31. Geyer, C. J. and Thompson, E. A. (1995) Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference. Journal of the American Statistical Association 90, 909–920.Google Scholar
  32. Gilks, W. R., Roberts, G. O. and Sahu, S. K. (1996) Adaptive Markov Chain Monte Carlo. Technical report, MRC Biostatistics Unit, Cambridge.Google Scholar
  33. Green, P. J. (1995) Reversible Jump MCMC Computation and Bayesian Model determination. Biometrika 82, 711–732.Google Scholar
  34. Hastings, W. K. (1970) Monte Carlo Sampling Methods using Markov Chains and their Applications. Biometrika 57, 97–109.Google Scholar
  35. Heidelberger, P. and Welch, P. D. (1981a) Adaptive Spectral Methods for Simulation Output Analysis. IBM Journal of Research and Development 25, 860–876.Google Scholar
  36. Heidelberger, P. and Welch, P. D. (1981b) A Spectral Method for Confidence Interval Generation and Run Length Control in Simulations. Communications of the Association Computing Machinery 24, 233–245.Google Scholar
  37. Heidelberger, P. and Welch, P. D. (1983) Simulation Run Length Control in the Presence of an Initial Transient. Operations Research 31, 1109–1144.Google Scholar
  38. Jennison, C. (1993) Discussion to Smith and Roberts (1993) Journal of the Royal Statistical Society, Series B 55, 54–56.Google Scholar
  39. Johnson, V. E. (1996) Studying Convergence of Markov chain Monte Carlo Algorithms using Coupled Sample Paths. Journal of the American Statistical Association 91, 154–166.Google Scholar
  40. Kendall, W. S. (1997) Perfect simulation for the area interaction process. In C. C. Heyde and L. Acardi (eds.), Probability Perspective, World Scientific Press.Google Scholar
  41. Kolmogorov, A. N. (1933) Sulla Determiazione Empirica delle Leggi di Probabilita. Giornal Istitutto Italia Attuari 4, 1–11.Google Scholar
  42. Lawler, G. F. and Sokal, A. D. (1988) Bounds on the L 2 spectrum for Markov Chains and their Applications. Transactions of the American Mathematical Society 309, 557–580.Google Scholar
  43. Lin, Z. Y. (1992) On the Increments of Partial Sums of a ø-mixing Sequence. Theory of Probability and its Applications 36, 316–326.Google Scholar
  44. Lindvall, T. (1992) Lectures on the Coupling Method. Wiley.Google Scholar
  45. Liu, C., J. Liu and Rubin, D. B. (1993) A Control Variable for Assessment the Convergence of the Gibbs Sampler. In Proceedings of the Statistical Computing Section of the American Statistical Association, pp. 74–78.Google Scholar
  46. Liu, J. S. (1994) Metropolized Independent Sampling with Comparisons to Rejection Sampling and Importance Sampling. Technical report, Harvard University.Google Scholar
  47. Müller, P. (1991) A Generic Approach to Posterior Integration and Gibbs Sampling. Technical report, Dept. of Statistics, Purdue University.Google Scholar
  48. Mengersen, K. L. and Tweedie, R. L. (1995) Rates of Convergence of the Hastings and Metropolis Algorithms. Annals of Statistics 24, 101–121.Google Scholar
  49. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953) Equations of State Calculations by Fast Computing Machines. Journal of Chemical Physics 21, 1087–1091.Google Scholar
  50. Meyn, S. P. and Tweedie, R. L. (1993) Markov Chains and Stochastic Stability. Springer-Verlag.Google Scholar
  51. Meyn, S. P. and Tweedie, R. L. (1994) Computable Bounds for geometric Convergence Rates of Markov Chains. Annals of Applied Probability 4, 981–1011.Google Scholar
  52. Mises von, R. (1931) Wahrscheinlichkeitschrechnung. Deutiche, Vienna.Google Scholar
  53. Møller, J. (1997) Markov chain Monte Carlo and spatial point processes. In W. S. Kendall and M. N. M. vanLieshout (eds.), Stochastic geometry, likelihood and computation, Chapman and Hall.Google Scholar
  54. Murdoch, D. J. and Green, P. J. (1997) Exact Sampling from a Continuous State Space. Technical report, Queen's University, Ontario.Google Scholar
  55. Mykland, P., Tierney, L., and Yu, B. (1995) Regeneration in Markov Chain Samplers. Journal of the American Statistical Association 90, 233–241.Google Scholar
  56. Nummelin, E. (1984) General Irreducible Markov Chains and Non-Negative Operators. Cambridge University Press.Google Scholar
  57. Peskun, P. H. (1973) Optimum Monte Carlo Sampling using Markov Chains. Biometrika 60, 607–612.Google Scholar
  58. Polson, N. G. (1996) Convergence of Markov Chain Monte Carlo Algorithms. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 5, Oxford University Press.Google Scholar
  59. Propp, J. G. and Wilson, D. B. (1996) Exact Sampling with Coupled Markov Chains and Applications to Statistical Mechanics. Random Structures and Algorithms 9, 223–252.Google Scholar
  60. Raftery, A. E. and Lewis, S. M. (1992) How Many Iterations in the Gibbs Sampler? In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, Oxford University Press.Google Scholar
  61. Reutter, A. and Johnson, V. (1995) General Strategies for Assessing Convergence of MCMC Algorithms Using Coupled Sample Paths. Technical report, Duke University.Google Scholar
  62. Ripley, B. D. (1987) Stochastic Simulation. John Wiley and Sons.Google Scholar
  63. Ritter, C. and Tanner, M. A. (1992) Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler. Journal of the American Statistical Association 87, 861–868.Google Scholar
  64. Robert, C. P. (1996) Convergence Assessments for Markov Chain Monte Carlo Methods. Statistical Science 10, 231–253.Google Scholar
  65. Roberts, G. O. (1992) Convergence Diagnostics of the Gibbs sampler. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, Oxford University Press.Google Scholar
  66. Roberts, G. O. (1994) Methods for Estimating L 2 Convergence of Markov Chain Monte Carlo. In D. Berry, K. Chaloner and J. Geweke (eds.), Bayesian Statistics and Econometrics: Essays in Honor of Arnold Zellner, North Holland: Amsterdam.Google Scholar
  67. Roberts, G. O. and Polson, N. G. (1994) On the Geometric Convergence of the Gibbs Sampler. Journal of the Royal Statistical Society, Series B 56, 377–384.Google Scholar
  68. Roberts, G. O. and Rosenthal, J. S. (1998) Convergence of Slice Sampler Markov Chains. Journal of the Royal Statistical Society, Series B In press.Google Scholar
  69. Roberts, G. O. and Sahu, S. K. (1996) Rate of Convergence of the Gibbs Sampler by Gaussian Approximation. Technical report, University of Cambridge.Google Scholar
  70. Roberts, G. O. and Sahu, S. K. (1997) Updating Schemes, Covariance Structure, Blocking and Parameterisation for the Gibbs Sampler. Journal of the Royal Statistical Society, Series B 59, 291–318.Google Scholar
  71. Roberts, G. O. and Smith, A. F. M. (1994) Simple Conditions for the Convergence of the Gibbs Sampler and Metropolis Hastings Algorithms. Stochastic Processes and Applications 49, 207–216.Google Scholar
  72. Roberts, G. O. and Tweedie, R. L. (1996a) Exponential Convergence of Langevin Diffusions and their Discrete Approximations. Bernoulli 2, 341–363.Google Scholar
  73. Roberts, G. O. and Tweedie, R. L. (1996b) Geometric Convergence and Central Limit Theorems, for Multidimensional Hastings and Metropolis Algorithms. Biometrika 83, 95–110.Google Scholar
  74. Roberts, G. O. and Tweedie, R. L. (1998) Bounds on Regeneration Times and Convergence Rates for Markov Chains. Technical report, University of Cambridge.Google Scholar
  75. Rosenthal, J. (1995a) Rates of Convergence for Gibbs Sampling for Variance Component Models. Annals of Statistics 23, 740–761.Google Scholar
  76. Rosenthal, J. S. (1995b) Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo. Journal of the American Statistical Association 90, 558–566.Google Scholar
  77. Rosenthal, J. S. (1995c) Rates of Convergence for Data Augmentation on Finite Sample Spaces. Annals of Applied Probability 3, 319–339.Google Scholar
  78. Schervish, M. J. and Carlin, B. P. (1992) On the Convergence of Successive Substitution Sampling. Journal of Computational and Graphical statistics 1, 111–127.Google Scholar
  79. Schruben, L., Singh, H. and Tierney, L. (1983) Optimal Tests for Initialization Bias in Simulation Output. Operations Research 31, 1167–1178.Google Scholar
  80. Schruben, L. W. (1982) Detecting Initialization Bias in Simulation Output. Operations Research 30, 569–590.Google Scholar
  81. Silverman, B. W. (1986) Density Estimation for Statistics and Data Analysis. London: Chapman and Hall.Google Scholar
  82. Sinclair, A. J. and Jerrum, M. R. (1988) Conductance and the Rapid Mixing Property for Markov Chains: The Approximation of the Permanent Resolved. In Proceedings of the 20th annual ACM symposium on the Theory of Computing.Google Scholar
  83. Smith, A. F. M. and Gelfand, A. E. (1992) Bayesian Statistics Without Tears: A Sampling-Resampling Perspective. The American Statistician 46, 84–88.Google Scholar
  84. Smith, R. L. and Tierney, L. (1996) Exact Transition Probabilities for the Independence Metropolis Sampler. Technical report, Statistical Laboratory, Cambridge.Google Scholar
  85. Tierney, L. (1994) Markov Chains for Exploring Posterior Distributions. Annals of Statistics 22, 1701–1762.Google Scholar
  86. Yu, B. (1995a) Discussion to Besag et al. (1995) Statistical Science 10, 3–66.Google Scholar
  87. Yu, B. (1995b) Estimating L 1 Error of Kernel Estimator: Monitoring Convergence of Markov Samplers. Technical report, Dept. of Statistics, University of California, Berkeley.Google Scholar
  88. Yu, B. and Mykland, P. (1998) Looking at Markov Sampler through Cusum Path Plots: A Simple Diagnostic Idea. Statistics and Computing 8, 275–286.Google Scholar
  89. Zellner, A. and Min, C. (1995) Gibbs Sampler Convergence Criteria. Journal of the American Statistical Association 90, 921–927.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • STEPHEN P. BROOKS
    • 1
    • 2
  • GARETH O. ROBERTS
    • 1
    • 2
  1. 1.School of MathematicsUniversity of Bristol, University WalkBristol
  2. 2.Department of Maths and StatisticsUniversity of Leicester

Personalised recommendations