Advertisement

Statistics and Computing

, Volume 4, Issue 3, pp 189–201 | Cite as

Simulating posterior Gibbs distributions: a comparison of the Swendsen-Wang and Gibbs sampler methods

  • A. J. Gray
Papers

Abstract

We show in detail how the Swendsen-Wang algorithm, for simulating Potts models, may be used to simulate certain types of posterior Gibbs distribution, as a special case of Edwards and Sokal (1988), and we empirically compare the behaviour of the algorithm with that of the Gibbs sampler. Some marginal posterior mode and simulated annealing image restorations are also examined. Our results demonstrate the importance of the starting configuration. If this is inappropriate, the Swendsen-Wang method can suffer from critical slowing in moderately noise-free situations where the Gibbs sampler convergence is very fast, whereas the reverse is true when noise level is high.

Keywords

Markov random fields posterior Gibbs distributions simulation Swendsen-Wang algorithm Gibbs sampler restoration monitoring convergence 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Besag, J. (1986). On the statistical analysis of dirty pictures (with Discussion). Journal of the Royal Statistical Society, B, 48, 259–302.Google Scholar
  2. Besag, J. E. and Clifford, P. (1989). Generalised Monte Carlo significance tests. Biometrika, 76, 633–642.Google Scholar
  3. Besag, J. E. and Clifford, P. (1991). Sequential Monte Carlo p-values. Biometrika, 78, 301–304.Google Scholar
  4. Besag, J. and Green, P. J. (1993). Spatial statistics and Bayesian computation. Journal of the Royal Statistical Society, B, 55, 25–37.Google Scholar
  5. Besag, J., York, J. and Mollie, A. (1991). Bayesian image restoration, with two applications in spatial statistics (with Discussion). Annals of the Institute of Statistical Mathematics.Google Scholar
  6. Chen, C. C. (1988). Markov random fields in image analysis. Ph.D. thesis, Michigan State University, East Lansing.Google Scholar
  7. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with Discussion). Journal of the Royal Statistical Society B, 39, 1–38.Google Scholar
  8. Dubes, R. C. and Jain, A. K. (1989). Random field models in image analysis. Journal of Applied Statistics, 16, 131–164.Google Scholar
  9. Edwards, R. G. and Sokal, A. D. (1988). Generalisation of the Fortuin-Kasteleyn-Swendsen-Wang representation and Monte Carlo algorithm. Physical Review D, 38, 2009–2012.Google Scholar
  10. Flinn, P. A. (1974). Monte Carlo calculation of phase separation in a 2-dimensional Ising system. Journal of Statistical Physics, 10, 89–97.Google Scholar
  11. Fortuin, C. M. and Kasteleyn, P. W. (1972). On the random cluster model. I. Introduction and relation to other models. Physica, 57, 536–564.Google Scholar
  12. Gelfand, A. E., Hills, S. E., Racine-Poon, A. and Smith, A. F. M. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling. Journal of the American Statistical Association, 85, 972–985.Google Scholar
  13. Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.Google Scholar
  14. Geman, S. and Graffigne, C. (1986). Markov random field image models and their application to computer vision. Proceedings of the International Conference of Mathematicians, ed. A. M. Gleason. American Mathematical Society.Google Scholar
  15. Geyer, C. J. and Thompson, E. A. (1992). Constrained Monte Carlo maximum likelihood for dependent data (with Discussion). Journal of the Royal Statistical Society, B, 54, 657–699.Google Scholar
  16. Gilks, W. R., Clayton, D. G., Spiegelhalter, D. J., Best, N. G., McNeil, A. J., Sharpies, L. D. and Kirby, A. J. (1993). Modelling complexity: applications of Gibbs sampling in medicine. Journal of the Royal Statistical Society, B, 55, 39–52.Google Scholar
  17. Gray, A. J. (1993). Contribution to the discussion of Smith and Roberts, Besag and Green, Gilks et al. (1993). The Gibbs sampler and other Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, B, 55, 58–61.Google Scholar
  18. Gray, A. J., Kay, J. W. and Titterington, D. M. (1994). An empirical study of the simulation of various models used for images. To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence.Google Scholar
  19. Green, P. J. (1991). A note on the Swendsen-Wang algorithm and ordered colours. Preprint.Google Scholar
  20. Green, P. J. and Han, X.-L. (1992). Metropolis methods, Gaussian proposals and antithetic variables. In Stochastic Models, Statistical Methods and Algorithms in Image Analysis. Ed. P. Barone, A. Frigessi and M. Piccioni, Lecture Notes in Statistics. Springer-Verlag, Berlin.Google Scholar
  21. Greig, D. M., Porteous, B. T. and Seheult, A. H. (1989). Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society, B, 51, 271–279.Google Scholar
  22. Kandel, D., Domany, E. and Brandt, A. (1989). Simulations without critical slowing down: Ising and three-state Potts models. Physical Review B, 40, 330–344.Google Scholar
  23. Kandel, D., Domany, E., Ron, D. and Brandt, A. (1988). Simulations without critical slowing down. Physical Review Letters, 60, 1591–1594.Google Scholar
  24. Kirkland, M. D. (1989). Simulation methods for Markov random fields. Ph.D. thesis, University of Strathclyde.Google Scholar
  25. Lakshmanan, S. and Derin, H. (1989). Simultaneous parameter estimation and segmentation of Gibbs random fields using simulated annealing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 799–813.Google Scholar
  26. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953). Equations of state calculations by fast computing machines. Journal of Chemical Physics, 21, 1087–1092.Google Scholar
  27. Pickard, D. K. (1987). Inference for discrete Markov fields: the simplest nontrivial case. Journal of the American Statistical Association, 82, 90–96.Google Scholar
  28. Potts, R B. (1952). Some generalised order-disorder transformations. Proceedings of the Cambridge Philosophical Society, 48, 106–109.Google Scholar
  29. Qian, W. and Titterington, D. M. (1992). Stochastic relaxations and EM algorithms for Markov random fields. Journal of Statistical Computation and Simulation, 40, 55–69.Google Scholar
  30. Ripley, B. D. (1988). Statistical Inference for Spatial Processes. Cambridge University Press.Google Scholar
  31. Ripley, B. D. and Kirkland, M. D. (1990). Iterative simulation methods. Journal of Computational and Applied Mathematics, 31, 165–172.Google Scholar
  32. Smith, A. F. M. and Roberts, G. O. (1993). Bayesian computation and related Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, B, 55, 3–23.Google Scholar
  33. Sokal, A. D. (1989). Monte Carlo methods in statistical mechanics: foundations and new algorithms. Technical report, Department of Physics, New York University, New York, USA.Google Scholar
  34. Swendsen, R. H. and Wang, J. S. (1987). Nonuniversal critical dynamics in Monte Carlo simulation. Physics Review Letters, 58, 86–88.Google Scholar

Copyright information

© Chapman & Hall 1994

Authors and Affiliations

  • A. J. Gray
    • 1
  1. 1.Department of Statistics and Modelling ScienceUniversity of StrathclydeGlasgowUK

Personalised recommendations