Skip to main content
SpringerLink
Log in

Polynomial Accelerated MCMC and Other Sampling Algorithms Inspired by Computational Optimization

  • Conference paper
  • First Online:
Monte Carlo and Quasi-Monte Carlo Methods 2012

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 65))

Abstract

Polynomial acceleration methods from computational optimization can be applied to accelerating MCMC. For example, a geometrically convergent MCMC may be accelerated to be a perfect sampler in special circumstances. An equivalence between Gibbs sampling of Gaussian distributions and classical iterative methods can be established using matrix splittings, allowing direct application of Chebyshev acceleration. The conjugate gradient method can also be adapted to give an accelerated sampler for Gaussian distributions, that is perfect in exact arithmetic.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adler, S.L.: Over-relaxation method for the Monte Carlo evaluation of the partition function for multiquadratic actions. Phys. Rev. D 23, 2901–2904 (1981)

    Article  Google Scholar 

  2. Amit, Y., Grenander, U.: Comparing sweep strategies for stochastic relaxation. J. Multivariate Anal. 37, 197–222 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  3. Axelsson, O.: Iterative Solution Methods. Cambridge University Press, Cambridge (1996)

    MATH  Google Scholar 

  4. Barone, P., Frigessi, A.: Improving stochastic relaxation for Gaussian random fields. Probab. Engrg. Inform. Sci. 4, 369–384 (1990)

    Article  MATH  Google Scholar 

  5. Ceriotti, M., Bussi, G., Parrinello, M.: Conjugate gradient heat bath for ill-conditioned actions. Phys. Rev. E 76, 026707-1–8 (2007)

    Google Scholar 

  6. Diaconis, P., Freedman, D.: Iterated random functions. SIAM Rev. 41, 45–76 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  7. Fox, C.: A conjugate direction sampler for normal distributions, with a few computed examples. Technical Reports from the Electronics Group, University of Otago (2008)

    Google Scholar 

  8. Fox, C., Parker, A.: Convergence in variance of Chebyshev accelerated Gibbs samplers. SIAM J. Sci. Comput. (2013, to appear)

    Google Scholar 

  9. Fox, C., Parker, A.: Gibbs sampling of normal distributions using matrix splittings and polynomial acceleration. (2013, in preparation)

    Google Scholar 

  10. Galli, A., Gao, H.: Rate of convergence of the Gibbs sampler in the Gaussian case. Mathematical Geology 33, 653–677 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  11. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)

    Article  MATH  Google Scholar 

  12. Glauber, R.: Time dependent statistics of the Ising model. J. Math. Phys. 4, 294–307 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  13. Golub, G.H., Varga, R.S.: Chebyshev semi-iterative methods, successive over-relaxation iterative methods, and second-order Richardson iterative methods, parts I and II. Numer. Math. 3, 147–156, 157–168 (1961)

    Article  MathSciNet  Google Scholar 

  14. Goodman, J., Sokal, A.D.: Multigrid Monte Carlo method. Conceptual foundations. Phys. Rev. D 40, 2035–2071 (1989)

    Google Scholar 

  15. Higdon, D.: A primer on space-time modelling from a Bayesian perspective. In: Finkenstadt, B., Held, L., Isham, V. (eds.) Statistics of Spatio-Temporal Systems, pp. 217–279. Chapman & Hall/CRC, New York (2006)

    Chapter  Google Scholar 

  16. Łatuszyński, K., Roberts, G.O.: CLTs and asymptotic variance of time-sampled Markov chains. Methodol. Comput. Appl. Probab. 15, 237–247 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Parker, A., Fox, C.: Sampling Gaussian distributions in Krylov spaces with conjugate gradients. SIAM J. Sci. Comput. 34, B312–B334 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Roberts, G.O., Sahu, S.K.: Updating schemes, correlation structure, blocking and parameterization for the Gibbs sampler. J. R. Stat. Soc. Ser. B. 59, 291–317 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  19. Rue, H.: Fast sampling of Gaussian Markov random fields. J. R. Stat. Soc. Ser. B. 63, 325–338 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  20. Schneider, M.K., Willsky, A.S.: A Krylov subspace method for covariance approximation and simulation of random processes and fields. Multidimens. Syst. Signal Process. 14, 295–318 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. Turchin, V.F.: On the computation of multidimensional integrals by the Monte Carlo method. Theory Probab. Appl. 16, 720–724 (1971)

    Article  Google Scholar 

  22. Young, D.M.: Iterative Solution of Large Linear Systems. Academic, New York (1971)

    MATH  Google Scholar 

Download references

Acknowledgements

Polynomial acceleration of Gibbs sampling is the brainchild of Al Parker, to whom I am indebted. This research was supported by Marsden contract UOO1015.

Author information

Authors and Affiliations

  1. Physics Department, University of Otago, 56, Dunedin, 9054, New Zealand

    Colin Fox

Authors
  1. Colin Fox
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Colin Fox .

Editor information

Editors and Affiliations

  1. The University of New South Wales School of Mathematics and Statistics, Sydney, New South Wales, Australia

    Josef Dick

  2. The University of New South Wales School of Mathematics and Statistics, Sydney, New South Wales, Australia

    Frances Y. Kuo

  3. The University of New South Wales School of Mathematics and Statistics, Sydney, New South Wales, Australia

    Gareth W. Peters

  4. The University of New South Wales School of Mathematics and Statistics, Sydney, New South Wales, Australia

    Ian H. Sloan

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fox, C. (2013). Polynomial Accelerated MCMC and Other Sampling Algorithms Inspired by Computational Optimization. In: Dick, J., Kuo, F., Peters, G., Sloan, I. (eds) Monte Carlo and Quasi-Monte Carlo Methods 2012. Springer Proceedings in Mathematics & Statistics, vol 65. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41095-6_15

Download citation

Publish with us

Policies and ethics

Search

Navigation