Statistics and Computing

, Volume 24, Issue 3, pp 339–349 | Cite as

Split Hamiltonian Monte Carlo

  • Babak Shahbaba
  • Shiwei Lan
  • Wesley O. Johnson
  • Radford M. Neal
Article

Abstract

We show how the Hamiltonian Monte Carlo algorithm can sometimes be speeded up by “splitting” the Hamiltonian in a way that allows much of the movement around the state space to be done at low computational cost. One context where this is possible is when the log density of the distribution of interest (the potential energy function) can be written as the log of a Gaussian density, which is a quadratic function, plus a slowly-varying function. Hamiltonian dynamics for quadratic energy functions can be analytically solved. With the splitting technique, only the slowly-varying part of the energy needs to be handled numerically, and this can be done with a larger stepsize (and hence fewer steps) than would be necessary with a direct simulation of the dynamics. Another context where splitting helps is when the most important terms of the potential energy function and its gradient can be evaluated quickly, with only a slowly-varying part requiring costly computations. With splitting, the quick portion can be handled with a small stepsize, while the costly portion uses a larger stepsize. We show that both of these splitting approaches can reduce the computational cost of sampling from the posterior distribution for a logistic regression model, using either a Gaussian approximation centered on the posterior mode, or a Hamiltonian split into a term that depends on only a small number of critical cases, and another term that involves the larger number of cases whose influence on the posterior distribution is small.

Keywords

Markov chain Monte Carlo Hamiltonian dynamics Bayesian analysis 

References

  1. Beskos, A., Pinski, F.J., Sanz-Serna, J.M., Stuart, A.M.: Hybrid Monte-Carlo on Hilbert spaces. Stoch. Process. Appl. 121, 2201–2230 (2011) CrossRefMATHMathSciNetGoogle Scholar
  2. de Campos, D.A., Bernardes, J., Garrido, A., de Sa, J.M., Pereira-Leite, L.: SisPorto 2.0 a program for automated analysis of cardiotocograms. J. Matern.-Fetal Med. 9, 311–318 (2000) Google Scholar
  3. Duane, S., Kennedy, A.D., Pendleton, B.J., Roweth, D.: Hybrid Monte Carlo. Phys. Lett. B 195(2), 216–222 (1987) CrossRefGoogle Scholar
  4. Geyer, C.J.: Practical Markov chain Monte Carlo. Stat. Sci. 7(4), 473–483 (1992) CrossRefMathSciNetGoogle Scholar
  5. Girolami, M., Calderhead, B.: Riemann manifold Langevin and Hamiltonian Monte Carlo methods (with discussion). J. R. Stat. Soc. B 73(2), 123–214 (2011) CrossRefMathSciNetGoogle Scholar
  6. Hoffman, M., Gelman, A.: The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. arXiv:1111.4246 (2011)
  7. Leimkuhler, B., Reich, S.: Simulating Hamiltonian Dynamics. Cambridge University Press, Cambridge (2004) MATHGoogle Scholar
  8. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953). doi:10.1063/1.1699114 CrossRefGoogle Scholar
  9. Neal, R.M.: Probabilistic inference using Markov chain Monte Carlo methods. Technical report CRG-TR-93-1, Department of Computer Science, University of Toronto (1993) Google Scholar
  10. Neal, R.M.: MCMC using Hamiltonian dynamics. In: Brooks, S., Gelman, A., Jones, G., Meng, X.L. (eds.) Handbook of Markov Chain Monte Carlo. Chapman and Hall/CRC, London/Boca Raton (2010) Google Scholar
  11. Polyanin, A.D., Zaitsev, V.F., Moussiaux, A.: Handbook of First Order Partial Differential Equations. Taylor & Francis, London (2002) MATHGoogle Scholar
  12. Thompson, M.B.: A comparison of methods for computing autocorrelation time. Technical report 1007 (2010) Google Scholar
  13. Tierney, L., Kadane, J.B.: Accurate approximations for posterior moments and marginal densities. J. Am. Stat. Assoc. 81, 82–86 (1986) CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Babak Shahbaba
    • 1
  • Shiwei Lan
    • 2
  • Wesley O. Johnson
    • 2
  • Radford M. Neal
    • 3
  1. 1.Department of Statistics and Department of Computer ScienceUniversity of CaliforniaIrvineUSA
  2. 2.Department of StatisticsUniversity of CaliforniaIrvineUSA
  3. 3.Department of Statistics and Department of Computer ScienceUniversity of TorontoTorontoCanada

Personalised recommendations