Advertisement

On Hitting Time, Mixing Time and Geometric Interpretations of Metropolis–Hastings Reversiblizations

  • Michael C. H. ChoiEmail author
  • Lu-Jing Huang
Article
  • 22 Downloads

Abstract

Given a target distribution \(\mu \) and a proposal chain with generator Q on a finite state space, in this paper, we study two types of Metropolis–Hastings (MH) generator \(M_1(Q,\mu )\) and \(M_2(Q,\mu )\) in a continuous-time setting. While \(M_1\) is the classical MH generator, we define a new generator \(M_2\) that captures the opposite movement of \(M_1\) and provide a comprehensive suite of comparison results ranging from hitting time and mixing time to asymptotic variance, large deviations and capacity, which demonstrate that \(M_2\) enjoys superior mixing properties than \(M_1\). To see that \(M_1\) and \(M_2\) are natural transformations, we offer an interesting geometric interpretation of \(M_1\), \(M_2\) and their convex combinations as \(\ell ^1\) minimizers between Q and the set of \(\mu \)-reversible generators, extending the results by Billera and Diaconis (Stat Sci 16(4):335–339, 2001). We provide two examples as illustrations. In the first one, we give explicit spectral analysis of \(M_1\) and \(M_2\) for Metropolized independent sampling, while in the second example, we prove a Laplace transform order of the fastest strong stationary time between birth–death \(M_1\) and \(M_2\).

Keywords

Markov chains Metropolis–Hastings algorithm Additive reversiblization Hitting time Mixing time Asymptotic variance Large deviations 

Mathematics Subject Classification (2010)

60J27 60J28 

Notes

Acknowledgements

The authors would like to thank the anonymous referee for constructive comments that improved the presentation of the manuscript. Michael Choi acknowledges support from the Chinese University of Hong Kong, Shenzhen grant PF01001143. Lu-Jing Huang acknowledges support from NSFC No. 11771047 and Probability and Statistics: Theory and Application (IRTL1704). The authors would also like to thank Professor Yong-Hua Mao for his hospitality during their visit to Beijing Normal University, where this work was initiated.

References

  1. 1.
    Aldous, D., Fill, J.A.: Reversible Markov Chains and Random Walks on Graphs, 2002. Unfinished Monograph, Recompiled. http://www.stat.berkeley.edu/~aldous/RWG/book.html (2014)
  2. 2.
    Bierkens, J.: Non-reversible Metropolis–Hastings. Stat. Comput. 26(6), 1213–1228 (2016)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Billera, L.J., Diaconis, P.: A geometric interpretation of the Metropolis–Hastings algorithm. Stat. Sci. 16(4), 335–339 (2001)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Chen, F., Lovász, L., Pak, I.: Lifting Markov chains to speed up mixing. In: Proceedings of the Thirty-First Annual ACM Symposium on Theory of Computing, pp. 275–281 (1999)Google Scholar
  5. 5.
    Chen, M.-F.: From Markov Chains to Non-equilibrium Particle Systems. World Scientific, Singapore (1992)zbMATHGoogle Scholar
  6. 6.
    Chen, T.-L., Hwang, C.-R.: Accelerating reversible Markov chains. Stat. Probab. Lett. 83(9), 1956–1962 (2013)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Chen, T.-L., Chen, W.-K., Hwang, C.-R., Pai, H.-M.: On the optimal transition matrix for Markov chain Monte Carlo sampling. SIAM J. Control Optim. 50(5), 2743–2762 (2012)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Choi, M.C.: Metropolis–Hastings reversiblizations of non-reversible Markov chains. arXiv:1706.00068 (2017)
  9. 9.
    Cui, H., Mao, Y.-H.: Eigentime identity for asymmetric finite Markov chains. Front. Math. China 5(4), 623–634 (2010)MathSciNetzbMATHGoogle Scholar
  10. 10.
    den Hollander, F.: Large Deviations, Volume 14 of Fields Institute Monographs. American Mathematical Society, Providence, RI (2000)Google Scholar
  11. 11.
    Diaconis, P., Miclo, L.: On characterizations of metropolis type algorithms in continuous time. ALEA Lat. Am. J. Probab. Math. Stat. 6, 199–238 (2009)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Diaconis, P., Saloff-Coste, L.: Separation cut-offs for birth and death chains. Ann. Appl. Probab. 16(4), 2098–2122 (2006)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Diaconis, P., Holmes, S., Neal, R.M.: Analysis of a nonreversible Markov chain sampler. Ann. Appl. Probab. 10(3), 726–752 (2000)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Doyle, P.-G.: Energy for Markov Chains. http://www.math.dartmouth.edu/doyle (1994)
  15. 15.
    Fill, J.A.: On hitting times and fastest strong stationary times for skip-free and more general chains. J. Theor. Probab. 22(3), 587–600 (2009)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Frigessi, A., Hwang, C.-R., Younes, L.: Optimal spectral structure of reversible stochastic matrices, Monte Carlo methods and the simulation of Markov random fields. Ann. Appl. Probab. 2(3), 610–628 (1992)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Gaudillière, A., Landim, C.: A Dirichlet principle for non reversible Markov chains and some recurrence theorems. Probab. Theory Relat. Fields 158(1–2), 55–89 (2014)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Geyer, C.-J., Mira, A.: On non-reversible Markov chains. Fields Inst. Commun. 26, 93–108 (2000)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Horn, R.A., Johnson, C.R.: Matrix Analysis, 2nd edn. Cambridge University Press, Cambridge (2013)zbMATHGoogle Scholar
  20. 20.
    Huang, L.-J., Mao, Y.-H.: Dirichlet principles for asymptotic variance of markov chains. (preprint)Google Scholar
  21. 21.
    Huang, L.-J., Mao, Y.-H.: On some mixing times for nonreversible finite Markov chains. J. Appl. Probab. 54(2), 627–637 (2017)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Huang, L.-J., Mao, Y.-H.: Variational principles of hitting times for non-reversible Markov chains. J. Math. Anal. Appl. 468(2), 959–975 (2018)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Hwang, C.-R., Hwang-Ma, S.-Y., Sheu, S.-J.: Accelerating diffusions. Ann. Appl. Probab. 15(2), 1433–1444 (2005)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Komorowski, T., Landim, C., Olla, S.: Fluctuations in Markov Processes, Volume 345 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer, Heidelberg (2012)Google Scholar
  25. 25.
    Leisen, F., Mira, A.: An extension of Peskun and Tierney orderings to continuous time Markov chains. Stat. Sin. 18(4), 1641–1651 (2008)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Liggett, T.-M.: Interacting Particle Systems. Springer, Berlin (1985)zbMATHGoogle Scholar
  27. 27.
    Liu, J.S.: Metropolized independent sampling with comparisons to rejection sampling and importance sampling. Stat Comput 6(2), 113–119 (1996)Google Scholar
  28. 28.
    Mao, Y.-H.: The eigentime identity for continuous-time ergodic Markov chains. J. Appl. Probab. 41(4), 1071–1080 (2004)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Metropolis, N., Rosenbluth, A.-W., Rosenbluth, M.-N., Teller, A.-H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)Google Scholar
  30. 30.
    Oliveira, R.I.: Mixing and hitting times for finite Markov chains. Electron. J. Probab. 17(70), 12 (2012)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Peskun, P.H.: Optimum Monte-Carlo sampling using Markov chains. Biometrika 60, 607–612 (1973)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Roberts, G.O., Rosenthal, J.S.: Geometric ergodicity and hybrid Markov chains. Electron. Commun. Probab. 2(2), 13–25 (1997). (electronic)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Roberts, G.O., Rosenthal, J.S.: General state space Markov chains and MCMC algorithms. Probab. Surv. 1, 20–71 (2004)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Sun, Y., Gomez, F., Schmidhuber, J.: Improving the asymptotic performance of Markov chain Monte-Carlo by inserting vortices. In: NIPS, USA, pp. 2235–2243 (2010)Google Scholar
  35. 35.
    Tierney, L.: A note on Metropolis–Hastings kernels for general state spaces. Ann. Appl. Probab. 8(1), 1–9 (1998)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institute for Data and Decision AnalyticsThe Chinese University of Hong KongShenzhenPeople’s Republic of China
  2. 2.College of Mathematics and InformaticsFujian Normal UniversityFuzhouPeople’s Republic of China

Personalised recommendations