Geometric Allocation Approach in Markov Chain Monte Carlo

Part of the Springer Theses book series (Springer Theses)


The general improvement of the Markov chain Monte Carlo method, which is a powerful tool for investigating systems with multiple degrees of freedom, will be shown in this chapter. There are three key points for the method to be effective: the choice of the ensemble, the selection of candidate configurations, and the determination of the transition probability (kernel). We focus our interest on the third optimization problem. After reviewing the previous work, we will invent a novel method that constructs a transition kernel by a geometric approach. This method finds solutions through a graphical procedure, namely weight allocation, instead of solving the detailed balance equation algebraically as before. Surprisingly, it is always possible to find a solution that minimizes the average rejection rate. Later, this geometric approach will be extended to continuous state space. We will assess the convergence and the autocorrelations for the ferromagnetic q-state Potts models and some simple models with continuous state variables. Then we will see the significant improvement by the new method, comparing to the conventional methods, such as the Metropolis and the heat bath algorithm. Our method is the first versatile scheme that satisfies the total balance without imposing the detailed balance.


Breaking of detailed balance Irreversible Markov chain Optimization of transition kernel Geometric weight allocation 


  1. 1.
    Adler, S. L. (1981). Over-relaxation method for the Monte Carlo evaluation of the partition function for multiquadratic actions. Physical Review D, 23, 2901.ADSCrossRefGoogle Scholar
  2. 2.
    Alet, F., Wessel, S., & Troyer, M. (2005). Generalized directed loop method for quantum Monte Carlo simulations. Physical Review E, 71, 036706. doi: 10.1103/PhysRevE.71.036706.ADSCrossRefGoogle Scholar
  3. 3.
    Andrieu, C., & Toms, J. (2008). A tutorial on adaptive MCMC. Statistics and Computing, 18, 343–373.MathSciNetCrossRefGoogle Scholar
  4. 4.
    Baldi, P., Frigessi, A., & Piccioni, M. (1993). Importance sampling for Gibbs random fields. Annals of Applied Probability, 3, 914–933.MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Barker, A. A. (1965). Monte Carlo calculations of the radial distribution functions for a proton-electron plasma. Australian Journal of Physics, 18, 119.ADSCrossRefGoogle Scholar
  6. 6.
    Bellman, R. E. (1957). Dynamic programming. Princeton: Princeton University Press.Google Scholar
  7. 7.
    Berg, B. A., & Neuhaus, T. (1992). Multicanonical ensemble: A new approach to simulate first-order phase transitions. Physical Review Letters, 68, 9.ADSCrossRefGoogle Scholar
  8. 8.
    Billera Louis, J., & Diaconis, P. (2001). A geometric interpretation of the Metropolis-Hastings algorithm. Statistical Science, 16, 335–339.MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Brown, F. R., & Woch, T. J. (1987). Overrelaxed heat-bath and Metropolis algorithms for accelerating pure gauge Monte Carlo calculations. Physical Review Letters, 58, 2394–2395.ADSCrossRefGoogle Scholar
  10. 10.
    Chiang, T. S., & Chow, Y. (1993). Asymptotic behavior of eigenvalues and random updating shemes. Applied Mathematics and Optimization, 28, 259–275.MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Creutz, M. (1980). Monte Carlo study of quantized SU(2) gauge theory. Physical Review D, 21, 2308.MathSciNetADSCrossRefGoogle Scholar
  12. 12.
    Creutz, M. (1987). Overrelaxation and Monte Carlo simulation. Physical Review D, 36, 515–519.MathSciNetADSCrossRefGoogle Scholar
  13. 13.
    Diaconis, P., Holmes, S., & Neal, R. M. (2000). Analysis of a nonreversible Markov chain sampler. Annals of Applied Probability, 10, 726.MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Dick, J., & Pillichshammer, F. (2010). Digital nets and sequences: Discrepancy theory and Quasi Monte Carlo integration. Cambridge: Cambridge University Press.Google Scholar
  15. 15.
    Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987). Hybrid Monte Carlo. Physics Letters B, 195, 216.ADSCrossRefGoogle Scholar
  16. 16.
    Evertz, H. G., Lana, G., & Marcu, M. (1993). Cluster algorithm for vertex models. Physical Review Letters, 70, 875.ADSCrossRefGoogle Scholar
  17. 17.
    Faure, H. (1982). Discrépance de suites associées à un systéme de numération (en dimension s). Acta Arithmetica, 41, 337–351.MathSciNetMATHGoogle Scholar
  18. 18.
    Franke, B., Hwang, C. R., Pai, H. M., & Sheu, S. J. (2010). The behavior of the spectral gap under growing drift. American Mathematical Society, 362, 1325–1350.MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Frenkel, D., & Smit, B. (2001). Understanding molecular simulation: From algorithms to applications (2nd ed.). San Diego: Academic Press.Google Scholar
  20. 20.
    Frigessi, A., Gåsemayr, J., & Rue, H. (2000). Antithetic coupling of two Gibbs sampler chains. Annals of Statistics, 28, 1128–1149.MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Frigessi, A., Hwang, C. R., & Younes, L. (1992). Optimal spectral structure of reversible stochastic matrices, Monte Carlo methods and the simulation of Markov random fields. Annals of Applied Probability, 2, 610–628.MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Frigessi, A., Martinelli, F., & Stander, J. (1997). Computational complexity of Markov chain Monte Carlo methods for finite Markov random fields. Biometrika, 84, 1–18.MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721.CrossRefMATHGoogle Scholar
  24. 24.
    Gilks, W. R., Richardson, S., & Spiegelhalter, D. J. (1996). Markov Chain Monte Carlo in Practice. London/New York: Chapman & Hall.CrossRefMATHGoogle Scholar
  25. 25.
    Green, P. J., & Mira, A. (2001). Delayed rejection in reversible jump Metropolis-Hastings. Biometrika, 88, 1035–1053.MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97.CrossRefMATHGoogle Scholar
  27. 27.
    Hlawka, E. (1961). Funktionen von beschränkter variation in der theorie der gleichverteilung. Annali di Matematica Pura ed Applicata, 54, 325–333.MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Hukushima, K., & Nemoto, K. (1996). Exchange Monte Carlo method and application to spin glass simulations. Journal of the Physical Society of Japan, 65, 1604.ADSCrossRefGoogle Scholar
  29. 29.
    Hwang, C. R. (2006). Nonreversible perturbations accelerate convergence. RIMS Kokyuroku, 1462, 26–34.Google Scholar
  30. 30.
    Hwang, C. R., Chen, T. L., Chen, W. K., & Pai, H. M. (2012). On the optimal transition matrix for MCMC sampling. SIAM Journal on Control and Optimization, 50, 2743–2762. Google Scholar
  31. 31.
    Hwang, C. R., Hwang-Ma, S. Y., & Sheu, S. J. (2005). Accelerating diffusions. Annals of Applied Probability, 15, 1433–1444.MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Landau, D. P., & Binder, K. (2005). A guide to Monte Carlo simulations in statistical physics (2nd ed.). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  33. 33.
    Lemieux, C. (2009). Monte Carlo and Quasi-Monte Carlo sampling. New York: Springer.MATHGoogle Scholar
  34. 34.
    Liu, J. S. (1996). Metropolized independent sampling with comparisons to rejection sampling and importance sampling. Statistics and Computing, 6, 113.CrossRefGoogle Scholar
  35. 35.
    Liu, J. S. (1996). Peskun’s theorem and a modified discrete-state Gibbs sampler. Biometrika, 83, 681–682.MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Liu, J. S. (2001). Monte Carlo strategies in scientific computing (1st ed.). Springer: New York.MATHGoogle Scholar
  37. 37.
    Liu, J. S., Liang, F., & Wong Wing, H. (2000). The multiple-try method and local optimization in Metropolis sampling. Journal of the American Statistical Association, 95, 121.MathSciNetCrossRefGoogle Scholar
  38. 38.
    Loison, D., Qin, C. L., Schotte, K. D., & Jin, X. F. (2004). Canonical local algorithms for spin systems: Heat bath and Hasting’s methods. European Physical Journal B, 41, 395–412.ADSCrossRefGoogle Scholar
  39. 39.
    Manousiouthakis, V. I., & Deem, M. W. (1999). Strict detailed balance is unnecessary in Monte Carlo simulation. Journal of Chemical Physics, 110, 2753.ADSCrossRefGoogle Scholar
  40. 40.
    Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953). Equation of state calculations by fast computing machines. Journal of Chemical Physics, 21, 1087.ADSCrossRefGoogle Scholar
  41. 41.
    Meyn, S. P., & Tweedie, R. L. (1993). Markov chains and stochastic stability. New York: Springer.CrossRefMATHGoogle Scholar
  42. 42.
    Mira, A. (2001). Ordering and improving the performance of Monte Carlo Markov chains. Statistical Science, 16, 340–350.MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Mira, A. (2006). Stationarity preserving and efficiency increasing probability mass transfers made possible. Computational Statistics, 21, 509–522.MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Mira, A., & Sargent Daniel, J. (2003). A new strategy for speeding Markov chain Monte Carlo algorithms. Statistical Methods and Applications, 12, 49–60.MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Neal, R. M. (1994). An improved acceptance procedure for the hybrid Monte Carlo algorithm. Journal of Computational Physics, 111, 194–203.MathSciNetADSCrossRefMATHGoogle Scholar
  46. 46.
    Neal, R. M. (1999). Suppressing random walks in Markov chain Monte Carlo using ordered overrelaxation. In M. I. Jordan (Ed.), Learning in graphical models (pp. 205–228). Cambridge: MIT Press.Google Scholar
  47. 47.
    Niederreiter, H. (1988). Low-discrepancy and low-dispersion sequences. Journal of Number Theory, 30, 51–70.MathSciNetCrossRefMATHGoogle Scholar
  48. 48.
    Peskun, P. H. (1973). Optimum Monte Carlo sampling using Markov chains. Biometrika, 60, 607.MathSciNetCrossRefMATHGoogle Scholar
  49. 49.
    Pollet, L., Rombouts, S. M. A., Van Houcke, K., & Heyde, K. (2004). Optimal Monte Carlo updating. Physical Review E, 70, 056705.ADSCrossRefGoogle Scholar
  50. 50.
    Prokof’ev, N. V., Svistunov, B. V., & Tupitsyn, I. S. (1998). Exact, complete, and universal continuous-time world-line Monte Carlo approach to the statistics of discrete quantum systems. Soviet Physics JETP, 87, 310.ADSCrossRefGoogle Scholar
  51. 51.
    Qin, Z. S., & Liu, J. S. (2001). Multipoint Metropolis method with application to hybrid Monte Carlo. Journal of Computational Physics, 172, 827.MathSciNetADSCrossRefMATHGoogle Scholar
  52. 52.
    Robert, C. P., & Casella, G. (2004). Monte Carlo statistical methods (2nd ed.). New York: Springer.CrossRefMATHGoogle Scholar
  53. 53.
    Sobol, I. M. (1967). Distribution of points in a cube and approximate evaluation of integrals. Z̆rnal Vyčislitel’noi Matematiki i Matematicheskoi Fiziki, 7, 784–802.Google Scholar
  54. 54.
    Suwa, H., & Todo, S. (2010). Markov chain Monte Carlo method without detailed balance. Physical Review Letters, 105, 120603.ADSCrossRefGoogle Scholar
  55. 55.
    Suwa, H., & Todo, S. (2011). Butsuri, 66, 370 (in Japanese).Google Scholar
  56. 56.
    Suwa, H., & Todo, S. (2012). Geometric allocation approach for transition kernel of Markov chain. In Monte Carlo methods and applications, (pp. 213–221). Berlin: De Gruyter.Google Scholar
  57. 57.
    Swendsen, R. H., & Wang, J. S. (1987). Nonuniversal critical dynamics in Monte Carlo simulations. Physical Review Letters, 58, 86.ADSCrossRefGoogle Scholar
  58. 58.
    Syljuasen, O. F., & Sandvik, A. W. (2002). Quantum Monte Carlo with directed loops. Physical Review E, 66, 046701.ADSCrossRefGoogle Scholar
  59. 59.
    Tierney, L. (1994). Markov chain for exploring posterior distributions. Annals of Statistics, 22, 1701.MathSciNetCrossRefMATHGoogle Scholar
  60. 60.
    Todo, S., & Kato, K. (2001). Cluster algorithms for general-\(S\) quantum spin systems. Physical Review Letters, 87, 047203.ADSCrossRefGoogle Scholar
  61. 61.
    Ulam, S., Richtmyer, R. D., & von Neumann, J. (1947). Statistical methods in neutron diffusion. Los Alamos, NM: Los Alamos Scientific Laboratory. (Report LAMS-551).Google Scholar
  62. 62.
    Whitmer, C. (1984). Over-relaxation methods for Monte Carlo simulations of quadratic and multiquadratic actions. Physical Review D, 29, 306–311.ADSCrossRefGoogle Scholar
  63. 63.
    Wu, F. Y. (1982). The Potts model. Reviews of Modern Physics, 54, 235.MathSciNetADSCrossRefGoogle Scholar
  64. 64.
    Zheng, B. (1998). Monte Carlo simulations of short-time critical dynamics. International Journal of Modern Physics B, 12, 1419.ADSCrossRefGoogle Scholar

Copyright information

© Springer Japan 2014

Authors and Affiliations

  1. 1.Department of PhysicsBoston UniversityBostonUSA

Personalised recommendations