Skip to main content
SpringerLink
Log in

Convergence of Gibbs Sampling: Coordinate Hit-and-Run Mixes Fast

  • Published:
Discrete & Computational Geometry Aims and scope Submit manuscript

Abstract

Gibbs sampling, also known as Coordinate Hit-and-Run (CHAR), is a Markov chain Monte Carlo algorithm for sampling from high-dimensional distributions. In each step, the algorithm selects a random coordinate and re-samples that coordinate from the distribution induced by fixing all the other coordinates. While this algorithm has become widely used over the past half-century, guarantees of efficient convergence have been elusive. We show that the Coordinate Hit-and-Run algorithm for sampling from a convex body K in \({\mathbb {R}}^n\) mixes in \(O^{*}(n^9 R^2/r^2)\) steps, where K contains a ball of radius r and R is the average distance of a point of K from its centroid. We also give an upper bound on the conductance of Coordinate Hit-and-Run, showing that it is strictly worse than Hit-and-Run or the Ball Walk in the worst case.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. The \(O^*\) notation suppresses logarithmic factors and dependence on other parameters like error bound.

References

  1. Andersen, H.C., Diaconis, P.: Hit and run as a unifying device. J. Soc. Fr. Stat. Rev. Stat. Appl. 148(4), 5–28 (2007)

    MathSciNet  MATH  Google Scholar 

  2. Boneh, A.: Preduce—a probabilistic algorithm identifying redundancy by a random feasible point generator (RFPG). In: Redundancy in Mathematical Programming. Lecture Notes in Economics and Mathematical Systems Book Series, vol. 206, pp. 108–134. Springer, Berlin (1983)

  3. Cousins, B., Vempala, S.: Volume-and-Sampling, v.2.2.1. MATLAB File Exchange (2013). https://www.mathworks.com/matlabcentral/fileexchange/43596-volume-and-sampling

  4. Cousins, B., Vempala, S.: Bypassing KLS: Gaussian cooling and an \({\rm O}^*(n^3)\) volume algorithm. In: 47th Annual ACM Symposium on Theory of Computing (Portland 2015), pp. 539–548. ACM, New York (2015)

  5. Cousins, B., Vempala, S.: A practical volume algorithm. Math. Program. Comput. 8(2), 133–160 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  6. Diaconis, P., Khare, K., Saloff-Coste, L.: Gibbs sampling, conjugate priors and coupling. Sankhya A 72(1), 136–169 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  7. Diaconis, P., Lebeau, G., Michel, L.: Gibbs/Metropolis algorithms on a convex polytope. Math. Z. 272(1–2), 109–129 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Emiris, I.Z., Fisikopoulos, V.: Efficient random-walk methods for approximating polytope volume. In: 30th Annual Symposium on Computational Geometry (Kyoto 2014), pp. 318–327. ACM, New York (2014)

  9. Finkel, J.R., Grenager, T., Manning, Ch.: Incorporating non-local information into information extraction systems by Gibbs sampling. In: 43rd Annual Meeting of the Association for Computational Linguistics (Ann Arbor 2005), pp. 363–370. ACL, Stroudsburg (2005)

  10. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6(6), 721–741 (1984)

    Article  MATH  Google Scholar 

  11. George, E.I., McCulloch, R.E.: Variable selection via Gibbs sampling. J. Am. Stat. Assoc. 88(423), 881–889 (1993)

    Article  Google Scholar 

  12. Kannan, R.: Rapid mixing in Markov chains (2003). arXiv:math/0304470

  13. Kannan, R., Lovász, L., Simonovits, M.: Isoperimetric problems for convex bodies and a localization lemma. Discrete Comput. Geom. 13(3–4), 541–559 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  14. Kannan, R., Lovász, L., Simonovits, M.: Random walks and an \(O^*(n^5)\) volume algorithm for convex bodies. Random Struct. Algorithms 11(1), 1–50 (1997)

    Article  MATH  Google Scholar 

  15. Lee, Y.T., Vempala, S.S.: Eldan’s stochastic localization and the KLS hyperplane conjecture: an improved lower bound for expansion. In: 58th Annual IEEE Symposium on Foundations of Computer Science (Berkeley 2017), pp. 998–1007. IEEE Computer Society, Los Alamitos (2017)

  16. Loomis, L.H., Whitney, H.: An inequality related to the isoperimetric inequality. Bull. Am. Math. Soc. 55, 961–962 (1949)

    Article  MathSciNet  MATH  Google Scholar 

  17. Lovász, L.: How to compute the volume? In: Jahresbericht der Deutschen Mathematiker-Vereinigung. Jubiläumstagung 100 Jahre DMV (Bremen 1990), pp. 138–151. Teubner, Stuttgart (1990)

  18. Lovász, L.: Hit-and-run mixes fast. Math. Program. 86(3), 443–461 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  19. Lovász, L., Simonovits, M.: Random walks in a convex body and an improved volume algorithm. Random Struct. Algorithms 4(4), 359–412 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  20. Lovász, L., Vempala, S.: Hit-and-run from a corner. SIAM J. Comput. 35(4), 985–1005 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  21. Lovász, L., Vempala, S.: Simulated annealing in convex bodies and an \(O^*(n^4)\) volume algorithm. J. Comput. Syst. Sci. 72(2), 392–417 (2006)

    Article  MATH  Google Scholar 

  22. Lovász, L., Vempala, S.: The geometry of logconcave functions and sampling algorithms. Random Struct. Algorithms 30(3), 307–358 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  23. Narayanan, H., Srivastava, P.: On the mixing time of coordinate hit-and-run. Combin. Probab. Comput. 31(2), 320–332 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  24. Sinclair, A., Jerrum, M.: Approximate counting, uniform generation and rapidly mixing Markov chains. Inform. Comput. 82(1), 93–133 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  25. Smith, R.L.: Efficient Monte Carlo procedures for generating points uniformly distributed over bounded regions. Oper. Res. 32(6), 1296–1308 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  26. Tkocz, T.: An upper bound for spherical caps. Am. Math. Mon. 119(7), 606–607 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  27. Turchin, V.F.: On the computation of multidimensional integrals by the Monte-Carlo method. Theory Probab. Appl. 16(4), 720–724 (1971)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by NSF awards DMS-1839323, CCF-1909756, and CCF-2007443. The authors thank Ben Cousins for helpful discussions.

Author information

Authors and Affiliations

  1. Georgia Institute of Technology, Atlanta, GA, USA

    Aditi Laddha & Santosh S. Vempala

Authors
  1. Aditi Laddha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Santosh S. Vempala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aditi Laddha.

Additional information

Editor in Charge: Kenneth Clarkson

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Laddha, A., Vempala, S.S. Convergence of Gibbs Sampling: Coordinate Hit-and-Run Mixes Fast. Discrete Comput Geom 70, 406–425 (2023). https://doi.org/10.1007/s00454-023-00497-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00454-023-00497-x

Keywords

Mathematics Subject Classification

Search

Navigation