On Approximating the Stationary Distribution of Time-Reversible Markov Chains

  • Marco BressanEmail author
  • Enoch Peserico
  • Luca Pretto
Part of the following topical collections:
  1. Special Issue on Theoretical Aspects of Computer Science (2018)


Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require \(\tilde {O}(\tau /\pi (v))\) operations to approximate the probability π(v) of a state v in a chain with mixing time τ, and even the best available techniques still have complexity \(\tilde {O}(\tau ^{1.5}/\pi (v)^{0.5})\); and since these complexities depend inversely on π(v), they can grow beyond any bound in the size of the chain or in its mixing time. In this paper we show that, for time-reversible Markov chains, there exists a simple randomized approximation algorithm that breaks this “small-π(v) barrier”.


Markov chains MCMC sampling Large graph algorithms Randomized algorithms Sublinear algorithms 



Marco Bressan is supported in part by the ERC Starting Grant DMAP 680153, by a Google Focused Research Award, and by the “Dipartimenti di Eccellenza 2018-2022” grant awarded to the Dipartimento di Informatica at Sapienza. Enoch Peserico is supported in part by Università degli Studi di Padova under Project CAEPAE.


  1. 1.
    Alon, N., Gurel-Gurevich, O., Lubetzky, E.: Choice-memory tradeoff in allocations. Ann. Appl. Probab. 20(4), 1470–1511 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Auger, A., Doerr, B. (eds.): Theory of Randomized Search Heuristics: Foundations and Recent Developments, vol. 1. World Scientific Publishing Co., Inc, Singapore (2011)Google Scholar
  3. 3.
    Banerjee, S., Lofgren, P.: Fast Bidirectional Probability Estimation in Markov Models. In: Proceedings Of NIPS, pp. 1423–1431 (2015)Google Scholar
  4. 4.
    Bonacich, P., Lloyd, P.: Eigenvector-like measures of centrality for asymmetric relations. Soc. Netw. 23(3), 191–201 (2001)CrossRefGoogle Scholar
  5. 5.
    Borgs, C., Brautbar, M., Chayes, J. T., Teng, S.: Multiscale matrix sampling and sublinear-time PageRank computation. Internet Math. 10(1-2), 20–48 (2014)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Borgs, C., Brautbar, M., Chayes, J. T., Teng, S. H.: A Sublinear Time Algorithm for PageRank Computations. In: Proceedings Of WAW, pp. 41–53. Springer (2012)Google Scholar
  7. 7.
    Bressan, M., Peserico, E., Pretto, L.: Brief Announcement: On Approximating PageRank Locally with Sublinear Query Complexity. In: Proceedings Of ACM SPAA, pp. 87–89 (2018)Google Scholar
  8. 8.
    Bressan, M., Peserico, E., Pretto, L.: Sublinear Algorithms for Local Graph Centrality Estimation. In: Proceedings Of IEEE FOCS, pp. 709–718 (2018)Google Scholar
  9. 9.
    Chung, K. M., Lam, H., Liu, Z., Mitzenmacher, M.: Chernoff-Hoeffding Bounds for Markov Chains: Generalized and Simplified. In: Proceedings Of STACS, pp. 124–135 (2012)Google Scholar
  10. 10.
    Freedman, D. A.: On tail probabilities for martingales. Ann. Probab. 3(1), 100–118 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Golub, G. H., Van Loan, C. F.: Matrix Computations. Johns Hopkins University Press, Baltimore (2012)zbMATHGoogle Scholar
  12. 12.
    Hastings, W. K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1), 97–109 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Lee, C. E., Ozdaglar, A., Shah, D.: Computing the Stationary Distribution, Locally. In: Proceedings Of NIPS, pp. 1376–1384 (2013)Google Scholar
  14. 14.
    Lee, C. E., Ozdaglar, A. E., Shah, D.: Solving systems of linear equations: Locally and asynchronously. arXiv:1411.2647 (2014)
  15. 15.
    Levin, D. A., Peres, Y., Wilmer, E. L.: Markov Chains and Mixing Times. American Mathematical Society (2009)Google Scholar
  16. 16.
    Lofgren, P., Banerjee, S., Goel, A.: Bidirectional PageRank Estimation: from Average-Case to Worst-Case. In: Proceedings Of WAW, pp. 164–176 (2015)Google Scholar
  17. 17.
    Lofgren, P. A., Banerjee, S., Goel, A., Seshadhri, C.: FAST-PPR: Scaling Personalized PageRank Estimation for Large Graphs. In: Proceedings Of ACM KDD, pp. 1436–1445 (2014)Google Scholar
  18. 18.
    Motwani, R., Panigrahy, R., Xu, Y.: Estimating Sum by Weighted Sampling. In: Proceedings Of ICALP, pp. 53–64 (2007)Google Scholar
  19. 19.
    Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, Cambridge (1995)CrossRefzbMATHGoogle Scholar
  20. 20.
    Panconesi, A., Srinivasan, A.: Randomized distributed edge coloring via an extension of the Chernoff–Hoeffding bounds. SIAM J. Comput. 26(2), 350–368 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Rubinfeld, R., Shapira, A.: Sublinear time algorithms. SIAM J. Discret. Math. 25(4), 1562–1588 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Shyamkumar, N., Banerjee, S., Lofgren, P.: Sublinear Estimation of a Single Element in Sparse Linear Systems. In: 2016 Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 856–860 (2016)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Dipartimento di InformaticaSapienza Università di RomaRomaItaly
  2. 2.Dipartimento di Ingegneria dell’InformazioneUniversità degli Studi di PadovaPadovaItaly

Personalised recommendations