Skip to main content
Log in

Hitting Times in Markov Chains with Restart and their Application to Network Centrality

  • Published:
Methodology and Computing in Applied Probability Aims and scope Submit manuscript

Abstract

Motivated by applications in telecommunications, computer science and physics, we consider a discrete-time Markov process with restart. At each step the process either with a positive probability restarts from a given distribution, or with the complementary probability continues according to a Markov transition kernel. The main contribution of the present work is that we obtain an explicit expression for the expectation of the hitting time (to a given target set) of the process with restart. The formula is convenient when considering the problem of optimization of the expected hitting time with respect to the restart probability. We illustrate our results with two examples in uncountable and countable state spaces and with an application to network centrality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alt H, Guibas L, Mehlhorn K, Karp R, Wigderson A (1996) A method for obtaining randomized algorithms with small tail probabilities. Algorithmica 16:543–547

    Article  MathSciNet  Google Scholar 

  • Andersson FK, Silvestrov S (2008) The mathematics of internet search engines. Acta Appl Math 104(2):211–242

    Article  MathSciNet  Google Scholar 

  • Asmussen S, Fiorini P, Lipsky L, Rolski T, Sheahan R (2008) Asymptotic behavior of total times for jobs that must start over if a failure occurs. Math Oper Res 33(4):932–944

    Article  MathSciNet  Google Scholar 

  • Asmussen S, Lipsky L, Thompson S (2014) Checkpointing in failure recovery in computing and data transmission. In: Proceedings of ASMTA’14, also in LNCS, vol 8499. Springer, Berlin, pp 253-272

    Chapter  Google Scholar 

  • Avrachenkov K, Filar JA, Howlett PG (2013) Analytic perturbation theory and its applications. SIAM, Philadelphia

    Book  Google Scholar 

  • Avrachenkov K, Litvak N (2006) The effect of new links on Google PageRank. Stoch Model 22(2):319–331

    Article  MathSciNet  Google Scholar 

  • Avrachenkov K, Litvak N, Pham KS (2008) A singular perturbation approach for choosing the PageRank damping factor. Internet Math 5(1-2):47–69

    Article  MathSciNet  Google Scholar 

  • Avrachenkov K, Mazalov V, Tsynguev B (2015) Beta current flow centrality for weighted networks. In: Proceedings of CSoNet 2015, also in Springer LNCS, vol 9197, pp 216–227

    Chapter  Google Scholar 

  • Avrachenkov K, Piunovskiy A, Zhang Y (2013) Markov processes with restart. J Appl Prob 50:960–968

    Article  MathSciNet  Google Scholar 

  • Baumgärtel H (1985) Analytic perturbation theory for matrices and operators. Birkhäuser, Basel

    MATH  Google Scholar 

  • Bertsekas D, Shreve S (1978) Stochastic optimal control: the discrete-time case. Academic Press, New York

    MATH  Google Scholar 

  • Bianchini M, Gori M, Scarselli F (2005) Inside PageRank. ACM Trans Internet Technol 5(1):92–128

    Article  Google Scholar 

  • Bonacich P (1987) Power and centrality: A family of measures. Amer J Sociol 92(5):1170–1182

    Article  Google Scholar 

  • Borgatti SP (2005) Centrality and network flow. Soc Netw 27(1):55–71

    Article  MathSciNet  Google Scholar 

  • Borgatti SP, Everett MG (2006) A graph-theoretic perspective on centrality. Soc Netw 28(4):466–484

    Article  Google Scholar 

  • Brin S, Page L (1998) The anatomy of a large-scale hypertextual Web search engine. Comput Netw ISDN Syst 30:107–117

    Article  Google Scholar 

  • Dumitriu I, Tetali P, Winkler P (2003) On playing golf with two balls. SIAM J Discr Math 16(4):604–615

    Article  MathSciNet  Google Scholar 

  • Engström C, Silvestrov S (2017) PageRank for networks, graphs and Markov chains. Theory Prob Math Stat 96:61–83

    MathSciNet  MATH  Google Scholar 

  • Evans MR, Majumdar SN (2011) Diffusion with stochastic resetting. Phys Rev Lett 106(16):160601

    Article  Google Scholar 

  • Evans MR, Majumdar SN (2011) Diffusion with optimal resetting. J Phys A: Math Theor 44(43):435001

    Article  Google Scholar 

  • Feller W (1968) An introduction to probability theory and its applications: Volume I. Wiley, New York

    MATH  Google Scholar 

  • Fercoq O, Akian M, Bouhtou M, Gaubert S (2013) Ergodic control and polyhedral approaches to PageRank optimization. IEEE Trans Autom Control 58 (1):134–148

    Article  MathSciNet  Google Scholar 

  • Freeman LC (1978) Centrality in social networks: Conceptual clarification. Soc Netw 1(3):215–239

    Article  Google Scholar 

  • González MC, Hidalgo CA, Barabási A-L (2008) Understanding individual human mobility patterns. Nature 453:779–782

    Article  Google Scholar 

  • Hernández-Lerma O, Lasserre J-B (1999) Further topics in discrete-time Markov control processes. Springer, New York

    Book  Google Scholar 

  • Hopcroft J, Sheldon D (2008) Manipulation-resistant reputations using hitting time. Internet Math 5(1-2):71–90

    Article  MathSciNet  Google Scholar 

  • Janson S, Peres Y (2012) Hitting times for random walks with restarts. SIAM J Discr Math 26(2):537–547

    Article  MathSciNet  Google Scholar 

  • Krishnamurthy B, Rexford J (2001) Web protocols and practice: HTTP/1.1, networking protocols, caching, and traffic measurement. Addison Wesley, Boston

    Google Scholar 

  • Kulkarni V, Nicola V, Trivedi K (1987) The completion time of a job on a multimode system. Adv Appl Prob 19:932–954

    Article  MathSciNet  Google Scholar 

  • Langville AN, Meyer CD (2004) Deeper inside PageRank. Internet Math 1 (3):335–380

    Article  MathSciNet  Google Scholar 

  • Langville AN, Meyer CD (2011) Google’s PageRank and beyond: The science of search engine rankings. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Liu BK, Parkes DC, Seuken S (2016) Personalized hitting time for informative trust mechanisms despite sybils. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, pp 1124–1132

  • Luby M, Sinclair A, Zuckerman D (1993) Optimal speedup of Las Vegas algorithms. Inf Process Lett 47:173–180

    Article  MathSciNet  Google Scholar 

  • Maurer SM, Huberman BA (2001) Restart strategies and Internet congestion. J Econ Dyn Control 25:641–654

    Article  Google Scholar 

  • Meyn S, Tweedie R (1993) Markov chains and stochastic stability. Springer, London

    Book  Google Scholar 

  • Montero M, Villarroel J (2013) Monotonic continuous-time random walks with drift and stochastic reset events. Phys Rev E 87(1):012116

    Article  Google Scholar 

  • Nummelin E (2002) MC’s for MCMC’ists. Int Stat Rev 70(2):215–240

    Article  Google Scholar 

  • Nummelin E, Tuominen P (1982) Geometric ergodicity of Harris recurrent Marcov chains with applications to renewal theory. Stoch Process Their Appl 12(2):187–202

    Article  Google Scholar 

  • Silvestrov DS (1996) Recurrence relations for generalized hitting times for semi-Markov processes. Ann Appl Prob 6(2):617–649

    Article  MathSciNet  Google Scholar 

  • Walsh PD, Boyer D, Crofoot MC (2010) Monkey and cell-phone-user mobilities scale similarly. Nat Phys 6:929–930

    Article  Google Scholar 

Download references

Acknowledgments

This work was partially supported by the European Commission within the framework of the CONGAS project FP7-ICT-2011-8-317672. Y.Zhang’s work was carried out with a financial grant from the Research Fund for Coal and Steel of the European Commission, within the INDUSE-2-SAFETY project (Grant No. RFSR-CT-2014-00025).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Konstantin Avrachenkov.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Avrachenkov, K., Piunovskiy, A. & Zhang, Y. Hitting Times in Markov Chains with Restart and their Application to Network Centrality. Methodol Comput Appl Probab 20, 1173–1188 (2018). https://doi.org/10.1007/s11009-017-9600-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11009-017-9600-5

Keywords

Mathematics Subject Classification (2010)

Navigation