Abstract
Motivated by applications in telecommunications, computer science and physics, we consider a discrete-time Markov process with restart. At each step the process either with a positive probability restarts from a given distribution, or with the complementary probability continues according to a Markov transition kernel. The main contribution of the present work is that we obtain an explicit expression for the expectation of the hitting time (to a given target set) of the process with restart. The formula is convenient when considering the problem of optimization of the expected hitting time with respect to the restart probability. We illustrate our results with two examples in uncountable and countable state spaces and with an application to network centrality.
Similar content being viewed by others
References
Alt H, Guibas L, Mehlhorn K, Karp R, Wigderson A (1996) A method for obtaining randomized algorithms with small tail probabilities. Algorithmica 16:543–547
Andersson FK, Silvestrov S (2008) The mathematics of internet search engines. Acta Appl Math 104(2):211–242
Asmussen S, Fiorini P, Lipsky L, Rolski T, Sheahan R (2008) Asymptotic behavior of total times for jobs that must start over if a failure occurs. Math Oper Res 33(4):932–944
Asmussen S, Lipsky L, Thompson S (2014) Checkpointing in failure recovery in computing and data transmission. In: Proceedings of ASMTA’14, also in LNCS, vol 8499. Springer, Berlin, pp 253-272
Avrachenkov K, Filar JA, Howlett PG (2013) Analytic perturbation theory and its applications. SIAM, Philadelphia
Avrachenkov K, Litvak N (2006) The effect of new links on Google PageRank. Stoch Model 22(2):319–331
Avrachenkov K, Litvak N, Pham KS (2008) A singular perturbation approach for choosing the PageRank damping factor. Internet Math 5(1-2):47–69
Avrachenkov K, Mazalov V, Tsynguev B (2015) Beta current flow centrality for weighted networks. In: Proceedings of CSoNet 2015, also in Springer LNCS, vol 9197, pp 216–227
Avrachenkov K, Piunovskiy A, Zhang Y (2013) Markov processes with restart. J Appl Prob 50:960–968
Baumgärtel H (1985) Analytic perturbation theory for matrices and operators. Birkhäuser, Basel
Bertsekas D, Shreve S (1978) Stochastic optimal control: the discrete-time case. Academic Press, New York
Bianchini M, Gori M, Scarselli F (2005) Inside PageRank. ACM Trans Internet Technol 5(1):92–128
Bonacich P (1987) Power and centrality: A family of measures. Amer J Sociol 92(5):1170–1182
Borgatti SP (2005) Centrality and network flow. Soc Netw 27(1):55–71
Borgatti SP, Everett MG (2006) A graph-theoretic perspective on centrality. Soc Netw 28(4):466–484
Brin S, Page L (1998) The anatomy of a large-scale hypertextual Web search engine. Comput Netw ISDN Syst 30:107–117
Dumitriu I, Tetali P, Winkler P (2003) On playing golf with two balls. SIAM J Discr Math 16(4):604–615
Engström C, Silvestrov S (2017) PageRank for networks, graphs and Markov chains. Theory Prob Math Stat 96:61–83
Evans MR, Majumdar SN (2011) Diffusion with stochastic resetting. Phys Rev Lett 106(16):160601
Evans MR, Majumdar SN (2011) Diffusion with optimal resetting. J Phys A: Math Theor 44(43):435001
Feller W (1968) An introduction to probability theory and its applications: Volume I. Wiley, New York
Fercoq O, Akian M, Bouhtou M, Gaubert S (2013) Ergodic control and polyhedral approaches to PageRank optimization. IEEE Trans Autom Control 58 (1):134–148
Freeman LC (1978) Centrality in social networks: Conceptual clarification. Soc Netw 1(3):215–239
González MC, Hidalgo CA, Barabási A-L (2008) Understanding individual human mobility patterns. Nature 453:779–782
Hernández-Lerma O, Lasserre J-B (1999) Further topics in discrete-time Markov control processes. Springer, New York
Hopcroft J, Sheldon D (2008) Manipulation-resistant reputations using hitting time. Internet Math 5(1-2):71–90
Janson S, Peres Y (2012) Hitting times for random walks with restarts. SIAM J Discr Math 26(2):537–547
Krishnamurthy B, Rexford J (2001) Web protocols and practice: HTTP/1.1, networking protocols, caching, and traffic measurement. Addison Wesley, Boston
Kulkarni V, Nicola V, Trivedi K (1987) The completion time of a job on a multimode system. Adv Appl Prob 19:932–954
Langville AN, Meyer CD (2004) Deeper inside PageRank. Internet Math 1 (3):335–380
Langville AN, Meyer CD (2011) Google’s PageRank and beyond: The science of search engine rankings. Princeton University Press, Princeton
Liu BK, Parkes DC, Seuken S (2016) Personalized hitting time for informative trust mechanisms despite sybils. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, pp 1124–1132
Luby M, Sinclair A, Zuckerman D (1993) Optimal speedup of Las Vegas algorithms. Inf Process Lett 47:173–180
Maurer SM, Huberman BA (2001) Restart strategies and Internet congestion. J Econ Dyn Control 25:641–654
Meyn S, Tweedie R (1993) Markov chains and stochastic stability. Springer, London
Montero M, Villarroel J (2013) Monotonic continuous-time random walks with drift and stochastic reset events. Phys Rev E 87(1):012116
Nummelin E (2002) MC’s for MCMC’ists. Int Stat Rev 70(2):215–240
Nummelin E, Tuominen P (1982) Geometric ergodicity of Harris recurrent Marcov chains with applications to renewal theory. Stoch Process Their Appl 12(2):187–202
Silvestrov DS (1996) Recurrence relations for generalized hitting times for semi-Markov processes. Ann Appl Prob 6(2):617–649
Walsh PD, Boyer D, Crofoot MC (2010) Monkey and cell-phone-user mobilities scale similarly. Nat Phys 6:929–930
Acknowledgments
This work was partially supported by the European Commission within the framework of the CONGAS project FP7-ICT-2011-8-317672. Y.Zhang’s work was carried out with a financial grant from the Research Fund for Coal and Steel of the European Commission, within the INDUSE-2-SAFETY project (Grant No. RFSR-CT-2014-00025).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Avrachenkov, K., Piunovskiy, A. & Zhang, Y. Hitting Times in Markov Chains with Restart and their Application to Network Centrality. Methodol Comput Appl Probab 20, 1173–1188 (2018). https://doi.org/10.1007/s11009-017-9600-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11009-017-9600-5