Abstract
The paper is devoted to studies of regularly and singularly perturbed Markov chains with damping component. In such models, a matrix of transition probabilities is regularised by adding a special damping matrix multiplied by a small damping (perturbation) parameter ε. We perform a detailed perturbation analysis for such Markov chains, particularly, give effective upper bounds for the rate of approximation for stationary distributions of unperturbed Markov chains by stationary distributions of perturbed Markov chains with regularised matrices of transition probabilities, asymptotic expansions for approximating stationary distributions with respect to damping parameter, explicit coupling type upper bounds for the rate of convergence in ergodic theorems for n-step transition probabilities, as well as ergodic theorems in triangular array mode.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Abola B, Biganda PS, Engström C, Mango JM, Kakuba G, Silvestrov S (2018a) Pagerank in evolving tree graphs. In: Silvestrov S, Ranc̆ić M, Malyarenko A (eds) Stochastic processes and applications, Chapter 16. Springer proceedings in mathematics & statistics, vol 271. Springer, Cham, pp 375–390
Abola B, Biganda PS, Engström C, Mango JM, Kakuba G, Silvestrov S (2018b) Updating of PageRank in evolving tree graphs. In: Skiadas C H (ed) Proceedings of the 5th stochastic modeling techniques and data analysis international conference with demographics workshop. Chania, Crete, Greece, 2018. ISAST: international society for the advancement of science and technology, pp 15–26
Andersson FK, Silvestrov SD (2008) The mathematics of internet search engines. Acta Applic Math 104(2):211–242
Avrachenkov K, Litvak N (2004) Decomposition of the google PageRank and optimal linking strategy. Research report, INRIA, 15
Avrachenkov K, Litvak N, Pham KS (2008) A singular perturbation approach for choosing the PageRank damping factor. Internet Math 5(1–2):47–69
Avrachenkov K, Filar J, Howlett PG (2013) Analytic perturbation theory and its applications. SIAM, Philadelphia, p xii+ 372
Avrachenkov K, Piunovskiy A, Zhang Y (2018) Hitting times in Markov chains with restart and their application to network centrality. Methodol Comput Appl Probab 20(4):1173–1188
Battiston S, Puliga M, Kaushik R, Tasca P, Caldarelli G (2012) Debtrank: too central to fail? Financial networks, the fed and systemic risk. Scientific Reports 2:541
Biganda PS, Abola B, Engström C, Silvestrov S (2017) Pagerank, connecting a line of nodes with multiple complete graphs. In: Skiadas C H (ed) Proceedings of the 17th applied stochastic models and data analysis international conference with the 6th demographics workshop. London, UK, 2017. ISAST: international society for the advancement of science and technology, pp 113–126
Biganda PS, Abola B, Engström C, Mango JM, Kakuba G, Silvestrov S (2018a) Exploring the relationship between ordinary PageRank, lazy PageRank and random walk with backstep PageRank for different graph structures. In: Skiadas CH (ed) Proceedings of the 5th stochastic modeling techniques and data analysis international conference with demographics workshop. Chania, Crete, Greece, 2018. ISAST: international society for the advancement of science and technology, pp 71–85
Biganda PS, Abola B, Engström C, Mango JM, Kakuba G, Silvestrov S (2018b) Traditional and lazy PageRanks for a line of nodes connected with complete graphs. In: Silvestrov S, Ranc̆ić M, Malyarenko A (eds) Stochastic processes and applications, Chapter 17. Springer proceedings in mathematics & statistics, vol 271. Springer, Cham, pp 391–412
Bini DA, Latouche G, Meini B (2005) Numerical methods for structured Markov chains. Numerical mathematics and scientific computation, Oxford Science Publications. Oxford University Press, New York, p xii+ 327
Brin S, Page L (1998) The anatomy of a large-scale hypertextual web search engine. Comp Networks, ISDN Syst 30(1-7):107–117
Dobrushin R (1956) Central limit theorem for non-stationary Markov chains. I Teor Veroyatnost i Primenen 1(1):72–89. (English translation in Theory Probab Appl 1(1):65–80)
Doeblin W (1940) ÉLéments d’une théorie générale des chaînes simples constantes de markoff. (French) Ann École Norm 57(3):61–111
Englund E (2001) Nonlinearly perturbed renewal equations with applications. Doctoral dissertation, Umeå University
Englund E, Silvestrov D (1997) Mixed large deviation and ergodic theorems for regenerative processes with discrete time. In: Jagers P, Kulldorff G, Portenko N, Silvestrov D (eds) Proceedings of the second Scandinavian–Ukrainian conference in mathematical statistics, Vol. I, Umeå, 1997. Theory Stoch. Process., 3(19), no. 1–2, 164–176
Engström C (2016) Pagerank in evolving networks and applications of graphs in natural language processing and biology. Doctoral dissertation 217, Mälardalen University, Västerås
Engström C, Silvestrov S (2014) Generalisation of the damping factor in PageRank for weighted networks. In: Silvestrov D, Martin-Löf A (eds) Modern problems in insurance mathematics, chapter 19. EAA series. Springer, Cham, pp 313–334
Engström C, Silvestrov S (2016a) Pagerank, a look at small changes in a line of nodes and the complete graph. In: Silvestrov S, Ranc̆ić M (eds) Engineering mathematics II. Algebraic, stochastic and analysis structures for networks, data classification and optimization, Chapter 11. Springer Proceedings in Mathematics & Statistics, vol 179. Springer, Cham, pp 223–248
Engström C, Silvestrov S (2016b) Pagerank, connecting a line of nodes with a complete graph. In: Silvestrov S, Ranc̆ić M (eds) Engineering mathematics II. Algebraic, stochastic and analysis structures for networks, data classification and optimization, Chapter 12. Springer Proceedings in Mathematics & Statistics, vol 179. Springer, Cham, pp 249–274
Engström C, Silvestrov S (2017) Pagerank for networks, graphs, and Markov chains. Teor Ĭmovirn Mat Stat 96:61–83. (Also, in Theor Probab Math Statist 96:59–82)
Feller W (1968) An introduction to probability theory and its applications, vol I, 3rd edn. Wiley, New York, p xviii+ 509
Filar J, Krieger HA, Syed Z (2002) Cesaro limits of analytically perturbed stochastic matrices. Linear Algeb Appl 353(1–3):227–243
Gambini A, Krzyanowski P, Pokarowski P (2008) Aggregation algorithms for perturbed Markov chains with applications to networks modeling. SIAM J Sci Comput 31(1):45–73
Gleich DF (2015) Pagerank beyond the Web. SIAM Rev 57(3):321–363
Griffeath D (1975) A maximal coupling for Markov chains. Z Wahrsch Verw Gebiete 31:95–106
Gyllenberg M, Silvestrov D (2000) Nonlinearly perturbed regenerative processes and pseudo-stationary phenomena for stochastic systems. Stoch Process Appl 86:1–27
Gyllenberg M, Silvestrov D (2008) Quasi-stationary phenomena in nonlinearly perturbed stochastic systems. de gruyter expositions in mathematics, 44. Walter de Gruyter, Berlin, p ix+ 579
Hartfiel DJ, Meyer CD (1998) On the structure of stochastic matrices with a subdominant eigenvalue near 1. Linear Algebra Appl 272(1-3):193–203
Haveliwala T, Kamvar S (2003) The second eigenvalue of the google matrix. Technical report 2003-20. Stanford University
Kalashnikov VV (1989) Coupling method, development and applications. Supplement to the Russian edition of the book by Nummelin, E. (1984). General lrreducible Markov Chains and Non-negative Operators. Cambridge University Press (Russian edition, MIR, Moscow, 1989, 176–190)
Kamvar SD, Schlosser MT, Garcia-Molina H (2003) The eigentrust algorithm for reputation management in p2p networks. In: Proceedings of the 12th international conference on world wide web. ACM, pp 640–651
Konstantinov M, Gu DW, Mehrmann V, Petkov P (2003) Perturbation theory for matrix equations. Studies in computational mathematics, 9, North-Holland, Amsterdam, pp xii+ 429
Koroliuk VS, Limnios N (2005) Stochastic systems in merging phase space. World Scientific, Singapore, p xv+ 331
Korolyuk VS, Korolyuk VV (1999) Stochastic models of systems. Mathematics and its applications, 469. Kluwer, Dordrecht, p xii+ 185
Langville AN, Meyer CD (2004) Deeper inside PageRank. Internet Math 1(3):335–338
Langville AN, Meyer CD (2011) Google’s pagerank and beyond: the science of search engine rankings. Princeton University Press, Princeton, p x + 224
Lindvall T (2002) Lectures on the coupling method. Wiley series in probability and mathematical statistics: probability and mathematical statistics. Wiley, New York, p xiv+ 257. (A revised reprint of the 1992 original)
Loève M (1978) Probability theory II. Fourth edition. Graduate texts in mathematics, springer, xvi+ 416pp. (Extended version of the first (1955) edition)
Markov AA (1906) Generalisation of the law of large numbers on dependent trials. Izv Kazan Fiz-Mat Obsch 15(4):135–156
Mitrophanov AY (2005) Sensitivity and convergence of uniformly ergodic Markov chains. J Appl Prob 42:1003–1014
Ni Y (2011) Nonlinearly perturbed renewal equations: asymptotic results and applications. Doctoral dissertation 106, Mälardalen University, Västerås
Ni Y, Silvestrov D, Malyarenko A (2008) Exponential asymptotics for nonlinearly perturbed renewal equation with non-polynomial perturbations. J Numer Appl Math 1(96):173–197
Petersson M (2016) Perturbed discrete time stochastic models. Doctoral dissertation. Stockholm University
Pitman JW (1979) On coupling of Markov chains. Z Wahrsch Verw Gebiete 35:315–322
Silvestrov D (1978) The renewal theorem in a series scheme. 1. Teor Veroyatn Mat Stat 18:144–161. (English translation in Theory Probab Math Statist 18:155–172)
Silvestrov D (1979) The renewal theorem in a series scheme 2. Teor Veroyatn Mat Stat 20:97–116. (English translation in Theory Probab Math Statist 20:113–130)
Silvestrov D (1983) Method of a single probability space in ergodic theorems for regenerative processes 1. Math Operat Statist, Ser Optim 14:285–299
Silvestrov D (1984a) Method of a single probability space in ergodic theorems for regenerative processes 2. Math Operat Statist, Ser Optim 15:601–612
Silvestrov D (1984b) Method of a single probability space in ergodic theorems for regenerative processes 3. Math Operat Statist, Ser Optim 15:613–622
Silvestrov D (1994) Coupling for Markov renewal processes and the rate of convergence in ergodic theorems for processes with semi-Markov switchings. Acta Applic Math 34:109–124
Silvestrov D (2018) Individual ergodic theorems for perturbed alternating regenerative processes. In: Silvestrov S, Ranc̆ić M, Malyarenko A (eds) Stochastic processes and applications, Chapter 3. Springer Proceedings in Mathematics & Statistics, vol 271. Springer, Cham, pp 23–89
Silvestrov D, Petersson M (2014) Exponential expansions for perturbed discrete time renewal equations. In: Karagrigoriou A, Lisnianski A, Kleyner A, Frenkel I (eds) Applied reliability engineering and risk analysis, Chapter 23. Probabilistic models and statistical inference. Wiley, New York, pp 349–362
Silvestrov D, Petersson M, Hössjer O (2018) Nonlinearly perturbed birth-death-type models. In: Silvestrov S, Ranc̆ić M, Malyarenko A (eds) Stochastic processes and applications, Chapter 3. Springer proceedings in mathematics & statistics, vol 271. Springer, Cham, pp 189–244
Silvestrov D, Pezinska G (1985) On maximally coinciding random variables. Teor Veroyatn Mat Stat 32:102–105. (English translation in Theory Probab Math Statist 32:113–115)
Silvestrov D, Silvestrov S (2016) Asymptotic expansions for stationary distributions of perturbed semi-Markov processes. In: Silvestrov S, Ranc̆ić M (eds) Engineering mathematics II. Algebraic, stochastic and analysis structures for networks, data classification and optimization, Chapter 10. Springer Proceedings in Mathematics & Statistics, vol 179. Springer, Cham, pp 151–222
Silvestrov D, Silvestrov S (2017a) Nonlinearly perturbed semi-markov processes. Springer briefs in probability and mathematical statistics. Springer, Cham, p xiv+ 143
Silvestrov D, Silvestrov S (2017b) Asymptotic expansions for power-exponential moments of hitting times for nonlinearly perturbed semi-Markov processes. Teor Ĭmovirn Mat Stat 97:171–187. (Also, in Theory Probab Math Statist 97:183–200)
Silvestrov D, Silvestrov S (2019a) Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes. 1. Methodol Comput Appl Probab 21(3):945–964. (First Online 2017)
Silvestrov D, Silvestrov S (2019b) Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes. 2. Methodol Comput Appl Probab 21(3):965–984. (First Online 2017)
Stewart GW (1998) Matrix algorithms. Vol. I. Basic decompositions. SIAM, Philadelphia, p xx+ 458
Stewart GW (2001) Matrix algorithms. Vol. II. Eigensystems. SIAM, Philadelphia, p xx+ 469
Stewart WJ (1994) Introduction to the numerical solution of markov chains. Princeton University Press, Princeton, p xx+ 539
Sun Y, Han J (2013) Mining heterogeneous information networks: a structural analysis approach. ACM SIGKDD Explor Newslet 14(2):20–28
Thorisson H (2000) Coupling, stationarity and regeneration. Probability and its applications. Springer, New York, p ivx+ 517
Yin GG, Zhang Q (2005) Discrete-time markov chains. Two-time-scale methods and applications. Stochastic modelling and applied probability, 55. Springer, New York, p xix+ 348
Yin GG, Zhang Q (2013) Continuous-time markov chains and applications. a two-time-scale approach. Second edition, stochastic modelling and applied probability, 37. Springer, New York, p xxii+ 427. (An extended variant of the first (1998) edition)
Acknowledgements
This research was supported by the Swedish International Development Cooperation Agency (Sida), International Science Programme (ISP) in Mathematical Sciences (IPMS) and Sida Bilateral Research Programmes for research and education capacity development in Mathematics in Uganda and Tanzania. The authors are also grateful to the research environment Mathematics and Applied Mathematics (MAM), Division of Applied Mathematics, Mälardalen University for providing an excellent and inspiring environment for research education and research.
Funding
Open access funding provided by Stockholm University.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Silvestrov, D., Silvestrov, S., Abola, B. et al. Perturbed Markov Chains with Damping Component. Methodol Comput Appl Probab 23, 369–397 (2021). https://doi.org/10.1007/s11009-020-09815-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11009-020-09815-9
Keywords
- Markov chain
- Damping component
- Information network
- Regular perturbation
- Singular perturbation
- Stationary distribution
- Asymptotic expansion
- Rate of convergence
- Coupling
- Ergodic theorem
- Triangular array mode