Advertisement

Metrika

, Volume 76, Issue 1, pp 107–133 | Cite as

Coefficients of ergodicity for Markov chains with uncertain parameters

  • D. Škulj
  • R. Hable
Article

Abstract

One of the central considerations in the theory of Markov chains is their convergence to an equilibrium. Coefficients of ergodicity provide an efficient method for such an analysis. Besides giving sufficient and sometimes necessary conditions for convergence, they additionally measure its rate. In this paper we explore coefficients of ergodicity for the case of imprecise Markov chains. The latter provide a convenient way of modelling dynamical systems where parameters are not determined precisely. In such cases a tool for measuring the rate of convergence is even more important than in the case of precisely determined Markov chains, since most of the existing methods of estimating the limit distributions are iterative. We define a new coefficient of ergodicity that provides necessary and sufficient conditions for convergence of the most commonly used class of imprecise Markov chains. This so-called weak coefficient of ergodicity is defined through an endowment of the structure of a metric space to the class of imprecise probabilities. Therefore we first make a detailed analysis of the metric properties of imprecise probabilities.

Keywords

Markov chain Imprecise Markov chain Coefficient of ergodicity Weak coefficient of ergodicity Uniform coefficient of ergodicity Lower expectation Upper expectation Hausdorff metric Convergence of Markov chains 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Beer G (1993) Topologies on closed and closed convex sets. Kluwer, DordrechtzbMATHGoogle Scholar
  2. Birkhoff G (1957) Extensions of Jentzsch’s theorem. Trans Am Math Soc 85(1): 219–227MathSciNetzbMATHGoogle Scholar
  3. Crossman RJ, Škulj D (2010) Imprecise Markov chains with absorption. Int J Approx Reason 51: 1085–1099. doi: 10.1016/j.ijar.2010.08.008 zbMATHCrossRefGoogle Scholar
  4. Crossman RJ, Coolen-Schrijner P, Coolen FPA (2009a) Time-homogeneous birth-death processes with probability intervals and absorbing state. J Stat Theory Practice 3(1): 103–118MathSciNetzbMATHCrossRefGoogle Scholar
  5. Crossman RJ, Coolen-Schrijner P, Škulj D, Coolen FPA (2009b) Imprecise Markov chains with an absorbing state. In: Augustin T, Coolen FPA, Moral S, Troffaes MCM (eds) ISIPTA’09: proceedings of the sixth international symposium on imprecise probability: theories and applications, SIPTA, Durham, UK, pp 119–128Google Scholar
  6. Darroch JN, Seneta E (1965) On quasi-stationary distributions in absorbing discrete-time finite Markov chains. J Appl Probab 2(1):88–100, http://www.jstor.org/stable/3211876 Google Scholar
  7. de Campos LD, Huete J, Moral S (1994) Probability intervals: a tool for uncertain reasoning. Int J Uncertain Fuzz Knowl Based Syst 2(2): 167–196zbMATHCrossRefGoogle Scholar
  8. De Cooman G, Hermans F, Quaeghebeur E (2009) Imprecise Markov chains and their limit behavior. Probab Eng Inform Sci 23(4): 597–635. doi: 10.1017/S0269964809990039 MathSciNetzbMATHCrossRefGoogle Scholar
  9. Dobrushin R (1956) Central limit theorem for non-stationary Markov chains, I, II. Theory Probab Appl 1(4): 329–383CrossRefGoogle Scholar
  10. Dunford N, Schwartz J (1988) Linear operators. Part I: general theory. Wiley, New YorkzbMATHGoogle Scholar
  11. Fan K (1953) Minimax theorems. Proc Natl Acad Sci USA 39: 42–47zbMATHCrossRefGoogle Scholar
  12. Hable R (2009) Data-based decisions under complex uncertainty. PhD thesis, Ludwig-Maximilians-Universität (LMU) Munich, http://edoc.ub.uni-muenchen.de/9874/
  13. Hable R (2010) Minimum distance estimation in imprecise probability models. J Stat Plan Inference 140: 461–479MathSciNetzbMATHCrossRefGoogle Scholar
  14. Harmanec D (2002) Generalizing Markov decision processes to imprecise probabilities. J Stat Plan Inference 105: 199–213MathSciNetzbMATHCrossRefGoogle Scholar
  15. Hartfiel D (1998) Markov set-chains. Springer, BerlinzbMATHGoogle Scholar
  16. Hartfiel D, Rothblum U (1998) Convergence of inhomogenous products of matrices and coefficients of ergodicity. Linear Algebra Appl 277: 1–9MathSciNetzbMATHCrossRefGoogle Scholar
  17. Hartfiel D, Seneta E (1994) On the theory of Markov set-chains. Adv Appl Probab 26(4): 947–964MathSciNetzbMATHCrossRefGoogle Scholar
  18. Holmes RB (1975) Geometric functional analysis and its applications. Springer, BerlinzbMATHCrossRefGoogle Scholar
  19. Itoh H, Nakamura K (2007) Partially observable Markov decision processes with imprecise parameters. Artif Intell 171(8–9): 453–490MathSciNetzbMATHCrossRefGoogle Scholar
  20. Kozine I, Utkin L (2002) Interval-valued finite Markov chains. Reliable Comput 8(2): 97–113MathSciNetzbMATHCrossRefGoogle Scholar
  21. Nilim A, Ghaoui LE (2005) Robust control of Markov decision processes with uncertain transition matrices. Oper Res 53: 780–798MathSciNetzbMATHCrossRefGoogle Scholar
  22. Paz A (1970) Ergodic theorems for infinite probabilistic tables. Ann Math Stat 41(2): 539–550MathSciNetzbMATHCrossRefGoogle Scholar
  23. Satia J, Lave R (1973) Markovian decision processes with uncertain transition probabilities. Oper Res 21(3): 728–740MathSciNetzbMATHCrossRefGoogle Scholar
  24. Seneta E (1979) Coefficients of ergodicity—structure and applications. Adv Appl Probab 11(2): 270–271MathSciNetCrossRefGoogle Scholar
  25. Seneta E (2006) Non-negative matrices and Markov chains. Springer, BerlinzbMATHGoogle Scholar
  26. Škulj D (2006) Finite discrete time Markov chains with interval probabilities. In: Lawry J, Miranda E, Bugarín A, Li S, Gil MA, Grzegorzewski P, Hryniewicz O (eds) SMPS. Advances in soft computing. Springer, Berlin, vol 37, pp 299–306Google Scholar
  27. Škulj D (2007) Regular finite Markov chains with interval probabilities. In: De Cooman G, Zaffalon M, Vejnarová J (eds) ISIPTA’07—proceedings of the fifth international symposium on imprecise probability: theories and applications, SIPTA, pp 405–413Google Scholar
  28. Škulj D (2009) Discrete time Markov chains with interval probabilities. Int J Approx Reason 50(8): 1314–1329. doi: 10.1016/j.ijar.2009.06.007 zbMATHCrossRefGoogle Scholar
  29. Škulj D, Hable R (2009) Coefficients of ergodicity for imprecise Markov chains. In: Augustin T, Coolen FPA, Moral S, Troffaes MCM (eds) ISIPTA’09: proceedings of the sixth international symposium on imprecise probability: theories and applications, SIPTA, Durham, UK, pp 377–386Google Scholar
  30. Walley P (1991) Statistical reasoning with imprecise probabilities. Chapman and Hall, LondonzbMATHGoogle Scholar
  31. Walley P (2000) Towards a unified theory of imprecise probability. Int J Approx Reason 24: 125–148MathSciNetzbMATHCrossRefGoogle Scholar
  32. Weichselberger K (2001) Elementare Grundbegriffe einer allgemeineren Wahrscheinlichkeitsrechnung. I: Intervallwahrscheinlichkeit als umfassendes Konzept. Physica-Verlag, HeidelbergzbMATHCrossRefGoogle Scholar
  33. White C, Eldeib H (1994) Markov decision processes with imprecise transition probabilities. Oper Res 42(4): 739–749MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Faculty of Social SciencesLjubljanaSlovenia
  2. 2.Department of MathematicsBayreuthGermany

Personalised recommendations