Advertisement

Transform-Domain Solutions of Poisson’s Equation with Applications to the Asymptotic Variance

  • Koen De Turck
  • Sofian De Clercq
  • Sabine Wittevrongel
  • Herwig Bruneel
  • Dieter Fiems
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7314)

Abstract

Recent years have seen a considerable increase of attention devoted to Poisson’s equation for Markov chains, which now has attained a central place in Markov chain theory, due to the extensive list of areas where Poisson’s equation pops up: perturbation analysis, Markov decision processes, limit theorems of Markov chains, etc. all find natural expression when viewed from the vantage point of Poisson’s equation.

We describe how the use of generating functions helps solve Poisson’s equation for different types of structured Markov chains and for driving functions, and point out some applications. In particular, we solve Poisson’s equation in the transform domain for skip-free Markov chains and Markov chains with linear displacement. Closed-form solutions are obtained for a class of driving functions encompassing polynomial functions and functions with finite support.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cohen, J.: The single-server queue. North-Holland Series in Appl Math. and Mech. (1969)Google Scholar
  2. 2.
    Takagi, H.: Queueing analysis, a foundation of performance evaluation. Discrete-time systems, vol. 3. Elsevier Science Publishers BV, Amsterdam (1993)Google Scholar
  3. 3.
    Bellman, R.: On the Theory of Dynamic Programming. Proceedings of the National Academy of Sciences (1952)Google Scholar
  4. 4.
    Glynn, P.W., Meyn, S.P.: A Liapounov bound for solutions of the Poisson equation. Ann. Probab. 24(2), 916–931 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Schweitzer, P.J.: Perturbation theory and finite Markov chains. J. Appl. Prob. 5, 401–403 (1968)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Meyn, S.P., Tweedie, R.L.: Markov chains and stochastic stability. Springer, London (2003)Google Scholar
  7. 7.
    Meyn, S.P.: Control Techniques for Complex Networks. Cambridge University Press (2007)Google Scholar
  8. 8.
    Asmussen, S.: Applied probability and queues, 2nd edn. Springer (2003)Google Scholar
  9. 9.
    Neveu, J.: Potentiel Markovien récurrent des chaînes de Harris. Ann. Inst. Fourier 22(7), 130 (1972)Google Scholar
  10. 10.
    Jones, G.L.: On the Markov chain central limit theorem. Probab. Surv. 1, 299–320 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Nummelin, E.: On the Poisson equation in the potential theory of a single kernel. Math. Scand. 68, 59–82 (1991)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Shwartz, A., Makowski, A.: On the Poisson equation for Markov chains: existence of solutions and parameter dependence. Technical report, Dept. Electrical Engineering, Technion – Israel Institute of Technology (1991)Google Scholar
  13. 13.
    Koole, G., Spieksma, F.: On deviation matrices for birth-death processes. Probability in the Engineering and Informational Sciences 15, 239–258 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Hordijk, A., Spieksma, F.M.: On ergodicity and recurrence properties of a Markov chain with an application. Adv. Appl. Probab. 24, 343–376 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Nazarathy, Y.: The Variance of Departure Processes: Puzzling Behavior and Open Problems. Queueing Systems 68, 385–394 (2011)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Kingman, J.F.C.: On Queues in Heavy Traffic. Journal of the Royal Statistical Society. Series B (Methodological) 24(2), 383–392 (1962)MathSciNetzbMATHGoogle Scholar
  17. 17.
    White, L.B.: A new policy evaluation algorithm for Markov decision processes with quasi birth-death structure. Stochastic Models 21, 785–797 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Lambert, J., van Houdt, B., Blondia, C.: A policy iteration algorithm for Markov decision processes skip-free in one direction. Numerical Methods for Structured Markov Chains (2007)Google Scholar
  19. 19.
    Fiems, D., Prabhu, B., De Turck, K.: Analytic approximations of queues with lightly- and heavily-correlated autoregressive service times. Annals of Operations ResearchGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Koen De Turck
    • 1
  • Sofian De Clercq
    • 1
  • Sabine Wittevrongel
    • 1
  • Herwig Bruneel
    • 1
  • Dieter Fiems
    • 1
  1. 1.Department of Telecommunications and Information ProcessingGhent UniversityGentBelgium

Personalised recommendations