Transform-Domain Solutions of Poisson’s Equation with Applications to the Asymptotic Variance
Recent years have seen a considerable increase of attention devoted to Poisson’s equation for Markov chains, which now has attained a central place in Markov chain theory, due to the extensive list of areas where Poisson’s equation pops up: perturbation analysis, Markov decision processes, limit theorems of Markov chains, etc. all find natural expression when viewed from the vantage point of Poisson’s equation.
We describe how the use of generating functions helps solve Poisson’s equation for different types of structured Markov chains and for driving functions, and point out some applications. In particular, we solve Poisson’s equation in the transform domain for skip-free Markov chains and Markov chains with linear displacement. Closed-form solutions are obtained for a class of driving functions encompassing polynomial functions and functions with finite support.
Unable to display preview. Download preview PDF.
- 1.Cohen, J.: The single-server queue. North-Holland Series in Appl Math. and Mech. (1969)Google Scholar
- 2.Takagi, H.: Queueing analysis, a foundation of performance evaluation. Discrete-time systems, vol. 3. Elsevier Science Publishers BV, Amsterdam (1993)Google Scholar
- 3.Bellman, R.: On the Theory of Dynamic Programming. Proceedings of the National Academy of Sciences (1952)Google Scholar
- 6.Meyn, S.P., Tweedie, R.L.: Markov chains and stochastic stability. Springer, London (2003)Google Scholar
- 7.Meyn, S.P.: Control Techniques for Complex Networks. Cambridge University Press (2007)Google Scholar
- 8.Asmussen, S.: Applied probability and queues, 2nd edn. Springer (2003)Google Scholar
- 9.Neveu, J.: Potentiel Markovien récurrent des chaînes de Harris. Ann. Inst. Fourier 22(7), 130 (1972)Google Scholar
- 12.Shwartz, A., Makowski, A.: On the Poisson equation for Markov chains: existence of solutions and parameter dependence. Technical report, Dept. Electrical Engineering, Technion – Israel Institute of Technology (1991)Google Scholar
- 18.Lambert, J., van Houdt, B., Blondia, C.: A policy iteration algorithm for Markov decision processes skip-free in one direction. Numerical Methods for Structured Markov Chains (2007)Google Scholar
- 19.Fiems, D., Prabhu, B., De Turck, K.: Analytic approximations of queues with lightly- and heavily-correlated autoregressive service times. Annals of Operations ResearchGoogle Scholar