Two extensions of asymptotic methods in controlled Markov chains
- 44 Downloads
The application of the Skorokhod representation of martingales and of the local asymptotic normality to derive limit inequalities for the cost in controlled finite state Markov chains is reviewed. The inequalities are usable in self-optimizing control. The methods are taken from the references listed but, with the exception of proposition 4, the results are formulated for Markov chains for the first time.
KeywordsMarkov Chain Asymptotic Normality Asymptotic Method State Markov Chain Limit Inequality
Unable to display preview. Download preview PDF.
- D. Florens-Zmirou and P. Mandl, Théorèmes asymptotiques d'une diffusion récurrente contrÔlée, Prépublications 88-30, Université de Paris-Sud, Mathématique (1988).Google Scholar
- D. Freedman,Brownian Motion and Diffusion (Springer, New York, 1983).Google Scholar
- I.A. Ibragimov and R.Z. Has' minskii,Statistical Estimation. Asymptotic Theory (Springer, New York, 1981).Google Scholar
- V. Lánská, A note on estimation in controlled diffusion processes, Kybernetika 22 (1986) 133–141.Google Scholar
- M. Laušmanová, On asymptotic inequalities in discrete time controlled linear systems, in:Proc. 4th Prague Symp. on Asymptotic Statistics, Charles University, Prague (1989) pp. 377–388.Google Scholar
- M. Laušmanová, A vanishing discount limit theorem for controlled Markov chains, Kybernetika 25 (1989) 366–374.Google Scholar
- P. Mandl, A connection between controlled Markov chains and martingales, Kybernetika 9 (1973) 237–241.Google Scholar
- P. Mandl, Local asymptotic normality in controlled Markov chains, Statist. Decision, Suppl. 2 (1985) 123–127.Google Scholar
- P. Mandl, Limit theorems of probability theory and optimality in linear controlled systems with quadratic cost, in:Lecture Notes in Control and Information Sciences 96 (Springer, Berlin, 1987) pp. 316–329.Google Scholar
- P. Mandl, On the arcsine law in the adaptive control of Markov chains, in:Proc. 1st World Congress Bernoulli Society, vol. 1 (VNU Science Press, Utrecht, 1987) pp. 331–335.Google Scholar
- P. Mandl and M.R. Romera Ayllón, On controlled Markov processes with average cost criterion, Kybernetika 23 (1987) 433–442.Google Scholar