Annals of Operations Research

, Volume 28, Issue 1, pp 67–79 | Cite as

Two extensions of asymptotic methods in controlled Markov chains

  • Petr Mandl
  • Monika Laušmanová


The application of the Skorokhod representation of martingales and of the local asymptotic normality to derive limit inequalities for the cost in controlled finite state Markov chains is reviewed. The inequalities are usable in self-optimizing control. The methods are taken from the references listed but, with the exception of proposition 4, the results are formulated for Markov chains for the first time.


Markov Chain Asymptotic Normality Asymptotic Method State Markov Chain Limit Inequality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    D. Florens-Zmirou and P. Mandl, Théorèmes asymptotiques d'une diffusion récurrente contrÔlée, Prépublications 88-30, Université de Paris-Sud, Mathématique (1988).Google Scholar
  2. [2]
    D. Freedman,Brownian Motion and Diffusion (Springer, New York, 1983).Google Scholar
  3. [3]
    I.A. Ibragimov and R.Z. Has' minskii,Statistical Estimation. Asymptotic Theory (Springer, New York, 1981).Google Scholar
  4. [4]
    V. Lánská, A note on estimation in controlled diffusion processes, Kybernetika 22 (1986) 133–141.Google Scholar
  5. [5]
    M. Laušmanová, On asymptotic inequalities in discrete time controlled linear systems, in:Proc. 4th Prague Symp. on Asymptotic Statistics, Charles University, Prague (1989) pp. 377–388.Google Scholar
  6. [6]
    M. Laušmanová, A vanishing discount limit theorem for controlled Markov chains, Kybernetika 25 (1989) 366–374.Google Scholar
  7. [7]
    P. Mandl, A connection between controlled Markov chains and martingales, Kybernetika 9 (1973) 237–241.Google Scholar
  8. [8]
    P. Mandl, Local asymptotic normality in controlled Markov chains, Statist. Decision, Suppl. 2 (1985) 123–127.Google Scholar
  9. [9]
    P. Mandl, Limit theorems of probability theory and optimality in linear controlled systems with quadratic cost, in:Lecture Notes in Control and Information Sciences 96 (Springer, Berlin, 1987) pp. 316–329.Google Scholar
  10. [10]
    P. Mandl, On the arcsine law in the adaptive control of Markov chains, in:Proc. 1st World Congress Bernoulli Society, vol. 1 (VNU Science Press, Utrecht, 1987) pp. 331–335.Google Scholar
  11. [11]
    P. Mandl and M.R. Romera Ayllón, On controlled Markov processes with average cost criterion, Kybernetika 23 (1987) 433–442.Google Scholar

Copyright information

© J.C. Baltzer A.G. Scientific Publishing Company 1991

Authors and Affiliations

  • Petr Mandl
    • 1
  • Monika Laušmanová
    • 1
  1. 1.Faculty of Mathematics and PhysicsCharles UniversityPragueCzechoslovakia

Personalised recommendations