Abstract
The self-tuning approach to adaptive control is applied to a class of Markov chains called nearest-neighbor motions. These have a countable state space and move from any state to at most finitely many neighboring states. For compact parameter and control spaces, the almost-sure optimality of the self-tuner for an ergodic cost criterion is established under two sets of assumptions.
Similar content being viewed by others
References
V. E. Beneš, Existence of optimal strategies based on specified information for a class of stochastic decision problems,SIAM J. Control Optim.,8 (1970), 179–188.
V. S. Borkar, Controlled Markov chains and stochastic networks,SIAM J. Control Optim.,21 (1983), 652–666.
V. S. Borkar, On minimum cost per unit time control of Markov chains,SIAM J. Control Optim.,22 (1984), 965–978.
V. S. Borkar, Control of Markov chains with long-run average cost criterion, inProceedings of the I.M.A. Workshop on Stochastic Differential Systems with Application to Electrical/Computer Engineering, Control Theory and Operations Research (W. Fleming and P. L. Lions, eds.), pp. 57–77, Springer-Verlag, New York, 1987.
V. S. Borkar, A convex analytic approach to Markov decision processes,Probab. Theory Related Fields,78 (1988), 583–602.
V. S. Borkar, Control of Markov chains with long-run average cost criterion: the dynamic programming equations,SIAM J. Control Optim.,27 (1989), 642–657.
V. S. Borkar, The Kumar-Becker-Lin scheme revisited,J. Optim. Theory Appl. (to appear in August 1990).
V. S. Borkar, and P. Varaiya, Adaptive control of Markov chain, I: Finite parameter set,IEEE Trans. Automat. Control,24 (1979), 953–957.
V. S. Borkar, and P. Varaiya, Identification and adaptive control of Markov chains,SIAM J. Control Optim.,20 (1982), 470–489.
B. Hajek, Hitting-time and occupation-time bounds implied by drift analysis with applications,Adv. in Appl. Probab.,14 (1982), 502–525.
O. Hernandez-Lerma and S. I. Marcus, Adaptive control of service in queueing systems,Systems Control Lett.,3 (1983), 283–289.
O. Hernandez-Lerma and S. I. Marcus, Optimal adaptive control of priority assignment in queueing systems,Systems Control Lett.,4, (1984), 65–72.
P. R. Kumar, Survey of results in stochastic adaptive control,SIAM J. Control Optim.,23 (1985), 329–380.
P. R. Kumar and W. Lin, Optimal adaptive controllers for unknown Markov chains,IEEE Trans. Automat. Control,27 (1982), 765–774.
P. Mandl, Estimation and control in Markov chains,Adv. in Appl. Probab.,6 (1974), 40–60.
B. Sagalovsky, Adaptive control and parameter estimation in Markov chains: a linear case,IEEE Trans. Automat. Control,27 (1982), 137–146.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Borkar, V.S., Ghosh, M.K. Ergodic and adaptive control of nearest-neighbor motions. Math. Control Signal Systems 4, 81–98 (1991). https://doi.org/10.1007/BF02551382
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02551382