Skip to main content
SpringerLink
Log in

Controlled Markov chains with constraints

  • Identification And Control Of Stochastic Systems
  • Published:
Sadhana Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

We consider the ergodic control of a Markov chain on a countable state space with a compact action space in presence of finitely many (say,m) ergodic constraints. Under a condition on the cost functions that penalizes instability, the existence of an optimal stable stationary strategy randomized at a maximum ofm states is established using convex analytic arguments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Altman E, Shwartz A 1990 Sensitivity of constrained Markov decision processes, EE Pub. No. 741, Dept. of Electrical Eng., Technion, Haifa, Israel

    Google Scholar 

  • Beutler F J, Ross K W 1985 Optimal policies for controlled Markov chains with a constraint,J. Math. Anal. Appl. 112: 236–252

    Article  MATH  MathSciNet  Google Scholar 

  • Billingsley P 1968Convergence of probability measures (New York: Wiley)

    MATH  Google Scholar 

  • Borkar V S 1989 Control of Markov chains with long-run average cost criterion: the dynamic programmin equations.SIAM J. Control Optim. 27: 642–657

    Article  MATH  MathSciNet  Google Scholar 

  • Borkar V S 1991 Topics in controlled Markov chains,Pitman research notes in mathematics (Harlow: Longman) Chap. 7

    Google Scholar 

  • Dubins L 1962 On extreme points of convex sets,J. Math. Anal. Appl. 5: 237–244

    Article  MATH  MathSciNet  Google Scholar 

  • Hordijk A, Kallenberg L C M 1984 Constrained undiscounted stochastic dynamic programming.Math. Oper. Res. 9: 276–289

    MATH  MathSciNet  Google Scholar 

  • Luenberger D 1967Optimization by vector space methods (New York: Wiley)

    Google Scholar 

  • Phelps R 1966Lectures on Choquet’s theorem (New York: Van Nostrand)

    MATH  Google Scholar 

  • Ross K W 1989 Randomized and past-dependent policies for Markov decision processes with multiple constraints.Oper. Res. 37: 474–477

    Article  MATH  MathSciNet  Google Scholar 

  • Schwartz L 1961Disintegration of measures (Bombay: Tata Institute of Fundamental Research)

    Google Scholar 

  • Witsenhausen 1980 Some aspects of convexity useful in information theory.IEEE Trans. Inf. Theory IT-26: 265–271

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Indian Institute of Science, 560012, Bangalore, India

    Vivek S Borkar

Authors
  1. Vivek S Borkar
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Borkar, V.S. Controlled Markov chains with constraints. Sadhana 15, 405–413 (1990). https://doi.org/10.1007/BF02811335

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02811335

Keywords

Search

Navigation