Skip to main content
SpringerLink
Log in
Book cover

Stochastic Differential Systems, Stochastic Control Theory and Applications pp 57–77Cite as

Control of Markov Chains with Long-Run Average Cost Criterion

  • Conference paper

Part of the book series: The IMA Volumes in Mathematics and Its Applications ((IMA,volume 10))

Abstract

The long-run average cost control problem for discrete time Markov chains is studied in an extremely general framework. Existence of stable stationary strategies which are optimal in the appropriate sense is established and these are characterized via the dynamic programming equations. The approach here differs from the conventional approach via the discounted cost problem and covers situations not covered by the latter.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. R.N. Bhattacharya, and M. Majumdar, preprint.

    Google Scholar 

  2. P. Billingsley, Convergence of Probability Measures, John Wiley, New York, 1968.

    MATH  Google Scholar 

  3. V. Borkar, Controlled Markov Chains and Stochastic Networks, SIAM J. Control and Opt., 21 (1983), pp. 652–666.

    Article  MathSciNet  Google Scholar 

  4. V. Borkar, On Minimum Cost Per Unit Time Control of Markov Chains, SIAM J. Control and Opt., 22 (1984), pp. 965–978.

    Article  MathSciNet  MATH  Google Scholar 

  5. V. Borkar and P. Varaiya, Identification and Adaptive Control of Markov Chains, SIAM J. Control and Opt., 20 (1982), pp. 470–489.

    Article  MathSciNet  MATH  Google Scholar 

  6. H. Deppe, Continuity of Mean Recurrence Times in Denumerable Semi-Markov Processes, Zeit. Wahr, 69 (1985), pp. 581–592.

    Article  MathSciNet  MATH  Google Scholar 

  7. M. Doxsxer, and S.R.S Varadhan, Asymptotic Evaluation of Certain Markov Process Expectations for Large Time III, Comm. in Pure and Appl. Math XXIX, 4 (1976), pp. 389–461.

    Google Scholar 

  8. A Federgruen, A. Horduk and H.C. Toms, A Note on Simultaneous Recurrence Conditions on a Set of Denumerable Stochastic Matrices, J. Appl. Prob., 15 (1978), pp. 842–847.

    Article  MATH  Google Scholar 

  9. B. Hajek, Hitting-Time and Occupation-Time Bounds Implied by Drift Analysis with Applications, Adv. Appl. Prob., 14 (1982), pp. 502–525.

    Article  MathSciNet  MATH  Google Scholar 

  10. H.J. Kushner, Introduction to Stochastic Control, Holt, Rinehart and Winston Inc., New York, 1971.

    MATH  Google Scholar 

  11. M. Losve, Probability Theory II, 4th Ed., Springer-Verlag, New York, 1978.

    Google Scholar 

  12. Z. Kosberg, P. Varaiya and J. Walrand, Optimal Control of Service in Tandem Queues, IEEE Trans. on Automatic Control, AC-27 (1982), pp. 600–609.

    Google Scholar 

  13. S. Ross, Applied Probability Models with Optimization Applications, Holden Day, San Francisco (1970).

    MATH  Google Scholar 

  14. J. Wessels and J. Van De Wal, Markov Decision Processes, Statistics Neerlandica, 39 (1985), pp. 219–233.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tata Institute of Fundamental Research, Bangalore Centre, P.O. Box 1234, I.I.Sc. Campus, Bangalore, 560012, India

    Vivek S. Borkar

  2. Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA

    Vivek S. Borkar

Authors
  1. Vivek S. Borkar
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Division of Applied Mathematics, Brown University, 02912, Providence, Rhode Island, USA

    Wendell Fleming

  2. Ceremade, Universite Paris-Dauphine, Place de Lattre de Tassigny, 75775, Paris Cedex 16, France

    Pierre-Louis Lions

Rights and permissions

Reprints and permissions

Copyright information

© 1988 Springer-Verlag New York Inc.

About this paper

Cite this paper

Borkar, V.S. (1988). Control of Markov Chains with Long-Run Average Cost Criterion. In: Fleming, W., Lions, PL. (eds) Stochastic Differential Systems, Stochastic Control Theory and Applications. The IMA Volumes in Mathematics and Its Applications, vol 10. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-8762-6_4

Download citation

  • DOI: https://doi.org/10.1007/978-1-4613-8762-6_4

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4613-8764-0

  • Online ISBN: 978-1-4613-8762-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation