Skip to main content
SpringerLink
Log in

On essential information in sequential decision processes

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

This paper provides sufficient conditions when certain information about the past of a stochastic decision processes can be ignored by a controller. We illustrate the results with particular applications to queueing control, control of semi-Markov decision processes with iid sojourn times, and uniformization of continuous-time Markov decision processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Balachandran KR (1973) Control policies for a single server system. Manage Sci 19:1013–1018

    MathSciNet  MATH  Google Scholar 

  • Bertesekas DP (2001) Dynamic programming and optimal control, 2nd edin. Scientific, Belmont MA

    Google Scholar 

  • Bertsekas DP, Shreve SE (1978) Stochastic optimal control: The discrete-time case. Academic, New York (republished by Athena Scientific, Belmont, MA 1997)

    MATH  Google Scholar 

  • Cassandras CG (1993) Discrete event systems: modeling and performance analysis. IRWIN, Boston

    Google Scholar 

  • Dynkin EB, Yushkevich AA (1979) Controlled Markov processes. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Fan-Orzechowski X, Feinberg EA (2004) Optimality of Randomized trunk reservation for a problem with a single constraint. Department of Applied Mathematics and Statistics, SUNY at Stony Brook, http://www.ams.sunysb.edu/∼feinberg/public/FanFeinberg1.pdf

  • Feinberg EA (2004) Continuous-time discounted jump-Markov decision processes: a discrete-event approach. Math Oper Res 29:492–524

    Article  MATH  MathSciNet  Google Scholar 

  • Feinberg EA, Kella O (2002) Optimality of D-policies for an M/G/1 queue. Queueing Syst 42:355–376

    Article  MATH  MathSciNet  Google Scholar 

  • Feinberg EA, Reiman MI (1994) Optimality of randomized trunk reservation. Prob Eng Informa Sci 8:463–489

    Article  Google Scholar 

  • Feinberg EA, Shwartz A. ed. (2002) Handbook of Markov decision processes. Kluwer, Boston

    MATH  Google Scholar 

  • Fristedt B, Gray L (1997) A modern approach to probability theory. Birkhäuser, Boston

    MATH  Google Scholar 

  • Gut A (1988) Stopped random walks. Limit theorems and applications. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  • Hinderer K (1970) Foundations of non-stationary dynamic programming with discrete time paramerter. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Hordijk A (1974) Dynamic programming and Markov potential theory. Mathematical centre tracts 51, Amsterdam

  • Kitayev MY (1985) Semi-Markov and jump Markov controlled models: average cost criterion. SIAM Theory Probab. Appl. 30:272-288

    Article  Google Scholar 

  • Kitayev MY Rykov VV (1995) Controlled queueing systems. CRC, New York

    Google Scholar 

  • Koole G (2005) Routing to parallel homogeneous queues. Math Methods Oper Res, (this issue)

  • Lippman SA (1975) Applying a new device in the optimization of exponential queueing systems. Oper Res 23:687-710

    Article  MATH  MathSciNet  Google Scholar 

  • Miller BL (1969) A queueing reward system with several customer classes. Manage Sci 16:235-245

    Google Scholar 

  • Puterman ML (1994) Markov decision processes. Wiley, New York

    MATH  Google Scholar 

  • Sennott LI (1999) Stochastic dynamic programming and the control of queueing systems. Wiley, New York

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Applied Mathematics and Statistics, State University of New York, Stony Brook, NY, 11794-3600, USA

    Eugene A. Feinberg

Authors
  1. Eugene A. Feinberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eugene A. Feinberg.

Additional information

Mathematics Subject Classification (2000): Primary 60K25, Secondary 90C40

Rights and permissions

Reprints and permissions

About this article

Cite this article

Feinberg, E.A. On essential information in sequential decision processes. Math Meth Oper Res 62, 399–410 (2005). https://doi.org/10.1007/s00186-005-0035-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-005-0035-3

Keywords

Search

Navigation