Skip to main content
SpringerLink
Log in

Numerical solution of a long-term average control problem for singular stochastic processes

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

This paper analyzes numerically a long-term average stochastic control problem involving a controlled diffusion on a bounded region. The solution technique takes advantage of an infinite-dimensional linear programming formulation for the problem which relates the stationary measures to the generators of the diffusion. The restriction of the diffusion to an interval is accomplished through reflection at one end point and a jump operator acting singularly in time at the other end point. Different approximations of the linear program are obtained using finite differences for the differential operators (a Markov chain approximation to the diffusion) and using a finite element method to approximate the stationary density. The numerical results are compared with each other and with dynamic programming.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Brenner SC, Scott LR (1994) The mathematical theory of finite element methods. Springer, New York

    MATH  Google Scholar 

  • Cho MJ, Stockbridge RH (2002) Linear programming formulation for optimal stopping problems. SIAM J Control Opt 40:1965–1982

    Article  MATH  MathSciNet  Google Scholar 

  • Helmes K, Stockbridge RH (2000) Numerical comparison of controls and verification of optimality for stochastic control problems. J Opt Theor Appl 106:107–127

    Article  MATH  MathSciNet  Google Scholar 

  • Kurtz TG, Stockbridge RH (2001) Stationary solutions and forward equations for controlled and singular martingale problems. Electron J Probab 6(14):1–52

    MathSciNet  Google Scholar 

  • Kurtz TG, Stockbridge RH (1998) Existence of Markov controls and characterization of optimal Markov controls. SIAM J Control Optim 36:609–653

    Article  MATH  MathSciNet  Google Scholar 

  • Kushner HJ, Dupuis P (2001) Numerical methods for stochastic control problems in continuous time, 2nd ed. Springer, New York

    MATH  Google Scholar 

  • Mendiondo MS, Stockbridge RH (2002) Long term average cost with cost based on the local time of a diffusion. In: Filar JA, Chen A, Hou Z (eds) Markov processes and controlled markov chains. Kluwer, Dordrecht, pp 425–441

    Google Scholar 

  • Øksendal B (2003) An introduction to stochastic differential equations, 6th edn. Springer, New York

    Google Scholar 

  • Shen X, Chen H, Dai JG, Dai W (2002) The finite element method for computing the stationary distribution of an SRBM in a hypercube with applications to finite buffer queueing networks. Queueing Syst 42:33–62

    Article  MATH  MathSciNet  Google Scholar 

  • Shreve SE (1988) An introduction to singular stochastic control. Stochastic differential systems, stochastic control theory and applications (Minneapolis, Minn, 1986), IMA Vol Math Appl, 10. Springer, New York, pp 513–528

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, University of Wisconsin-Milwaukee, Milwaukee, USA

    P. Kaczmarek, G. A. Rus, R. H. Stockbridge & B. A. Wade

  2. Queens’ College, University of Cambridge, Cambridge, UK

    S. T. Kent

Authors
  1. P. Kaczmarek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. T. Kent
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. G. A. Rus
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. H. Stockbridge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. B. A. Wade
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. H. Stockbridge.

Additional information

This research has been supported in part by the U.S. National Security Agency under Grant Agreement Number H98230-05-1-0062. The United States Government is authorized to reproduce and distribute reprints notwithstanding any copyright notation herein.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kaczmarek, P., Kent, S.T., Rus, G.A. et al. Numerical solution of a long-term average control problem for singular stochastic processes. Math Meth Oper Res 66, 451–473 (2007). https://doi.org/10.1007/s00186-007-0166-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-007-0166-9

Keywords

AMS 2000 Classifications

Search

Navigation