Skip to main content
SpringerLink
Log in

Max-Plus Stochastic Control and Risk-Sensitivity

  • Published:
Applied Mathematics and Optimization Submit manuscript

Abstract

In the Maslov idempotent probability calculus, expectations of random variables are defined so as to be linear with respect to max-plus addition and scalar multiplication. This paper considers control problems in which the objective is to minimize the max-plus expectation of some max-plus additive running cost. Such problems arise naturally as limits of some types of risk sensitive stochastic control problems. The value function is a viscosity solution to a quasivariational inequality (QVI) of dynamic programming. Equivalence of this QVI to a nonlinear parabolic PDE with discontinuous Hamiltonian is used to prove a comparison theorem for viscosity sub- and super-solutions. An example from mathematical finance is given, and an application in nonlinear H-infinity control is sketched.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Akian, M., Gaubert, S., Walsh, C.: Discrete max-plus spectral theory. In: Litvinov, G.L., Maslov, V.P. (eds.) Idempotent Mathematics and Mathematical Physics. Contemporary Math., vol. 377, pp. 53–78. American Mathematical Society, Providence (2005)

    Google Scholar 

  2. Akian, M., Quadrat, J.-P., Viot, M.: Bellman processes. In: Cohen, G., Quadrat, J.-P. (eds.) Lecture Notes in Control and Inform. Sci., vol. 199 (1994)

  3. Baccelli, F., Cohen, G., Olsder, G.J., Quadrat, J.-P.: Synchronization and Linearity: An Algebra for Discrete Event Systems. Wiley, New York (1992)

    MATH  Google Scholar 

  4. Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997)

    Book  MATH  Google Scholar 

  5. Barles, G., Perthame, B.: Discontinuous solutions of deterministic optimal stopping time problems. Math. Model. Numer. Anal. 21, 557–579 (1987)

    MATH  MathSciNet  Google Scholar 

  6. Barron, E.N., Ishii, H.: The Bellman equation for minimizing the maximum cost. Nonlinear Anal. Theory Methods Appl. 9, 1067–1090 (1989)

    Article  MathSciNet  Google Scholar 

  7. Billingsley, P.: Convergence of Probability Measures. Wiley, New York (1968)

    MATH  Google Scholar 

  8. Del Moral, P., Doisy, M.: Maslov idempotent probability calculus I. Theory Probab. Appl. 43(4), 562–576 (1999)

    Article  MathSciNet  Google Scholar 

  9. Dower, P.M., McEneaney, W.M.: A max-plus affine power method for approximation of a class of mixed L ∞/L 2 value functions. In: Proc. 42nd IEEE Conf. on Decision and Control, Maui, pp. 2573–2578 (2003)

  10. Elliott, R.J., Kalton, N.J.: The existence of value in differential games. Mem. Am. Math. Soc. 126 (1972)

  11. Evans, L.C., Souganidis, P.E.: Differential games and representation formulas of Hamilton-Jacobi-Isaacs equations. Indiana Univ. Math. J. 33(5), 773–797 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  12. Fleming, W.H.: Max-plus stochastic control. In: Pasik-Duncan, B. (ed.) Stochastic Theory and Control. Lecture Notes in Control and Inform. Sci., vol. 280, pp. 111–119. Springer, New York (2002)

    Chapter  Google Scholar 

  13. Fleming, W.H.: Max-plus stochastic processes. Appl. Math. Optim. 49, 159–181 (2004)

    MATH  MathSciNet  Google Scholar 

  14. Fleming, W.H., Rishel, R.W.: Deterministic and Stochastic Optimal Control. Springer, New York (1975)

    MATH  Google Scholar 

  15. Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  16. Freidlin, M.I., Wentzell, A.D.: Random Perturbation of Dynamical Systems. Springer, New York (1984)

    Google Scholar 

  17. Helton, J.W., James, M.R.: Extending H ∞ Control to Nonlinear Systems. SIAM, Philadelphia (1999)

    MATH  Google Scholar 

  18. Kaise, H., Sheu, S.-J.: Differential games of inf-sup type and Isaacs equations. Appl. Math. Optim. 52, 1–22 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  19. Maslov, V.P., Samborskiĭ, S.N. (eds.): Idempotent Analysis. American Mathematical Society, Providence (1992)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Applied Mathematics and Lefschetz Center for Dynamical Systems, Brown University, Providence, RI, 02912, USA

    Wendell H. Fleming

  2. Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan

    Hidehiro Kaise

  3. Institute of Mathematics, Academia Sinica, Nankang, Taipei, 11529, Taiwan

    Shuenn-Jyi Sheu

Authors
  1. Wendell H. Fleming
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hidehiro Kaise
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuenn-Jyi Sheu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hidehiro Kaise.

Additional information

H. Kaise’s research is supported by Grant-in-Aid for Young Scientists (B), No. 20740052, MEXT.

S.-J. Sheu’s research is supported by NSC96-2119-M-001-002.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fleming, W.H., Kaise, H. & Sheu, SJ. Max-Plus Stochastic Control and Risk-Sensitivity. Appl Math Optim 62, 81–144 (2010). https://doi.org/10.1007/s00245-010-9097-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-010-9097-6

Keywords

Search

Navigation