Skip to main content
SpringerLink
Log in

Ergodic Type Bellman Equations of First Order with Quadratic Hamiltonian

  • Published:
Applied Mathematics and Optimization Submit manuscript

Abstract

We consider Bellman equations of ergodic type in first order. The Hamiltonian is quadratic on the first derivative of the solution. We study the structure of viscosity solutions and show that there exists a critical value among the solutions. It is proved that the critical value has the representation by the long time average of the kernel of the max-plus Schrödinger type semigroup. We also characterize the critical value in terms of an invariant density in max-plus sense, which can be understood as a counterpart of the characterization of the principal eigenvalue of the Schrödinger operator by an invariant measure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Akian, M., Quadrat, J.-P., Viot, M.: Bellman processes. In: Cohen, G., Quadrat, J.-P. (eds.) Lecture Notes in Control and Info. Sci., vol. 199 (1994)

  2. Baccelli, F., Cohen, G., Olsder, G.J., Quadrat, J.-P.: Synchronization and Linearity: An Algebra for Discrete Event Systems. Wiley, New York (1992)

    MATH  Google Scholar 

  3. Bardi, M., Cappuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997)

    MATH  Google Scholar 

  4. Başar, T., Bernhard, P.: H ∞-Optimal Control and Related Minimax Design Problems, 2nd edn. Birkhäuser, Boston (1995)

    Google Scholar 

  5. Bensoussan, A.: Perturbation Methods in Optimal Control. Wiley, New York (1988)

    MATH  Google Scholar 

  6. Del Moral, P., Doisy, M.: Maslov idempotent probability calculus I. Theory Probab. Appl. 43(4), 562–576 (1999)

    Article  MathSciNet  Google Scholar 

  7. Deuschel, J.-D., Stroock, D.W.: Large Deviations. Academic, New York (1989)

    MATH  Google Scholar 

  8. Donsker, M.D., Varadhan, S.R.S.: On the principal eigenvalue of second-order elliptic differential operators. Commun. Pure Appl. Math. 29, 595–621 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  9. Fathi, A., Siconolfi, A.: PDE aspects of Aubry-Mather theory for quasiconvex Hamiltonians. Calc. Var. 22, 185–228 (2005)

    Article  MathSciNet  Google Scholar 

  10. Fleming, W.H.: Max-plus stochastic processes. Appl. Math. Optim. 49, 159–181 (2004)

    MATH  MathSciNet  Google Scholar 

  11. Fleming, W.H., James, M.R.: The risk-sensitive index and the H 2 and H ∞ norms for nonlinear systems. Math. Control Signals Syst. 8, 199–221 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  12. Fleming, W.H., McEneaney, W.M.: Risk-sensitive control on an infinite horizon. SIAM J. Control Optim. 33, 1881–1915 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  13. Fleming, W.H., Sheu, S.-J.: Asymptotics for the principal eigenvalue and eigenfunction of a nearly first-order operator with large potential. Ann. Probab. 25(4), 1953–1994 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  14. Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems, 2nd edn. Springer, New York (1998)

    MATH  Google Scholar 

  15. Helton, J.W., James, M.R.: Extending H ∞ Control to Nonlinear Systems. SIAM, Philadelphia (1999)

    MATH  Google Scholar 

  16. Hislop, P.D., Sigal, I.M.: Introduction to Spectral Theory: With Applications to Schrödinger Operators. Springer, New York (1996)

    MATH  Google Scholar 

  17. Ishii, H.: Comparison results for Hamilton-Jacobi equations without growth condition on solutions from above. Appl. Anal. 67, 357–372 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  18. Ishii, H.: Lecture Notes on the Weak KAM Theorem. Hokkaido Univ. (2004)

  19. Ishii, H.: Asymptotic solutions for large time of Hamilton-Jacobi equations in Euclidean n space. Ann. Inst. H. Poincaré Anal. Non Linéaire 25, 231–266 (2008)

    Article  MATH  Google Scholar 

  20. Ishii, H., Nagai, H., Teramoto, F.: A singular limit on risk sensitive control and semi-classical analysis. In: Probab. Theory and Math. Stat., Proc. of the 7th Japan-Russia Symp., pp. 164–173 (1996)

  21. Kaise, H., Nagai, H.: Bellman-Isaacs equations of ergodic type related to risk-sensitive control and their singular limits. Asymptot. Anal. 16, 347–362 (1998)

    MATH  MathSciNet  Google Scholar 

  22. Kaise, H., Nagai, H.: Ergodic type Bellman equations of risk-sensitive control with large parameters and their singular limits. Asymptot. Anal. 20, 279–299 (1999)

    MATH  MathSciNet  Google Scholar 

  23. Kaise, H., Sheu, S.-J.: Risk sensitive optimal investment: solutions for the dynamical programming equation. AMS Contemp. Math. 351, 217–230 (2004)

    MathSciNet  Google Scholar 

  24. Kaise, H., Sheu, S.-J.: On the structure of solutions of ergodic type Bellman equations related to risk-sensitive control. Ann. Probab. 34, 284–320 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  25. Kaise, H., Sheu, S.-J.: Evaluation of large time expectations for diffusion processes. Preprint

  26. Lions, P.-L.: Generalized Solutions of Hamilton-Jacobi Equations. Research Notes in Mathematics, vol. 69. Pitman, Boston (1982)

    MATH  Google Scholar 

  27. Lions, P.-L., Souganidis, P.E.: Differential games, optimal control and directional derivatives of viscosity solutions of Bellman’s and Isaacs’ equations. SIAM J. Control Optim. 23(4), 566–582 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  28. Lions, P.-L., Papanicolaou, G., Varadhan, S.R.S.: Homogenization of Hamilton-Jacobi equations. Unpublished preprint (1987)

  29. Maslov, V.P., Samborskiĭ, S.N. (eds.): Idempotent Analysis. AMS, Providence (1992)

    MATH  Google Scholar 

  30. McEneaney, W.M.: Max-plus eigenvector representations for solution of nonlinear H ∞ problems: basic concept. IEEE Trans. Autom. Control 48(7), 1150–1163 (2003)

    Article  MathSciNet  Google Scholar 

  31. Pinsky, R.G.: Positive Harmonic Functions and Diffusion. Cambridge University Press, Cambridge (1995)

    MATH  Google Scholar 

  32. Simon, B.: Semiclassical analysis of low lying eigenvalues, I. Non-degenerate minima: asymptotic expansion. Ann. Inst. H. Poincaré 40(3), 295–307 (1983)

    Google Scholar 

  33. Simon, B.: Semiclassical analysis of low lying eigenvalues, II. Tunneling. Ann. Math. 120, 89–118 (1984)

    Article  Google Scholar 

  34. Stroock, D.W.: An Introduction to Theory of Large Deviations. Springer, New York (1984)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan

    Hidehiro Kaise

  2. Institute of Mathematics, Academia Sinica, Nankang, Taipei, 11529, Taiwan, ROC

    Shuenn-Jyi Sheu

Authors
  1. Hidehiro Kaise
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuenn-Jyi Sheu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hidehiro Kaise.

Additional information

H. Kaise supported by Grant-in-Aid for Young Scientists, No.17740052, JSPS.

S.-J. Sheu supported by National Science Council of Taiwan, NSC96-2119-M-001-002.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kaise, H., Sheu, SJ. Ergodic Type Bellman Equations of First Order with Quadratic Hamiltonian. Appl Math Optim 59, 37–73 (2009). https://doi.org/10.1007/s00245-008-9043-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-008-9043-z

Keywords

Search

Navigation