Skip to main content
SpringerLink
Log in

A Framework for the Dynamic Programming Principle and Martingale-Generated Control Correspondences

  • Published:
Applied Mathematics & Optimization Aims and scope Submit manuscript

Abstract

We construct an abstract framework in which the dynamic programming principle (DPP) can be readily proven. It encompasses a broad range of common stochastic control problems in the weak formulation, and deals with problems in the “martingale formulation” with particular ease. We give two illustrations; first, we establish the DPP for general controlled diffusions and show that their value functions are viscosity solutions of the associated Hamilton–Jacobi–Bellman equations under minimal conditions. After that, we show how to treat singular control on the example of the classical monotone-follower problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bather, J., Chernoff, H.: Sequential decisions in the control of a spaceship. In: Proc. Fifth Berkeley Sympos. Mathematical Statistics and Probability (Berkeley, Calif., 1965/66), vol. III, Physical Sciences, pp. 181–207. Univ. California Press, Berkeley (1967)

  2. Bayraktar, E., Sîrbu, M.: Stochastic Perron’s method and verification without smoothness using viscosity comparison: the linear case. Proc. Am. Math. Soc. 140(10), 3645–3654 (2012)

    Article  MathSciNet  Google Scholar 

  3. Bayraktar, E., Sîrbu, M.: Stochastic Perron’s method for Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim. 51(6), 4274–4294 (2013)

    Article  MathSciNet  Google Scholar 

  4. Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control, vol. 139 of Mathematics in Science and Engineering. Academic Press Inc. [Harcourt Brace Jovanovich Publishers], New York (1978) (The discrete time case)

  5. Borkar, V.S.: Optimal Control of Diffusion Processes, vol. 203 of Pitman Research Notes in Mathematics Series. Longman Scientific & Technical, Harlow (1989)

  6. Bouchard, B., Nutz, M.: Weak dynamic programming for generalized state constraints. SIAM J. Control Optim. 50(6), 3344–3373 (2012)

    Article  MathSciNet  Google Scholar 

  7. Bouchard, B., Touzi, N.: Weak dynamic programming principle for viscosity solutions. SIAM J. Control Optim. 49(3), 948–962 (2011)

    Article  MathSciNet  Google Scholar 

  8. Bouchard, B., Vu, T.N.: The obstacle version of the geometric dynamic programming principle: application to the pricing of American options under constraints. Appl. Math. Optim. 61(2), 235–265 (2010)

    Article  MathSciNet  Google Scholar 

  9. Daley, D.J., Vere-Jones, D.: An introduction to the Theory of Point Processes, vol. I of Probability and Its Applications (New York), 2nd edn. Springer, New York (2003) (Elementary theory and methods)

  10. El Karoui, N.: Les aspects probabilistes du contrôle stochastique. In: Ninth Saint Flour Probability Summer School—1979 (Saint Flour, 1979), vol. 876 of Lecture Notes in Math., pp. 73–238. Springer, Berlin (1981)

  11. El Karoui, N., Tan, X.: Capacities, Measurable Selection and Dynamic Programming Part I: Abstract Framework (2013). arXiv:1310.3363

  12. El Karoui, N., Tan, X.: Capacities, Measurable Selection and Dynamic Programming Part II: Application in Stochastic Control Problems (2013). arXiv:1310.3364

  13. Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions, vol. 25 of Applications of Mathematics (New York). Springer, New York (1993). ISBN 0-387-97927-1

  14. Karatzas, I., Shreve, S.E.: Connections between optimal stopping and singular stochastic control. I. Monotone follower problems. SIAM J. Control Optim. 22(6), 856–877 (1984)

    Article  MathSciNet  Google Scholar 

  15. Karatzas, I., Shreve, S.E.: Brownian Motion and Stochastic Calculus, vol. 113 of Graduate Texts in Mathematics, 2nd edn. Springer, New York (1991)

  16. Nutz, M., van Handel, R.: Constructing sublinear expectations on path space. Stoch. Process. Appl. 123(8), 3100–3121 (2013)

    Article  MathSciNet  Google Scholar 

  17. Soner, H.M., Touzi, N.: Dynamic programming for stochastic target problems and geometric flows. J. Eur. Math. Soc. (JEMS) 4(3), 201–236 (2002)

    Article  MathSciNet  Google Scholar 

  18. Soner, H.M., Touzi, N.: Stochastic target problems, dynamic programming, and viscosity solutions. SIAM J. Control Optim. 41(2), 404–424 (2002)

    Article  MathSciNet  Google Scholar 

  19. Srivastava, S.M.: A Course on Borel Sets, vol. 180. Springer, New York (1998)

    Book  Google Scholar 

  20. Stricker, C.: Quasimartingales, martingales locales, semimartingales et filtration naturelle. Probab. Theory Relat. Fields 39, 55–63 (1977)

    MathSciNet  MATH  Google Scholar 

  21. Žitković, G.: Dynamic programming for controlled markov families: abstractly and over martingale measures. SIAM J. Control Optim. 52(3), 1597–1621 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Mihai Sîrbu and Kasper Larsen for valuable conversations and acknowledge the support by the National Science Foundation under Grants DMS-0706947 (2007–2010), DMS-09556194 (2010–2015), DMS-1107465 (2012–2017) and DMS-1516165 (2015–2018). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).

Author information

Authors and Affiliations

  1. Department of Mathematics, The University of Texas at Austin, Austin, TX, USA

    Roman Fayvisovich & Gordan Žitković

Authors
  1. Roman Fayvisovich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gordan Žitković
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gordan Žitković.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fayvisovich, R., Žitković, G. A Framework for the Dynamic Programming Principle and Martingale-Generated Control Correspondences. Appl Math Optim 83, 1311–1352 (2021). https://doi.org/10.1007/s00245-019-09589-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-019-09589-8

Keywords

Mathematics Subject Classification

Search

Navigation