Martingale methods in stochastic control

  • M. H. A. Davis
Part I: Survey Lectures
First Online:
Part of the Lecture Notes in Control and Information Sciences book series (LNCIS, volume 16)

Keywords

Stochastic Differential Equation Stochastic Control Minimum Principle Jump Process Bellman Equation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

9. References

  1. [1]
    V.E. Beneš, Existence of optimal strategies based on specified information, for a class of stochastic decision problems, —SICON 8 (1970) 179–188Google Scholar
  2. [2]
    V.E. Beneš, Existence of optimal stochastic control laws, —SICON 9 (1971) 446–475Google Scholar
  3. [3]
    V.E. Beneš, Full "bang" to reduce predicted miss is optimal, —SICON 15 (1976) 52–83Google Scholar
  4. [4]
    A. Bensoussan and J.L. Lions, Applications des inéquations varationelles en contrôle stochastique, Dunod, Paris, 1978Google Scholar
  5. [5]
    J.M. Bismut, Théorie probabiliste du contrôle des diffusions, Mem. Americ. Math. Soc. 4 (1976), no. 167Google Scholar
  6. [6]
    J.M. Bismut, Duality methods in the control of densities, —SICON 16 (1978) 771–777Google Scholar
  7. [7]
    J.M. Bismut, An introductory approach to duality in optimal stochastic control, SIAM Review 20 (1978) 62–78Google Scholar
  8. [8]
    J.M. Bismut and B. Skalli, Temps d'arrêt optimal, théorie général des processus et processus de Markov, —ZW 39 (1977) 301–313Google Scholar
  9. [9]
    R. Boel and M. Kohlmann, Stochastic control over double martingales, in "Analysis and Optimization of Stochastic Systems" ed. O.L.R. Jacobs, Academic Press, New York/London 1979Google Scholar
  10. [10]
    R. Boel and M. Kohlmann, Semimartingale models of stochastic optimal control with applications to double martingales, preprint, Institut für Angewandte Mathematik der Universität Bonn, 1977Google Scholar
  11. [11]
    R. Boel and P. Varaiya, Optimal control of jump processes, —SICON 15 (1977) 92–119Google Scholar
  12. [12]
    R. Boel, P. Varaiya and E. Wong, Martingales on jump processes I and II, —SICON 13 (1975) 999–1061Google Scholar
  13. [13]
    P. Brémaud and J. Jacod, Processus ponctuels et martingales: resultats recents sur la modelisation et le filtrage, Adv. Appl. Prob. 9 (1977) 362–416Google Scholar
  14. [14]
    P. Brémaud and J.M. Pietri, The role of martingale theory in continuous-time dynamic programming, Tech. report, IRIA, Le Chesnay, France, 1978Google Scholar
  15. [15]
    J.M.C. Clark, The representation of functionals of Brownian motion by stochastic integrals, Ann. Math. Stat. 41 (1970) 1285–1295Google Scholar
  16. [16]
    M.H.A. Davis, On the existence of optimal policies in stochastic control, —SICON 11 (1973) 507–594Google Scholar
  17. [17]
    M.H.A. Davis, The representation of martingales of jump processes, —SICON 14 (1976) 623–638Google Scholar
  18. [18]
    M.H.A. Davis, The separation principle in stochastic control via Girsanov solutions, —SICON 14 (1976) 176–188Google Scholar
  19. [19]
    M.H.A. Davis, Functionals of diffusion processes as stochastic integrals, submitted to Math. Proc. Camb. Phil. Soc.Google Scholar
  20. [20]
    M.H.A. Davis, Nonlinear semigroups in the control of partially-observable stochastic systems, in Measure Theory and Applications to Stochastic Analysis, ed. G. Kallianpur and D. Kölzow, Lecture Notes in Mathematics, Springer-Verlag, to appearGoogle Scholar
  21. [21]
    M.H.A. Davis and J.M.C. Clark, "Predicted Miss" problems in stochatic control, Stochastics 2 (1979)Google Scholar
  22. [22]
    M.H.A. Davis and R.J. Elliott, Optimal control of a jump process, —ZW 40 (1977) 183–202Google Scholar
  23. [23]
    M.H.A. Davis and M. Kohlmann, Stochastic control by measure transformation: a general existence result, preprint, Institut für Angewandte Mathematik der Universität Bonn (1978)Google Scholar
  24. [24]
    M.H.A. Davis and P.P. Varaiya, Information states for linear stochastic systems, J. Math. Anal. Appl. 37 (1972) 387–402Google Scholar
  25. [25]
    M.H.A. Davis and P.P. Varaiya, Dynamic programming conditions for partially-observable stochastic systems, —SICON 11 (1973) 226–261Google Scholar
  26. [26]
    M.H.A. Davis and C.B. Wan, The principle of optimality for Markov jump processes, in "Analysis and Optimization of Stochastic Systems" ed. O.L.R. Jacobs, Academic Press, New York/London, 1979Google Scholar
  27. [27]
    C. Dellacherie, Capacités et processus stochastiques, Springer-Verlag, Berlin, 1972Google Scholar
  28. [28]
    C. Doléans-Dade, Quelques applications de la formule de changement de variables pour les semimartingales, —ZW 16 (1970) 181–194Google Scholar
  29. [29]
    T.E. Duncan, Dynamic programming criteria for stochastic systems in Riemannian manifolds, Applied Math. & Opt. 3 (1977) 191–208Google Scholar
  30. [30]
    T.E. Duncan and P.P. Varaiya, On the solutions of a stochastic control system, —SICON 9 (1971) 354–371Google Scholar
  31. [31]
    T.E. Duncan and P.P. Varaiya, On the solutions of a stochastic control system II, —SICON 13 (1975) 1077–1092Google Scholar
  32. [32]
    R.J. Elliott, The existence of value in stochastic differential games, —SICON 14 (1976) 85–94Google Scholar
  33. [33]
    R.J. Elliott, Double martingales, —ZW 34 (1976) 17–28Google Scholar
  34. [34]
    R.J. Elliott, The optimal control of a stochastic system, —SICON 15 (1977) 756–778Google Scholar
  35. [35]
    R.J. Elliott, The existence of optimal strategies and saddle points in stochastic differential games, in Differential Games and Applications, ed. P. Hagedorn, Lecture Notes in Control and Information Sciences 3, Springer-Verlag, Berlin, 1977Google Scholar
  36. [36]
    R.J. Elliott, Lévy systems and absolutely continuous changes of measure, J. Math. Anal. Appl. 61 (1977) 785–796Google Scholar
  37. [37]
    R.J. Elliott, The optimal control of a semimartingale, 3rd Kingston Conference on Control Theory, Kingston, RI (1978)Google Scholar
  38. [38]
    R.J. Elliott, The martingale calculus and its applications, this volumeGoogle Scholar
  39. [39]
    R.J. Elliott and P.P. Varaiya, A sufficient condition for the optimal control of a partially observed stochastic system, in "Analysis and Optimization of Stochastic Systems," ed. O.L.R. Jacobs, Academic Press, New York/London, 1979Google Scholar
  40. [40]
    W.H. Fleming, Optimal continuous-parameter stochastic control, SIAM Rev. 11 (1969) 470–509Google Scholar
  41. [41]
    W.H. Fleming and R.W. Rishel, Deterministic and stochastic optimal control, Springer-Verlag, New York, 1975Google Scholar
  42. [42]
    M. Fujisaki, G. Kallianpur and H. Kunita, Stochastic differential equations for the nonlinear filtering problem, Osaka J. Math. 9 (1972) 19–40Google Scholar
  43. [43]
    I.V. Girsanov, On transforming a certain class of stochastic processes by absolutely continuous substitution of measures, Theory of Prob. and Appls. 5, (1960) 285–301Google Scholar
  44. [44]
    U.G. Haussmann, On the stochastic maximum principle, —SICON 16 (1978) 236–251Google Scholar
  45. [45]
    U.G. Haussmann, Functionals of Ito processes as stochastic integrals, —SICON 16 (1978) 252–269Google Scholar
  46. [46]
    U.G. Haussmann, On the integral representation of functionals of Ito processes, Stochastic 2 (1979)Google Scholar
  47. [47]
    J. Jacod, Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales, —ZW 31 (1975) 235–253Google Scholar
  48. [48]
    J. Jacod and J. Memin, Caracteristiques locales et conditions de continuité absolue pour les semimartingales, —ZW 35 (1976) 1–37Google Scholar
  49. [49]
    M. Kohlmann, A game with Wiener noise and jump process disturbances, submitted to StochasticsGoogle Scholar
  50. [50]
    M. Kohlmann, On control of jump process, preprint, Institut für Angewandte Mathematik der Universität Bonn, 1978Google Scholar
  51. [51]
    N.V. Krylov, Control of a solution of a stochastic integral equation Theory of Prob. and Appls. 17 (1972) 114–131Google Scholar
  52. [52]
    H. Kunita and S. Watanabe, On square-integrable martingales, Nagoya Math. Journal 30 (1967) 209–245Google Scholar
  53. [53]
    H.J. Kushner, Necessary conditions for continuous-parameter stochastic optimization problems, —SICON 10 (1972) 550–565Google Scholar
  54. [54]
    H.J. Kushner, Probability Methods for Approximations in Stochastic Control and for Elliptic Equations, Academic Press, New York, 1977Google Scholar
  55. [55]
    H.J. Kushner, Optimality conditions for the average cost per unit time problem with a diffusion model, —SICON 16 (1978) 330–346Google Scholar
  56. [56]
    E. Lenglart, Transformation des martingales locales par changement absolument continu de probabilités, —ZW 39 (1977) 65–70Google Scholar
  57. [57]
    D. Lepingle and J. Memin, Sur l'integrabilité uniforme des martingales exponentielles, —ZW 42 (1978) 175–203Google Scholar
  58. [58]
    J.P. Lepeltier and B. Marchal, Sur l'existence de politiques optimales dans le contrôle integro-differentiel, Ann. Inst. H. Poincaré 13 (1977) 45–97Google Scholar
  59. [59]
    J.P. Lepeltier and B. Marchal, Techniques probabilistes dans le contrôle impulsionelle, Stochastics 2 (1979)Google Scholar
  60. [60]
    R.S. Liptser and A.N. Shiryayev, Statistics of Random Processes, vols I and II, Springer-Verlag, New York, 1977Google Scholar
  61. [61]
    J. Memin, Conditions d'optimalité pour un problème de contrôle portant sur une famille de probabilités dominées par une probabilité P, preprint, University of Rennes, 1977Google Scholar
  62. [62]
    J.F. Mertens, Processus stochastiques généraux, applications aux surmartingales, —ZW 22 (1972) 45–68Google Scholar
  63. [63]
    P.A. Meyer, Probability and Potentials, Blaisdell, Waltham, MA, 1966Google Scholar
  64. [64]
    P.A. Meyer, Un cours sur les intégrales stochastiques, —SP10/LNMGoogle Scholar
  65. [65]
    R.E. Mortensen, Stochastic optimal control with noisy observations, Int. J. Control 4 (1966) 455–464Google Scholar
  66. [66]
    M. Nisio, On a nonlinear semigroup attached to stochastic control, Pub. Res. Inst. Math. Sci., Kyoto, 12 (1976) 513–537Google Scholar
  67. [67]
    A.A. Novikov, On an identity for stochastic integrals, Theor. Probability Appl. 17 (1972) 717–720Google Scholar
  68. [68]
    R. Rishel, Necessary and sufficient conditions for continuous-time stochastic optimal control, —SICON 8 (1970) 559–571Google Scholar
  69. [69]
    J.H. van Schuppen and E. Wong, Transformations of local martingales under a change of law, Ann. Prob. 2 (1974) 879–888Google Scholar
  70. [70]
    C. Striebel, Martingale conditions for the optimal control of continuous-time stochastic systems, 5th Symposium on nonlinear estimation and applications, San Diego, CA, 1974Google Scholar
  71. [71]
    D.W. Stroock and S.R.S. Varadhan, Diffusion processes with continuous coefficients, Comm. Pure and Appl. Math. 22 (1969) 345–400, 479–530Google Scholar
  72. [72]
    B.S. Tsyrelson, An example of a stochastic equation having no strong solution, Theory of Prob. and Appls. 20 (1975) 427–430Google Scholar
  73. [73]
    K. Uchida, On the existence of a Nash equilibrium point in N-person nonzero sum stochastic differential games, —SICON 16 (1978) 142–149Google Scholar
  74. [74]
    K. Uchida, A note on the existence of a Nash equlibrium point in stochastic differential games, to appear in SICON SIAM Journal on Control (and Optimization)Google Scholar
  75. [75]
    P. Varaiya, Differential games, Proc. VI Berkeley Symposium on Math. Stat. and Prob., vol. 3, 687–697, Univ. of California Press, Berkeley, 1972Google Scholar
  76. [76]
    P.P. Varaiya, N-person stochastic differential games, —SICON 14 (1976) 538–545Google Scholar
  77. [77]
    C.B. Wan and M.H.A. Davis, existence of optimal controls for stochastic jump processes, to appear in SICON SIAM Journal on Control (and Optimization)Google Scholar
  78. [78]
    W.M. Wonham, On the separation theorem of stochastic control, —SICON 6 (1968) 312–326Google Scholar
  79. [79]
    C. Yoeurp, Decompositions des martingales locales et formules exponentielles, —SP10/LNM 511 (1976) 432–480Google Scholar

Copyright information

© Springer-Verlag 1979

Authors and Affiliations

  • M. H. A. Davis
    • 1
  1. 1.Laboratory for Information and Decision SystemsMassachusetts Institute of TechnologyCambridge

Personalised recommendations