Abstract
A new approach to the optimal control of diffusion processes based on Lagrange functionals is presented. The method is conceptually and technically simpler than existing ones. A first class of functionals allows to obtain optimality conditions without any resort to stochastic calculus and functional analysis. A second class, which requires Ito's rule, allows to establish optimality in a larger class of problems. Calculations in these two methods are sometimes akin to those in minimum principles and in dynamic programming, but the thinking behind them is new. A few examples are worked out to illustrate the power and simplicity of this approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bensoussan, A.: Variational methods in stochastic control, in S. K. Mitter and A. Moro (eds),Nonlinear Filtering and Stochastic Control, Lecture Notes in Mathematics 972, Springer-Verlag, Berlin, 1983.
Berkovitz, L. D.:Optimal Control Theory, Springer-Verlag, Berlin, 1974.
Bismut, J.-M.: Conjugate convex functions in optimal stochastic control,J. Math. Anal. Appl. 44 (1973), 384–404.
Bismut, J.-M.: An introductory approach to duality in optimal stochastic control,SIAM Rev. 20 (1978), 62–78.
Dai Pra, P.: A stochastic control approach to reciprocal diffusion processes,Appl. Math. Optim. 23 (1991), 313–329.
Dai Pra, P. and Pavon, M.: On the Markov processes of Schrödinger, the Feynman-Kac formula and stochastic control, in M. A. Kaashoek, J. van Schuppen, and A. C. M. Ran (eds),Realization and Modeling in System Theory, Birkäuser, Boston, 1990, pp. 497–504.
Davis, M. H. A. and Varaya, P. P.: Dynamic programming conditions for partially observed stochastic systems,SIAM J. Control Optim. 11 (1973), 226–261.
Dawson, D., Gorostiza, L. and Wakolbinger, A.: Schrödinger processes and large deviations,J. Math. Phys. 31 (1990), 2385–2388.
Elliot, R. J.: The optimal control of diffusions.Appl. Math. Optim. 22 (1990), 229–240.
Fleming, W. H.: Exit probabilities and optimal stochastic control,Appl. Math. Optim. 4 (1978), 329–346.
Fleming, W. H.: Logarithmic transformation and stochastic control, in W. H. Fleming and L. G. Gorostiza (eds),Advances in Filtering and Optimal Stochastic Control, Lecture Notes in Control and Information Sciences 42, Springer-Verlag, Berlin, 1982, pp. 131–141.
Fleming, W. H. and Rishel, R. W.:Deterministic and Stochastic Optimal Control, Springer-Verlag, Berlin, 1975.
Fleming W. H. and Vermes, D.: Convex duality approach to the optimal control of diffusions,SIAM J. Contr. Optim. 27 (1989), 1136–1155.
Guerra, F. and Morato, L.: Quantization of dynamical systems and stochastic control theory,Phys. Rev. D 27 (1983), 1771–1786.
Haussmann, U. G.: Some examples of optimal stochastic controls or: The stochastic maximum principle at work,SIAM Rev. 23 (1981), 292–307.
Haussmann, U. G.: Extremal controls for completely observable diffusions, in W. H. Fleming and L. G. Gorostiza (eds),Advances in Filtering and Optimal Stochastic Control, Lecture Notes in Control and Information Sciences 42, Springer-Verlag, Berlin, 1982, pp. 149–160.
Hewitt, E. and Stromberg, K.:Real and Abstract Analysis, Springer-Verlag, Berlin, 1969.
Holmes, R. B.:A Course on Optimization and Best Approximation, Lecture Notes in Mathematics 257, Springer-Verlag, Berlin, 1972.
Karatzas, I.: On a stochastic representation of the principal eigenvalue of a second-order differential equation,Stochastics 3 (1980), 305–321.
Karatzas, I. and Shreve, S.E.:Brownian Motion and Stochastic Calculus, Springer-Verlag, Berlin, 1987.
Karatzas, I.: Optimization problems in the theory of continuous trading,SIAM J. Control Optim. 27 (1989), 1221–1259.
Kosmol, I.:Optimierung und Approximation, De Gruyter Lehrbuch, Berlin, 1991.
Kushner, H. J.: Necessary conditions for continuous parameter stochastic optimization problems,SIAM J. Control 10 (1972), 550–562.
Nagasawa, M.: Transformations of diffusions and Schrödinger processes,Probab. Theory Related Fields 82 (1989), 109–136.
Pavon, M.: Stochastic control and nonequilibrium thermodynamical systems,Appl. Math. Optim. 19 (1989), 187–202.
Pavon, M. and Wakolbinger, A.: On free energy, stochastic control, and Schrödinger processes, in G. B. Di Masi, A. Gombani and A. Kurzhanski (eds),Modeling, Estimation and Control of Systems with Uncertainty, Birkäuser, Boston, 1991, pp. 334–348.
Wakolbinger, A.: A simplified variational characterization of Schrödinger processes,J. Math. Phys. 30 (1989), 2943–2946.
Weierstrass, K.:Mathematische Werke von Karl Weierstrass, Siebenter Band, Vorlesungen über Variationsrechnung, Akademische Verlagsgesellschaft, Leipzig, 1927.
Author information
Authors and Affiliations
Additional information
Research performed at the Mathematisches Seminar der Universität Kiel with support provided by an Alexander von Humboldt Foundation fellowship.
Rights and permissions
About this article
Cite this article
Kosmol, P., Pavon, M. Lagrange approach to the optimal control of diffusions. Acta Appl Math 32, 101–122 (1993). https://doi.org/10.1007/BF00998149
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00998149