Journal of Optimization Theory and Applications

, Volume 35, Issue 4, pp 611–633 | Cite as

Optimal stationary linear control of the Wiener process

  • V. E. Beneš
  • I. Karatzas
Contributed Papers

Abstract

In the present paper, we consider the following stochastic control problem: to minimize the average expected total cost
$$J(x,u) = \mathop {\lim \inf }\limits_{T \to \infty } (1/T)E_x^u \int_0^T {\left[ {\phi (\xi _t ) + |u_t (\xi )|} \right]} dt,$$
〈subject to
$$d\xi _t = u_1 (\xi )dt + dw_t , \xi _0 = x, |u| \leqslant 1,$$
(wt) a Wiener process, with all measurable functions on the past of the state process {ξ s ;st} and bounded by unity, admissible as controls. It is proved that, under very mild conditions on the running cost function φ(·), the optimal law is of the form
$$\begin{gathered} u_t^* (\xi ) = - sign\xi _t , |\xi _t | > b, \hfill \\ u_t^* (\xi ) = 0, |\xi _t | > b. \hfill \\ \end{gathered} $$
The cutoff pointb and the performance rate of the optimal lawu* are simultaneously determined in terms of the function φ(·) through a simple system of integrotranscendental equations.

Key Words

Stationary stochastic control Bellman equation invariant measures nonanticipative laws dead-zone controllers 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Wonham, W. M.,Optimal Stationary Control of a Linear System with State-Dependent Noise, SIAM Journal on Control, Vol. 3, pp. 486–500, 1967.Google Scholar
  2. 2.
    Kushner, H. J.,Optimality Conditions for the Average Cost per Unit Time Problem with a Diffusion Model, SIAM Journal on Control and Optimization, Vol. 16, pp. 330–346, 1978.Google Scholar
  3. 3.
    Girsanov, I. V.,On Transforming a Certain Class of Stochastic Processes by Absolutely Continuous Substitution of Measures, Theory of Probability and Applications, Vol. 5, pp. 285–301, 1960.Google Scholar
  4. 4.
    Liptser, R. S., andShiryayev, A. N.,Statistics of Random Processes, Vol. 1, General Theory, Springer-Verlag, Berlin, Germany, 1977.Google Scholar
  5. 5.
    Zvonkin, A. K.,A Transformation of the Phase Space of a Diffusion Process that Removes the Drift, Mathematics of the USSR (Sbornik), Vol. 22, pp. 129–149, 1974.Google Scholar
  6. 6.
    Gihman, I. I., andSkorohod, A. V.,Stochastic Differential Equations, Springer-Verlag, Berlin, Germany, 1972.Google Scholar
  7. 7.
    Wonham, W. M.,Liapunov Criteria for Weak Stochastic Stability, Journal of Differential Equations, Vol. 2, pp. 195–207, 1966.Google Scholar
  8. 8.
    Khas'minskii, I. Z.,Ergodic Properties of Recurrent Diffusion Processes and Stabilization of the Solution to the Cauchy Problem for Parabolic Equations, Theory of Probability and Applications, Vol. 5, pp. 179–195, 1960.Google Scholar
  9. 9.
    Friedman, A.,Partial Differential Equations of Parabolic Type, Prentice Hall, Englewood Cliffs, New Jersey, 1964.Google Scholar
  10. 10.
    Fleming, W. H., andRishel, R. W.,Deterministic and Stochastic Optimal Control, Springer-Verlag, Berlin, Germany, 1975.Google Scholar
  11. 11.
    Ikeda, N., andWatanabe, S.,A Comparison Theorem for Solutions of Stochastic Differential Equations and Its Applications, Osaka Journal of Mathematics, Vol. 14, pp. 619–633, 1977.Google Scholar
  12. 12.
    Chow, Y. S., andTeicher, H.,Probability Theory: Independence, Interchangeability, Martingales, Springer-Verlag, Berlin, Germany, 1978.Google Scholar

Copyright information

© Plenum Publishing Corporation 1981

Authors and Affiliations

  • V. E. Beneš
    • 1
  • I. Karatzas
    • 2
  1. 1.Bell Telephone LaboratoriesMurray Hill
  2. 2.Division of Applied Mathematics and Lefschetz Center for Dynamical SystemsBrown UniversityProvidence

Personalised recommendations