Skip to main content
SpringerLink
Log in

On the convergence of policy iteration for controlled diffusions

  • Contributed Papers
  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

The convergence of an approximation scheme known as policy iteration has been demonstrated for controlled diffusions by Fleming, Puterman, and Bismut. In this paper, we show that this approximation scheme is equivalent to the Newton-Kantorovich iteration for solving the optimality equation and exploit this equivalence to obtain a new proof of convergence. Estimates of the rate of convergence of this procedure are also obtained.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Fleming, W. H.,Some Markovian Optimization Problems, Journal of Mathematics and Mechanics, Vol. 12, pp. 131–140, 1963.

    Google Scholar 

  2. Puterman, M. L.,Optimal Control of Diffusion Processes with Reflection, Journal of Optimization Theory and Applications, Vol. 22, 103–116, 1977.

    Google Scholar 

  3. Bismut, J.,An Approximation Method in Optimal Stochastic Control, SIAM Journal on Control and Optimization, Vol. 16, pp. 122–130, 1978.

    Google Scholar 

  4. Puterman, M. L., andBrumelle, S. L.,On the Convergence of Policy Iteration in Stationary Dynamic Programming, Mathematics of Operations Research, Vol. 4, pp. 60–69, 1979.

    Google Scholar 

  5. Stroock, D. W., andVaradhan, S. R. S.,Diffusion Processes with Boundary Conditions, Communications on Pure and Applied Mathematics, Vol. 26, pp. 147–226, 1971.

    Google Scholar 

  6. Kantorovich, L. V., andAkilov, G. P.,Functional Analysis in Normed Spaces, The Macmillan Company, New York, New York, 1964.

    Google Scholar 

  7. Stroock, D. W., andVaradhan, S. R. S.,Diffusion Processes with Continuous Coefficients, II, Communications on Pure and Applied Mathematics, Vol. 22, pp. 479–530, 1969.

    Google Scholar 

  8. Fleming, W. H., andRishel, R.,Deterministic and Stochastic Optimal Control, Springer-Verlag, New York, New York, 1975.

    Google Scholar 

  9. Mandl, P.,Analytic Treatment of One-Dimensional Markov Processes, Springer-Verlag, New York, New York, 1968.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Commerce and Business Administration, University of British Columbia, Vancouver, British Columbia, Canada

    M. L. Puterman (Associate Professor)

Authors
  1. M. L. Puterman
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Communicated by R. Rishel

This research was partially supported by NRC Grant No. A-3609.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Puterman, M.L. On the convergence of policy iteration for controlled diffusions. J Optim Theory Appl 33, 137–144 (1981). https://doi.org/10.1007/BF00935182

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00935182

Key Words

Search

Navigation