Advertisement

Mathematical Methods of Operations Research

, Volume 65, Issue 2, pp 239–259 | Cite as

A policy iteration algorithm for fixed point problems with nonexpansive operators

  • Jean-Philippe Chancelier
  • Marouen Messaoud
  • Agnès Sulem
Original Article

Abstract

The aim of this paper is to solve the fixed point problems:
$$ v = \mathcal{O}v,\quad \hbox{with}\, \mathcal{O}v(x) \mathop{=}^{\rm def} \max (Lv(x), Bv(x) ), x \in \varepsilon, \quad (1)$$
where \(\varepsilon\) is a finite set, L is contractive and B is a nonexpansive operator and
$$ v = \mathcal{O}v,\quad \hbox{with} \mathcal{O}v(x) \mathop{=}^{\rm def} \max\left(\sup_{w \in \mathcal{W}} L^{w} v(x) ,\sup_{z \in \mathcal{Z}} B^{z} v(x)\right), x \in \varepsilon, \quad (2)$$
where \(\mathcal{W}\) and \(\mathcal{Z}\) are general control sets, the operators L w are contractive and operators B z are nonexpansive. For these two problems, we give conditions which imply existence and uniqueness of a solution and provide a policy iteration algorithm which converges to the solution. The proofs are slightly different for the two problems since the set of controls is finite for (1) while it is not necessary the case for problem (2). Equation (2) typically arises in numerical analysis of quasi variational inequalities and variational inequalities associated to impulse or singular stochastic control.

Keywords

Howard algorithm Policy iteration Impulse control Quasi-variational inequalities Fixed point problems Optimal control of Markov Chains Nonexpansive operators 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bertsekas DP (2001) Dynamic programming and optimal control, vol I and II. Athena Scientific, BelmontGoogle Scholar
  2. Chancelier J-Ph, Øksendal B, Sulem A (2002) Combined stochastic control and optimal stopping, and application to numerical approximation of combined stochastic and impulse control. In: Stochastic Financial Mathematics, Proceedings of Steklov Math. Inst., Moscou, vol. 237, editeur A.N. Shiryaev, pp 149 –173 2002Google Scholar
  3. Gaubert S, Gunawardena J (privately circuled draft) Existence of the cycle time for some subtopical functionsGoogle Scholar
  4. Kushner HJ, Dupuis P (2001) Numerical methods for stochastic control problems in continuous time, 2nd edn. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  5. Øksendal B, Sulem A (2005) Applied stochastic control of jump diffusions. Universitext. Springer, Berlin Heidelberg New YorkGoogle Scholar
  6. Øksendal B, Sulem A (2002) Optimal consumption and portfolio with both fixed and proportional transaction costs. SIAM J Control Optim 40(6): 1765–1790CrossRefMathSciNetGoogle Scholar
  7. Puterman ML (1994) Markov decision processes: discrete stochastics markov decision processes: Discrete Stochastics Dynamic Programming. Probability and Mathematical Statistics: applied probability and statistics section. Wiley, New YorkGoogle Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  • Jean-Philippe Chancelier
    • 1
  • Marouen Messaoud
    • 2
  • Agnès Sulem
    • 2
  1. 1.Ceramics, École Nationale des Ponts et Chaussées, ParitechMarne la Vallée CedexFrance
  2. 2.Inria, Mathfi project, Domaine de VoluceauLe Chesnay CedexFrance

Personalised recommendations