Mathematical Methods of Operations Research

, Volume 65, Issue 2, pp 239–259

# A policy iteration algorithm for fixed point problems with nonexpansive operators

• Jean-Philippe Chancelier
• Marouen Messaoud
• Agnès Sulem
Original Article

## Abstract

The aim of this paper is to solve the fixed point problems:
$$v = \mathcal{O}v,\quad \hbox{with}\, \mathcal{O}v(x) \mathop{=}^{\rm def} \max (Lv(x), Bv(x) ), x \in \varepsilon, \quad (1)$$
where $$\varepsilon$$ is a finite set, L is contractive and B is a nonexpansive operator and
$$v = \mathcal{O}v,\quad \hbox{with} \mathcal{O}v(x) \mathop{=}^{\rm def} \max\left(\sup_{w \in \mathcal{W}} L^{w} v(x) ,\sup_{z \in \mathcal{Z}} B^{z} v(x)\right), x \in \varepsilon, \quad (2)$$
where $$\mathcal{W}$$ and $$\mathcal{Z}$$ are general control sets, the operators L w are contractive and operators B z are nonexpansive. For these two problems, we give conditions which imply existence and uniqueness of a solution and provide a policy iteration algorithm which converges to the solution. The proofs are slightly different for the two problems since the set of controls is finite for (1) while it is not necessary the case for problem (2). Equation (2) typically arises in numerical analysis of quasi variational inequalities and variational inequalities associated to impulse or singular stochastic control.

## Keywords

Howard algorithm Policy iteration Impulse control Quasi-variational inequalities Fixed point problems Optimal control of Markov Chains Nonexpansive operators

## References

1. Bertsekas DP (2001) Dynamic programming and optimal control, vol I and II. Athena Scientific, BelmontGoogle Scholar
2. Chancelier J-Ph, Øksendal B, Sulem A (2002) Combined stochastic control and optimal stopping, and application to numerical approximation of combined stochastic and impulse control. In: Stochastic Financial Mathematics, Proceedings of Steklov Math. Inst., Moscou, vol. 237, editeur A.N. Shiryaev, pp 149 –173 2002Google Scholar
3. Gaubert S, Gunawardena J (privately circuled draft) Existence of the cycle time for some subtopical functionsGoogle Scholar
4. Kushner HJ, Dupuis P (2001) Numerical methods for stochastic control problems in continuous time, 2nd edn. Springer, Berlin Heidelberg New York
5. Øksendal B, Sulem A (2005) Applied stochastic control of jump diffusions. Universitext. Springer, Berlin Heidelberg New YorkGoogle Scholar
6. Øksendal B, Sulem A (2002) Optimal consumption and portfolio with both fixed and proportional transaction costs. SIAM J Control Optim 40(6): 1765–1790
7. Puterman ML (1994) Markov decision processes: discrete stochastics markov decision processes: Discrete Stochastics Dynamic Programming. Probability and Mathematical Statistics: applied probability and statistics section. Wiley, New YorkGoogle Scholar

## Authors and Affiliations

• Jean-Philippe Chancelier
• 1
Email author
• Marouen Messaoud
• 2
• Agnès Sulem
• 2
1. 1.Ceramics, École Nationale des Ponts et Chaussées, ParitechMarne la Vallée CedexFrance
2. 2.Inria, Mathfi project, Domaine de VoluceauLe Chesnay CedexFrance