Finance and Stochastics

, Volume 13, Issue 4, pp 591–611 | Cite as

MDP algorithms for portfolio optimization problems in pure jump markets



We consider the problem of maximizing the expected utility of the terminal wealth of a portfolio in a continuous-time pure jump market with general utility function. This leads to an optimal control problem for piecewise deterministic Markov processes. Using an embedding procedure we solve the problem by looking at a discrete-time contracting Markov decision process. Our aim is to show that this point of view has a number of advantages, in particular as far as computational aspects are concerned. We characterize the value function as the unique fixed point of the dynamic programming operator and prove the existence of optimal portfolios. Moreover, we show that value iteration as well as Howard’s policy improvement algorithm works. Finally, we give error bounds when the utility function is approximated and when we discretize the state space. A numerical example is presented and our approach is compared to the approximating Markov chain method.


Portfolio optimization Piecewise deterministic Markov processes Markov decision process Operator fixed points Approximation algorithms 

Mathematics Subject Classification (2000)

91B28 93E20 90C39 60G55 

JEL Classification

G11 C61 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Almudevar, A.: A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes. SIAM J. Control Optim. 40, 525–539 (2001) MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Bäuerle, N., Rieder, U.: Portfolio optimization with Markov-modulated stock prices and interest rates. IEEE Trans. Autom. Control 49, 442–447 (2004) CrossRefGoogle Scholar
  3. 3.
    Bäuerle, N., Rieder, U.: Portfolio optimization with jumps and unobservable intensity process. Math. Finance 17, 205–224 (2007) MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control: The Discrete-Time Case. Mathematics in Science and Engineering, vol. 139. Academic Press, New York (1978) MATHGoogle Scholar
  5. 5.
    Davis, M.H.A.: Markov Models and Optimization. Monographs on Statistics and Applied Probability, vol. 49. Chapman & Hall, London (1993) MATHGoogle Scholar
  6. 6.
    Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Applications of Mathematics, vol. 25. Springer, New York (1993) MATHGoogle Scholar
  7. 7.
    Jacobsen, M.: Point Process Theory and Applications. Probability and its Applications. Birkhäuser, Boston (2006) MATHGoogle Scholar
  8. 8.
    Jouini, E., Napp, C.: Convergence of utility functions and convergence of optimal strategies. Finance Stoch. 8, 133–144 (2004) MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Kirch, M., Runggaldier, W.J.: Efficient hedging when asset prices follow a geometric Poisson process with unknown intensities. SIAM J. Control Optim. 43, 1174–1195 (2005) CrossRefMathSciNetGoogle Scholar
  10. 10.
    Kushner, H.J., Dupuis, P.: Numerical Methods for Stochastic Control Problems in Continuous Time. Applications of Mathematics, vol. 24. Springer, New York (2001) MATHGoogle Scholar
  11. 11.
    Norberg, R.: The Markov chain market. Astin Bull. 33, 265–287 (2003) MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Øksendal, B., Sulem, A.: Applied Stochastic Control of Jump Diffusions. Universitext/Springer, Berlin (2005) Google Scholar
  13. 13.
    Rieder, U., Bäuerle, N.: Portfolio optimization with unobservable Markov-modulated drift process. J. Appl. Probab. 42, 362–378 (2005) MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Schäl, M.: Control of ruin probabilities by discrete-time investments. Math. Methods Oper. Res. 62, 141–158 (2005) MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.Institut für StochastikUniversität Karlsruhe (TH)KarlsruheGermany
  2. 2.Institut für Optimierung und Operations ResearchUniversität UlmUlmGermany

Personalised recommendations