Applied Mathematics & Optimization

, Volume 68, Issue 3, pp 311–331 | Cite as

Risk-Sensitive Control of Pure Jump Process on Countable Space with Near Monotone Cost

  • K. Suresh KumarEmail author
  • Chandan Pal


In this article, we study risk-sensitive control problem with controlled continuous time pure jump process on a countable space as state dynamics. We prove multiplicative dynamic programming principle, elliptic and parabolic Harnack’s inequalities. Using the multiplicative dynamic programing principle and the Harnack’s inequalities, we prove the existence and a characterization of optimal risk-sensitive control under the near monotone condition.


Risk-sensitive control Controlled Markov chain Multiplicative dynamic programming principle Harnack’s inequality Near monotone cost 



The authors are thankful to the referee for giving several suggestions which improved the readability of the paper.


  1. 1.
    Araposthathis, A., Borkar, V.S., Ghosh, M.K.: Ergodic Control of Diffusion Processes. Encyclopedia of Mathematics and Its Applications, vol. 143. Cambridge University Press, Cambridge (2012) Google Scholar
  2. 2.
    Biswas, A., Borkar, V.S., Suresh Kumar, K.: Risk-sensitive control with near monotone cost. Appl. Math. Optim. 62(2), 145–163 (2010) MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Borkar, V.S., Meyn, S.P.: Risk-sensitive optimal control for Markov decision processes with monotone cost. Math. Oper. Res. 27, 192–209 (2002) MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Di Masi, G.B., Stettner, L.: Infinite horizon risk-sensitive control of discrete time Markov processes under minorization property. SIAM J. Control Optim. 46(1), 231–252 (2007) MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Ghosh, M.K., Saha, S.: Risk-sensitive control of continuous time Markov chain. Preprint (2012) Google Scholar
  6. 6.
    Guo, Y.P., Hernaández-Lerma, O.: Continuous Time Markov Decision Processes. Stochastic Modelling and Applied Probability, vol. 62. Springer, Berlin (2009) CrossRefzbMATHGoogle Scholar
  7. 7.
    Hernández-Hernández, D., Markus, S.I.: Risk-sensitive control of Markov processes in countable state space. Syst. Control Lett. 29, 147–155 (1996) CrossRefzbMATHGoogle Scholar
  8. 8.
    Menaldi, J.L., Robin, M.: Remarks on risk-sensitive control problems. Appl. Math. Optim. 52, 297–310 (2005) MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Meyn, S.P., Tweedie, R.L.: Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab. 25, 518–548 (1993) MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Protter, P.: Stochastic Integration and Differential Equations, 2nd edn. Springer, Berlin (2004) zbMATHGoogle Scholar
  11. 11.
    Rishel, R.: Necessary and sufficient dynamic programming conditions for continuous time stochastic optimal control. SIAM J. Control 8(4), 559–571 (1970) MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Striebel, C.: Martingale conditions for the optimal control of continuous time stochastic systems. Stoch. Process. Appl. 18, 329–347 (1984) MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Department of MathematicsIndian Institute of Technology BombayMumbaiIndia

Personalised recommendations