Mathematical Methods of Operations Research

, Volume 54, Issue 3, pp 455–469 | Cite as

Stopped Markov decision processes with multiple constraints

  • Masayuki Horiguchi


In this paper, a optimization problem for stopped Markov decision processes with vector-valued terminal reward and multiple running cost constraints is considered. Applying the idea of occupation measures and using the scalarization technique for vector maximization problems we obtain the equivalent Mathematical Programming problem and show the existence of a Pareto optimal pair of stationary policy and stopping time requiring randomization in at most k states, where k is the number of constraints. Moreover Lagrange multiplier approaches are considered. The saddle-point statements are given, whose results are applied to obtain a related parametric Mathematical Programming, by which the problem is solved. Numerical examples are given.

Key words: Stopped Markov decision process multi-objective multiple constraints Mathematical Programming formulation Lagrange multiplier 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Masayuki Horiguchi
    • 1
  1. 1.Division of Mathematical Sciences and Physics, Graduate School of Science and Technology, Chiba University, 33, Yayoi-cho 1-chome, Inage-ku, Chiba 263-8522, Japan (e-mail:

Personalised recommendations