Mean, variance, and probabilistic criteria in finite Markov decision processes: A review

  • D. J. White
Survey Paper

DOI: 10.1007/BF00938524

Cite this article as:
White, D.J. J Optim Theory Appl (1988) 56: 1. doi:10.1007/BF00938524


This paper is a survey of papers which make use of nonstandard Markov decision process criteria (i.e., those which do not seek simply to optimize expected returns per unit time or expected discounted return). It covers infinite-horizon nondiscounted formulations, infinite-horizon discounted formulations, and finite-horizon formulations. For problem formulations in terms solely of the probabilities of being in each state and taking each action, policy equivalence results are given which allow policies to be restricted to the class of Markov policies or to the randomizations of deterministic Markov policies. For problems which cannot be stated in such terms, in terms of the primitive state setI, formulations involving a redefinition of the states are examined.

Key Words

Markov decision processesinfinite horizonfinite horizonmeanvarianceprobabilistic criteria

Copyright information

© Plenum Publishing Corporation 1988

Authors and Affiliations

  • D. J. White
    • 1
  1. 1.University of ManchesterManchesterEngland