Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards

Abstract.

This paper is the second part of our study of Blackwell optimal policies in Markov decision chains with a Borel state space and unbounded rewards. We prove that a stationary policy is Blackwell optimal in the class of all history-dependent policies if it is Blackwell optimal in the class of stationary policies.

 We also develop recurrence and drift conditions which ensure ergodicity and integrability assumptions made in the previous paper, and which are more suitable for applications. As an example we study a cash-balance model.