Postponing Collapse: Ergodic Control with a Probabilistic Constraint

Abstract

We consider the long run average or ‘ergodic’ control of a discrete time Markov process with a probabilistic constraint in terms of a bound on the exit rate from a bounded subset of the state space. This is a natural counterpart of the more common probabilistic constraints in the finite horizon control problems. Using a recent characterization by Anantharam and the first author of risk-sensitive reward as the value of an average cost ergodic control problem, this problem is mapped to a constrained ergodic control problem that seeks to maximize an ergodic reward subject to a constraint on another ergodic reward. However, unlike the classical constrained ergodic reward/cost problems, this problem has some non-classical features due to a non-standard coupling between between the primary ergodic reward and the one that gets constrained. This renders the problem inaccessible to standard solution methodologies. A brief discussion of possible ways out is included.