Discretization Procedures

Abstract

We begin in Section 6.2 by introducing the control model (CM) we will be dealing with, together with some general assumptions. In Section 6.3 we consider non-adaptive CM’s. We start with a non-recursive procedure and then show how it can be made recursive; furthermore, in both cases, recursive and non-recursive, we obtain asymptotically discount optimal (ADO) control policies. The results in Section 6.3 can be seen as “discretized” versions of the Nonstationary Value Iteration (NVI) schemes NVI-1 and NVI-2 in Section 2.4. Next, in Section 6.4, the discretizations in Section 6.3 are extended to adaptive CM’s and, in particular, we obtain “discretized” forms of the Principle of Estimation and Control (PEC) and the NVI adaptive policies in Section 2.5. The proofs of the theorems in Sections 6.3 and 6.4 have many common arguments, and therefore, all the proofs are collected in Section 6.5. We close in Section 6.6 with some comments on relevant references.