Mathematical Methods of Operations Research

, Volume 54, Issue 3, pp 491–505 | Cite as

Adaptive policies for time-varying stochastic systems under discounted criterion

  • Nadine Hilgert
  • J. Adolfo Minjárez-Sosa


We consider a class of time-varying stochastic control systems, with Borel state and action spaces, and possibly unbounded costs. The processes evolve according to a discrete-time equation x n + 1=G n (x n , a n , ξn), n=0, 1, … , where the ξn are i.i.d. ℜk-valued random vectors whose common density is unknown, and the G n are given functions converging, in a restricted way, to some function G as n→∞. Assuming observability of ξn, we construct an adaptive policy which is asymptotically discounted cost optimal for the limiting control system x n+1=G (x n , a n , ξn).

AMS 1991 subject classifications: 93E20 90C40. 
Key words: Non-homogeneous Markov control processes; discrete-time stochastic systems; discounted cost criterion; optimal adaptive policy 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Nadine Hilgert
    • 1
  • J. Adolfo Minjárez-Sosa
    • 2
  1. 1.Laboratoire de Biométrie, INRA-ENSA.M, 2 place Viala, 34060 Montpellier CEDEX 1, France. ( The research of this author was performed while she was visiting the Departamento de Matemáticas, CINVESTAV-IPN, México, DF.MX
  2. 2.Departamento de Matemáticas, Universidad de Sonora, Rosales s/n, Col. Centro, 83000, Hermosillo, Sonora, México. (

Personalised recommendations