Machine Learning

, Volume 22, Issue 1, pp 59–94

Feature-based methods for large scale dynamic programming

  • John N. Tsitsiklis
  • Benjamin van Roy

DOI: 10.1007/BF00114724

Cite this article as:
Tsitsiklis, J.N. & van Roy, B. Mach Learn (1996) 22: 59. doi:10.1007/BF00114724


We develop a methodological framework and present a few different ways in which dynamic programming and compact representations can be combined to solve large scale stochastic control problems. In particular, we develop algorithms that employ two types of feature-based compact representations; that is, representations that involve feature extraction and a relatively simple approximation architecture. We prove the convergence of these algorithms and provide bounds on the approximation error. As an example, one of these algorithms is used to generate a strategy for the game of Tetris. Furthermore, we provide a counter-example illustrating the difficulties of integrating compact representations with dynamic programming, which exemplifies the shortcomings of certain simple approaches.


Compact representationcurse of dimensionalitydynamic programmingfeaturesfunction approximationneuro-dynamic programmingreinforcement learning
Download to read the full article text

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • John N. Tsitsiklis
    • 1
  • Benjamin van Roy
    • 1
  1. 1.Laboratory for Information and Decision SystemsMassachusetts Institute of Technology