Compositional Models for Reinforcement Learning

  • Nicholas K. Jong
  • Peter Stone
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5781)

Abstract

Innovations such as optimistic exploration, function approximation, and hierarchical decomposition have helped scale reinforcement learning to more complex environments, but these three ideas have rarely been studied together. This paper develops a unified framework that formalizes these algorithmic contributions as operators on learned models of the environment. Our formalism reveals some synergies among these innovations, and it suggests a straightforward way to compose them. The resulting algorithm, Fitted R-MAXQ, is the first to combine the function approximation of fitted algorithms, the efficient model-based exploration of R-MAX, and the hierarchical decompostion of MAXQ.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Nicholas K. Jong
    • 1
  • Peter Stone
    • 1
  1. 1.The University of Texas at AustinAustinUnited States

Personalised recommendations