Learning Rates for Q-Learning

  • Eyal Even-Dar
  • Yishay Mansour
Conference paper

DOI: 10.1007/3-540-44581-1_39

Part of the Lecture Notes in Computer Science book series (LNCS, volume 2111)
Cite this paper as:
Even-Dar E., Mansour Y. (2001) Learning Rates for Q-Learning. In: Helmbold D., Williamson B. (eds) Computational Learning Theory. COLT 2001. Lecture Notes in Computer Science, vol 2111. Springer, Berlin, Heidelberg

Abstract

In this paper we derive convergence rates for Q-learning. We show an interesting relationship between the convergence rate and the learning rate used in the Q-learning. For a polynomial learning rate, one which is 1/tω at time t where ω ε (1/2, 1), we show that that the convergence rate is polynomial in 1/(1 - γ), where γ is the discount factor. In contrast we show that for a linear learning rate, one which is 1/t at time t, the convergence rate has an exponential dependence on 1/(1 - γ). In addition we show a simple example that proves that this exponential behavior is inherent for a linear learning rate.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Eyal Even-Dar
    • 1
  • Yishay Mansour
    • 1
  1. 1.School of Computer ScienceTel-Aviv UniversityIsrael

Personalised recommendations