Volume 2111 of the series Lecture Notes in Computer Science pp 589604
Learning Rates for QLearning
 Eyal EvenDarAffiliated withSchool of Computer Science, TelAviv University
 , Yishay MansourAffiliated withSchool of Computer Science, TelAviv University
Abstract
In this paper we derive convergence rates for Qlearning. We show an interesting relationship between the convergence rate and the learning rate used in the Qlearning. For a polynomial learning rate, one which is 1/t ^{ω} at time t where ω ε (1/2, 1), we show that that the convergence rate is polynomial in 1/(1  γ), where γ is the discount factor. In contrast we show that for a linear learning rate, one which is 1/t at time t, the convergence rate has an exponential dependence on 1/(1  γ). In addition we show a simple example that proves that this exponential behavior is inherent for a linear learning rate.
 Title
 Learning Rates for QLearning
 Book Title
 Computational Learning Theory
 Book Subtitle
 14th Annual Conference on Computational Learning Theory, COLT 2001 and 5th European Conference on Computational Learning Theory, EuroCOLT 2001 Amsterdam, The Netherlands, July 16–19, 2001 Proceedings
 Pages
 pp 589604
 Copyright
 2001
 DOI
 10.1007/3540445811_39
 Print ISBN
 9783540423430
 Online ISBN
 9783540445814
 Series Title
 Lecture Notes in Computer Science
 Series Volume
 2111
 Series ISSN
 03029743
 Publisher
 Springer Berlin Heidelberg
 Copyright Holder
 SpringerVerlag Berlin Heidelberg
 Additional Links
 Topics
 Industry Sectors
 eBook Packages
 Editors

 David Helmbold ^{(1)}
 Bob Williamson ^{(2)}
 Editor Affiliations

 1. School of Engineering, Department of Computer Science, University of California, Santa Cruz
 2. Research School of Information Sciences and Engineering Department of Telecommunications Engineering, Australian National University
 Authors

 Eyal EvenDar ^{(5)}
 Yishay Mansour ^{(5)}
 Author Affiliations

 5. School of Computer Science, TelAviv University, Israel
Continue reading...
To view the rest of this content please follow the download PDF link above.