Volume 2063 of the series Lecture Notes in Computer Science pp 133150
Chess Neighborhoods, Function Combination, and Reinforcement Learning
 Robert LevinsonAffiliated withUniversity of California Santa Cruz
 , Ryan WeberAffiliated withUniversity of California Santa Cruz
Abstract
Over the years, various research projects have attempted to develop a chess program that learns to play well given little prior knowledge beyond the rules of the game. Early on it was recognized that the key would be to adequately represent the relationships between the pieces and to evaluate the strengths or weaknesses of such relationships. As such, representations have developed, including a graphbased model. In this paper we extend the work on graph representation to a precise type of graph that we call a piece or square neighborhood. Specifically, a chessboard is represented as 64 neighborhoods, one for each square. Each neighborhood has a center, and 16 satellites corresponding to the pieces that are immediately close on the 4 diagonals, 2 ranks, 2 files, and 8 knight moves related to the square. Games are played and training values for boards are developed using temporal difference learning, as in other reinforcement learning systems. We then use a 2layer regression network to learn. At the lower level the values (expected probability of winning) of the neighborhoods are learned and at the top they are combined based on their product and entropy. We report on relevant experiments including a learning experience on the Internet Chess Club (ICC) from which we can estimate a rating for the new program. The level of chess play achieved in a few days of training is comparable to a few months of work on previous systems such as Morph which is described as “one of the best fromscratch game learning systems, perhaps the best” [22].
Keywords
linear regression value function approximation temporal difference learning reinforcement learning computer chess exponentiated gradient gradient descent multilayer neural nets Title
 Chess Neighborhoods, Function Combination, and Reinforcement Learning
 Book Title
 Computers and Games
 Book Subtitle
 Second International Conference, CG 2000 Hamamatsu, Japan, October 26–28, 2000 Revised Papers
 Book Part
 Part 2
 Pages
 pp 133150
 Copyright
 2001
 DOI
 10.1007/3540455795_9
 Print ISBN
 9783540430803
 Online ISBN
 9783540455790
 Series Title
 Lecture Notes in Computer Science
 Series Volume
 2063
 Series ISSN
 03029743
 Publisher
 Springer Berlin Heidelberg
 Copyright Holder
 SpringerVerlag Berlin Heidelberg
 Additional Links
 Topics
 Keywords

 linear regression
 value function approximation
 temporal difference learning
 reinforcement learning
 computer chess
 exponentiated gradient
 gradient descent
 multilayer neural nets
 Industry Sectors
 eBook Packages
 Editors

 Tony Marsland ^{(4)}
 Ian Frank ^{(5)}
 Editor Affiliations

 4. Department of Computer Science, University of Alberta
 5. Future University  Hakodate
 Authors

 Robert Levinson ^{(6)}
 Ryan Weber ^{(6)}
 Author Affiliations

 6. University of California Santa Cruz, Santa Cruz, CA, 95064, USA
Continue reading...
To view the rest of this content please follow the download PDF link above.