GP-Gammon: Genetically Programming Backgammon Players
- 134 Downloads
We apply genetic programming to the evolution of strategies for playing the game of backgammon. We explore two different strategies of learning: using a fixed external opponent as teacher, and letting the individuals play against each other. We conclude that the second approach is better and leads to excellent results: Pitted in a 1000-game tournament against a standard benchmark player—Pubeval—our best evolved program wins 62.4% of the games, the highest result to date. Moreover, several other evolved programs attain win percentages not far behind the champion, evidencing the repeatability of our approach.
Keywordsgenetic programming backgammon self-learning
Unable to display preview. Download preview PDF.
- 1.J. R. Anderson, and C. Lebiere, The Atomic Components of Thought, Lawrence Erlbaum Associates: Mahwah, NJ, 1998.Google Scholar
- 2.K. Chellapilla, “A preliminary investigation into evolving modular programs without subtree crossover,”. in, Genetic Programming 1998: Proceedings of the Third Annual Conference, J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (Eds.). University of Wisconsin, Madison, Wisconsin, USA, 1998, pp. 23–31.Google Scholar
- 3.F. Dahl, “JellyFish Backgammon,” 1998–2004. http://www.jellyfish-backgammon.com.
- 4.P. Darwen, “Why co-evolution beats temporal-difference learning at backgammon for a linear architecture, but not a non-linear architecture,” in Proceedings of the 2001 Congress on Evolutionary Computation (CEC-01). Seoul Korea, 2001, pp. 1003–1010.Google Scholar
- 5.R. Gross, K. Albrecht, W. Kantschik, and W. Banzhaf, “Evolving chess playing programs,” in, GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, W. B. Langdon, E. Cantú-Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. A. Potter, A. C. Schultz, J. F. Miller, E. Burke, and N. Jonoska (Eds.). New York, 2002, pp. 740–747.Google Scholar
- 6.A. Hauptman, and M. Sipper, “GP-EndChess: Using genetic programming to evolve chess endgame players,” in Proceedings of 8th European Conference on Genetic Programming (EuroGP2005), M. Keijzer, A. Tettamanzi, P. Collet, J. van Hemert, and M. Tomassini, (Eds), vol. 3447 of Lecture Notes in Computer Science, Springer-Verlag, Heidelberg, 2005, pp. 120–131.Google Scholar
- 7.J. R. Koza, Genetic programming: On the Programming of Computers by Means of Natural Selection. MIT Press: Cambridge, MA, 1992.Google Scholar
- 8.J. R. Koza, Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press: Cambridge, Massachusetts, 1994.Google Scholar
- 9.J. R. Koza, F. H. Bennett III, D. Andre, and M. A. Keane, Genetic Programming III: Darwinian Invention and Problem Solving, Morgan Kaufmann: San Francisco, California, 1999.Google Scholar
- 10.D. J. Montana, “Strongly typed genetic programming,” Evolutionary Computation, vol. 3, no. 2, pp. 199–230, 1995.Google Scholar
- 11.J. B. Pollack, A. D. Blair, and M. Land, “Coevolution of a backgammon player,” in, Artificial Life V: Proceedings of the Fifth International Workshop on the Synthesis and Simulation of Living Systems, C. G. Langton and K. Shimohara (Eds.), Cambridge, MA, 1997, pp. 92–98.Google Scholar
- 12.J. B. Pollack, A. D. Blair, and M. Land, “DEMO Lab”s HC-Gammon”, 1997. http://demo.cs.brandeis.edu/bkg.html.
- 13.D. Qi, and R. Sun, “Integrating reinforcement learning, bidding and genetic algorithms,” in Proceedings of the International Conference on Intelligent Agent Technology (IAT-2003), 2003, pp. 53–59.Google Scholar
- 15.S. Sanner, J. R. Anderson, C. Lebiere, and M. Lovett, “Achieving efficient and cognitively plausible learning in backgammon,” in, Proceedings of the 17th International Conference on Machine Learning (ICML-2000), P. Langley (Ed.), Stanford, CA, 2000, pp. 823–830.Google Scholar
- 16.Y. Shichel, E. Ziserman, and M. Sipper, “GP-Robocode: Using genetic programming to evolve robocode players,” in Proceedings of 8th European Conference on Genetic Programming (Euro GP2005), M. Keijzer, A. Tettamanzi, P. Collet, J. van Hemert, and M. Tomassini, (Eds), vol. 3447 of Lecture Notes in Computer Science, Springer-Verlag, Heidelberg, 2005, pp. 143–154.Google Scholar
- 18.G. Tesauro, “NEUROGAMMON: A neural-network backgammon learning program,” Heuristic Programming in Artificial Intelligence, vol. 1, no. 7, pp. 78–80, 1989.Google Scholar
- 19.G. Tesauro, “Software–Source Code Benchmark player ‘pubeval.c’”. http://www.bkgm.com/rgb/rgb.cgi?view+610 1993.
- 21.X. Yao, “Evolving artificial neural networks,” Proceedings of the IEEE, vol. 87, no. 9, pp. 1423–1447, 1999.Google Scholar