Advertisement

Genetic Programming and Evolvable Machines

, Volume 6, Issue 3, pp 283–300 | Cite as

GP-Gammon: Genetically Programming Backgammon Players

  • Yaniv AzariaEmail author
  • Moshe Sipper
Article

Abstract

We apply genetic programming to the evolution of strategies for playing the game of backgammon. We explore two different strategies of learning: using a fixed external opponent as teacher, and letting the individuals play against each other. We conclude that the second approach is better and leads to excellent results: Pitted in a 1000-game tournament against a standard benchmark player—Pubeval—our best evolved program wins 62.4% of the games, the highest result to date. Moreover, several other evolved programs attain win percentages not far behind the champion, evidencing the repeatability of our approach.

Keywords

genetic programming backgammon self-learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    J. R. Anderson, and C. Lebiere, The Atomic Components of Thought, Lawrence Erlbaum Associates: Mahwah, NJ, 1998.Google Scholar
  2. 2.
    K. Chellapilla, “A preliminary investigation into evolving modular programs without subtree crossover,”. in, Genetic Programming 1998: Proceedings of the Third Annual Conference, J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (Eds.). University of Wisconsin, Madison, Wisconsin, USA, 1998, pp. 23–31.Google Scholar
  3. 3.
    F. Dahl, “JellyFish Backgammon,” 1998–2004. http://www.jellyfish-backgammon.com.
  4. 4.
    P. Darwen, “Why co-evolution beats temporal-difference learning at backgammon for a linear architecture, but not a non-linear architecture,” in Proceedings of the 2001 Congress on Evolutionary Computation (CEC-01). Seoul Korea, 2001, pp. 1003–1010.Google Scholar
  5. 5.
    R. Gross, K. Albrecht, W. Kantschik, and W. Banzhaf, “Evolving chess playing programs,” in, GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, W. B. Langdon, E. Cantú-Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. A. Potter, A. C. Schultz, J. F. Miller, E. Burke, and N. Jonoska (Eds.). New York, 2002, pp. 740–747.Google Scholar
  6. 6.
    A. Hauptman, and M. Sipper, “GP-EndChess: Using genetic programming to evolve chess endgame players,” in Proceedings of 8th European Conference on Genetic Programming (EuroGP2005), M. Keijzer, A. Tettamanzi, P. Collet, J. van Hemert, and M. Tomassini, (Eds), vol. 3447 of Lecture Notes in Computer Science, Springer-Verlag, Heidelberg, 2005, pp. 120–131.Google Scholar
  7. 7.
    J. R. Koza, Genetic programming: On the Programming of Computers by Means of Natural Selection. MIT Press: Cambridge, MA, 1992.Google Scholar
  8. 8.
    J. R. Koza, Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press: Cambridge, Massachusetts, 1994.Google Scholar
  9. 9.
    J. R. Koza, F. H. Bennett III, D. Andre, and M. A. Keane, Genetic Programming III: Darwinian Invention and Problem Solving, Morgan Kaufmann: San Francisco, California, 1999.Google Scholar
  10. 10.
    D. J. Montana, “Strongly typed genetic programming,” Evolutionary Computation, vol. 3, no. 2, pp. 199–230, 1995.Google Scholar
  11. 11.
    J. B. Pollack, A. D. Blair, and M. Land, “Coevolution of a backgammon player,” in, Artificial Life V: Proceedings of the Fifth International Workshop on the Synthesis and Simulation of Living Systems, C. G. Langton and K. Shimohara (Eds.), Cambridge, MA, 1997, pp. 92–98.Google Scholar
  12. 12.
    J. B. Pollack, A. D. Blair, and M. Land, “DEMO Lab”s HC-Gammon”, 1997. http://demo.cs.brandeis.edu/bkg.html.
  13. 13.
    D. Qi, and R. Sun, “Integrating reinforcement learning, bidding and genetic algorithms,” in Proceedings of the International Conference on Intelligent Agent Technology (IAT-2003), 2003, pp. 53–59.Google Scholar
  14. 14.
    E. M. A. Ronald, M. Sipper, and M. S. Capcarrère, “Design, Observation, Surprise! A Test of Emergence,” Artificial Life vol. 5, no. 3, pp. 225–239, 1999.CrossRefGoogle Scholar
  15. 15.
    S. Sanner, J. R. Anderson, C. Lebiere, and M. Lovett, “Achieving efficient and cognitively plausible learning in backgammon,” in, Proceedings of the 17th International Conference on Machine Learning (ICML-2000), P. Langley (Ed.), Stanford, CA, 2000, pp. 823–830.Google Scholar
  16. 16.
    Y. Shichel, E. Ziserman, and M. Sipper, “GP-Robocode: Using genetic programming to evolve robocode players,” in Proceedings of 8th European Conference on Genetic Programming (Euro GP2005), M. Keijzer, A. Tettamanzi, P. Collet, J. van Hemert, and M. Tomassini, (Eds), vol. 3447 of Lecture Notes in Computer Science, Springer-Verlag, Heidelberg, 2005, pp. 143–154.Google Scholar
  17. 17.
    M. Sipper, “A success story or an old wives' Tale? On judging experiments in evolutionary computation,” Complexity, vol. 5, no. 4, pp. 31–33, 2000.CrossRefGoogle Scholar
  18. 18.
    G. Tesauro, “NEUROGAMMON: A neural-network backgammon learning program,” Heuristic Programming in Artificial Intelligence, vol. 1, no. 7, pp. 78–80, 1989.Google Scholar
  19. 19.
    G. Tesauro, “Software–Source Code Benchmark player ‘pubeval.c’”. http://www.bkgm.com/rgb/rgb.cgi?view+610 1993.
  20. 20.
    G. Tesauro, “Temporal difference learning and TD-Gammon,” Communications of the ACM, vol. 38, no. 3, pp. 58–68, 1995.CrossRefGoogle Scholar
  21. 21.
    X. Yao, “Evolving artificial neural networks,” Proceedings of the IEEE, vol. 87, no. 9, pp. 1423–1447, 1999.Google Scholar

Copyright information

© Springer Science + Business Media, Inc 2005

Authors and Affiliations

  1. 1.Department of Computer ScienceBen-Gurion UniversityBeer-ShevaIsrael

Personalised recommendations