Scalable Neural Networks for Board Games

  • Tom Schaul
  • Jürgen Schmidhuber
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5768)


Learning to solve small instances of a problem should help in solving large instances. Unfortunately, most neural network architectures do not exhibit this form of scalability. Our Multi-Dimensional Recurrent LSTM Networks, however, show a high degree of scalability, as we empirically show in the domain of flexible-size board games. This allows them to be trained from scratch up to the level of human beginners, without using domain knowledge.


Random Network Recurrent Neural Network Board Size Neural Network Architecture Board Game 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    van der Werf, E., van den Herik, H.J., Uiterwijk, J.: Solving Go on small boards. International Computer Games Association Journal 26 (2003)Google Scholar
  2. 2.
    Richards, N., Moriarty, D.E., Miikkulainen, R.: Evolving neural networks to play Go. Applied Intelligence 8, 85–96 (1997)CrossRefGoogle Scholar
  3. 3.
    Runarsson, T.P., Lucas, S.M.: Co-evolution versus Self-play Temporal Difference Learning for Acquiring Position Evaluation in Small-board Go. IEEE Transactions on Evolutionary Computation, 628–640 (2005)Google Scholar
  4. 4.
    Wu, L., Baldi, P.: A Scalable Machine Learning Approach to Go. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems 19, pp. 1521–1528. MIT Press, Cambridge (2007)Google Scholar
  5. 5.
    Stanley, K.O., Miikkulainen, R.: Evolving a Roving Eye for Go. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 1226–1238. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Schaul, T., Schmidhuber, J.: A Scalable Neural Network Architecture for Board Games. In: Proceedings of the IEEE Symposium on Computational Intelligence in Games. IEEE Press, Los Alamitos (2008)Google Scholar
  7. 7.
    Grüttner, M.: Evolving Multidimensional Recurrent Neural Networks for the Capture Game in Go. Bachelor Thesis, Techniche Universität München (2008)Google Scholar
  8. 8.
    Graves, A., Fernández, S., Schmidhuber, J.: Multidimensional Recurrent Neural Networks. In: Proceedings of the 2007 International Conference on Artificial Neural Networks (September 2007)Google Scholar
  9. 9.
    Graves, A.: Supervised Sequence Labelling with Recurrent Neural Networks, Ph.D. in Informatics, Fakultat für Informatik – Technische Universität München (2008)Google Scholar
  10. 10.
    Silver, D., Sutton, R.S., Müller, M.: Reinforcement Learning of Local Shape in the Game of Go. In: IJCAI, pp. 1053–1058 (2007)Google Scholar
  11. 11.
    Lecun, Y., Bengio, Y.: Convolutional Networks for Images, Speech and Time Series, pp. 255–258. The MIT Press, Cambridge (1995)Google Scholar
  12. 12.
    Schraudolph, N.N., Dayan, P., Sejnowski, T.J.: Temporal Difference Learning of Position Evaluation in the Game of Go. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, vol. 6, pp. 817–824. Morgan Kaufmann, San Francisco (1994)Google Scholar
  13. 13.
    Freisleben, B., Luttermann, H.: Learning to Play the Game of Go-Moku: A Neural Network Approach. Australian Journal of Intelligent Information Processing Systems 3(2), 52–60 (1996)Google Scholar
  14. 14.
    Gauci, J., Stanley, K.: Generating large-scale neural networks through discovering geometric regularities. In: GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 997–1004 (2007)Google Scholar
  15. 15.
    Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45, 2673–2681 (1997)CrossRefGoogle Scholar
  16. 16.
    Baldi, P., Pollastri, G.: The principled design of large-scale recursive neural network architectures DAG-RNNs and the protein structure prediction problem. Journal of Machine Learning Research 4, 575–602 (2003)zbMATHGoogle Scholar
  17. 17.
    Graves, A., Schmidhuber, J.: Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks. In: NIPS (2008)Google Scholar
  18. 18.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(9), 1735–1780 (1997)CrossRefGoogle Scholar
  19. 19.
    Gherman, S.: Atari-Go Applet (2000),
  20. 20.
    Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)CrossRefGoogle Scholar
  21. 21.
    Gomez, F., Miikkulainen, R.: Incremental Evolution of Complex General Behavior. Adaptive Behavior 5, 317–342 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Tom Schaul
    • 1
  • Jürgen Schmidhuber
    • 1
  1. 1.IDSIAManno-LuganoSwitzerland

Personalised recommendations