Abstract
A game is called “Monte-Carlo perfect” when in each position pure Monte-Carlo search converges to perfect play as the number of simulations tends toward infinity. We exhibit three families of Monte-Carlo perfect single-player and two-player games where this convergence is not monotonic. We for example give a class of MC-perfect games in which MC(1) performs arbitrarily well against MC(1,000).
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
For Brevity, we use ‘he’ and ‘his’ whenever ‘he or she’ and ‘his or her’ are meant.
References
Althöfer, I.: On games with random-turn order and Monte Carlo perfectness. ICGA J. 34, 179–190 (2011)
Browne, C.: On the dangers of random playouts. ICGA J. 34, 25–26 (2011)
Lorentz, R.J.: Amazons discover Monte-Carlo. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) CG 2008. LNCS, vol. 5131, pp. 13–24. Springer, Heidelberg (2008)
Coulom, R.: Efficient selectivity and backup operators in Monte-Carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M.J. (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007)
Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)
Althöfer, I.: On the laziness of Monte Carlo game tree search in non-tight positions. Technical report, Friedrich Schiller University of Jena (2009). http://www.althofer.de/mc-laziness.pdf
Althöfer, I.: Game self-play with pure Monte Carlo: the basin structure. Technical report, Friedrich Schiller University of Jena (2010). http://www.althofer.de/monte-carlo-basins-althoefer.pdf
Fischer, T.: Exakte Analyse von Heuristiken für kombinatorische Spiele. Ph.D. thesis, Fakultaet für Mathematik und Informatik, FSU Jena (2011). http://www.althofer.de/dissertation_thomas-fischer.pdf (in German language) Single step and double step races are covered in Chap. 2 of the dissertation
Nau, D.: Quality of decision versus depth of search on game trees. Ph.D. thesis, Duke University (1979)
Beal, D.: An analysis of minimax. In: Clarke, M. (ed.) Advances in Computer Chess, vol. 2, pp. 103–109. Edinburgh University Press, Edinburgh (1980)
Schrüfer, G.: Presence and absence of pathology on game trees. In: Beal, D. (ed.) Advances in Computer Chess, vol. 4, pp. 101–112. Elsevier, Amsterdam (1986)
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 235–256 (2002)
Kahn, J.: Personal communication with I. Althöfer (2013)
Klee, V., Minty, G.J.: How good is the simplex algorithm? In: Shisha, O. (ed.) Proceedings of the Third Symposium on Inequalities, pp. 159–175. Academic Press, New York-London (1972)
Finnsson, H., Björnsson, Y.: Game-tree properties and MCTS performance. In: IJCAI’11 Workshop on General Intelligence in Game Playing Agents, pp. 23–30 (2011)
Peres, Y., Schramm, O., Sheffield, S., Wilson, D.B.: Random-turn hex and other selection games. Amer. Math. Mon. 114, 373–387 (2007)
Wilson, D.B.: Hexamania - a computer program for playing random-turn hex, random-turn tripod, and the harmonic explorer game (2005). http://dbwilson.com/hex/
Brügmann, B.: Monte Carlo go (1993). Not published officially, but available at several places online including. http://www.althofer.de/Bruegmann-MonteCarloGo.pdf
Acknowledgments
This paper is dedicated to Prof. Bernd Brügmann. Twenty years ago in his report from 1993 [18] he introduced the approach of pure Monte-Carlo search to the game of Go and also created the name “Monte-Carlo Go”. Brügmann now holds a chair in theoretical physics for gravitational theory. His special topic is the understanding of mergers of black holes. This motivated us to call the jump-outs in our Double Step Races “black holes”.
Back in 2005, Jörg Sameith designed his beautiful tool McRandom, which allows one to design and test new board games with Monte-Carlo game search. With the help of McRandom we found the “DSR-3 with 16 Black Holes”. Also the graphic in the upper part of Fig. 1 is that of McRandom. In a workshop on search methodologies in September 2012, Soren Riis asked several questions on our preliminary anomaly result. This motivated us to do the research presented in this paper. Participants of the computer-go mailing list gave helpful feedback; here special thanks go to Jonas Kahn, Cameron Browne, and Stefan Kaitschick. Thanks to Matthias Beckmann and Michael Hartisch for proofreading an earlier version, and to the three anonymous referees for their constructive criticism and helpful proposals. Finally, thanks to Jaap van den Herik for his proposals in giving the paper a much clearer structure.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Althöfer, I., Turner, W.M. (2014). Anomalies of Pure Monte-Carlo Search in Monte-Carlo Perfect Games. In: van den Herik, H., Iida, H., Plaat, A. (eds) Computers and Games. CG 2013. Lecture Notes in Computer Science(), vol 8427. Springer, Cham. https://doi.org/10.1007/978-3-319-09165-5_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-09165-5_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09164-8
Online ISBN: 978-3-319-09165-5
eBook Packages: Computer ScienceComputer Science (R0)