Abstract
We study the multi-armed bandit problem, where the aim is to minimize the simple regret with a fixed budget. The Sequential Halving algorithm is known to tackle it efficiently. We present a more elaborate version of this algorithm to integrate some exterior knowledge or “scores”, that can for instance be provided by a neural network or a heuristic such as all-moves-as-first (AMAF) in the context of a Monte-Carlo Tree Search. We provide both theoretical justifications and experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bouzy, B., Helmstetter, B.: Monte-Carlo Go developments. In: ACG, pp. 159–174 (2003)
Cazenave, T.: Generalized rapid action value estimation. In: IJCAI, pp. 754–760 (2015)
Cazenave, T.: Sequential halving applied to trees. IEEE Trans. Comput. Intell. Games 7(1), 102–105 (2015)
Cazenave, T.: Improving model and search for computer Go. In: IEEE CoG (2021)
Cazenave, T.: Mobile networks for computer Go. IEEE Trans. Games 14, 76–84 (2021)
Coulom, R.: Efficient selectivity and backup operators in Monte-Carlo tree search. In: Computers and Games, pp. 72–83 (2006)
Sunklodas, J.K.: On the normal approximation of a binomial random sum. Lithuanian Mathematical Journal 54(3), 356–365 (2014). https://doi.org/10.1007/s10986-014-9248-6
Gelly, S., Silver, D.: Combining online and offline knowledge in UCT. In: ICML, pp. 273–280 (2007)
Gelly, S., Silver, D.: Monte-Carlo tree search and rapid action value estimation in computer Go. Artif. Intell. 175(11), 1856–1875 (2011)
Karnin, Z.S., Koren, T., Somekh, O.: Almost optimal exploration in multi-armed bandits. In: ICML, pp. 1238–1246 (2013)
Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: ECML, pp. 282–293 (2006)
Pepels, T., Cazenave, T., Winands, M.H.M., Lanctot, M.: Minimizing simple and cumulative regret in Monte-Carlo tree search. In: Cazenave, T., Winands, M.H.M., Björnsson, Y. (eds.) CGW 2014. CCIS, vol. 504, pp. 1–15. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-14923-3_1
Pepels, T., Cazenave, T., Winands, M.H.M.: Sequential Halving for Partially Observable Games. In: Cazenave, T., et al. (eds.) CGW/GIGA -2015. CCIS, vol. 614, pp. 16–29. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39402-2_2
Silver, D., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Acknowledgment
This work was supported in part by the French government under the management of the Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR19-P3IA-0001 (PRAIRIE 3IA Institute).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Fabiano, N., Cazenave, T. (2022). Sequential Halving Using Scores. In: Browne, C., Kishimoto, A., Schaeffer, J. (eds) Advances in Computer Games. ACG 2021. Lecture Notes in Computer Science, vol 13262. Springer, Cham. https://doi.org/10.1007/978-3-031-11488-5_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-11488-5_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11487-8
Online ISBN: 978-3-031-11488-5
eBook Packages: Computer ScienceComputer Science (R0)