Abstract
Today planning algorithms are among the most sought after. One of the main such algorithms is Monte Carlo Tree Search. However, this architecture is complex in terms of parallelization and development. We presented possible approximations for the MCTS algorithm, which allowed us to significantly increase the learning speed of the agent.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sutton, R.S.: Dyna, an integrated architecture for learning, planning, and reacting. SIGART Bull. 2(4), 160–163 (1991)
Coulom, R.: Efficient selectivity and backup operators in monte-carlo tree search. In: International Conference on Computers and Games, pp. 72–83. Springer (2006)
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al.: Mastering the game of go without human knowledge. Nature 550(7676), 354 (2017)
Kocsis, L., Szepesvári, C.: Bandit based monte-carlo planning. In: European Conference on Machine Learning, pp. 282–293. Springer (2006)
Gelly, S., Silver, D.: Monte-carlo tree search and rapid action value estimation in computer go. Artif. Intell. 175(11), 1856–1875 (2011)
Lecarpentier, E., Infantes, G., Lesire, C., Rachelson, E.: Open loop execution of tree-search algorithms. arXiv preprint arXiv:1805.01367 (2018)
Guez, A., Weber, T., Antonoglou, I., Simonyan, K., Vinyals, O., Wierstra, D., Munos, R., Silver, D.: Learning to search with MCTSNets. arXiv preprint arXiv:1802.04697 (2018)
Racanière, S., Weber, T., Reichert, D., Buesing, L., Guez, A., Rezende, D.J., Badia, A.P., Vinyals, O., Heess, N., Li, Y., et al.: Imagination-augmented agents for deep reinforcement learning. In: Advances in Neural Information Processing Systems, pp. 5690–5701 (2017)
Chaslot, G.M.J.-B., Winands, M.H.M., van Den Herik, H.J.: Parallel monte-carlo tree search. In: International Conference on Computers and Games, pp. 60–71. Springer (2008)
Ali Mirsoleimani, S., Plaat, A., van den Herik, J., Vermaseren, J.: A new method for parallel Monte Carlo tree search. arXiv preprint arXiv:1605.04447 (2016)
Schrader, M.-P.B.: gym-sokoban (2018). https://github.com/mpSchrader/gym-sokoban
Edelkamp, S., Gath, M., Greulich, C., Humann, M., Herzog, O., Lawo, M.: Monte-Carlo tree search for logistics. In: Commercial Transport, pp. 427–440. Springer (2016)
Acknowledgements
The reported study was supported by RFBR, research Projects No. 17-29-07079 and No. 18-29-22047.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Aksenov, K., Panov, A.I. (2020). Approximation Methods for Monte Carlo Tree Search. In: Kovalev, S., Tarassov, V., Snasel, V., Sukhanov, A. (eds) Proceedings of the Fourth International Scientific Conference “Intelligent Information Technologies for Industry” (IITI’19). IITI 2019. Advances in Intelligent Systems and Computing, vol 1156. Springer, Cham. https://doi.org/10.1007/978-3-030-50097-9_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-50097-9_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50096-2
Online ISBN: 978-3-030-50097-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)