Skip to main content

Strength Improvement and Analysis for an MCTS-Based Chinese Dark Chess Program

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 9525)

Abstract

Monte-Carlo tree search (MCTS) has been successfully applied to Chinese dark chess (CDC). In this paper, we study how to improve and analyze the playing strength of an MCTS-based CDC program, named DarkKnight, which won the CDC tournament in the 17th Computer Olympiad. We incorporate the three recent techniques, early playout terminations, implicit minimax backups, and quality-based rewards, into the program. For early playout terminations, playouts end when reaching states with likely outcomes. Implicit minimax backups use heuristic evaluations to help guide selections of MCTS. Quality-based rewards adjust rewards based on online collected information. Our experiments showed that the win rates against the original DarkKnight were 60.75 %, 70.90 % and 59.00 %, respectively for incorporating the three techniques. By incorporating all together, we obtained a win rate of 76.70 %.

Keywords

  • Monte Carlo Tree Search (MCTS)
  • Playout
  • Dark Knight
  • Heuristic Evaluation
  • Chance Nodes

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-27992-3_4
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-27992-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   69.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

References

  1. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)

    CrossRef  MATH  Google Scholar 

  2. Baier, H., Winands, M.H.: Monte-Carlo tree search and minimax hybrids with heuristic evaluation functions. In: Cazenave, T., Winands, M.H., Björnsson, Y. (eds.) CGW 2014. CCIS, vol. 504, pp. 45–63. Springer, Heidelberg (2014)

    CrossRef  Google Scholar 

  3. Björnsson, Y., Finnsson, H.: CadiaPlayer: a simulation-based general game player. IEEE Trans. Comput. Intell. AI Games 1(1), 4–15 (2009)

    CrossRef  Google Scholar 

  4. Borsboom, J., Saito, J.-T., Chaslot, G., Uiterwijk, J.: A comparison of Monte-Carlo methods for phantom go. In: Proceedings of BeNeLux Conference on Artificial Intelligence, Utrecht, The Netherlands, pp. 57–64 (2007)

    Google Scholar 

  5. Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S.: A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)

    CrossRef  Google Scholar 

  6. Chang, H.-J., Hsu, T.-S.: A quantitative study of 2 × 4 Chinese dark chess. In: van den Herik, H., Iida, H., Plaat, A. (eds.) CG 2013. LNCS, vol. 8427, pp. 151–162. Springer, Heidelberg (2014)

    Google Scholar 

  7. Chen, B.-N., Hsu, T.-S.: Automatic generation of opening books for dark chess. In: van den Herik, H., Iida, H., Plaat, A. (eds.) CG 2013. LNCS, vol. 8427, pp. 221–232. Springer, Heidelberg (2014)

    Google Scholar 

  8. Chen, B.-N., Shen, B.-J., Hsu, T.-S.: Chinese dark chess. ICGA J. 33(2), 93–106 (2010)

    CrossRef  Google Scholar 

  9. Chen, J.-C., Lin, T.-Y., Chen, B.-N., Hsu, T.-S.: Equivalence classes in chinese dark chess endgames. IEEE Trans. Comput. Intell. AI Games 7(2), 109–122 (2015)

    CrossRef  Google Scholar 

  10. Chen, J.-C., Lin, T.-Y., Hsu, S.-C., Hsu, T.-S.: Design and implementation of computer Chinese dark chess endgame database. In: Proceeding of TCGA Workshop 2012, pp. 5–9, Hualien, Taiwan (2012) (in Chinese)

    Google Scholar 

  11. Enzenberger, M., Müller, M., Arneson, B., Segal, R.: Fuego: an open-source framework for board games and go engine based on Monte Carlo tree search. IEEE Trans. Comput. Intell. AI Games 2(4), 259–270 (2010)

    CrossRef  Google Scholar 

  12. Finnsson, H.: Generalized Monte-Carlo tree search extensions for general game playing. In: The Twenty-Sixth AAAI Conference on Artificial Intelligence, pp. 1550–1556, Toronto, Canada (2012)

    Google Scholar 

  13. Gelly, S., Silver, D.: Monte-Carlo tree search and rapid action value estimation in computer go. Artif. Intell. 175(11), 1856–1875 (2011)

    MathSciNet  CrossRef  Google Scholar 

  14. Gelly, S., Wang, Y., Munos, R., Teytaud, O.: Modification of UCT with patterns in Monte-Carlo go. Technical report, HAL - CCSd - CNRS, France (2006)

    Google Scholar 

  15. Jouandeau, N.: Varying complexity in CHINESE DARK CHESS stochastic game. In: Proceeding of TCGA Workshop 2014, pp. 86, Taipei, Taiwan (2014)

    Google Scholar 

  16. Jouandeau, N., Cazenave, T.: Monte-Carlo tree reductions for stochastic games. In: Cheng, S.-M., Day, M.-Y. (eds.) TAAI 2014. LNCS, vol. 8916, pp. 228–238. Springer, Heidelberg (2014)

    CrossRef  Google Scholar 

  17. Jouandeau, N., Cazenave, T.: Small and large MCTS playouts applied to Chinese dark chess stochastic game. In: Cazenave, T., Winands, M.H., Björnsson, Y. (eds.) CGW 2014. CCIS, vol. 504, pp. 78–89. Springer, Heidelberg (2014)

    CrossRef  Google Scholar 

  18. Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)

    CrossRef  Google Scholar 

  19. Lanctot, M., Winands, M.H.M., Pepels, T., Sturtevant, N.R.: Monte Carlo tree search with heuristic evaluations using implicit minimax backups. In: 2014 IEEE Conference on Computational Intelligence and Games, CIG 2014, pp. 1–8 (2014)

    Google Scholar 

  20. Lin, Y.-S., Wu, I.-C., Yen, S.-J.: TAAI 2011 computer-game tournaments. ICGA J. 34(4), 248–250 (2011)

    CrossRef  Google Scholar 

  21. Lorentz, R.J.: Amazons discover Monte-Carlo. In: van den Herik, H., Xu, X., Ma, Z., Winands, M.H. (eds.) CG 2008. LNCS, vol. 5131, pp. 13–24. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  22. Lorentz, R.: Early playout termination in MCTS. In: The 14th Conference on Advances in Computer Games (ACG2015), Leiden, The Netherlands (2015)

    Google Scholar 

  23. Lorentz, R., Horey, T.: Programming breakthrough. In: van den Herik, H., Iida, H., Plaat, A. (eds.) CG 2013. LNCS, vol. 8427, pp. 49–59. Springer, Heidelberg (2014)

    Google Scholar 

  24. Pepels, T., Tak, M.J., Lanctot, M., Winands, M.H.M.: Quality-based rewards for Monte-Carlo tree search simulations. In: 21st European Conference on Artificial Intelligence, Prague, Czech Republic (2014)

    Google Scholar 

  25. Saffidine, A., Jouandeau, N., Buron, C., Cazenave, T.: Material symmetry to partition endgame tables. In: van den Herik, H., Iida, H., Plaat, A. (eds.) CG 2013. LNCS, vol. 8427, pp. 187–198. Springer, Heidelberg (2014)

    Google Scholar 

  26. Su, T.-C., Yen, S.-J., Chen, J.-C., Wu, I.-C.: TAAI 2012 computer game tournaments. ICGA J. 37(1), 33–35 (2014)

    CrossRef  Google Scholar 

  27. Theory of computer games, a course in National Taiwan University taught by Tsu, T.-S. http://www.iis.sinica.edu.tw/~tshsu/tcg/index.html

  28. Tseng, W.-J., Chen, J.-C., Chen, L.-P., Yen, S.-J., Wu, I.-C.: TCGA 2013 computer game tournament report. ICGA J. 36(3), 166–168 (2013)

    CrossRef  Google Scholar 

  29. Van Lishout, F., Chaslot, G., Uiterwijk, J.W.: Monte-Carlo tree search in Backgammon. In: Computer Games Workshop, pp. 175–184, Amsterdam, The Netherlands (2007)

    Google Scholar 

  30. Winands, M.H.M., Björnsson, Y., Saito, J.-T.: Monte Carlo tree search in lines of action. IEEE Trans. Comput. Intell. AI Games 2(4), 239–250 (2010)

    CrossRef  Google Scholar 

  31. Winands, M.H., Björnsson, Y., Saito, J.-T.: Monte-Carlo tree search solver. In: van den Herik, H., Xu, X., Ma, Z., Winands, M.H. (eds.) CG 2008. LNCS, vol. 5131, pp. 25–36. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  32. Yang, J.-K., Su, T.-C., Wu, I.-C.: TCGA 2012 computer game tournament report. ICGA J. 35(3), 178–180 (2012)

    CrossRef  Google Scholar 

  33. Yen, S.-J., Chou, C.-W., Chen, J.-C., Wu, I.-C., Kao, K.-Y.: Design and implementation of Chinese dark chess programs. IEEE Trans. Comput. Intell. AI Games 7(1), 66–74 (2015)

    CrossRef  Google Scholar 

  34. Yen, S.-J., Chen, J.-C., Chen, B.-N., Tseng, W.-J.: DarkKnight wins Chinese dark chess tournament. ICGA J. 36(3), 175–176 (2013)

    CrossRef  Google Scholar 

  35. Yen, S.-J., Su, T.-C., Wu, I.-C.: The TCGA 2011 computer-games tournament. ICGA J. 34(2), 108–110 (2011)

    CrossRef  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Ministry of Science and Technology of the Republic of China (Taiwan) for financial support of this research under contract numbers MOST 102-2221-E-009-069-MY2, 102-2221-E-009-080-MY2, 104-2221-E-009-127-MY2, and 104-2221-E-009-074-MY2.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to I-Chen Wu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Hsueh, CH., Wu, IC., Tseng, WJ., Yen, SJ., Chen, JC. (2015). Strength Improvement and Analysis for an MCTS-Based Chinese Dark Chess Program. In: Plaat, A., van den Herik, J., Kosters, W. (eds) Advances in Computer Games. ACG 2015. Lecture Notes in Computer Science(), vol 9525. Springer, Cham. https://doi.org/10.1007/978-3-319-27992-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27992-3_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27991-6

  • Online ISBN: 978-3-319-27992-3

  • eBook Packages: Computer ScienceComputer Science (R0)