Skip to main content
SpringerLink
Log in

Learning to Trade in Strategic Board Games

  • Conference paper
  • First Online:
Book cover Computer Games (CGW 2015, GIGA 2015)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 614))

  • 595 Accesses

Abstract

Automated agents in multiplayer board games often need to trade resources with their opponents—and trading strategically can lead to higher winning rates. While rule-based agents can be used for such a purpose, here we opt for a data-driven approach based on examples from human players for automatic trading in the game “Settlers of Catan”. Our experiments are based on data collected from human players trading in text-based Natural Language. We compare the performance of Bayesian Networks, Conditional Random Fields, and Random Forests in the task of ranking trading offers, and evaluate them both in an offline setting and online while playing the game against a rule-based baseline. Experimental results show that agents trained from data from average human players can outperform rule-based trading behavior, and that the Random Forest model achieves the best results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The baseline agent (referred to as ‘rule-based’) included the following parameters in all agents, see [16] for further details: TRY_N_BEST_BUILD_PLANS:0, FAVOUR_DEV_CARDS:-5.

References

  1. Afantenos, S., Asher, N., Benamara, F., Cadilhac, A., Dégremont, C., Denis, P., Guhe, M., Keizer, S., Lascarides, A., Lemon, O., Muller, P., Paul, S., Rieser, V., Vieu, L.: Developing a corpus of strategic conversation in the Settlers of Catan. In: Workshop on the Semantics and Pragmatics of Dialogue SeineDial, Paris, France (2012). https://hal.inria.fr/hal-00750618

  2. Asher, N., Lascarides, A.: Commitments, beliefs and intentions in dialogue. In: Proceedings of SemDial, pp. 35–42 (2008)

    Google Scholar 

  3. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  4. Choi, J., Park, J., Park, H., Park, J.I.: iHand: an interactive bare-hand-based augmented reality interface on commercial mobile phones. Opt. Eng. 52(2), 027206 (2013)

    Article  MathSciNet  Google Scholar 

  5. Cooper, G., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)

    MATH  Google Scholar 

  6. Cozman, F.G.: Generalizing variable elimination in Bayesian networks. In: Workshop on Probabilistic Reasoning in Artificial Intelligence, pp. 27–32 (2000)

    Google Scholar 

  7. Criminisi, A., Shotton, J., Konukoglu, E.: Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found. Trends Comput. Graph. Vis. 7(2–3), 81–227 (2012)

    MATH  Google Scholar 

  8. Cuayáhuitl, H., van Otterlo, M., Dethlefs, N., Frommberger, L.: Machine learning for interactive systems and robots: a brief introduction. In: 2nd Workshop on Machine Learning for Interactive Systems (MLIS), pp. 19–28. ACM (2013)

    Google Scholar 

  9. Dethlefs, N., Cuayáhuitl, H.: Hierarchical reinforcement learning for situated natural language generation. Nat. Lang. Eng. 21(3), 391–435 (2015)

    Article  Google Scholar 

  10. Efstathiou, I., Lemon, O.: Learning to manage risk in non-cooperative dialogues. In: Proceedings of SEMDIAL (2014)

    Google Scholar 

  11. Efstathiou, I., Lemon, O.: Learning non-cooperative dialogue behaviours. In: SIGDIAL (2014)

    Google Scholar 

  12. Efstathiou, I., Lemon, O.: Learning non-cooperative dialogue policies to beat opponent models: ‘the good, the bad and the ugly’. In: Proceedings of SEMDIAL (2015)

    Google Scholar 

  13. Fürnkranz, J.: Machine learning in games: a survey. In: Machines that Learn to Play Games, Chapter 2, pp. 11–59. Nova Science Publishers (2000)

    Google Scholar 

  14. Georgila, K., Nelson, C., Traum, D.: Single-agent vs. multi-agent techniques for concurrent reinforcement learning of negotiation dialogue policies. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, USA, pp. 500–510, September 2014

    Google Scholar 

  15. Georgila, K., Traum, D.: Reinforcement learning of argumentation dialogue policies in negotiation. In: Proceedings of INTERSPEECH (2011)

    Google Scholar 

  16. Guhe, M., Lascarides, A.: Game strategies for the Settlers of Catan. In: 2014 IEEE Conference on Computational Intelligence and Games, CIG 2014, Dortmund, Germany, 26–29 August 2014, pp. 1–8 (2014)

    Google Scholar 

  17. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference and Prediction, 2nd edn. Springer, New York (2009)

    Book  MATH  Google Scholar 

  18. Hiraoka, T., Georgila, K., Nouri, E., Traum, D., Nakamura, S.: Reinforcement learning in multi-party trading dialog. In: Proceedings of the SIGDIAL 2015 Conference, Prague, Czech Republic, pp. 32–41, September 2015

    Google Scholar 

  19. Kudo, T.: CRF++: Yet another CRF toolkit (2005). http://crfpp.sourceforge.net

  20. Lemon, O.: Adaptive natural language generation in dialogue using reinforcement learning. In: Proceedings of SEMDIAL (2008)

    Google Scholar 

  21. Maddison, C.J., Huang, A., Sutskever, I., Silver, D.: Move evaluation in go using deep convolutional neural networks. CoRR abs/1412.6564 (2014)

    Google Scholar 

  22. McFarlin, M.: 10 great board games for traders. Futures Magazine (2013). http://www.futuresmag.com/2013/10/02/10-great-board-games-for-traders

  23. Pfeiffer, M.: Reinforcement learning of strategies for Settlers of Catan. In: International Conference on Computer Games: Artificial Intelligence, Design and Education (2004)

    Google Scholar 

  24. Pietquin, O., Lopez, M.: Machine learning for interactive systems: challenges and future trends. In: Proceedings of the Workshop Affect, Compagnon Artificiel (WACAI) (2014)

    Google Scholar 

  25. Runarsson, T.P., Lucas, S.M.: Preference learning for move prediction and evaluation function approximation in Othello. IEEE Trans. Comput. Intell. AI Games 6(3), 300–313 (2014)

    Article  Google Scholar 

  26. Shim, J., Arkin, R.: A taxonomy of robot deception and its benefits in HRI. In: Proceedings of IEEE Systems, Man and Cybernetics Conference (2013)

    Google Scholar 

  27. Szita, I., Chaslot, G., Spronck, P.: Monte-Carlo tree search in Settlers of Catan. In: van den Herik, H.J., Spronck, P. (eds.) ACG 2009. LNCS, vol. 6048, pp. 21–32. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  28. Tesauro, G.: Temporal difference learning and TD-Gammon. Commun. ACM 38(3), 58–68 (1995)

    Article  Google Scholar 

  29. Thomas, R., Hammond, K.J.: Java settlers: a research environment for studying multi-agent negotiation. In: Intelligent User Interfaces (IUI), p. 240 (2002)

    Google Scholar 

  30. Traum, D.: Extended abstract: computational models of non-cooperative dialogue. In: Proceedings of SIGdial Workshop on Discourse and Dialogue (2008)

    Google Scholar 

Download references

Acknowledgments

Funding from the European Research Council (ERC) project “STAC: Strategic Conversation” no. 269427 is gratefully acknowledged. See http://www.irit.fr/STAC/. We would like to thank the following members of the STAC project for helpful discussions: Markus Guhe, Eric Kow, Mihai Dobre, Ioannis Efstathiou, Verena Rieser, Alex Lascarides, and Nicholas Asher.

Author information

Authors and Affiliations

  1. School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, UK

    Heriberto Cuayáhuitl, Simon Keizer & Oliver Lemon

Authors
  1. Heriberto Cuayáhuitl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Keizer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oliver Lemon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heriberto Cuayáhuitl .

Editor information

Editors and Affiliations

  1. Université Paris-Dauphine, Paris, France

    Tristan Cazenave

  2. Maastricht University, Maastricht, The Netherlands

    Mark H.M. Winands

  3. Universität Bremen, Bremen, Bremen, Germany

    Stefan Edelkamp

  4. Reykjavik University, Reykjavik, Iceland

    Stephan Schiffel

  5. The University of New South Wales, Sydney, New South Wales, Australia

    Michael Thielscher

  6. New York University, Brooklyn, New York, USA

    Julian Togelius

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Cuayáhuitl, H., Keizer, S., Lemon, O. (2016). Learning to Trade in Strategic Board Games. In: Cazenave, T., Winands, M., Edelkamp, S., Schiffel, S., Thielscher, M., Togelius, J. (eds) Computer Games. CGW GIGA 2015 2015. Communications in Computer and Information Science, vol 614. Springer, Cham. https://doi.org/10.1007/978-3-319-39402-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-39402-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-39401-5

  • Online ISBN: 978-3-319-39402-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation