Skip to main content
SpringerLink
Log in

Virtual player design using self-learning via competitive coevolutionary algorithms

  • Published:
Natural Computing Aims and scope Submit manuscript

Abstract

The Google Artificial Intelligence (AI) Challenge is an international contest the objective of which is to program the AI in a two-player real time strategy (RTS) game. This AI is an autonomous computer program that governs the actions that one of the two players executes during the game according to the state of play. The entries are evaluated via a competition mechanism consisting of two-player rounds where each entry is tested against others. This paper describes the use of competitive coevolutionary (CC) algorithms for the automatic generation of winning game strategies in Planet Wars, the RTS game associated with the 2010 contest. Three different versions of a prime algorithm have been tested. Their common nexus is not only the use of a Hall-of-Fame (HoF) to keep note of the winners of past coevolutions but also the employment of an archive of experienced players, termed the hall-of-celebrities (HoC), that puts pressure on the optimization process and guides the search to increase the strength of the solutions; their differences come from the periodical updating of the HoF on the basis of quality and diversity metrics. The goal is to optimize the AI by means of a self-learning process guided by coevolutionary search and competitive evaluation. An empirical study on the performance of a number of variants of the proposed algorithms is described and a statistical analysis of the results is conducted. In addition to the attainment of competitive bots we also conclude that the incorporation of the HoC inside the primary algorithm helps to reduce the effects of cycling caused by the use of HoF in CC algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://wp.me/p2cObl-60.

  2. http://planetwars.aichallenge.org.

  3. http://anyself.wordpress.com/.

  4. http://dnemesis.lcc.uma.es/wordpress/.

References

  • Angeline PJ, Pollack JB (1993) Competitive environments evolve better solutions for complex tasks. In: Forrest S (ed) 5th International conference on genetic algorithms (ICGA93). Morgan Kaufmann, San Mateo, pp 264–270

  • Ashlock D, Ashlock W, Samothrakis S, Lucas SM, Lee C (2012) From competition to cooperation: co-evolution in a rewards continuum. In: IEEE conference on computational Intelligence and games (CIG 2012). IEEE, Granada, pp 33–40

  • Avery P, Louis SJ (2010) Coevolving team tactics for a real-time strategy game. In: IEEE congress on evolutionary computation (CEC 2010). IEEE, Barcelona, pp 1–8

  • Avery P et al (2008) Coevolving a computer player for resource allocation games: using the game of Tempo as a test space. PhD Thesis, School of Computer Science, University of Adelaide

  • Avery PM, Greenwood GW, Michalewicz Z (2008) Coevolving strategic intelligence. In: IEEE congress on evolutionary computation (CEC 2008). IEEE, Hong Kong, pp 3523–3530

  • Avery PM, Michalewicz Z (2007) Static experts and dynamic enemies in coevolutionary games. In: IEEE congress on evolutionary computation (CEC 2007). IEEE, pp 4035–4042

  • Buro M (2004) Call for AI research in RTS games. In: Fu D, Orkin J (eds) AAAI workshop on challenges in game AI, San Jose, pp 139–141

  • Cook M, Colton S, Gow J (2012) Initial results from co-operative co-evolution for automated platformer design. In: Chio CD, Agapitos A, Cagnoni S, Cotta C, de Vega FF, Caro GAD, Drechsler R, Ekárt A, Esparcia-Alcázar AI, Farooq M, Langdon WB, Guervós JJM, Preuss M, Richter H, Silva S, Simões A, Squillero G, Tarantino E, Tettamanzi A, Togelius J, Urquhart N, Uyar S, Yannakakis GN (eds) Applications of evolutionary computation—EvoApplications (EvoGAMES). Lecture notes in computer science, vol 7248. Springer, Berlin, pp 194–203

  • Corruble V, Madeira CAG, Ramalho G (2002) Steps toward building of a good AI for complex Wargame-Type simulation games. In: Mehdi QH, Gough NE (eds) 3rd International conference on intelligent games and simulation (GAME-ON 2002), London, pp 155–159

  • de Jong E (2004) Towards a bounded Pareto-Coevolution archive. In: Congress on evolutionary computation (CEC2004), vol 2. IEEE, Portland, pp 2341–2348

  • Dziuk A, Miikkulainen R (2011) Creating intelligent agents through shaping of coevolution. In: IEEE congress on evolutionary computation (CEC 2011), IEEE, New Orleans, pp 1077–1083

  • Ebner M, Watson RA, Alexander J (2010) Coevolutionary dynamics of interacting species. In: Chio CD, Cagnoni S, Cotta C, Ebner M, Ekárt A, Esparcia-Alcázar A, Goh CK, Guervós JJM, Neri F, Preuss M, Togelius J, Yannakakis GN (eds) Applications of evolutionary computation, EvoApplicatons 2010 (EvoApplications (1)). Lecture notes in computer science, vol 6024. Springer, Berlin, pp 1–10

  • Fernández-Ares A, Mora AM, Guervós JJM, García-Sánchez P, Fernandes C (2011) Optimizing player behavior in a real-time strategy game using evolutionary algorithms. In: IEEE congress on evolutionary computation (CEC 2011), IEEE, New Orleans, pp 2017–2024

  • Ficici SG, Bucci A (2007) Advanced tutorial on coevolution. In: 2007 GECCO conference companion on genetic and evolutionary computation. ACM, New York, pp 3172–3204

  • Gutiérrez JAG, Cotta C, Fernández Leiva AJ (2011) Design of emergent and adaptive virtual players in a war RTS game. In: Ferrández JM, Sánchez JRÁ, de la Paz F, Toledo FJ (eds) 4th International work-conference on the interplay between natural and artificial computation (IWINAC 2011). Lecture notes in computer science, vol 6686. Springer, La Palma, pp 372–382

  • Jaskowski W, Krawiec K (2010) Coordinate system archive for coevolution. In: IEEE congress on evolutionary computation (CEC 2010), IEEE, Barcelona, pp 1–10

  • Johnson RW, Melich ME, Michalewicz Z, Schmidt M (2004) Coevolutionary Tempo game. In: Congress on evolutionary computation (CEC 2004), vol 2, IEEE, Portland, pp 1610–1617

  • Kruskal W, Wallis W (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621

    Article  MATH  Google Scholar 

  • Lichocki P (2008) Evolving players for a real-time strategy game using gene expression programming. Master Thesis, Poznan University of Technology

  • Lidén L (2004) Artificial stupidity: the art of intentional mistakes. In: AI game programming wisdom 2. Charles River Media, Inc., Hingham, pp 41–48

  • Livingstone D (2005) Coevolution in hierarchical ai for strategy games. In: IEEE conference on computational intelligence and games (CIG 2005). IEEE, Colchester, Essex

  • Lucas SM (2009) Computational intelligence and AI in games: a new IEEE transactions. IEEE Trans Comput Intell AI Games 1(1):1–3

    Article  Google Scholar 

  • Lucas SM, Mateas M, Preuss M, Spronck P, Togelius J (2012) Artificial and computational intelligence in games (Dagstuhl seminar 12191). Dagstuhl Rep 2(5):43–70

    Google Scholar 

  • Miconi T (2009) Why coevolution doesn’t “Work”: superiority and progress in coevolution. In: Vanneschi L, Gustafson S, Moraglio A, Falco ID, Ebner M (eds) Genetic programming, 12th European conference (EuroGP 2009). Lecture notes in computer science, vol 5481. Springer, Berlin, pp 49–60

  • Miles C, Louis SJ (2006) Co-evolving real-time strategy game playing influence map trees with genetic algorithms. In: International congress on evolutionary computation (CEC 2006). IEEE, Vancouver

  • Mora AM, Fernández-Ares A, Guervós JJM, García-Sánchez P, Fernandes CM (2012) Effect of noisy fitness in real-time strategy games player behaviour optimisation using evolutionary algorithms. J Comput Sci Technol 27(5):1007–1023

    Article  Google Scholar 

  • Nerome M, Yamada K, Endo S, Miyagi H (1998) Competitive co-evolution based game-strategy acquisition with the packaging. In: Jain LC, Jain RK (eds) Knowledge-based intelligent electronic systems, 2nd international conference (KES (3)). IEEE, Adelaide, pp 184–189

  • Nogueira M, Gálvez J, Cotta C, Fernández-Leiva A (2012) Hall of Fame based competitive coevolutionary algorithms for optimizing opponent strategies in a new RTS game. In: Fernández-Leiva A et al A (ed) 13th Annual European conference on simulation and AI in computer games (GAME-ON 2012). Eurosis, Málaga, pp 71–78

  • Nogueira M, Gálvez J, Cotta C, Fernández-Leiva A (2013) An analysis of hall-of-fame strategies in competitive coevolutionary algorithms for self-learning in RTS games. In: International conference on learning and intelligent optimization (LION7), LNCS. Springer, Catania

  • Reynolds C (1994) Competition, coevolution and the game of tag. In: Brooks R, Maes P (eds) Proceedings of artificial life IV. MIT Press, Cambridge, pp 59–69

    Google Scholar 

  • Rosin C, Belew R (1997) New methods for competitive coevolution. Evol Comput 5(1):1–29

    Article  Google Scholar 

  • Rosin CD, Belew RK (1995) Methods for competitive co-evolution: finding opponents worth beating. In: Eshelman LJ (ed) 6th International conference on genetic algorithms (ICGA). Morgan Kaufmann, San Mateo, pp 373–381

  • Samothrakis S, Lucas SM, Runarsson TP, Robles D (2013) Coevolving game-playing agents: measuring performance and intransitivities. IEEE Trans Evol Comput 17(2):213–226

    Article  Google Scholar 

  • Sims K (1994) Evolving 3D morphology and behavior by competition. Artif Life 1(4):353–372

    Article  Google Scholar 

  • Smith G, Avery P, Houmanfar R, Louis SJ (2010) Using co-evolved RTS opponents to teach spatial tactics. In: Yannakakis GN, Togelius J (eds) IEEE conference on computational intelligence and games (CIG 2010). IEEE, Copenhagen, pp 146–153

  • Sokal Robert R, Rohlf James F (1995) Biometry: the principles and practice of statistics in biological research. W.H. Freeman and Company, New York

    Google Scholar 

  • Szita I, Ponsen MJV, Spronck P (2009) Effective and diverse adaptive game AI. IEEE Trans Comput Intell AI Games 1(1):16–27

    Article  Google Scholar 

  • Togelius J, Yannakakis GN, Stanley KO, Browne C (2011) Search-based procedural content generation: a taxonomy and survey. IEEE Trans Comput Intell AI Games 3(3):172–186

    Article  Google Scholar 

Download references

Acknowledgments

This work is partially supported by Spanish MICINN under Project ANYSELF (TIN2011-28627-C04-01),Footnote 3 by Junta de Andalucía under Project P10-TIC-6083 (DNEMESIS)Footnote 4 and by Universidad de Málaga, Campus de Excelencia Internacional Andalucía Tech.

Author information

Authors and Affiliations

  1. University of Computer Science, Havana, Cuba

    Mariela Nogueira Collazo

  2. University of Málaga, Málaga, Spain

    Carlos Cotta & Antonio J. Fernández-Leiva

Authors
  1. Mariela Nogueira Collazo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Cotta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio J. Fernández-Leiva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Cotta.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nogueira Collazo, M., Cotta, C. & Fernández-Leiva, A.J. Virtual player design using self-learning via competitive coevolutionary algorithms. Nat Comput 13, 131–144 (2014). https://doi.org/10.1007/s11047-014-9411-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11047-014-9411-3

Keywords

Search

Navigation