It’s Time to Stop: A Comparison of Termination Conditions in the Evolution of Game Bots

  • A. Fernández-AresEmail author
  • P. García-Sánchez
  • Antonio M. Mora
  • Pedro A. Castillo
  • J. J. Merelo
  • María Isabel G. Arenas
  • Gustavo Romero
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9028)


Evolutionary Algorithms (EAs) are frequently used as a mechanism for the optimization of autonomous agents in games (bots), but knowing when to stop the evolution, when the bots are good enough, is not as easy as it would a priori seem. The first issue is that optimal bots are either unknown (and thus unusable as termination condition) or unreachable. In most EAs trying to find optimal bots fitness is evaluated through game playing. Many times it is found to be noisy, making its use as a termination condition also complicated. A fixed amount of evaluations or, in the case of games, a certain level of victories does not guarantee an optimal result. Thus the main objective of this paper is to test several termination conditions in order to find the one that yields optimal solutions within a restricted amount of time, and that allows researchers to compare different EAs as fairly as possible. To achieve this we will examine several ways of finishing an EA who is finding an optimal bot design process for a particular game, Planet Wars in this case, with the characteristics described above, determining the capabilities of every one of them and, eventually, selecting one for future designs.


Videogames RTS Evolutionary algorithms Termination criteria Noisy fitness 



This work has been supported in part by SIPESCA (Programa Operativo FEDER de Andalucía 2007–2013), TIN2011-28627-C04-02 (Spanish Ministry of Economy and Competitivity), SPIP2014-01437 (Dirección General de Tráfico), PRY142/14 (Fundación Pública Andaluza Centro de Estudios Andaluces en la IX Convocatoria de Proyectos de Investigación) and PYR-2014-17 GENIL project (CEI-BIOTIC Granada).


  1. 1.
    Small, R., Bates-Congdon, C.: Agent Smith: Towards an evolutionary rule-based agent for interactive dynamic games. In: IEEE Congress on Evolutionary Computation, CEC 2009, pp. 660–666 (2009)Google Scholar
  2. 2.
    Mora, A.M., Montoya, R., Merelo, J.J., Sánchez, P.G., Castillo, P.A., Laredo, J.L.J., Martínez, A.I., Espacia, A.: Evolving bot AI in unreal\(^{\rm TM}\). In: Di Chio, C., et al. (eds.) EvoApplicatons 2010, Part I. LNCS, vol. 6024, pp. 171–180. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  3. 3.
    Esparcia-Alcazár, A.I., Martínez-García, A., Mora, A.M., Merelo, J.J., García-Sánchez, P.: Genetic evolution of fuzzy finite state machines to control bots in a first-person shooter game. In: GECCO 2010: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 829–830. ACM (2010)Google Scholar
  4. 4.
    Fernández-Ares, A., García-Sánchez, P., Mora, A.M., Guervós, J.J.M.: Adaptive bots for real-time strategy games via map characterization. In: CIG, pp. 417–721. IEEE (2012)Google Scholar
  5. 5.
    Bäck, T.: Evolutionary algorithms in theory and practice. Oxford University Press, New York (1996)zbMATHGoogle Scholar
  6. 6.
    Roche, D., Gil, D., Giraldo, J.: Detecting loss of diversity for an efficient termination of eas. In: 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2013, Timisoara, Romania, 23–26 September, pp. 561–566. IEEE (2013)Google Scholar
  7. 7.
    Merelo, J.J., Castillo, P.A., Mora, A., Fernández-Ares, A., Esparcia-Alcázar, A.I., Cotta, C., Rico, N.: Studying and tackling noisy fitness in evolutionary design of game characters. In: Rosa, A., Merelo, J.J., Filipe, J. (eds.) ECTA 2014 - Proceedings of the International Conference on Evolutionary Computation Theory and Applications, pp. 76–85 (2014)Google Scholar
  8. 8.
    Mora, A.M., Fernández-Ares, A., Guervós, J.J.M., García-Sánchez, P., Fernandes, C.M.: Effect of noisy fitness in real-time strategy games player behaviour optimisation using evolutionary algorithms. J. CST 27(5), 1007–1023 (2012)Google Scholar
  9. 9.
    Fernández-Ares, A., Mora, A.M., Guervós, J.J.M., García-Sánchez, P., Fernandes, C.: Optimizing player behavior in a real-time strategy game using evolutionary algorithms. In: IEEE Conference on Evolutionary Computation, pp. 2017–2024. IEEE (2011)Google Scholar
  10. 10.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  11. 11.
    García-Sánchez, P., Fernández-Ares, A., Mora, A.M., Castillo, P.A., González, J., Guerv, J.J.M.: Tree depth influence in genetic programmingfor generation of competitive agentsfor RTS games. In: Esparcia-Alcázar, A.I., Mora, A.M. (eds.) EvoApplications 2014. LNCS, vol. 8602, pp. 409–419. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  12. 12.
    Lara-Cabrera, R., Cotta, C., Fernández-Leiva, A.: On balance and dynamism in procedural content generation with self-adaptive evolutionary algorithms. Nat. Comput. 13(2), 157–168 (2014)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Nogueira-Collazo, M., Fernández-Leiva, A.: Virtual player design using self-learning via competitive coevolutionary algorithms. Nat. Comput. 13(2), 131–144 (2014)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments - a survey. IEEE Trans. Evol. Comput. 9(3), 303–317 (2005)CrossRefGoogle Scholar
  15. 15.
    Qian, C., Yu, Y., Zhou, Z.H.: Analyzing evolutionary optimization in noisy environments. CoRR: abs/1311.4987 (2013).
  16. 16.
    Branke, J.: Creating robust solutions by means of evolutionary algorithms. In: Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, H.-P. (eds.) PPSN 1998. LNCS, vol. 1498, pp. 119–128. Springer, Heidelberg (1998) CrossRefGoogle Scholar
  17. 17.
    Branke, J.: Evolutionary optimization in dynamic environments, pp. 125–172. Kluwer Academic Publisher (2001)Google Scholar
  18. 18.
    Markon, S., Arnold, D., Bäck, T., Beielstein, T., Beyer, H.: Thresholding - a selection operator for noisy ES. In: Kim, J.-H., Zhang, B.-T., Fogel, G., Kuscu, I. (eds.) Proceedings of 2001 Congress on Evolutionary Computation (CEC 2001), pp. 465–472. IEEE Press, Piscataway (2001)Google Scholar
  19. 19.
    Bhattacharya, M., Islam, R., Mahmood, A.: Uncertainty and evolutionary optimization: A novel approach. CoRR abs/1407.4000 (2014)Google Scholar
  20. 20.
    Goel, T., Stander, N.: A non-dominance-based online stopping criterion for multi-objective evolutionary algorithms. Int. J. Numer. Meth. Eng. 84(6), 661–684 (2010)CrossRefzbMATHGoogle Scholar
  21. 21.
    Wagner, T., Trautmann, H.: Online convergence detection for evolutionary multi-objective algorithms revisited. In: Fogel, G., Ishibuchi, H. (eds.) Proceedings of International Congress on Evolutionary Computation (CEC 2010), pp. 3554–3561. IEEE press (2010)Google Scholar
  22. 22.
    Safe, M., Carballido, J.A., Ponzoni, I., Brignole, N.B.: On stopping criteria for genetic algorithms. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 405–413. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  23. 23.
    Rudolph, G.: Convergence analysis of canonical genetic algorithms. IEEE Trans. Neural Netw. 5, 96–101 (1994)CrossRefGoogle Scholar
  24. 24.
    Hart, W., Baden, S., Belew, R., Kohn, S.: Analysis of the numerical effects of parallelism on a parallel genetic algorithm. In: Proceedings of the 10th International Parallel Processing Symposium, pp. 606–612. IEEE Computer Society (1996)Google Scholar
  25. 25.
    Sena, G., Megherbi, D., Isern, G.: Implementation of a parallel genetic algorithm on a cluster of workstations: travelling salesman problem, a case study. Future Gener. Comput. Syst. 17, 477–488 (2001)CrossRefzbMATHGoogle Scholar
  26. 26.
    Esparcia-Alcázar, A.I., Moravec, J.: Fitness approximation for bot evolution in genetic programming. Soft Comput. 17(8), 1479–1487 (2013)CrossRefGoogle Scholar
  27. 27.
    Harper, R.: Evolving robocode tanks for Evo robocode. Genet. Programm. Evolvable Mach. 15(4), 403–431 (2014)CrossRefMathSciNetGoogle Scholar
  28. 28.
    Fernández Leiva, A.J., O’Valle Barragán, J.L.: Decision tree-based algorithms for implementing bot AI in UT2004. In: Ferrández, J.M., Álvarez Sánchez, J.R., de la Paz, F., Toledo, F.J. (eds.) IWINAC 2011, Part I. LNCS, vol. 6686, pp. 383–392. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  29. 29.
    Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • A. Fernández-Ares
    • 1
    Email author
  • P. García-Sánchez
    • 1
  • Antonio M. Mora
    • 1
  • Pedro A. Castillo
    • 1
  • J. J. Merelo
    • 1
  • María Isabel G. Arenas
    • 1
  • Gustavo Romero
    • 1
  1. 1.Department of Computer Architecture and TechnologyUniversity of GranadaGranadaSpain

Personalised recommendations