Soft Computing

, Volume 20, Issue 2, pp 607–620 | Cite as

Training multi-agent teams from zero knowledge with the competitive coevolutionary team-based particle swarm optimiser

  • Christiaan Scheepers
  • Andries P. EngelbrechtEmail author
Methodologies and Application


A new competitive coevolutionary team-based particle swarm optimiser (CCPSO(t)) algorithm is developed to train multi-agent teams from zero knowledge. The CCPSO(t) algorithm is applied to train a team of agents to play simple soccer. The algorithm uses the charged particle swarm optimiser in a competitive and cooperative coevolutionary training environment to train neural network controllers for the players. The CCPSO(t) algorithm makes use of the FIFA league ranking relative fitness function to gather detailed performance metrics from each game played. The training performance and convergence behaviour of the particle swarm are analysed. A hypothesis is presented that explains the lack of convergence in the particle swarms. After applying a clustering algorithm on the particle positions, a detailed visual and quantitative analysis of the player strategies is presented. The final results show that the CCPSO(t) algorithm is capable of evolving complex gameplay strategies for a complex non-deterministic game.


Cooperative coevolution Competitive coevolution Neural networks Charged particle swarm optimiser Zero knowledge Multi-agent system Simple soccer 


  1. Blackwell T (2003) Swarms in dynamic environments. In: Proceedings of the 2003 genetic and evolutionary computation conference, Springer, Berlin, pp 1–12Google Scholar
  2. Blackwell T, Bentley P (2002a) Dont push me! Collision-avoiding swarms. In: Proceedings of the 2002 congress on evolutionary computation, vol 2, pp 1691–1696Google Scholar
  3. Blackwell T, Bentley P (2002b) Dynamic search with charged swarms. In: Proceedings of the 2002 genetic and evolutionary computation conference. Morgan Kaufmann Publishers, Los Altos, pp 19–26Google Scholar
  4. Conradie J, Engelbrecht A (2006) Training bao game-playing agents using coevolutionary particle swarm optimization. In: Proceedings of the 2006 IEEE symposium on neural networks, pp 67–74, doi: 10.1109/CIG.2006.311683
  5. Davis D, Chalabi T, Berbank-Green B (2000) Artificial-life, agents and GO. In: Mohammadian M (ed) New frontiers in computational intelligence and its applications. IOS Press, Amsterdam, pp 125–139Google Scholar
  6. Franken C (2009) Visual exploration of algorithm parameter space. In: Proceedings of IEEE congress on evolutionary computation pp 389–398. doi: 10.1109/CEC.2009.4982973
  7. Franken C, Engelbrecht A (2003a) Comparing PSO structures to learn the game of checkers from zero knowledge. In: Proceedings of the 2003 congress on evolutionary computation, pp 234–241Google Scholar
  8. Franken C, Engelbrecht A (2003b) Evolving intelligent game-playing agents. In: Proceedings of the 2003 annual research conference of the South African institute of computer scientists and information technologists on enablement through technology, South African Institute for Computer Scientists and Information Technologists, pp 102–110Google Scholar
  9. Franken C, Engelbrecht A (2004) PSO approaches to coevolve IPD strategies. In: Proceedings of the 2004 congress on evolutionary computation, vol 1, pp 356–363. doi: 10.1109/CEC.2004.1330879
  10. Haykin S (1998) Neural networks: a comprehensive foundation. Prentice-Hall, Englewood CliffsGoogle Scholar
  11. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the 1995 IEEE international conference on neural networks, vol 4, pp 1942–1948. doi: 10.1109/TSMCB.2010.2043527
  12. Kennedy J, Eberhart R (2001) Swarm intelligence. Morgan Kaufmann, Los AltosGoogle Scholar
  13. Kitano H (1993) Massively parallel artificial intelligence and grand challenge aI applications. Tech. rep., SS-93-04, Association for the Advancement of Artificial IntelligenceGoogle Scholar
  14. Krink T,Vesterstrøm J, Riget J (2002) Particle swarm optimisation with spatial particle extension. In: Proceedings of the fourth congress on evolutionary computation 2:1474–1479Google Scholar
  15. Laubach S, Burdick J, Matthies L (1998) An autonomous path planner implemented on the rocky 7 prototype microrover. In: Proceedings of the 1998 IEEE international conference on robotics and automation, vol 1, pp 292–297Google Scholar
  16. Messerschmidt L, Engelbrecht A (2002) Learning to play games using a PSO-based competitive learning approach. In: Proceedings of the 4th Asia-Pacific conference on simulated evolution and learning, pp 444–448Google Scholar
  17. Pelleg D, Moore A (2000) X-means: extending K-means with efficient estimation of the number of clusters. In: Proceedings of the seventeenth international conference on machine learning, pp 727–734Google Scholar
  18. Rosin C, Belew R (1995) Methods for competitive co-evolution: finding opponents worth beating. In: Proceedings of the sixth international conference on genetic algorithms. Morgan Kaufmann Publishers Inc, Los Altos, pp 373–380Google Scholar
  19. Rosin C, Belew R (1997) New methods for competitive coevolution. Evolut Comput 5(1):1–29CrossRefGoogle Scholar
  20. Scheepers C (2013) Coevolution of neuro-controllers to train multi-agent teams from zero knowledge. Msc thesis, University of Pretoria,
  21. Scheepers C, Engelbrecht A (2014) Competitive coevolutionary training of simple soccer agents from zero knowledge. In: Proceedings of the 2014 IEEE congress on evolutionary computation, pp 1210–1217. doi: 10.1109/CEC.2014.6900236
  22. van Wyk A, Engelbrecht A (2010) Overfitting by PSO trained feedforward neural networks. In: Proceedings of the 2010 IEEE congress on evolutionary computation, pp 1–8. doi: 10.1109/CEC.2010.5586333

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of Computer Science, School of Information TechnologyUniversity of PretoriaPretoriaSouth Africa

Personalised recommendations