Abstract
This paper shows the design and improvement of an autonomous agent based in using evolutionary methods to improve behavioural models (finite state machines), which are part of the individuals to evolve. This leads to the obtention of a so-called bot that follows the Gameplay track rules of the international Mario AI Championship and is able to autonomously complete different scenarios on a simulator of Super Mario Bros. game. Mono- and multi-seed approaches (evaluation in one play or multiple plays respectively) have been analysed, in order to compare respectively the performance of an approach focused in solving a specific scenario, and another more general, devoted to obtain an agent which can play successfully in different scenarios. The analysis considers the machine resources consumption, which turns in a bottleneck in some experiments. However, the methods yield agents which can finish several stages of different difficulty levels, and playing much better than an expert human player, since they can deal with very difficult situations (several enemies surrounding Mario, for instance) in real time. According to the results and considering the competition’s restrictions (time limitations) and objectives (complete scenarios up to difficulty level 3), these agents have enough performance to participate in this competition track.
Similar content being viewed by others
Notes
Designer and producer of Nintendo Ltd., and winner of the 2012 Príncipe de Asturias Prize in Humanities and Communication.
References
Bäck T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, Oxford
Baumgarten R. Mario AI A* agent. http://www.doc.ic.ac.uk/rb1006/projects:marioai
Bojarski S, Bates-Congdon C (2011) REALM: a rule-based evolutionary computation agent that learns to play Mario. In: Proceedings of the 2011 IEEE symposium on computational intelligence and games (CIG 2011), IEEE Press, pp 83–90
Booth TL (1967) Sequential machines and automata theory, 1st edn. Wiley, New York
Esparcia-Alcázar AI, Martínez-García AI, Mora AM, Merelo JJ, García-Sánchez P (2010) Controlling bots in a first person shooter game using genetic algorithms. In: Proceedings of the 2010 IEEE congress on evolutionary computation (CEC 2010), pp 1–8
Esparcia-Alcázar AI, Martínez-García AI, Mora AM, Merelo JJ, García-Sánchez P (2010) Genetic evolution of fuzzy finite state machines to control bots in a first-person shooter game. In: Proceedings of the GECCO 2010, pp 829–830
Fernández AJ, González-Jiménez J (2005) Action games: evolutive experiences. In: Computational intelligence, theory and applications advances in soft computing, vol 33, pp 487–501
Fernández-Ares A, Mora AM, Merelo JJ, García-Sánchez P, Fernandes C (2011) Optimizing player behavior in a real-time strategy game using evolutionary algorithms. In: Proceedings of the 2011 IEEE congress on evolutionary computation, CEC ’11, pp 2017–2024
Fu D, Houlette R (2004) The ultimate guide to FSMs in games. AI Game Programming Wisdom 2, Charles River Media, pp 283–302
Goldberg DE, Korb B, Deb K (1989) Messy genetic algorithms: motivation, analysis, and first results. Complex Syst 3(5):493–530
Hagelbäck J (2012) Potential-field based navigation in StarCraft. In: Proceedings of the 2012 IEEE symposium on computational intelligence and games (CIG 2012), IEEE Press, pp 388–393
Jang SH, Yoon JW, Cho SB (2009) Optimal strategy selection of non-player character on real time strategy game using a speciated evolutionary algorithm. In: Proceedings of the 5th IEEE symposium on computational intelligence and games (CIG’09), IEEE Press, pp 75–79
Karakovskiy S, Togelius J (2012) The Mario AI benchmark and competitions. IEEE Trans Comput Intell AI Games 4(1):55–67
Laird JE (2001) Using a computer game to develop advanced AI. Computer 34(7):70–75
Martín E, Martínez M, Recio G, Saez Y (2010) Pac-mAnt: optimization based on ant colonies applied to developing an agent for Ms. Pac-Man. In: Proceedings of the 2010 IEEE conference on computational intelligence and games (CIG 2010), IEEE Press, pp 458–464
Mora AM, Montoya R, Merelo JJ, García-Sánchez P, Castillo PA, Laredo JLJ, Martínez A, Esparcia AI (2010) Evolving bot AI in unreal. In: CDC et al (eds) Applications of evolutionary computing, part I. Vol 6024 of LNCS., Springer, New York, pp 170–179
Mora AM, Moreno MA, Merelo JJ, Castillo PA, García-Arenas MI, Laredo JLJ (2010) Evolving the cooperative behaviour in UnrealTM bots. In: Proceedings of the 2010 IEEE conference on computational intelligence and games (CIG 2010), IEEE Press, pp 241–248
Mora AM, Fernández-Ares A, Merelo JJ, García-Sánchez P (2012) Dealing with noisy fitness in the design of a RTS game bot. In: Proceedings of the applications of evolutionary computing: EvoApplications 2012, LNCS, vol 7248, Springer, New York, pp 234–244
Nareyek A (2004) AI in computer games. ACM Queue J 1(10):58–65
Pedersen C, Togelius J, Yannakakis G (2009) Modeling player experience in Super Mario Bros. In: Proceedings of the 2009 IEEE symposium on computational intelligence and games (CIG’09), IEEE Press, pp 132–139
Pepels T, Windans HM (2012) Enhancements for Monte-Carlo Tree Search in Ms Pac-Man. In: Proceedings of the 2012 IEEE conference on computational intelligence and games (CIG 2012), IEEE Press, pp 265–272
Ponsen M, Munoz-Avila H, Spronck P, Aha DW (2006) Automatically generating game tactics through evolutionary learning. AI Mag 27(3):75–84
Shaker N, Togelius J, Yannakakis GN, Weber B, Shimizu T, Hashiyama T, Sorenson N, Pasquier P, Mawhorter P, Takahashi G, Smith G, Baumgarten R (2011) The 2010 Mario AI championship: level generation track. IEEE Trans Comput Intell AI Games 3(4):332–347
Shaker N, Nicolau M, Yannakakis GN, Togelius J, O’neill M (2012) Evolving levels for Super Mario Bros using grammatical evolution. In: Proceedings of the 2012 IEEE symposium on computational intelligence and games (CIG 2012), IEEE Press, pp 304–311
Shaker N, Togelius J, Yannakakis G, Poovanna L, Ethiraj VS, Johansson SJ, Reynolds RG, Heether LK, Schumann T, Gallagher M (2013) The Turing test track of the 2012 Mario AI championship: entries and evaluation. In: Proceedings of the 2013 IEEE symposium on computational intelligence and games (CIG 2013), IEEE Press, pp 1–8
Small R, Bates-Congdon C (2009) Agent Smith: towards an evolutionary rule-based agent for interactive dynamic games. In: Proceedings of the 2009 IEEE congress on evolutionary computation (CEC’09), pp 660–666
Spronck P, Sprinkhuizen-Kuyper I, Postma E (2003) Improving opponent intelligence through offline evolutionary learning. Int J Intell Games Simul 2(1):20–27
Synnaeve G, Bessière P (2011) A Bayesian model for RTS units control applied to StarCraft. In: Proceedings of the 2011 IEEE symposium on computational intelligence and games (CIG 2011), IEEE Press, pp 190–196
Togelius J, Karakovskiy S, Koutnik J, Schmidhuber J (2009) Super Mario evolution. In: Proceedings of the 2009 IEEE symposium on computational intelligence and games (CIG’09), IEEE Press, pp 156–161
Wikipedia. Super Mario series. http://en.wikipedia.org/wiki/Super_Mario_(series)#Games
Acknowledgments
This work has been supported in part by the P08-TIC-03903 project awarded by the Andalusian Regional Government, the FPU Grant 2009-2942, the TIN2011-28627-C04-02 project, awarded by the Spanish Ministry of Science and Innovation, and project #83 from CEI-BioTIC at the University of Granada.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mora, A.M., Merelo, J.J., García-Sánchez, P. et al. Creating autonomous agents for playing Super Mario Bros game by means of evolutionary finite state machines. Evol. Intel. 6, 205–218 (2014). https://doi.org/10.1007/s12065-014-0105-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12065-014-0105-7