Abstract
The imitation of human playing style has been gaining relevance in both the Artificial Intelligence for Games research community and the Digital Game industry over the last decade, achieving a special importance in recent years. The goal of these virtual players is to deceive real players and be perceived just as another human player. Although this challenge can be addressed using different Imitation Learning techniques, classic supervised learning approaches do not usually work well due to the violation of the independent and identically distributed assumption for random variables. No regret algorithms in online learning settings seem to outperform previous approaches. In this work we describe an interactive and online case-based reasoning system in which the bot gives control to the human player when it reaches game states that are not well represented by cases in its case base, and regains control when the game states are known again. Results show that (1) the amount of human intervention decreases rapidly, (2) the case base needed to achieve reasonable imitation is considerable smaller than that used in a non-interactive approach (3) the resulting agent outperforms other agents using non-interactive CBR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Floyd, M.W., Davoust, A., Esfandiari, B.: Considerations for real-time spatially-aware case-based reasoning: a case study in robotic soccer imitation. In: Althoff, K.-D., Bergmann, R., Minor, M., Hanft, A. (eds.) ECCBR 2008. LNCS (LNAI), vol. 5239, pp. 195–209. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85502-6_13
Floyd, M.W., Esfandiari, B.: An active approach to automatic case generation. In: McGinty, L., Wilson, D.C. (eds.) ICCBR 2009. LNCS (LNAI), vol. 5650, pp. 150–164. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02998-1_12
Floyd, M.W., Esfandiari, B., Lam, K.: A case-based reasoning approach to imitating RoboCup players. In: Proceedings of the Twenty-First International Florida Artificial Intelligence Research Society Conference, 15–17 May 2008, Coconut Grove, Florida, USA, pp. 251–256 (2008)
Geisler, B.: An empirical study of machine learning algorithms applied to modeling player behavior in a “first person shooter" video game. Ph.D. thesis, Citeseer (2002)
Goldberg, H.: All Your Base are Belong to Us: How 50 Years of Videogames Conquered Pop Culture. Three Rivers Press, New York (2011)
Gorman, B., Thurau, C., Bauckhage, C., Humphrys, M.: Believability testing and Bayesian imitation in interactive computer games. In: SAB (2006)
Harmer, J., Gisslén, L., Holst, H., Bergdahl, J., Olsson, T., Sjöö, K., Nordin, M.: Imitation learning with concurrent actions in 3D games. CoRR abs/1803.05402 (2018)
Hingston, P.: A new design for a turing test for bots. In: Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games, CIG 2010, Copenhagen, Denmark, 18–21 August 2010, pp. 345–350 (2010)
Iii, H.D., Langford, J., Marcu, D.: Search-based structured prediction. Mach. Learn. 75(3), 297–325 (2009)
Lam, K., Esfandiari, B., Tudino, D.: A scene-based imitation framework for RoboCup clients. In: MOO-Modeling Other Agents from Observations (2006)
Lamontagne, L., Rugamba, F., Mineau, G.: Acquisition of cases in sequential games using conditional entropy. In: ICCBR 2012 Workshop on TRUE: Traces for Reusing Users’ Experience (2012)
Livingstone, D.: Turing’s test and believable AI in games. Comput. Entertain. 4, 6 (2006)
Lucas, S.M.: Ms Pac-Man versus ghost-team competition. In: Proceedings of the 2009 IEEE Symposium on Computational Intelligence and Games, CIG 2009, 7–10 September 2009, Milano, Italy (2009)
Miranda, M., Sánchez-Ruiz, A.A., Peinado, F.: A neuroevolution approach to imitating human-like play in ms. pac-man video game. In: Proceedings of the 3rd Congreso de la Sociedad Española para las Ciencias del Videojuego, 29 June 2016, Barcelona, Spain, pp. 113–124 (2016)
Miranda, M., Sánchez-Ruiz, A.A., Peinado, F.: A CBR approach for imitating human playing style in Ms. Pac-Man video game. In: Cox, M.T., Funk, P., Begum, S. (eds.) ICCBR 2018. LNCS (LNAI), vol. 11156, pp. 292–308. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01081-2_20
Ontañón, S., Montaña, J.L., Gonzalez, A.J.: A dynamic-Bayesian network framework for modeling and evaluating learning from observation. Expert Syst. Appl. 41(11), 5212–5226 (2014)
Ortega, J., Shaker, N., Togelius, J., Yannakakis, G.N.: Imitating human playing styles in super mario bros. Entertain. Comput. 4(2), 93–104 (2013)
Packard, B., Ontañón, S.: Policies for active learning from demonstration. In: 2017 AAAI Spring Symposia, Stanford University, 27–29 March 2017, Palo Alto, California, USA (2017)
Packard, B., Ontañón, S.: Learning behavior from limited demonstrations in the context of games. In: Proceedings of the Thirty-First International Florida Artificial Intelligence Research Society Conference, FLAIRS 2018, 21–23 May 2018, Melbourne, Florida, USA, pp. 86–91 (2018)
Rohlfshagen, P., Liu, J., Pérez-Liébana, D., Lucas, S.M.: Pac-Man conquers academia: two decades of research using a classic arcade game. IEEE Trans. Games 10, 233–256 (2018)
Ross, S., Bagnell, D.: Efficient reductions for imitation learning. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, 13–15 May 2010, Chia Laguna Resort, Sardinia, Italy, pp. 661–668 (2010)
Ross, S., Gordon, G.J., Bagnell, D.: A reduction of imitation learning and structured prediction to no-regret online learning. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2011, 11–13 April 2011, Fort Lauderdale, USA, pp. 627–635 (2011)
Sagredo-Olivenza, I., Gómez-Martín, P.P., Gómez-Martín, M.A., González-Calero, P.A.: Using program by demonstration and visual scripting to supporting game design. In: Benferhat, S., Tabia, K., Ali, M. (eds.) IEA/AIE 2017. LNCS (LNAI), vol. 10351, pp. 33–39. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60045-1_5
Soni, B., Hingston, P.: Bots trained to play like a human are more fun. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 363–369 (2008)
Togelius, J., Nardi, R.D., Lucas, S.M.: Towards automatic personalised content creation for racing games. In: 2007 IEEE Symposium on Computational Intelligence and Games, pp. 252–259 (2007)
Togelius, J., Yannakakis, G.N., Karakovskiy, S., Shaker, N.: Assessing believability. In: Hingston, P. (ed.) Believable Bots, pp. 215–230. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-32323-2_9
Williams, P.R., Liebana, D.P., Lucas, S.M.: Ms. Pac-Man versus ghost team CIG 2016 competition. In: IEEE Conference on Computational Intelligence and Games, CIG 2016, 20–23 September 2016, Santorini, Greece, pp. 1–8 (2016)
Wooldridge, M.: Introduction to multiagent systems. Cell 757(239), 8573 (2002)
Yannakakis, G.N., Maragoudakis, M.: Player modeling impact on player’s entertainment in computer games. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 74–78. Springer, Heidelberg (2005). https://doi.org/10.1007/11527886_11
Acknowledgements
This work has been partially supported by the Spanish Committee of Economy and Competitiveness (TIN2017-87330-R) and the UCM (Group 921330), project ComunicArte: Comunicación Efectiva a través de la Realidad Virtual y las Tecnologías Educativas, funded by Ayudas Fundación BBVA a Equipos de Investigación Científica 2017, and project NarraKit VR: Interfaces de Comunicación Narrativa para Aplicaciones de Realidad Virtual (PR41/17-21016), funded by Ayudas para la Financiación de Proyectos de Investigación Santander-UCM 2017.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Miranda, M., Sánchez-Ruiz, A.A., Peinado, F. (2019). Towards Human-Like Bots Using Online Interactive Case-Based Reasoning. In: Bach, K., Marling, C. (eds) Case-Based Reasoning Research and Development. ICCBR 2019. Lecture Notes in Computer Science(), vol 11680. Springer, Cham. https://doi.org/10.1007/978-3-030-29249-2_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-29249-2_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29248-5
Online ISBN: 978-3-030-29249-2
eBook Packages: Computer ScienceComputer Science (R0)