Abstract
This paper presents CBRetaliate, an agent that combines Case-Based Reasoning (CBR) and Reinforcement Learning (RL) algorithms. Unlike most previous work where RL is used to improve accuracy in the action selection process, CBRetaliate uses CBR to allow RL to respond more quickly to changing conditions. CBRetaliate combines two key features: it uses a time window to compute similarity and stores and reuses complete Q-tables for continuous problem solving. We demonstrate CBRetaliate on a team-based first-person shooter game, where our combined CBR+RL approach adapts quicker to changing tactics by an opponent than standalone RL.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bowling, M.H., Veloso, M.M.: Multiagent learning using a variable learning rate. Artificial Intelligence 136(2) (2002)
Salustowicz, R.P., Wiering, M.A., Schmidhuber, J.: Learning team strategies: Soccer case studies. Mach. Learn. 33(2-3) (1998)
Ponsen, M., Spronck, P.: Improving adaptive game AI with evolutionary learning. In: Proceedings of Computer Games: Artificial Intelligence, Design and Education (CGAIDE 2004) (2004)
Tesauro, G.: Temporal dierence learning and TD-Gammon. Communications ofthe ACM 38(3) (1995)
Smith, M., Lee-Urban, S., Muñoz-Avila, H.: RETALIATE: Learning winning policiesin rst-person shooter games. In: Proceedings of the Seventeenth Innovative Applications of Articial Intelligence Conference (IAAI 2007), AAAI Press, Menlo Park (2007)
Hoang, H., Lee-Urban, S., Muñoz-Avila, H.: Hierarchical plan representations for encoding strategic game AI. In: Proceedings of the rst Arti cial Intelligence andInteractive Digital Entertainment Conference (AIIDE 2005). AAAI Press, Menlo Park (2005)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Bridge, D.: The virtue of reward: Performance, reinforcement and discovery in case-based reasoning. In: Invited Talk at the 6th International Conference on Case-Based Reasoning (ICCBR 2005) (2005)
Aha, D.W., Molineaux, M., Ponsen, M.J.V.: Learning to win: Case-based plan selection in a real-time strategy game. In: Proceedings of the 6th International Conference on Case-Based Reasoning (ICCBR 2005) (2005)
Ortañón, S., Mishra, K., Sugandh, N., Ram, A.: Case-based planning and execution for real-time strategy games. In: Proceedings of the 7th International Conference on Case-Based Reasoning Research and Development (ICCBR 2007) (2007)
Sharma, M., Holmes, M., Santamaría, J.C., Irani, J.A., Isbell, C., Ram, A.: Transferlearning in real-time strategy games using hybrid CBR/RL. In: Proceedings of the20th International Joint Conference on Articial Intelligence (IJCAI 2007) (2007)
Erol, K., Hendler, J., Nau, D.S.: HTN planning: complexity and expressivity. In: Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI 1994) (1994)
Ram, A., Santamaria, J.C.: Continuous case-based reasoning. Artificial Intelligence 90(1-2) (1997)
Ros, R., Veloso, M.M., de Mántares, R.L., Sierra, C., Arcos, J.L.: Retrievingand reusing game plays for robot soccer. In: Proceedings of the 8th European Conference on Advances in Case-Based Reasoning (ECCBR 2006) (2006)
Kuter, U., Nau, D.: Using domain-congurable search control in probabilistic planners. In: Proceedings of the The Twentieth National Conference on Artificial Intelligence (AAAI 2005) (2005)
Ulam, P., Goel, A., Jones, J., Murdock, J.W.: Using model-based re ection toguide reinforcement learning. In: Proceedings of the Nineteenth International JointConference on Articial Intelligence (IJCAI 2005) Workshop on Reasoning, Representation and Learning in Computer Games (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Auslander, B., Lee-Urban, S., Hogg, C., Muñoz-Avila, H. (2008). Recognizing the Enemy: Combining Reinforcement Learning with Strategy Selection Using Case-Based Reasoning. In: Althoff, KD., Bergmann, R., Minor, M., Hanft, A. (eds) Advances in Case-Based Reasoning. ECCBR 2008. Lecture Notes in Computer Science(), vol 5239. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85502-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-85502-6_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85501-9
Online ISBN: 978-3-540-85502-6
eBook Packages: Computer ScienceComputer Science (R0)