Play Ms. Pac-Man Using an Advanced Reinforcement Learning Agent
Reinforcement Learning (RL) algorithms have been promising methods for designing intelligent agents in games. Although their capability of learning in real time has been already proved, the high dimensionality of state spaces in most game domains can be seen as a significant barrier. This paper studies the popular arcade video game Ms. Pac-Man and outlines an approach to deal with its large dynamical environment. Our motivation is to demonstrate that an abstract but informative state space description plays a key role in the design of efficient RL agents. Thus, we can speed up the learning process without the necessity of Q-function approximation. Several experiments were made using the multiagent MASON platform where we measured the ability of the approach to reach optimum generic policies which enhances its generalization abilities.
KeywordsIntelligent Agents Reinforcement Learning Ms. Pac-Man
Unable to display preview. Download preview PDF.
- 2.Sutton, R.: Learning to predict by the method of temporal differences. Machine Learning 3(1), 9–44 (1988)Google Scholar
- 3.Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Inteligence Research 4, 237–285 (1996)Google Scholar
- 4.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
- 5.Szita, I.: Reinforcement learning in games. In: Reinforcement Learning, pp. 539–577 (2012)Google Scholar
- 7.Lucas, S.M.: Evolving a neural network location evaluator to play ms. pac-man. In: Proc. of IEEE Symposium on Computational Intelligence and Games (CIG 2005), pp. 203–210 (2005)Google Scholar
- 8.Bom, L., Henken, R., Wiering, M.A.: Reinforcement learning to train ms. pac-man using higher-order action-relative inputs. In: Proc. of IEEE Intern. Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 156–163 (2013)Google Scholar
- 9.Alhejali, A.M., Lucas, S.M.: Evolving diverse ms. pac-man playing agents using genetic programming. In: Proc. of IEEE Symposium on Computational Intelligence and Games (CIG 2010), pp. 53–60 (2010)Google Scholar
- 12.Torrey, L., Taylor, M.: Teaching on a budget: Agents advising agents in reinforcement learning. In: Intern. Conferecene on Autonomous Agents and Multi-agent Systems (AAMAS), pp. 1053–1060 (2013)Google Scholar
- 13.Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley (2005)Google Scholar
- 14.Singh, S., Sutton, R.S., Kaelbling, P.: Reinforcement learning with replacing eligibility traces, pp. 123–158 (1996)Google Scholar