Reinforcement Learning in Tower Defense

Abstract

Reinforcement learning is a machine learning technique that makes a decision based on a sequence of actions. This allows changing a game agent’s behavior through feedback, such as rewards or penalties for their actions. Recent work has been demonstrating the use of reinforcement learning to train agents capable of playing electronic games and obtain scores even higher than professional human players. These intelligent agents can also assume other roles, such as creating more complex challenges to players, improving the ambiance of more complex interactive games and even testing the behavior of players when the game is in development. Some literature has been using a deep learning technique to process an image of the game. This is known as the deep Q network and is used to create an intermediate representation and then process it by layers of neural network. These layers are capable of mapping game situations into actions that aim to maximize a reward over time. However, this method is not feasible in modern games, rendered in high resolution with an increasing frame rate. In addition, this method does not work for training agents who are not shown on the screen. In this work we propose a reinforcement learning pipeline based on neural networks, whose input is metadata, selected directly in the game state, and the actions are mapped directly into high-level actions by the agent. We propose this architecture for a tower defense player agent, a real time strategy game whose agent is not represented on the screen directly.