Lifelong Reinforcement Learning

Abstract

This chapter discusses lifelong reinforcement learning. Reinforcement learning (RL) is the problem where an agent learns actions through trial-and-error interactions with a dynamic environment [Kaelbling et al., 1996, Sutton and Barto, 1998]. In each interaction step, the agent receives input on the current state of the environment. It chooses an action from a set of possible actions. The action changes the state of the environment. Then, the agent gets the value of this state transition, which can be a reward or penalty. This process repeats as the agent learns a trajectory of actions to optimize its objective, e.g., to maximize the long-run sum of rewards. The goal of RL is to learn an optimal policy that maps states to actions (possibly stochastically). There is a recent surge in research in RL due to its successful use in the computer program called AlphaGo [Silver et al., 2016], which won 4–1 against one of the legendary professional Go players Lee Sedol in March 2016.¹ More recently, AlphaGo Zero [Silver et al., 2017]² was designed to learn to master the game of Go from scratch without human knowledge, and it has achieved superhuman performance.