Machine Learning

, Volume 84, Issue 1, pp 7-49

First online:

On the analysis and design of software for reinforcement learning, with a survey of existing systems

  • Tim KovacsAffiliated withIntelligent Systems Laboratory, University of Bristol Email author 
  • , Robert EggintonAffiliated withIntelligent Systems Laboratory, University of Bristol


Reinforcement Learning (RL) is a very complex domain and software for RL is correspondingly complex. We analyse the scope, requirements, and potential for RL software, discuss relevant design issues, survey existing software, and make recommendations for designers. We argue that broad and flexible libraries of reusable software components are valuable from a scientific, as well as practical, perspective, as they allow precise control over experimental conditions, encourage comparison of alternative methods, and allow a fuller exploration of the RL domain.


Reinforcement learning Software engineering