Applying Reinforcement Learning on Real-World Data with Practical Examples in Python

1. Front Matter

   Pages i-xvii

   PDF

2. Background and Definitions

   • Philip Osborne, Kajal Singh, Matthew E. Taylor

   Pages 1-16

3. Reinforcement Learning Theory

   • Philip Osborne, Kajal Singh, Matthew E. Taylor

   Pages 17-25

4. A Robot Cleaner Example

   • Philip Osborne, Kajal Singh, Matthew E. Taylor

   Pages 27-60

5. The Classroom Environment

   • Philip Osborne, Kajal Singh, Matthew E. Taylor

   Pages 61-75

6. Industry Applications

   • Philip Osborne, Kajal Singh, Matthew E. Taylor

   Pages 77-84

7. Conclusion

   • Philip Osborne, Kajal Singh, Matthew E. Taylor

   Pages 85-86

8. Back Matter

   Pages 87-92

   PDF

Reinforcement learning is a powerful tool in artificial intelligence in which virtual or physical agents learn to optimize their decision making to achieve long-term goals. In some cases, this machine learning approach can save programmers time, outperform existing controllers, reach super-human performance, and continually adapt to changing conditions. This book argues that these successes show reinforcement learning can be adopted successfully in many different situations, including robot control, stock trading, supply chain optimization, and plant control. However, reinforcement learning has traditionally been limited to applications in virtual environments or simulations in which the setup is already provided. Furthermore, experimentation may be completed for an almost limitless number of attempts risk-free. In many real-life tasks, applying reinforcement learning is not as simple as (1) data is not in the correct form for reinforcement learning, (2) data is scarce, and (3) automation has limitations in the real-world. Therefore, this book is written to help academics, domain specialists, and data enthusiast alike to understand the basic principles of applying reinforcement learning to real-world problems. This is achieved by focusing on the process of taking practical examples and modeling standard data into the correct form required to then apply basic agents. To further assist with readers gaining a deep and grounded understanding of the approaches, the book shows hand-calculated examples in full and then how this can be achieved in a more automated manner with code. For decision makers who are interested in reinforcement learning as a solution but are not technically proficient we include simple, non-technical examples in the introduction and case studies section. These provide context of what reinforcement learning offer but also the challenges and risks associated with applying it in practice. Specifically, the book illustrates the differences between reinforcement learning and other machine learning approaches as well as how well-known companies have found success using the approach to their problems.

• The University of Manchester, England

  Philip Osborne

• University of Oxford, England

  Kajal Singh

• The Univeristy of Alberta, Canada

  Matthew E. Taylor

• Alberta Machine Intelligence Institute (Amii), Canada

  Matthew E. Taylor

• Canada CIFAR AI, Amii, Canada

  Matthew E. Taylor

Philip Osborne is a doctoral student currently studying Artificial Intelligence at the University of Manchester with a Master’s Degree in Data Science and a Bachelor’s Degree in Mathematics. The primary focus of his research relates to the application of Reinforcement Learning to real-world tasks with the integration of Natural Language. During his doctorate, Philip has authored and co-authored peer-reviewed papers that have been accepted to top computer science conferences. He has also given lectures on reinforcement learning at both the University of Manchester and the University of Oxford. Philip first applied Reinforcement Learning in a commercial environment with his Master’s dissertation to recommend the order and design of data visualizations for client presentations within an insurance consulting business. Since then, he has demonstrated his other ideas publicly including meal planning and recommending strategy decisions within a popular video game. These public demonstrations have gained notoriety within the data science community, including two separate monetary awards from Kaggle (Google) for their novelty, which has put him at the forefront of the field Kajal Singh is a Full Stack Machine Learning Engineer working in the IT industry in Germany. Kajal is also a Python and Machine Learning mentor/tutor and guest speaker at the University of Oxford for online courses. She has worked on a range of problems, including anomaly detection, sentiment analysis, big data processing, document digitization, and project automation. Kajal has been a part of multiple hackathons conducted while working within industry. She was awarded with an Amazon Pride Card for her research contribution to “Women in AI” project of IIIT, India. She has been recognized for her project on Transactional AI assistants and has been honored as “Master Hacker” in Makeathon at a regional level in India. Matthew E. Taylor (Matt) received his doctorate from the University of Texas at Austin in the summer of 2008, supervised by Peter Stone. He then completed a 2-year postdoctoral research position at the University of Southern California with Milind Tambe and spent 2.5 years as an assistant professor at Lafayette College. He was then an assistant professor at Washington State University, where he held the Allred Distinguished Professorship in Artificial Intelligence. In 2017, he temporarily left academia to help start an artificial intelligence lab in Edmonton, Alberta, with Borealis AI, the artificial intelligence research lab for the Royal Bank of Canada. He is now a tenured associate professor in computer science at the University of Alberta, a Fellow-in-Residence at the Alberta Machine Intelligence Institute, and remains an adjunct professor at Washington State University. He has (co-)supervised 8 graduated Ph.D. students and 10 graduated M.S. students as well as published over 125 peer-reviewed conference papers and journal articles. His current fundamental and applied research interests are in reinforcement learning, human-in-the-loop AI, multi-agent systems, and robotics.