Skip to main content
  • Book
  • © 2021

Handbook of Reinforcement Learning and Control

  • Enriches understanding of the applications of reinforcement learning for control of dynamic systems

  • Collates research from a wide-range of experts, creating a comprehensive guide

  • Discusses both theoretical and practical aspects of machine learning and control

Part of the book series: Studies in Systems, Decision and Control (SSDC, volume 325)

Buy it now

Buying options

eBook USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

This is a preview of subscription content, log in via an institution to check for access.

Table of contents (26 chapters)

  1. Front Matter

    Pages i-xxiv
  2. Theory of Reinforcement Learning for Model-Free and Model-Based Control and Games

    1. Front Matter

      Pages 1-1
    2. Reinforcement Learning for Distributed Control and Multi-player Games

      • Bahare Kiumarsi, Hamidreza Modares, Frank Lewis
      Pages 7-27
    3. Fundamental Design Principles for Reinforcement Learning Algorithms

      • Adithya M. Devraj, Ana Bušić, Sean Meyn
      Pages 75-137
    4. Mixed Density Methods for Approximate Dynamic Programming

      • Max L. Greene, Patryk Deptula, Rushikesh Kamalapurkar, Warren E. Dixon
      Pages 139-172
    5. Model-Free Linear Quadratic Regulator

      • Hesameddin Mohammadi, Mahdi Soltanolkotabi, Mihailo R. Jovanović
      Pages 173-185
  3. Constraint-Driven and Verified RL

    1. Front Matter

      Pages 187-187
    2. Adaptive Dynamic Programming in the Hamiltonian-Driven Framework

      • Yongliang Yang, Donald C. Wunsch II, Yixin Yin
      Pages 189-214
    3. Reinforcement Learning for Optimal Adaptive Control of Time Delay Systems

      • Syed Ali Asad Rizvi, Yusheng Wei, Zongli Lin
      Pages 215-242
    4. Optimal Adaptive Control of Partially Uncertain Linear Continuous-Time Systems with State Delay

      • Rohollah Moghadam, S. Jagannathan, Vignesh Narayanan, Krishnan Raghavan
      Pages 243-272
    5. Dissipativity-Based Verification for Autonomous Systems in Adversarial Environments

      • Aris Kanellopoulos, Kyriakos G. Vamvoudakis, Vijay Gupta, Panos Antsaklis
      Pages 273-291
  4. Multi-agent Systems and RL

    1. Front Matter

      Pages 319-319
    2. Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

      • Kaiqing Zhang, Zhuoran Yang, Tamer Başar
      Pages 321-384
    3. Computational Intelligence in Uncertainty Quantification for Learning Control and Differential Games

      • Mushuang Liu, Yan Wan, Zongli Lin, Frank L. Lewis, Junfei Xie, Brian A. Jalaian
      Pages 385-418
    4. A Top-Down Approach to Attain Decentralized Multi-agents

      • Alex Tong Lin, Guido Montúfar, Stanley J. Osher
      Pages 419-431
  5. Bounded Rationality and Value of Information in RL and Games

    1. Front Matter

      Pages 465-465

About this book

This handbook presents state-of-the-art research in reinforcement learning, focusing on its applications in the control and game theory of dynamic systems and future directions for related research and technology.

The contributions gathered in this book deal with challenges faced when using learning and adaptation methods to solve academic and industrial problems, such as optimization in dynamic environments with single and multiple agents, convergence and performance analysis, and online implementation. They explore means by which these difficulties can be solved, and cover a wide range of related topics including:

  • deep learning;
  • artificial intelligence;
  • applications of game theory;
  • mixed modality learning; and
  • multi-agent reinforcement learning.

Practicing engineers and scholars in the field of machine learning, game theory, and autonomous control will find the Handbook of Reinforcement Learning and Control to be thought-provoking, instructive and informative. 

Editors and Affiliations

  • The Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, USA

    Kyriakos G. Vamvoudakis

  • Department of Electrical Engineering, The University of Texas at Arlington, Arlington, USA

    Yan Wan, Frank L. Lewis

  • Army Research Office, Durham, USA

    Derya Cansever

About the editors

Kyriakos G. Vamvoudakis serves as an Assistant Professor at The Daniel Guggenheim School of Aerospace Engineering at Georgia Tech. He received the Diploma in Electronic and Computer Engineering from the Technical University of Crete, Greece in 2006. He received his M.S. and Ph.D. in Electrical Engineering in 2008 and 2011 respectively from the University of Texas, Arlington. During the period from 2012 to 2016 he was a project research scientist at the Center for Control, Dynamical Systems and Computation at the University of California, Santa Barbara. He was an assistant professor at the Kevin T. Crofton Department of Aerospace and Ocean Engineering at Virginia Tech until 2018. His research interests include reinforcement learning, control theory, and safe/assured autonomy. He is the recipient of a 2019 ARO YIP award, a 2018 NSF CAREER award, and of several international awards including the 2016 International Neural Network Society Young Investigator Award. He currently is an Associate Editor of: Automatica; IEEE Computational Intelligence Magazine; IEEE Transactions on Systems, Man, and Cybernetics: Systems; Neurocomputing; Journal of Optimization Theory and Applications; and of IEEE Control Systems Letters.

Yan Wan is currently an Associate Professor in the Electrical Engineering Department at the University of Texas at Arlington. She received her Ph.D. degree in Electrical Engineering from Washington State University in 2009 and then did postdoctoral training at the University of California, Santa Barbara. Her research interests lie in the modeling, evaluation, and control of large-scale dynamical networks, cyber-physical system and stochastic networks. She has been recognized by several prestigious awards, including the NSF CAREER Award, RTCA William E. Jackson Award and U.S. Ignite and GENI demonstration awards. She currently serves as the Associate Editor for IEEE Transactions on Control of Network Systems, Transactions of the Institute of Measurement and Control, and Journal of Advanced Control for Applications.

Frank L. Lewis is a Distinguished Scholar Professor and Moncrief-O’Donnell Chair at University of Texas at Arlington’s Automation & Robotics Research Institute. He obtained his Bachelor’s Degree in Physics/EE and MSEE at Rice University, his MS in Aeronautical Engineering from Univ. W. Florida, and his Ph.D. at Ga. Tech. He received the Fulbright Research Award, the Outstanding Service Award from Dallas IEEE Section, and was selected as Engineer of the year by Ft. Worth IEEE Section. He is an elected Guest Consulting Professor at South China University of Technology and Shanghai Jiao Tong University. He is a Fellow of the IEEE, Fellow of IFAC, Fellow of the U.K. Institute of Measurement & Control, and a U.K. Chartered Engineer. His current research interests include distributed control on graphs, neural and fuzzy systems, and intelligent control.

Derya Cansever is a Program Manager at the US Army Research Office. Prior to that, he was the Chief Engineer of the Communication Networks and Networking Division at US Army CERDEC, where he conducts research in Tactical, Mission Aware and Software Defined Networks. Dr. Cansever has also worked at Johns Hopkins University Applied Physics Laboratory, AT&T Bell Labs, and GTE Laboratory. He taught courses on Data Communications and Network Security at Boston University and University of Massachusetts. He has a Ph.D. in Electrical and Computer Engineering from the University of Illinois at Urbana Champaign.

Bibliographic Information

Buy it now

Buying options

eBook USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access