Encyclopedia of Systems and Control

2015 Edition
| Editors: John Baillieul, Tariq Samad

Neural Control and Approximate Dynamic Programming

Reference work entry
DOI: https://doi.org/10.1007/978-1-4471-5058-9_224


There has been great interest recently in “universal model-free controllers” that do not need a mathematical model of the controlled plant, but mimic the functions of biological processes to learn about the systems they are controlling online, so that performance improves automatically. Neural network (NN) control has had two major thrusts: approximate dynamic programming, which uses NN to approximately solve the optimal control problem, and NN in closed-loop feedback control.


Adaptive control Learning systems Neural networks Optimal control Reinforcement learning 
This is a preview of subscription content, log in to check access.



This material is based upon the work supported by NSF. Grant Number: ECCS-1128050, ARO. Grant Number: W91NF-05-1-0314, AFOSR. Grant Number: FA9550-09-1-0278.


  1. Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791MathSciNetGoogle Scholar
  2. Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern Part B 38(4):943–949Google Scholar
  3. Lewis FL, Liu D (2012) Reinforcement learning and approximate dynamic programming for feedback control. IEEE Press computational intelligence series. Wiley-Blackwell, OxfordGoogle Scholar
  4. Lewis FL, Vamvoudakis KG (2011) Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Trans Syst Man Cybern Part B 41(1):14–25Google Scholar
  5. Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor and Francis, LondonGoogle Scholar
  6. Lewis FL, Campos J, Selmic R (2002) Neuro-fuzzy control of industrial systems with actuator nonlinearities. Society of Industrial and Applied Mathematics Press, PhiladelphiaGoogle Scholar
  7. Lewis FL, Vrabie D, Syrmos VL (2012a) Optimal control. Wiley, New YorkGoogle Scholar
  8. Lewis FL, Vrabie D, Vamvoudakis KG (2012b) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105MathSciNetGoogle Scholar
  9. Slotine JJE, Li W (1987) On the adaptive control of robot manipulators. Int J Robot Res 6(3):49–59Google Scholar
  10. Sutton RS, Barto AG (1998) Reinforcement learning – an introduction. MIT, CambridgeGoogle Scholar
  11. Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888MathSciNetGoogle Scholar
  12. Vamvoudakis KG, Lewis FL (2011) Multi-player non zero sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations. Automatica 47(8):1556–1569MathSciNetGoogle Scholar
  13. Vamvoudakis KG, Lewis FL (2012) Online solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int J Robust Nonlinear Control 22(13):1460–1483MathSciNetGoogle Scholar
  14. Vamvoudakis KG, Lewis FL, Hudas GR (2012a) Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8):1598–1611MathSciNetGoogle Scholar
  15. Vamvoudakis KG, Lewis FL, Johnson M, Dixon WE (2012b) Online learning algorithm for Stackelberg games in problems with hierarchy. In: Proceedings of the 51st IEEE conference on decision and control, Maui pp 1883–1889Google Scholar
  16. Vamvoudakis KG, Vrabie D, Lewis FL (2013) Online adaptive algorithm for optimal control with integral reinforcement learning. Int J Robust Nonlinear Control, Wiley. doi: 10.1002/rnc.3018Google Scholar
  17. Vrabie D, Pastravanu O, Lewis FL, Abu-Khalaf M (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 45(2):477–484MathSciNetGoogle Scholar
  18. Vrabie D, Vamvoudakis KG, Lewis FL (2012) Optimal adaptive control and differential games by reinforcement learning principles. Control engineering series. IET Press, LondonGoogle Scholar
  19. Werbos PJ (1989) Neural networks for control and system identification. In: Proceedings of the IEEE conference on decision and control, TampaGoogle Scholar
  20. Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control. Van Nostrand Reinhold, New YorkGoogle Scholar

Copyright information

© Springer-Verlag London 2015

Authors and Affiliations

  1. 1.Arlington Research InstituteUniversity of TexasFort WorthUSA
  2. 2.Center for Control, Dynamical Systems and Computation (CCDC)University of CaliforniaSanta BarbaraUSA