Intelligent Critic Control with Disturbance Attenuation for a Micro-Grid System

  • Ding WangEmail author
  • Chaoxu Mu
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 167)


In this chapter, a computationally efficient framework for intelligent critic control design and application of continuous-time input-affine systems is established with the purpose of disturbance attenuation. The described problem is formulated as a two-player zero-sum differential game and the adaptive critic mechanism with intelligent component is employed to solve the minimax optimization problem. First, a neural identifier is developed to reconstruct the unknown dynamical information incorporating stability analysis. Next, the optimal control law and the worst-case disturbance law are designed by introducing and tuning a critic neural network. Moreover, the closed-loop system is proved to possess the uniform ultimate boundedness. At last, the present method is applied to a smart micro-grid and then is further adopted to control a general nonlinear system via simulation, thereby substantiating the performance of disturbance attenuation.


  1. 1.
    Abu-Khalaf, M., Lewis, F.L., Huang, J.: Policy iterations on the Hamilton-Jacobi-Isaacs equation for \(H_{\infty }\) state feedback control with input saturation. IEEE Trans. Autom. Control 51(12), 1989–1995 (2006)Google Scholar
  2. 2.
    Basar, T., Bernhard, P.: \(H_{\infty }\)-Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach, 2nd edn. Birkhauser, Boston (2008)Google Scholar
  3. 3.
    Bian, T., Jiang, Y., Jiang, Z.P.: Decentralized adaptive optimal control of large-scale systems with application to power systems. IEEE Trans. Ind. Electron. 62(4), 2439–2447 (2015)CrossRefGoogle Scholar
  4. 4.
    Cheng, L., Liu, W., Hou, Z.G., Yu, J., Tan, M.: Neural-network-based nonlinear model predictive control for piezoelectric actuators. IEEE Trans. Ind. Electron. 62(12), 7717–7727 (2015)CrossRefGoogle Scholar
  5. 5.
    Corless, M.J., Leitmann, G.: Continuous state feedback guaranteeing uniform ultimate boundedness for uncertain dynamic systems. IEEE Trans. Autom. Control 26(5), 1139–1144 (1981)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Cucuzzella, M., Incremona, G.P., Ferrara, A.: Design of robust higher order sliding mode control for microgrids. IEEE J. Emerg. Sel. Top. Circuits Syst. 5(3), 393–401 (2015)CrossRefGoogle Scholar
  7. 7.
    Dierks, T., Thumati, B.T., Jagannathan, S.: Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence. Neural Netw. 22(5–6), 851–860 (2009)CrossRefGoogle Scholar
  8. 8.
    Francis, R., Chidambaram, I.A.: Optimized PI+ load-frequency controller using BWNN approach for an interconnected reheat power system with RFB and hydrogen electrolyser units. Int. J. Electr. Power Energy Syst. 67, 381–392 (2015)CrossRefGoogle Scholar
  9. 9.
    Gao, H., Wu, J., Shi, P.: Robust sampled-data \(H_{\infty }\) control with stochastic sampling. Automatica 45(7), 1729–1736 (2009)Google Scholar
  10. 10.
    Gao, W., Jiang, Z.P.: Adaptive dynamic programming and adaptive optimal output regulation of linear systems. IEEE Trans. Autom. Control 61(12), 4164–4169 (2016)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice-Hall, New Jersey (1999)Google Scholar
  12. 12.
    He, W., Zhang, S., Ge, S.S.: Adaptive control of a flexible crane system with the boundary output constraint. IEEE Trans. Ind. Electron. 61(8), 4126–4133 (2014)CrossRefGoogle Scholar
  13. 13.
    Heydari, A., Balakrishnan, S.N.: Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans. Neural Netw. Learn. Syst. 24(1), 145–157 (2013)CrossRefGoogle Scholar
  14. 14.
    Jafarzadeh, S., Fadali, M.S.: On the stability and control of continuous-time TSK fuzzy systems. IEEE Trans. Cybern. 43(3), 1073–1087 (2013)CrossRefGoogle Scholar
  15. 15.
    Jiang, Y., Jiang, Z.P.: Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Autom. Control 60(11), 2917–2929 (2015)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Kahrobaeian, A., Mohamed, Y.A.R.I.: Suppression of interaction dynamics in DG converter-based microgrids via robust system-oriented control approach. IEEE Trans. Smart Grid 3(4), 1800–1811 (2012)CrossRefGoogle Scholar
  17. 17.
    Kahrobaeian, A., Mohamed, Y.A.R.I.: Analysis and mitigation of low-frequency instabilities in autonomous medium-voltage converter-based microgrids with dynamic loads. IEEE Trans. Ind. Electron. 61(4), 1643–1658 (2014)CrossRefGoogle Scholar
  18. 18.
    Kamwa, I., Grondin, R., Hebert, Y.: Wide-area measurement based stabilizing control of large power systems-a decentralized/hierarchical approach. IEEE Trans. Power Syst. 16(1), 136–153 (2001)CrossRefGoogle Scholar
  19. 19.
    Khalil, H.: Nonlinear Systems, 3rd edn. Prentice-Hall, Upper Saddle River (2002)Google Scholar
  20. 20.
    Krstic, M., Kanellakopoulos, I., Kokotovic, P.: Nonlinear and Adaptive Control Design. Wiley, New York (1995)Google Scholar
  21. 21.
    Lewis, F.L., Liu, D.: Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. Wiley, New Jersey (2013)Google Scholar
  22. 22.
    Liang, J., Venayagamoorthy, G.K., Harley, R.G.: Wide-area measurement based dynamic stochastic optimal power flow control for smart grids with high variability and uncertainty. IEEE Trans. Smart Grid 3(1), 59–69 (2012)CrossRefGoogle Scholar
  23. 23.
    Luo, B., Wu, H.N., Huang, T.: Off-policy reinforcement learning for \(H_{\infty }\) control design. IEEE Trans. Cybern. 45(1), 65–76 (2015)Google Scholar
  24. 24.
    Mahmud, M.A., Hossain, M.J., Pota, H.R., Oo, A.M.T.: Robust nonlinear distributed controller design for active and reactive power sharing in islanded microgrids. IEEE Trans. Energy Convers. 29(4), 893–903 (2014)CrossRefGoogle Scholar
  25. 25.
    Modares, H., Lewis, F.L., Sistani, M.B.N.: Online solution of nonquadratic two-player zero-sum games arising in the \(H_{\infty }\) control of constrained input systems. Int. J. Adapt. Control Signal Process. 28(3–5), 232–254 (2014)Google Scholar
  26. 26.
    Mohamed, Y.A.R.I., Zeineldin, H.H., Salama, M.M.A., Seethapathy, R.: Seamless formation and robust control of distributed generation microgrids via direct voltage control and optimized dynamic power sharing. IEEE Trans. Power Electron. 27(3), 1283–1294 (2012)CrossRefGoogle Scholar
  27. 27.
    Mu, C., Tang, Y., He, H.: Observer-based sliding mode frequency control for micro-grid with photovoltaic energy integration. In: Proceedings of IEEE Power and Energy Society General Meeting Boston, pp. 1–5 (2016)Google Scholar
  28. 28.
    Mu, C., Wang, D., Sun, C., Zong, Q.: Robust adaptive critic control design with network-based event-triggered formulation. Nonlinear Dyn. 90(3), 2023–2035 (2017)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Pandey, S.K., Mohanty, S.R., Kishor, N.: A literature survey on load-frequency control for conventional and distribution generation power systems. Renew. Sustain. Energy Rev. 25(5), 318–334 (2013)CrossRefGoogle Scholar
  30. 30.
    Parmar, K.P.S., Majhi, S., Kothari, D.P.: Load frequency control of a realistic power system with multi-source power generation. Int. J. Electr. Power Energy Syst. 42(1), 426–433 (2012)CrossRefGoogle Scholar
  31. 31.
    Polyakov, A., Fridman, L.: Stability notions and Lyapunov functions for sliding mode control systems. J. Frankl. Inst. 351(4), 1831–1865 (2014)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Precup, R.E., Radac, M.B., Tomescu, M.L., Petriu, E.M., Preitl, S.: Stable and convergent iterative feedback tuning of fuzzy controllers for discrete-time SISO systems. Expert Syst. Appl. 40(1), 188–199 (2013)CrossRefGoogle Scholar
  33. 33.
    Qin, C., Zhang, H., Wang, Y., Luo, Y.: Neural network-based online \(H_{\infty }\) control for discrete-time affine nonlinear system using adaptive dynamic programming. Neurocomputing 198, 91–99 (2016)Google Scholar
  34. 34.
    Romero-Cadaval, E., Spagnuolo, G., Franquelo, L.G., Ramos-Paja, C.A., Suntio, T., Xiao, W.M.: Grid-connected photovoltaic generation plants: components and operation. IEEE Ind. Electron. Mag. 7(3), 6–20 (2013)CrossRefGoogle Scholar
  35. 35.
    Ruderman, M., Iwasaki, M.: Observer of nonlinear friction dynamics for motion control. IEEE Trans. Ind. Electron. 62(9), 5941–5949 (2015)CrossRefGoogle Scholar
  36. 36.
    Sonmez, S., Ayasun, S., Nwankpa, C.O.: An exact method for computing delay margin for stability of load frequency control systems with constant communication delays. IEEE Trans. Power Syst. 31(1), 370–377 (2016)CrossRefGoogle Scholar
  37. 37.
    Sun, W., Zhao, Z., Gao, H.: Saturated adaptive robust control for active suspension systems. IEEE Trans. Ind. Electron. 60(9), 3889–3896 (2013)CrossRefGoogle Scholar
  38. 38.
    Tang, Y., He, H., Wen, J., Liu, J.: Power system stability control for a wind farm based on adaptive dynamic programming. IEEE Trans. Smart Grid 6(1), 166–177 (2015)CrossRefGoogle Scholar
  39. 39.
    Vamvoudakis, K.G., Lewis, F.L.: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5), 878–888 (2010)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Vamvoudakis, K.G., Lewis, F.L.: Online solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int. J. Robust Nonlinear Control 22(13), 1460–1483 (2012)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Wang, C., Liu, D., Wei, Q., Zhao, D., Xia, Z.: Iterative adaptive dynamic programming approach to power optimal control for smart grid with energy storage devices. Acta Autom. Sin. 40(9), 1984–1990 (2014)zbMATHGoogle Scholar
  42. 42.
    Wang, J., Xu, X., Liu, D., Sun, Z., Chen, Q.: Self-learning cruise control using Kernel-based least squares policy iteration. IEEE Trans. Control Syst. Technol. 22(3), 1078–1087 (2014)CrossRefGoogle Scholar
  43. 43.
    Wang, D., Liu, D., Zhang, Q., Zhao, D.: Data-based adaptive critic designs for nonlinear robust optimal control with uncertain dynamics. IEEE Trans. Syst. Man Cybern. Syst. 46(11), 1544–1555 (2016)CrossRefGoogle Scholar
  44. 44.
    Wang, D., He, H., Mu, C., Liu, D.: Intelligent critic control with disturbance attenuation for affine dynamics including an application to a microgrid system. IEEE Trans. Ind. Electron. 64(6), 4935–4944 (2017)CrossRefGoogle Scholar
  45. 45.
    Werbos, P.J.: Approximate dynamic programming for real-time control and neural modeling. Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493–526 (1992)Google Scholar
  46. 46.
    Xu, B.: Robust adaptive neural control of flexible hypersonic flight vehicle with dead-zone input nonlinearity. Nonlinear Dyn. 80(3), 1509–1520 (2015)MathSciNetCrossRefGoogle Scholar
  47. 47.
    Yang, X., Liu, D., Wang, D.: Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. Int. J. Control 87(3), 553–566 (2014)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Zhang, H., Liu, D., Luo, Y., Wang, D.: Adaptive Dynamic Programming for Control: Algorithms and Stability. Springer, London (2013)Google Scholar
  49. 49.
    Zhang, H., Zhang, J., Yang, G.H., Luo, Y.: Leader-based optimal coordination control for the consensus problem of multiagent differential games via fuzzy adaptive dynamic programming. IEEE Trans. Fuzzy Syst. 23(1), 152–163 (2015)CrossRefGoogle Scholar
  50. 50.
    Zhao, Q., Xu, H., Jagannathan, S.: Near optimal output feedback control of nonlinear discrete-time systems based on reinforcement neural network learning. IEEE/CAA J. Autom. Sin. 1(4), 372–384 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.The State Key Laboratory of Management and Control for Complex SystemsInstitute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.School of Electrical and Information EngineeringTianjin UniversityTianjinChina

Personalised recommendations