Skip to main content
SpringerLink
Log in

An online fault tolerant actor-critic neuro-control for a class of nonlinear systems using neural network HJB approach

  • Regular Papers
  • Control Theory
  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

In this paper, we propose an actor-critic neuro-control for a class of continuous-time nonlinear systems under nonlinear abrupt faults, which is combined with an adaptive fault diagnosis observer (AFDO). Together with its estimation laws, an AFDO scheme, which estimates the faults in real time, is designed based on Lyapunov analysis. Then, based on the designed AFDO, a fault tolerant actor- critic control scheme is proposed where the critic neural network (NN) is used to approximate the value function and the actor NN updates the fault tolerant policy based on the approximated value function in the critic NN. The weight update laws for critic NN and actor NN are designed using the gradient descent method. By Lyapunov analysis, we prove the uniform ultimately boundedness (UUB) of all the states, their estimation errors, and NN weights of the fault tolerant system under the unpredictable faults. Finally, we verify the effectiveness of the proposed method through numerical simulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Y. Zhang and J. Jiang, “Issues on integration of fault diagnosis and reconfigurable control in active fault tolerant control systems,” Proc. of the 6th IFAC Symposium of Fault Detection Supervision and Safety for Technical Processes, pp. 1513–524, 2006.

    Google Scholar 

  2. H. Niemann and J. Stoustrup, “Passive fault tolerant control of a double inverted pendulum-a case study,” Control Engineering Practice, vol. 13, no. 8, pp. 1047–1059, 2005.

    Article  Google Scholar 

  3. M. M. Polycarpou and J. Helmicki, “Automated fault detection and accommodation: a learning systems approach,” IEEE Trans. Systems, Man, And Cybernetics, vol. 25, no. 11, pp. 1447–1458, 1995.

    Article  Google Scholar 

  4. B. Jiang, J. L. Wang, and Y. C. Soh, “An adaptive technique for robust diagnosis of faults with independent effects on system outputs,” International Journal of Control, vol. 75, no. 11, pp. 792–802, 2002.

    Article  MATH  MathSciNet  Google Scholar 

  5. C. P. Tan and C. Edwards, “Sliding mode observers for robust detection and reconstruction of actuator and sensor faults,” International Journal of Robust and Nonlinear Control, vol. 13, no. 5, pp. 443–463, 2003.

    Article  MATH  MathSciNet  Google Scholar 

  6. R. Sreedhar, B. Fernandez, and G. Y. Masada, “Robust fault detection in nonlinear systems using sliding mode observers,” Proc. of IEEE Conf. on Control Applications, pp. 715–721, 1993.

    Chapter  Google Scholar 

  7. F. L. Lewis, Optimal Control, John Wiley, 1986.

    MATH  Google Scholar 

  8. M. M. Polycarpou, “Stable adaptive neural control scheme for nonlinear Systems,” IEEE Trans. on Automatic Control, vol. 41, no. 3, pp. 447–451, 1996.

    Article  MATH  MathSciNet  Google Scholar 

  9. V. Nevistib and J. A. Primbs, Constrained nonlinear optimal control: a converse HJB approach, California Institute of Technology Pasadena, CA 91125, Tech rep. CIT-CDS 96-021.

  10. Y. Wang, D. Zhou, S. J. Qin, and H. Wang, “Active fault-tolerant control for a class of nonlinear systems with sensor faults,” International Journal of Control, Automation, and System, vol. 6, no. 3, pp. 339–350, 2008.

    Google Scholar 

  11. Z. F. Gao, B. Jiang, P. Shi, and Y. H. Cheng, “Sensor fault estimation and compensation for Microsatellite attitude control systems,” International Journal of Control, Automation, and System, vol. 8, no. 2, pp. 228–237, 2010.

    Article  Google Scholar 

  12. P. Werbos, “Approximate dynamic programming for real-time control and neural modeling,” in Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. Van Nostrand Reinhold, New York, 1992.

    Google Scholar 

  13. A. Al-Tamimi and F. L. Lewis, “Discrete-time Nonlinear HJB solution using approximate dynamic programming: convergence proof,” IEEE Trans. Syst. Man Cybern. Part B Cybern, vol. 38, no. 4, pp. 943–949, August 2008.

    Article  Google Scholar 

  14. D. V. Prokhorov and D. C. Wunsch, “Adaptive critic designs,” IEEE Trans. Neural Networks, vol. 8, no. 5, pp. 997–1007, September 1997.

    Article  Google Scholar 

  15. R. E. Bellman, Dynamic Programming, Princeton Univ. Press, Princeton, NJ, 1957.

    MATH  Google Scholar 

  16. S. E. Dreyfus and A. M. Law, The Art and Theory of Dynamic Programming, Academic, New York, NY, 1977.

    MATH  Google Scholar 

  17. W. B. Powell, Approximate Dynamic Programming Solving the Curses of Dimensionality, Wiley, Princeton, NJ, 2007.

    MATH  Google Scholar 

  18. S. Mohahegi, G. K. Venayagamoorth, and R. G. Harley, “Fully evolvable optimal neurofuzzy controller using adaptive critic designs,” IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 1450–1461, 2008.

    Article  Google Scholar 

  19. S. Mohahegi, G. K. Venayagamoorth, and R. G. Harley, “Adaptive critic design based neuro-fuzzy controller for a static compensator in a multi machine power system,” IEEE Trans. Power Syst., vol. 21, no. 4, pp. 1744–1754, 2006.

    Article  Google Scholar 

  20. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, “Adaptive dynamic programming,” IEEE Trans. Syst., Man., Cybern. C, vol. 32, no. 2, pp. 140–153, 2002.

    Article  Google Scholar 

  21. J. Campos and F. Lewis, “Adaptive critic neural network for feedforward compensation,” Proc. IEEE Am. Control Conf., vol. 4, 1999.

  22. S. Bhasin, M. Johnson, and W. E. Dixon, “A modelfree robust policy iteration algorithm for optimal control of nonlinear systems,” Proc. IEEE Conf. Decis. Control, 2010.

    Google Scholar 

  23. Y. Xu, B. Jiang, G. Tao, and Z. Gao, “Fault tolerant control for a class of nonlinear systems with application to near space vehicle,” Circuits. Syst. Signal. Process, vol. 30, no. 3, pp. 655–672, 2011.

    Article  MATH  MathSciNet  Google Scholar 

  24. S. M. Nail, P. R. Kumar, and B. E. Ydstie, “Robust continuous-time adaptive control by parameter projection,” IEEE Trans. Automat. Contr., vol. 37, no. 2, pp. 182–197, 1992.

    Article  Google Scholar 

  25. K. Doya, “Reinforcement learning in continuous time and space,” Neural Comput., vol. 12, no. 1, pp. 219–245, 2000.

    Article  Google Scholar 

  26. Y. Xu, Y. Li, and S. Tong, “Fuzzy adaptive actuator failure compensation dynamic surface control of multi-input and multi-output nonlinear systems,” Int. J. Innovative Comput. Inf. Control., vol. 9, no. 12, pp. 4875–4888, 2013.

    Google Scholar 

  27. C. Dobre, “A cluster-enhanced fault tolerant peerto-peer system,” Int. J. Innovative Comput. Inf. Control., vol. 10, no. 2, pp. 417–436, 2014.

    Google Scholar 

  28. C. Dobre, “Dynamic route guidance algorithm based on improved hopfield neural network and genetic algorithm,” Int. J. Innovative Comput. Inf. Control., vol. 10, no. 2, pp. 811–822, 2014.

    Google Scholar 

  29. L. Wu, Z. Feng, and W. X. Zheng, “Exponential stability analysis for delayed neural networks with switching parameters: average dwell time approach,” IEEE Trans. Neural Net, vol. 21, no. 9, pp. 1396–1407, 2010.

    Article  Google Scholar 

  30. L. Wu, Z. Feng, and J. Lam, “Stability and synchronization of discrete-time neural networks with switching parameters and time-varying delays,” IEEE Trans. Neural Net, vol. 24, no. 12, pp. 1957–1972, 2013.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, Yonsei University, Shinchon-dong, Seodaemum-gu, Seoul, 120-749, Korea

    Seung Jin Chang, Jae Young Lee & Jin Bae Park

  2. Department of Electronic Engineering, Kyonggi University, Suwon, Kyonggi-do, 443-760, Korea

    Yoon Ho Choi

Authors
  1. Seung Jin Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jae Young Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin Bae Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yoon Ho Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin Bae Park.

Additional information

Seung Jin Chang received his B.S. degree in Electrical and Electronic Engineering from Yonsei University, Seoul, Korea, in 2007. Since 2010, he has been working as a Research Assistant in Control Engineering Laboratory, Yonsei University, Seoul, where he is currently working toward a Ph.D. degree in Electrical and Electronic Engineering. His research interests include dynamic programming applied to fault tolerant control and condition monitoring/diagnosis of cables and signal processing techniques, time-frequency analysis, and estimation theory.

Jae Young Lee received his B.S. degree in Information and Control Engineering from Kwangwoon University, Seoul, Korea, in 2006. He is currently pursuing a Ph.D. degree in Electrical and Electronic Engineering with the Control Engineering Laboratory, Yonsei University, Seoul. He has been a Research Assistant with the Control Engineering Laboratory since 2006. His current research interests include approximate dynamic programming/reinforcement learning, optimal/adaptive control, nonlinear control theories, neural networks, and applications to unmanned vehicles, multiagent systems, robotics, and power systems.

Jin Bae Park received his B.S. degree in Electrical Engineering from Yonsei University, Seoul, Korea, and his M.S. and Ph.D. degrees in Electrical Engineering from Kansas State University, Manhattan, KS, USA, in 1977, 1985, and 1990, respectively. Since 1992, he has been with the Department of Electrical and Electronic Engineering, Yonsei University, where he is currently a Professor. His major research interests include robust control and filtering, nonlinear control, intelligent mobile robot, fuzzy logic control, neural networks, chaos theory, and genetic algorithms. He served as the Editor-in- Chief (2006-2010) for the International Journal of Control, Automation, and Systems, the Vice-President (2009-2011) for Institute of Control, Robot, and Systems Engineers (ICROS), and the President for the ICROS (2013).

Yoon Ho Choi received his B.S., M.S., and Ph.D. degrees in Electrical Engineering from Yonsei University, Seoul, Korea, in 1980, 1982, and 1991, respectively. Since 1993, he has been with Department of Electronic Engineering, Kyonggi University, Suwon, Korea, where he is currently a Professor. He was with Department of Electrical Engineering, The Ohio State University, where he was a Visiting Scholar (2000–2002, 2009–2010). His research interests include nonlinear control, intelligent control, multi-legged and mobile robots, networked control systems, and ADP based control. Prof. Choi was the Director (2003–2004, 2007–2008) of the Institute of Control, Robotics and Systems (ICROS). He is serving as the Vice-President for the ICROS (2012-present).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, S.J., Lee, J.Y., Park, J.B. et al. An online fault tolerant actor-critic neuro-control for a class of nonlinear systems using neural network HJB approach. Int. J. Control Autom. Syst. 13, 311–318 (2015). https://doi.org/10.1007/s12555-014-0034-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-014-0034-3

Keywords

Search

Navigation