Skip to main content
SpringerLink
Log in

Online learning based on adaptive learning rate for a class of recurrent fuzzy neural network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a novel structure of a recurrent interval type-2 TSK fuzzy neural network (RIT2-TSK-FNN) controller based on a reinforcement learning scheme for improving the performance of nonlinear systems using a less number of rules. The parameters of the proposed RIT2-TSK-FNN controller are leaned online using the reinforcement actor–critic method. The controller performance is improved over the time as a result of the online learning algorithm. The controller learns from its own mistakes and faults through the reward and punishment signal from the external environment and seeks to reinforce the RIT2-TSK-FNN controller parameters to converge. In order to obtain less number of rules, the structure learning is performed and thus the RIT2-TSK-FNN rules are obtained online based on the type-2 fuzzy clustering. The online adaptation of the proposed RIT2-TSK-FNN controller parameters is developed using the Levenberg–Marquardt method with adaptive learning rates. The stability analysis is discussed using the Lyapunov theorem. The obtained results show that the proposed RIT2-TSK-FNN controller using the reinforcement actor–critic technique is more preferable than the RIT2-TSK-FNN controller without the actor–critic method under the same conditions. The proposed controller is applied to a nonlinear mathematical system and an industrial process such as a heat exchanger to clarify the robustness of the proposed structure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. El-Nagar AM, El-Bardini M (2014) Practical realization for the interval type-2 fuzzy PD + I controller using a low-cost microcontroller. Arab J Sci Eng 39(8):6463–6476

    Article  Google Scholar 

  2. Shaheen O, El-Nagar AM, El-Bardini M, El-Rabaie NM (2018) Probabilistic fuzzy logic controller for uncertain nonlinear systems. J Frankl Inst 355(3):1088–1106

    Article  MathSciNet  MATH  Google Scholar 

  3. El-Nagar AM, El-Bardini M (2016) Hardware-in-the-loop simulation of interval type-2 fuzzy PD controller for uncertain nonlinear system using low cost microcontroller. Appl Math Model 40(3):2346–2355

    Article  MATH  Google Scholar 

  4. Sozhamadevi N, Sathiyamoorthy S (2015) A probabilistic fuzzy inference system for modeling and control of nonlinear process. Arab J Sci Eng 40(6):1777–1791

    Article  Google Scholar 

  5. Chemachema M (2012) Output feedback direct adaptive neural network control for uncertain SISO nonlinear systems using a fuzzy estimator of the control error. Neural Netw 36:25–34

    Article  MATH  Google Scholar 

  6. Rossomando FG, Soria CM (2017) Discrete-time sliding mode neuro-adaptive controller for SCARA robot arm. Neural Comput Appl 28(12):3837–3850

    Article  Google Scholar 

  7. Wen G, Ge SS, Tu F (2018) Optimized backstepping for tracking control of strict-feedback systems. IEEE Trans Neural Netw Learn Syst 29(8):3850–3862

    Article  MathSciNet  Google Scholar 

  8. Aghababa MP (2016) Optimal design of fractional-order PID controller for five bar linkage robot using a new particle swarm optimization algorithm. Soft Comput 20(10):4055–4067

    Article  Google Scholar 

  9. Kiumarsi B, Vamvoudakis KG, Modares H, Lewis FL (2018) Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans Neural Netw Learn Syst 29(6):2042–2062

    Article  MathSciNet  Google Scholar 

  10. Liu YJ, Li S, Tong S, Chen CP (2018) Adaptive reinforcement learning control based on neural approximation for nonlinear discrete-time systems with unknown nonaffine dead-zone input. IEEE Trans Neural Netw Learn Syst 99:1–11

    Google Scholar 

  11. Khan SG, Herrmann G, Lewis FL, Pipe T, Melhuish C (2012) Reinforcement learning and optimal adaptive control: an overview and implementation examples. Annu Rev Control 36(1):42–59

    Article  Google Scholar 

  12. Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans Syst Man Cybern Part C (Appl Rev) 32(2):140–153

    Article  Google Scholar 

  13. Khater AA, El-Bardini M, El-Rabaie NM (2015) Embedded adaptive fuzzy controller based on reinforcement learning for dc motor with flexible shaft. Arab J Sci Eng 40(8):2389–2406

    Article  MATH  Google Scholar 

  14. Radac MB, Precup RE, Roman RC (2017) Model-free control performance improvement using virtual reference feedback tuning and reinforcement Q-learning. J Syst Sci 48(5):1071–1083

    Article  MathSciNet  MATH  Google Scholar 

  15. Boubertakh H, Tadjine M, Glorennec PY, Labiod S (2010) Tuning fuzzy PD and PI controllers using reinforcement learning. ISA Trans 49(4):543–551

    Article  Google Scholar 

  16. Lewis FL, Liu D (2013) Reinforcement learning and approximate dynamic programming for feedback control. Wiley, Hoboken

    Google Scholar 

  17. Hendzel Z, Szuster M (2011) Discrete neural dynamic programming in wheeled mobile robot control. Commun Nonlinear Sci Numer Simul 16(5):2355–2362

    Article  MathSciNet  MATH  Google Scholar 

  18. Zhang J, Zhang H, Luo Y, Feng T (2014) Model-free optimal control design for a class of linear discrete-time systems with multiple delays using adaptive dynamic programming. Neurocomputing 135:163–170

    Article  Google Scholar 

  19. Wang D, Liu D, Wei Q, Zhao D (2012) Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48(8):1825–1832

    Article  MathSciNet  MATH  Google Scholar 

  20. Zhong X, He H, Zhang H, Wang Z (2015) A neural network based online learning and control approach for Markov jump systems. Neurocomputing 149:116–123

    Article  Google Scholar 

  21. Maharajan C, Raja R, Cao J, Rajchakit G (2018) Novel global robust exponential stability criterion for uncertain inertial-type BAM neural networks with discrete and distributed time-varying delays via Lagrange sense. J Frankl Inst 355(11):4727–4754

    Article  MathSciNet  MATH  Google Scholar 

  22. Sowmiya C, Raja R, Cao J, Rajchakit G (2018) Enhanced result on stability analysis of randomly occurring uncertain parameters, leakage, and impulsive BAM neural networks with time-varying delays: discrete-time case. Int J Adapt Control Signal Process 32(7):1010–1039

    Article  MathSciNet  MATH  Google Scholar 

  23. Sowmiya C, Raj R, Cao J, Li X, Rajchakit G (2018) Discrete-time stochastic impulsive BAM neural networks with leakage and mixed time delays: an exponential stability problem. J Frankl Inst 355(10):4404–4435

    Article  MathSciNet  MATH  Google Scholar 

  24. Sowmiya C, Raja R, Cao J, Rajchakit G, Alsaedi A (2018) Exponential stability of discrete-time cellular uncertain BAM neural networks with variable delays using halanay-type inequality. Appl Math Inf Sci 12(3):545–558

    Article  MathSciNet  Google Scholar 

  25. Sundara V, Raja R, Agarwal R, Rajchakit G (2018) A novel controllability analysis of impulsive fractional linear time invariant systems with state delay and distributed delays in control. Discontin Nonlinearity Complex 7(3):275–290

    Article  MATH  Google Scholar 

  26. Saravanakumar R, Rajchakit G, Ahn CK, Karimi HR (2017) Exponential stability, passivity, and dissipativity analysis of generalized neural networks with mixed time-varying delays. IEEE Trans Syst Man Cybern Syst 49(2):395–405

    Article  Google Scholar 

  27. Song R, Xiao W, Zhang H, Sun C (2014) Adaptive dynamic programming for a class of complex-valued nonlinear systems. IEEE Trans Neural Netw Learn Syst 25(9):1733–1739

    Article  Google Scholar 

  28. Si J, Wang YT (2001) Online learning control by association and reinforcement. IEEE Trans Neural Netw 12(2):264–276

    Article  Google Scholar 

  29. Huang X, Naghdy F, Du H, Naghdy G, Todd C (2015) Reinforcement learning neural network (RLNN) based adaptive control of fine hand motion rehabilitation robot. In: IEEE conference on control applications (CCA), pp 941–946

  30. Shen H, Guo C (2016) Path-following control of underactuated ships using actor-critic reinforcement learning with MLP neural networks. In: IEEE conference information science and technology (ICIST), pp 317–321

  31. Niedzwiedz C, Elhanany I, Liu Z, Livingston S (2008) A consolidated actor-critic model with function approximation for high-dimensional POMDPs. In: AAAI conference, pp 37–42

  32. He H, Ni Z, Fu J (2012) A three-network architecture for on-line learning and optimization based on adaptive dynamic programming. Neurocomputing 78(1):3–13

    Article  Google Scholar 

  33. Ni Z, Tang Y, Sui X, He H, Wen J (2016) An adaptive neuro-control approach for multimachine power systems. Int J Electr Power Energy Syst 75:108–116

    Article  Google Scholar 

  34. Lv Y, Na J, Ren X (2017) Online H∞ control for completely unknown nonlinear systems via an identifier–critic-based ADP structure. Int J Control 92:1–12

    MathSciNet  MATH  Google Scholar 

  35. Khater AA, El-Nagar AM, El-Bardini M, El-Rabaie NM (2018) Adaptive TS fuzzy controller using reinforcement learning based on Lyapunov stability. J Frankl Inst 355(14):6390–6415

    Article  MathSciNet  MATH  Google Scholar 

  36. Fung RF, Lin FJ, Wai RJ, Lu PY (2000) Fuzzy neural network control of a motor-quick-return servomechanism. Mechatronics 10:145–167

    Article  Google Scholar 

  37. Juang CF, Huang RB, Lin YY (2009) A recurrent self-evolving interval type-2 fuzzy neural network for dynamic system processing. IEEE Trans Fuzzy Syst 17(5):1092–1105

    Article  Google Scholar 

  38. Lin CJ, Chin CC (2004) Prediction and identification using wavelet-based recurrent fuzzy neural networks. IEEE Trans Syst Man Cybern Part B (Cybern) 34(5):2144–2154

    Article  Google Scholar 

  39. El-Nagar AM, El-Bardini M (2014) Simplified interval type-2 fuzzy logic system based on new type-reduction. J Intell Fuzzy Syst 27(4):1999–2010

    Article  MathSciNet  MATH  Google Scholar 

  40. El-Nagar AM (2016) Embedded intelligent adaptive PI controller for an electromechanical system. ISA Trans 64:314–327

    Article  Google Scholar 

  41. Deng Z, Choi KS, Cao L, Wang S (2014) T2FELA: type-2 fuzzy extreme learning algorithm for fast training of interval type-2 TSK fuzzy logic system”. IEEE Trans Neural Netw Learn Syst 25(4):664–676

    Article  Google Scholar 

  42. Zhang Z (2018) Trapezoidal interval type-2 fuzzy aggregation operators and their application to multiple attribute group decision making. Neural Comput Appl 29(4):1039–1054

    Article  Google Scholar 

  43. Lin CM, La VH, Le TL (2018) DC–DC converters design using a type-2 wavelet fuzzy cerebellar model articulation controller. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3755-z

    Article  Google Scholar 

  44. El-Bardini M, El-Nagar AM (2014) Interval type-2 fuzzy PID controller: analytical structures and stability analysis. Arab J Sci Eng 39(10):7443–7458

    Article  MATH  Google Scholar 

  45. El-Bardini M, El-Nagar AM (2014) Interval type-2 fuzzy PID controller for uncertain nonlinear inverted pendulum system. ISA Trans 53(3):732–743

    Article  Google Scholar 

  46. Juang CF, Tsao YW (2008) A self-evolving interval type-2 fuzzy neural network with online structure and parameter learning. IEEE Trans Fuzzy Syst 16(6):1411–1424

    Article  Google Scholar 

  47. Lin YY, Chang JY, Lin CT (2014) A TSK-type-based self-evolving compensatory interval type-2 fuzzy neural network (TSCIT2FNN) and its applications. IEEE Trans Ind Electron 61(1):447–459

    Article  Google Scholar 

  48. El-Nagar AM (2018) Nonlinear dynamic systems identification using recurrent interval type-2 TSK fuzzy neural network—A novel structure. ISA Trans 72:205–217

    Article  Google Scholar 

  49. Lin YY, Liao SH, Chang JY, Lin CT (2014) Simplified interval type-2 fuzzy neural networks. IEEE Trans Neural Netw Learn Syst 25(5):959–969

    Article  Google Scholar 

  50. Wiering M, Van Otterlo M (2012) Reinforcement learning. Adapt Learn Optim 12:3

    Article  Google Scholar 

  51. Juang CF, Lin CT (1998) An online self-constructing neural fuzzy inference network and its applications. IEEE Trans Fuzzy Syst 6(1):12–32

    Article  Google Scholar 

  52. Kermani BG, Schiffman SS, Nagle HT (2005) Performance of the Levenberg–Marquardt neural network training method in electronic nose applications. Sensors Actuators B Chem 110(1):13–22

    Article  Google Scholar 

  53. Fu X, Li S, Fairbank M, Wunsch DC, Alonso E (2015) Training recurrent neural networks with the Levenberg–Marquardt algorithm for optimal control of a grid-connected converter. IEEE Trans Neural Netw Learn Syst 26(9):1900–1912

    Article  MathSciNet  Google Scholar 

  54. Liu H (2010) On the Levenberg–Marquardt training method for feed-forward neural networks. In: IEEE international conference on natural computation (ICNC), vol 1. pp 456–460

  55. Astrom KJ, Wittenmark B (2013) Adaptive control. Courier Corporation

  56. Zhang X, Zhang H, Sun Q, Luo Y (2012) Adaptive dynamic programming-based optimal control of unknown nonaffine nonlinear discrete-time systems with proof of convergence. Neurocomputing 91:48–55

    Article  Google Scholar 

  57. Xu D, Jiang B, Shi P (2014) Adaptive observer based data-driven control for nonlinear discrete-time processes. IEEE Trans Autom Sci Eng 11(4):1037–1045

    Google Scholar 

  58. Eskinat E, Johnson SH, Luyben WL (1991) Use of Hammerstein models in identification of nonlinear systems. AIChE J 37(2):255–268

    Article  Google Scholar 

  59. Berger MA, da Fonseca Neto JV (2013) Neurodynamic programming approach for the PID controller adaptation. IFAC Proc 46(11):534–539

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Electronics and Control Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32852, Egypt

    A. Aziz Khater, Ahmad M. El-Nagar, Mohammad El-Bardini & Nabila M. El-Rabaie

Authors
  1. A. Aziz Khater
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmad M. El-Nagar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad El-Bardini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nabila M. El-Rabaie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmad M. El-Nagar.

Ethics declarations

Conflict of interest

There is no conflict of interest between the authors to publish this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A

The term \(\frac{{\partial O^{(6)} \left( n \right)}}{{\partial \theta_{a} (n)}}\) for the actor network is described by the following equations:

$$\begin{aligned} \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial c_{j}^{i} \left( n \right)}} & = \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 4 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 4 \right)} \left( n \right)}}{{\partial c_{j}^{i} \left( n \right)}} + \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 4 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 4 \right)} \left( n \right)}}{{\partial c_{j}^{i} \left( n \right)}}, \\ & = \frac{{\left( {1 - H_{l} + H_{r} } \right) \cdot \bar{O}_{i}^{\left( 3 \right)} \left( n \right) + \left( {1 - H_{r} + H_{l} } \right)\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} }} \cdot O_{j}^{\left( 1 \right)} (n), \\ \end{aligned}$$
(53)
$$\begin{aligned} \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial s_{j}^{i} \left( n \right)}} & = \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 4 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 4 \right)} \left( n \right)}}{{\partial s_{j}^{i} \left( n \right)}} + \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 4 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 4 \right)} \left( n \right)}}{{\partial s_{j}^{i} \left( n \right)}}, \\ & = \frac{{\left( {1 - H_{r} - H_{l} } \right) \cdot \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) - \left( {1 - H_{l} - H_{r} } \right) \cdot \bar{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} }} \cdot \left| {O_{j}^{\left( 1 \right)} (n)} \right|, \\ \end{aligned}$$
(54)
$$\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial m_{j1}^{i} \left( n \right)}} = \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\partial m_{j1}^{i} \left( n \right)}} + \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial m_{j1}^{i} \left( n \right)}},$$
(55)
$$\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial m_{j2}^{i} \left( n \right)}} = \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\partial m_{j2}^{i} \left( n \right)}} + \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial m_{j2}^{i} \left( n \right)}},$$
(56)
$$\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \sigma_{j}^{i} \left( n \right)}} = \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underline{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \bar{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\partial \sigma_{j}^{i} \left( n \right)}} + \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \overline{O}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \sigma_{j}^{i} \left( n \right)}}$$
(57)
$$\begin{aligned} \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial l^{i} \left( n \right)}} & = \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}} + \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \underline{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\partial l^{i} \left( n \right)}} \\ & \quad + \,\left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}} + \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial l^{i} \left( n \right)}}, \\ \end{aligned}$$
(58)
$$\begin{aligned} \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \alpha_{j} \left( n \right)}} = \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}} + \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\partial \alpha_{j} \left( n \right)}}\, \hfill \\ + \left( {\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}} + \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}} \right)\frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \alpha_{j} \left( n \right)}}, \hfill \\ \end{aligned}$$
(59)
$$\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial H_{l} \left( n \right)}} = \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial H_{l} \left( n \right)}} = \frac{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 4 \right)} \left( n \right)\left( {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) - \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} \right)} }}{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} }},$$
(60)
$$\frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial H_{r} \left( n \right)}} = \frac{{\partial O^{\left( 6 \right)} \left( n \right)}}{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial H_{r} \left( n \right)}} = \frac{{\sum\nolimits_{i = 1}^{M} {\bar{O}_{i}^{\left( 4 \right)} \left( n \right)\left( {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} \right)} }}{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} }}.$$
(61)

The derivatives in (55)–(61) are defined as:

$$\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}} = \frac{{H_{l} \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 4 \right)} \left( n \right) - \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} }},$$
(62)
$$\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}_{i}^{\left( 3 \right)} \left( n \right)}} = \frac{{\left( {1 - H_{l} } \right)\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 4 \right)} \left( n \right) - \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 5 \right)} \left( n \right)}}{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} }},$$
(63)
$$\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}} = \frac{{\,\left( {1 - H_{r} \,} \right)\bar{O}_{i}^{\left( 4 \right)} \left( n \right) - \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)\,} }},$$
(64)
$$\frac{{\partial \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\partial \bar{O}_{i}^{\left( 3 \right)} \left( n \right)}} = \frac{{H_{r} \bar{O}_{i}^{\left( 4 \right)} \left( n \right) - \bar{O}^{\left( 5 \right)} \left( n \right)}}{{\sum\nolimits_{i = 1}^{M} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right) + \bar{O}_{i}^{\left( 3 \right)} \left( n \right)} }},$$
(65)
$$\frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial m_{j1}^{i} \left( n \right)}} = \frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial {\bar{\varOmega }}^{i} \left( n \right)}}\frac{{\partial {\bar{\varOmega }}^{i} \left( n \right)}}{{\partial \bar{O}_{ij}^{\left( 2 \right)} \left( n \right)}}\frac{{\partial \bar{O}_{ij}^{\left( 2 \right)} \left( n \right)}}{{\partial m_{j1}^{i} \left( n \right)}} = \left\{ {\begin{array}{*{20}l} {l^{i} {\bar{\varOmega }}^{i} \left( n \right)\frac{{O_{j}^{1} \left( n \right) - m_{j1}^{i} \left( n \right)}}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{2} }},} \hfill & {O_{j}^{1} \left( n \right) < m_{j1}^{i} \left( n \right)} \hfill \\ {0,} \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right.,$$
(66)
$$\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 3 \right)} \left( n \right)}}{{\partial m_{j1}^{i} \left( n \right)}} = \frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{ij}^{\left( 2 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{ij}^{\left( 2 \right)} \left( n \right)}}{{\partial m_{j1}^{i} \left( n \right)}} = \left\{ {\begin{array}{*{20}l} {l^{i} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega } }^{i} \left( n \right)\frac{{O_{j}^{1} \left( n \right) - m_{j1}^{i} \left( n \right)}}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{2} }},} \hfill & {O_{j}^{1} \left( n \right) > \frac{{m_{j1}^{i} \left( n \right) + m_{j2}^{i} \left( n \right)}}{2}} \hfill \\ {0,} \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right.,$$
(67)
$$\frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial m_{j2}^{i} \left( n \right)}} = \frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial {\bar{\varOmega }}^{i} \left( n \right)}}\frac{{\partial {\bar{\varOmega }}^{i} \left( n \right)}}{{\partial \bar{O}_{ij}^{\left( 2 \right)} \left( n \right)}}\frac{{\partial \bar{O}_{ij}^{\left( 2 \right)} \left( n \right)}}{{\partial m_{j2}^{i} \left( n \right)}} = \left\{ {\begin{array}{*{20}l} {l^{i} {\bar{\varOmega }}^{i} \left( n \right)\frac{{O_{j}^{1} \left( n \right) - m_{j2}^{i} \left( n \right)}}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{2} }},} \hfill & {O_{j}^{1} \left( n \right) > m_{j2}^{i} \left( n \right)} \hfill \\ {0,} \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right.,$$
(68)
$$\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 3 \right)} \left( n \right)}}{{\partial m_{j2}^{i} \left( n \right)}} = \frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 3 \right)} \left( n \right)}}{{\partial {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega } }^{i} \left( n \right)}}\frac{{\partial {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega } }^{i} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{ij}^{\left( 2 \right)} \left( n \right)}}\frac{{\partial \underline{O}_{ij}^{\left( 2 \right)} \left( n \right)}}{{\partial m_{j2}^{i} \left( n \right)}} = \left\{ {\begin{array}{*{20}l} {l^{i} {\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega } }^{i} \left( n \right)\frac{{O_{j}^{1} \left( n \right) - m_{j2}^{i} \left( n \right)}}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{2} }},} \hfill & {O_{j}^{1} \left( n \right) \le \frac{{m_{j1}^{i} \left( n \right) + m_{j2}^{i} \left( n \right)}}{2}} \hfill \\ {0,} \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right.,$$
(69)
$$\frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \sigma_{j}^{i} \left( n \right)}} = \frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial {\bar{\varOmega }}^{i} \left( n \right)}}\frac{{\partial {\bar{\varOmega }}^{i} \left( n \right)}}{{\partial \bar{O}_{ij}^{\left( 2 \right)} \left( n \right)}}\frac{{\partial \bar{O}_{ij}^{\left( 2 \right)} \left( n \right)}}{{\partial \sigma_{j}^{i} \left( n \right)}} = \left\{ {\begin{array}{*{20}l} {l^{i} {\bar{\varOmega }}^{i} \left( n \right)\frac{{\left( {O_{j}^{1} \left( n \right) - m_{j1}^{i} \left( n \right)} \right)^{2} }}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{3} }},} \hfill & {O_{j}^{1} \left( n \right) < m_{j1}^{i} \left( n \right)} \hfill \\ {l^{i} {\bar{\varOmega }}^{i} \left( n \right)\frac{{\left( {O_{j}^{1} \left( n \right) - m_{j2}^{i} \left( n \right)} \right)^{2} }}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{3} }},} \hfill & {O_{j}^{1} \left( n \right) < m_{j2}^{i} \left( n \right)} \hfill \\ {0,} \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right.,$$
(70)
$$\begin{aligned} & \frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \sigma_{j}^{i} \left( n \right)}} = \frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{ij}^{\left( 2 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{ij}^{\left( 2 \right)} \left( n \right)}}{{\partial \sigma_{j}^{i} \left( n \right)}} \\ & = \left\{ {\begin{array}{*{20}l} {l^{i} \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)\frac{{\left( {O_{j}^{1} \left( n \right) - m_{j2}^{i} \left( n \right)} \right)^{2} }}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{3} }},} \hfill & {O_{j}^{1} \left( n \right) \le \frac{{m_{j1}^{i} \left( n \right) + m_{j2}^{i} \left( n \right)}}{2}} \hfill \\ {l^{i} \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)\frac{{\left( {O_{j}^{1} \left( n \right) - m_{j1}^{i} \left( n \right)} \right)^{2} }}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{3} }},} \hfill & {O_{j}^{1} \left( n \right) > \frac{{m_{j1}^{i} \left( n \right) + m_{j2}^{i} \left( n \right)}}{2}} \hfill \\ \end{array} ,} \right. \\ \end{aligned}$$
(71)
$$\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\partial l^{i} \left( n \right)}} = \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right) - \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{i}^{(3)} \left( {n - 1} \right),\frac{{\partial \bar{O}_{i}^{\left( 3 \right)} \left( n \right)}}{{\partial l^{i} \left( n \right)}} = {\bar{\varOmega }}^{i} \left( n \right) - \bar{O}_{i}^{(3)} \left( {n - 1} \right),$$
(72)
$$\begin{aligned} & \frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \alpha_{j} \left( n \right)}} = \frac{{\partial \bar{O}^{\left( 3 \right)} \left( n \right)}}{{\partial {\bar{\varOmega }}^{i} \left( n \right)}}\,\frac{{\partial {\bar{\varOmega }}^{i} \left( n \right)}}{{\partial \bar{O}_{ij}^{\left( 2 \right)} \left( n \right)}}\frac{{\partial \bar{O}_{ij}^{\left( 2 \right)} \left( n \right)}}{{\partial O_{j}^{1} \left( n \right)}}\frac{{\partial O_{j}^{1} \left( n \right)}}{{\partial \alpha_{j} \left( n \right)}} \\ & = \left\{ {\begin{array}{*{20}l} { - l^{i} {\bar{\varOmega }}^{i} \left( n \right)O_{j}^{1} \left( {n - 1} \right)\frac{{\left( {O_{j}^{1} \left( n \right) - m_{j1}^{i} \left( n \right)} \right)^{2} }}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{3} }},} \hfill & {O_{j}^{1} \left( n \right) < m_{j1}^{i} \left( n \right)} \hfill \\ { - l^{i} {\bar{\varOmega }}^{i} \left( n \right)O_{j}^{1} \left( {n - 1} \right)\frac{{\left( {O_{j}^{1} \left( n \right) - m_{j2}^{i} \left( n \right)} \right)^{2} }}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{3} }},} \hfill & {O_{j}^{1} \left( n \right) < m_{j2}^{i} \left( n \right)} \hfill \\ {0,} \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right., \\ \end{aligned}$$
(73)
$$\begin{aligned} & \frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \alpha_{j} \left( n \right)}} = \frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}^{\left( 3 \right)} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)}}{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{ij}^{\left( 2 \right)} \left( n \right)}}\frac{{\partial \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{O}_{ij}^{\left( 2 \right)} \left( n \right)}}{{\partial O_{j}^{1} \left( n \right)}}\frac{{\partial O_{j}^{1} \left( n \right)}}{{\partial \alpha_{j} \left( n \right)}} \\ & = \left\{ {\begin{array}{*{20}l} { - l^{i} \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)O_{j}^{1} \left( {n - 1} \right)\frac{{\left( {O_{j}^{1} \left( n \right) - m_{j2}^{i} \left( n \right)} \right)^{2} }}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{3} }},} \hfill & {O_{j}^{1} \left( n \right) \le \frac{{m_{j1}^{i} \left( n \right) + m_{j2}^{i} \left( n \right)}}{2}} \hfill \\ { - l^{i} \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\varOmega }^{i} \left( n \right)O_{j}^{1} \left( {n - 1} \right)\frac{{\left( {O_{j}^{1} \left( n \right) - m_{j1}^{i} \left( n \right)} \right)^{2} }}{{\left( {\sigma_{j}^{i} \left( n \right)} \right)^{3} }},} \hfill & {O_{j}^{1} \left( n \right) > \frac{{m_{j1}^{i} \left( n \right) + m_{j2}^{i} \left( n \right)}}{2}} \hfill \\ \end{array} .} \right. \\ \end{aligned}$$
(74)

Appendix B

Proof

Consider the following Lyapunov candidate:

$$V_{g} \left( n \right) = \frac{1}{2}e_{g}^{2} \left( n \right).$$
(75)

□

For stable training algorithm, \(\Delta V_{g} \left( n \right)\) should be less than zero. Hence, the \(\Delta V_{g} \left( n \right)\) is calculated as the following equation:

$$\begin{aligned} \Delta V_{g} \left( n \right) & = \frac{1}{2}\left( {e_{g}^{2} \left( {n + 1} \right) - e_{g}^{2} \left( n \right)} \right) \\ & = \frac{1}{2}\left( {\left( {e_{g} \left( n \right) + \Delta e_{g} \left( n \right)} \right)^{2} - e_{g}^{2} \left( n \right)} \right) \\ & = \Delta e_{g} \left( n \right)\left( {e_{g} \left( n \right) + \frac{1}{2}\Delta e_{g} \left( n \right)} \right). \\ \end{aligned}$$
(76)

The \(\Delta V_{g} \left( n \right)\) can be rewritten as:

$$\Delta V_{g} \left( n \right) = \left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)\Delta \theta_{g} \left( n \right)\left( {e_{g} \left( n \right) + \frac{1}{2}\left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)\Delta \theta_{g} \left( n \right)} \right).$$
(77)

Thus,

$$\left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)\Delta \theta_{g} \left( n \right) = \left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)\Phi \left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)^{\rm T} e_{g} \left( n \right),$$
(78)

where

$$\Phi = \left( {\left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)^{\rm T} \frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}} + \lambda_{g} \left( n \right)I} \right)^{ - 1} .$$
(79)

By using a matrix inversion lemma, we get [55]:

$$\left( {E + FGH} \right)^{ - 1} = E^{ - 1} - E^{ - 1} F\left( {G^{ - 1} + HE^{ - 1} F} \right)^{ - 1} \,HE^{ - 1} .$$
(80)

According to (80), (79) can be rewritten as:

$$\begin{aligned}\Phi & = \lambda_{g}^{ - 1} \left( n \right)I - \lambda_{g}^{ - 2} \left( n \right)\,\left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)^{\rm T} \left( {I + \frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}\lambda_{g}^{ - 1} \left( n \right)\,\left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)^{\rm T} } \right)^{ - 1} \frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}} \\ & = \lambda_{g}^{ - 1} \left( n \right)I - \lambda_{g}^{ - 1} \left( n \right)\,\left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)^{\rm T} \left( {\lambda_{g} \left( n \right)\,I + \frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}\left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)^{\rm T} } \right)^{ - 1} \frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}. \\ \end{aligned}$$
(81)

Using (81) and (78), we obtain

$$\left( {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right)\Delta \theta_{g} \left( n \right) = \lambda_{g}^{ - 1} \left( n \right)\,\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} \,e_{g} \left( n \right)\, - \lambda_{g}^{ - 1} \left( n \right)\,\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{4} \left( {\lambda_{g} + \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} } \right)^{ - 1} e_{g} \left( n \right).$$
(82)

The time difference of the Lyapunov function \(\Delta V_{g} \left( n \right)\) can be given as:

$$\begin{aligned} \Delta V_{g} \left( n \right) & = - \frac{1}{2}e_{g}^{2} \left( n \right)\,\left( {\lambda_{g}^{ - 1} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} - \lambda_{g}^{ - 1} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{4} } \right.\left. {\left[ {\lambda_{g} \left( n \right) + \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} } \right]^{ - 1} } \right) \\ & \quad \times \,\left( {2 - \lambda_{g}^{ - 1} \left( n \right)\,\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} } \right. + \left. {\lambda_{g}^{ - 1} \left( n \right)\,\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{4} \left[ {\lambda_{g} \left( n \right) + \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} } \right]^{ - 1} } \right). \\ \end{aligned}$$
(83)

Since

$$0 \le \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} ,$$
(84)

thus

$$\lambda_{g}^{ - 1} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{4} \le \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} + \lambda_{g}^{ - 1} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{4} .$$
(85)

By multiplying both sides of (85) by \(\left( {\lambda_{g} \left( n \right) + \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} } \right)^{ - 1}\), we get

$$\lambda_{g}^{ - 1} \left( n \right)\,\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{4} \left( {\lambda_{g} \left( n \right) + \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} } \right)^{ - 1} \le \lambda_{g} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} ,$$
(86)

so that

$$0 \le \lambda_{g} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\| - \lambda_{g}^{ - 1} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{4} \left( {\lambda_{g} \left( n \right) + \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} } \right)^{ - 1} .$$
(87)

Hence, in order that \(\Delta V_{g} \left( n \right) \le 0\),

$$0 \le \left( {2 - \lambda_{g}^{ - 1} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} + \lambda_{g}^{ - 1} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{4} \left( {\lambda_{g} \left( n \right) + \left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} } \right)^{ - 1} } \right).$$
(88)

Thus, the following constraint for the stability is

$$0 \le 2 - \lambda_{g}^{ - 1} \left( n \right)\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} ,$$
(89)

which means

$$\lambda_{g} \left( n \right) \ge \frac{1}{2}\left\| {\frac{{\partial e_{g} \left( n \right)}}{{\partial \theta_{g} \left( n \right)}}} \right\|^{2} .$$
(90)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khater, A.A., El-Nagar, A.M., El-Bardini, M. et al. Online learning based on adaptive learning rate for a class of recurrent fuzzy neural network. Neural Comput & Applic 32, 8691–8710 (2020). https://doi.org/10.1007/s00521-019-04372-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04372-w

Keywords

Search

Navigation