Skip to main content

Fractional-Based Stochastic Gradient Algorithms for Time-Delayed ARX Models

Abstract

In this study, two fractional-based stochastic gradient (FSG) algorithms for time-delayed auto-regressive exogenous (ARX) models are proposed. By combining momentum and adaptive methods, a momentum-based FSG and an adaptive-based FSG algorithms are developed. These two FSG algorithms have faster convergence rates when compared with the stochastic gradient algorithm. The mechanism of the convergence is proved in theory. Furthermore, two simulated examples are presented to illustrate the efficiency of the new proposed algorithms.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Data Availability Statement

All data generated or analyzed during this study are included in this article.

References

  1. 1.

    H. Akcay, P.P. Khargonekar, The least squares algorithm, parametric system identification and bounded noise. Automatica 29(6), 1535–1540 (1993)

    MathSciNet  MATH  Article  Google Scholar 

  2. 2.

    R. Almeida, A caputo fractional derivative of a function with respect to another function. Commun. Nonlinear Sci. Numer. Simul. 44, 460–481 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  3. 3.

    Y. Bengio, P. Simard, P. Frasconi, Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural. Netw. 5(2), 157–166 (1994)

    Article  Google Scholar 

  4. 4.

    I.R. Birs, C.I. Muresan, S. Folea, O. Prodan, A comparison between integer and fractional order Pd\(\mu \) controllers for vibration suppression. Appl. Math. Comput. 1(1), 273–282 (2016)

    MathSciNet  MATH  Google Scholar 

  5. 5.

    L. Bottou, Large-scale machine learning with stochastic gradient descent, in Proceedings of Compstat-2010, pp. 177–186 (2010)

  6. 6.

    N.I. Chaudhary, S. Zubair, M.A.Z. Raja, A new computing approach for power signal modeling using fractional adaptive algorithms. ISA Trans. 68, 189–202 (2017)

    Article  Google Scholar 

  7. 7.

    J. Chen, F. Ding, Y.J. Liu, Q.M. Zhu, Multi-step-length gradient iterative algorithm for equation-error type models. Syst. Control Lett. 115, 15–21 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  8. 8.

    J. Chen, B. Huang, F. Ding, Y. Gu, Variational Bayesian approach for ARX systems with missing observations and varying time-delays. Automatica 94, 194–204 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  9. 9.

    J. Chen, Y.J. Liu, Variational Bayesian-based iterative algorithm for ARX models with random missing outputs. Circuits Syst. Signal Process. 37(4), 1594–1608 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  10. 10.

    J. Chen, Q.M. Zhu, M.F. Hu, L.X. Guo, P. Narayan, Improved gradient descent algorithms for time-delay rational state-space systems: intelligent search method and momentum method. Nonlinear Dyn. 101(1), 361–373 (2020)

    Article  Google Scholar 

  11. 11.

    S.S. Cheng, Study on Fractional Order LMS Adaptive Filtering Algorithm (University of Science and Technology of China, Anhui, 2018)

    Google Scholar 

  12. 12.

    F. Ding, H.B. Chen, L. Xu, J.Y. Dai, Q.S. Li, T. Hayat, A hierarchical least squares identification algorithm for Hammerstein nonlinear systems using the key term separation. J. Frankl. Inst. 355(8), 3737–3752 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  13. 13.

    N. Doelman, The minimum of the time-delay wavefront error in adaptive optics. Mon. Notices R. Astron. Soc. 491(4), 4719–4723 (2019)

    Google Scholar 

  14. 14.

    R.Q. Dong, Y. Zhang, A.G. Wu, Weighted hierarchical stochastic gradient identification algorithms for ARX models. Int. J. Syst. Sci. 52(2), 363–373 (2021)

    MathSciNet  Article  Google Scholar 

  15. 15.

    J.X. Feng, D. Lu, Stochastic gradient-based particle filtering method for ARX models with nonlinear communication output sub-model. Int. J. Model. Identif. Control 31(4), 331–336 (2019)

    Article  Google Scholar 

  16. 16.

    M. Gan, Y. Guan, G.Y. Chen, C.P. Chen, Recursive variable projection algorithm for a class of separable nonlinear models. IEEE Trans. Neural Netw. Learn. Syst. 1–12 (2020). https://doi.org/10.1109/TNNLS.2020.3026482

  17. 17.

    M. Gan, C.L.P. Chen, G.Y. Chen, L. Chen, On some separated algorithms for separable nonlinear squares problems. IEEE Trans. Cybern. 48(10), 2866–2874 (2018)

    Article  Google Scholar 

  18. 18.

    P. Gao, Y. Gao, Quadrilateral interval Type-2 fuzzy regression analysis for data outlier detection. Math. Probl. Eng. 2019, 4914593 (2019)

    MathSciNet  MATH  Google Scholar 

  19. 19.

    Y. Gao, J. Liu, Z. Wang, L. Wu, Interval type-2 FNN-based quantized tracking control for hypersonic flight vehicles with prescribed performance. IEEE Trans. Syst. Man Cybern. Syst. 51(3), 1–13 (2019)

    Google Scholar 

  20. 20.

    J. Gmez, H. Pinedo, C. Uzctegui, The open mapping principle for partial actions of Polish groups. J. Math. Anal. Appl. 462(1), 337–346 (2018)

    MathSciNet  Article  Google Scholar 

  21. 21.

    T.T. Hartley, C.F. Lorenzo, Fractional-order system identification based on continuous order-distributions. Signal Process. 83(11), 2287–2300 (2003)

    MATH  Article  Google Scholar 

  22. 22.

    M.A. Henson, D.E. Seborg, Time delay compensation for nonlinear processes. Ind. Eng. Chem. Res. 33(6), 1493–1500 (1994)

    Article  Google Scholar 

  23. 23.

    M. Jiao, D. Wang, Y. Yang, F. Liu, More intelligent and robust estimation of battery state-of-charge with an improved regularized extreme learning machine. Eng. Appl. Artif. Intell. 104, 104407 (2021)

    Article  Google Scholar 

  24. 24.

    M. Jiao, D. Wang, J. Qiu, A GRU-RNN based momentum optimized algorithm for SOC estimation. J. Power Sources 459, 228051 (2020)

    Article  Google Scholar 

  25. 25.

    Z.A. Khan, S. Zubair, H. Alquhayz, M. Azeem, A. Ditta, Design of momentum fractional stochastic gradient descent for recommender systems. IEEE Access 7, 179575–179590 (2019)

    Article  Google Scholar 

  26. 26.

    A.M. Khan, R.K. Kumbhat, A. Chouhan, A. Alaria, Generalized fractional integral operators and M-series. J. Math. 2016, 1–10 (2016)

    MathSciNet  MATH  Article  Google Scholar 

  27. 27.

    M.H. Li, X.M. Liu, F. Ding, Filtering-based maximum likelihood gradient iterative estimation algorithm for bilinear systems with autoregressive moving average noise. Circuits Syst. Signal Process. 37(11), 5023–5048 (2018)

    MathSciNet  Article  Google Scholar 

  28. 28.

    P. Ma, F. Ding, A. Alsaedi, T. Hayat, Decomposition-based gradient estimation algorithms for multivariate equation-error autoregressive systems using the multi-innovation theory. Circuits Syst. Signal Process. 37(5), 1846–1862 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  29. 29.

    J.G. Milton, Time delays and the control of biological systems: an overview. IFAC-Pap. OnLine 48(12), 87–92 (2015)

    Article  Google Scholar 

  30. 30.

    D. Needell, N. Srebro, R. Ward, Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm. Math. Program. 155(1), 549–573 (2016)

    MathSciNet  MATH  Article  Google Scholar 

  31. 31.

    K. Owolabi, Riemann–Liouville fractional derivative and application to model chaotic differential equations. Prog. Fract. Differ. Appl. 4, 99–110 (2018)

    Article  Google Scholar 

  32. 32.

    X.Y. Peng, L. Li, F.Y. Wang, Accelerating minibatch stochastic gradient descent using typicality sampling. IEEE Trans. Neural Netw. Learn. Syst. 31(11), 4649–4659 (2020)

    MathSciNet  Article  Google Scholar 

  33. 33.

    R. Scherer, S.L. Kalla, Y.F. Tang, J.F. Huang, The Grunwald–Letnikov method for fractional differential equations. Comput. Math. with Appl. 62(3), 902–917 (2011)

    MathSciNet  MATH  Article  Google Scholar 

  34. 34.

    J. Schoukens, L. Ljung, Nonlinear system identification: a user-oriented road map. IEEE Control Syst. Mag. 39(6), 28–99 (2019)

    MathSciNet  Google Scholar 

  35. 35.

    M.D.L. Sen, About robust stability of Caputo linear fractional dynamic systems with time delays through fixed point theory. Fixed Point Theory Appl. 2011(1), 867932 (2011)

    MathSciNet  MATH  Article  Google Scholar 

  36. 36.

    M.D.L. Sen, N.S. Luo, Discretization and FIR filtering of continuous linear systems with internal and external point delays. Int. J. Control 60(6), 1223–1246 (1994)

    MathSciNet  MATH  Article  Google Scholar 

  37. 37.

    S.M. Shah, R. Samar, N.M. Khan, M.A.Z. Raja, Fractional-order adaptive signal processing strategies for active noise control systems. Nonlinear Dyn. 85(3), 1363–1376 (2016)

    MathSciNet  Article  Google Scholar 

  38. 38.

    C. Song, Y.D. Wang, B.Q. Xu, Research on time delay of high-speed EMU network control system based on BP neural network. Technol. Innov. Appl. 9, 1–5 (2020)

    Google Scholar 

  39. 39.

    Y. Tan, Z. He, B. Tian, A novel generalization of modified LMS algorithm to fractional order. IEEE Signal Process. Lett. 22(9), 1244–1248 (2015)

    Article  Google Scholar 

  40. 40.

    L.J. Wan, F. Ding, Decomposition-and gradient-based iterative identification algorithms for multivariable systems using the multi-innovation theory. Circuits Syst. Signal Process. 38(7), 2971–2991 (2019)

    Article  Google Scholar 

  41. 41.

    D.Q. Wang, Q.H. Fan, Y. Ma, An interactive maximum likelihood estimation method for multivariable Hammerstein systems. J. Frankl. Inst. 357, 12986–13005 (2020)

    MathSciNet  MATH  Article  Google Scholar 

  42. 42.

    D.Q. Wang, L.W. Li, Y. Ji, Y.R. Yan, Model recovery for Hammerstein systems using the auxiliary model based orthogonal matching pursuit method. Appl. Math. Model. 54, 537–550 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  43. 43.

    C. Wang, K.C. Li, Aitken-based stochastic gradient algorithm for ARX models with time delay. Circuits Syst. Signal Process. 38(6), 2863–2876 (2019)

    Article  Google Scholar 

  44. 44.

    J. Wang, W. Luo, J. Liu, L. Wu, Adaptive type-2 FNN-based dynamic sliding mode control of DC–DC boost converters. IEEE Trans. Syst. Man Cybern. Syst. 99, 1–12 (2019)

    Google Scholar 

  45. 45.

    D.Q. Wang, F. Ding, Extended stochastic gradient identification algorithms for Hammerstein–Wiener ARMAX systems. Comput. Math. Appl. 56, 3157–3164 (2008)

    MathSciNet  MATH  Article  Google Scholar 

  46. 46.

    D.Q. Wang, S. Zhang, M. Gan, J.L. Qiu, A novel EM identification method for Hammerstein systems with missing output data. IEEE Trans. Ind. Inf. 16(4), 2500–2508 (2020)

    Article  Google Scholar 

  47. 47.

    Y.H. Wei, M. Zhu, C. Peng, Y. Wang, Robust stability criteria for uncertain fractional order systems with time delay. Control Decis. 3, 511–516 (2014)

    Google Scholar 

  48. 48.

    H.F. Xia, Y.Q. Yang, F. Ding, A. Alsaedi, T. Hayat, Maximum likelihood-based recursive least-squares estimation for multivariable systems using the data filtering technique. Int. J. Syst. Sci. 50(6), 1121–1135 (2019)

    MathSciNet  Article  Google Scholar 

  49. 49.

    H. Yan, X.F. Zhang, Adaptive fractional multi-scale edge-preserving decomposition and saliency detection fusion algorithm. ISA Trans. 107, 160–172 (2020)

    Article  Google Scholar 

  50. 50.

    S. Zhang, D.Q. Wang, Y.R. Yan, Instrumental variable-based OMP identification algorithm for Hammerstein systems. Complexity 2018, 1–10 (2018)

    MATH  Google Scholar 

  51. 51.

    S.H. Zhang, F. Zheng, X. Li, An attitude algorithm based on variable-step-size momentum gradient descent method. Electron. Opt. Control 27(9), 66–70 (2020)

    Google Scholar 

  52. 52.

    Y.T. Zhao, Z.Y. Dan, H.F. Long, H.P. Liu, X.C. Hao, Dynamic soft measurement modeling method of T-LSSVR in systems with time delay. Acta Metrological Sinica 40(1), 146–152 (2019)

    Google Scholar 

Download references

Acknowledgements

The authors would like to express their gratitude to the editors and anonymous reviewers for their helpful comments and constructive suggestions regarding the revision of this paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jing Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by the National Natural Science Foundation of China (No. 61973137), the Fundamental Research Funds for the Central Universities (No. JUSRP22016) and the Funds of the Science and Technology on Near-Surface Detection Laboratory (No. TCGZ2019A001)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xu, T., Chen, J., Pu, Y. et al. Fractional-Based Stochastic Gradient Algorithms for Time-Delayed ARX Models. Circuits Syst Signal Process (2021). https://doi.org/10.1007/s00034-021-01874-8

Download citation

Keywords

  • Parameter estimation
  • Time delay
  • Fractional derivative
  • ARX model
  • Momentum method
  • Adaptive method