Maximum Likelihood-based Multi-innovation Stochastic Gradient Method for Multivariable Systems

  • Huafeng XiaEmail author
  • Yan Ji
  • Yanjun Liu
  • Ling Xu


This paper considers the parameter estimation problems for multivariable controlled autoregressive moving average systems. By means of the decomposition technique, a multivariable system is transformed into several identification submodels according to the number of outputs. A maximum likelihood extended stochastic gradient identification algorithm is derived for identifying each subsystem by using the maximum likelihood principle. In order to improve the convergence rate, a multivariable maximum likelihood-based muti-innovation stochastic gradient algorithm is proposed. The proposed algorithms can generate more accurate parameter estimates compared with the multivariable extended stochastic gradient algorithm. The illustrative simulation results show that the proposed methods work well.


Maximum likelihood multi-innovation identification theory multivariable system parameter estimation stochastic gradient 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Y. Cao, P. Li, and Y. Zhang, “Parallel processing algorithm for railway signal fault diagnosis data based on cloud computing,” Future Generation Computer Systems, vol. 88, pp. 279–283, November 2018.Google Scholar
  2. [2]
    Y. Z. Zhang, Y. Cao, Y. H. Wen, L. Liang, and F. Zou, “Optimization of information interaction protocols in cooperative vehicle–infrastructure systems,” Chinese Journal of Electronics, vol. 27, no. 2, pp. 439–444, March 2018.Google Scholar
  3. [3]
    Y. Cao, L. C. Ma, S. Xiao, X. Zhang, and W. Xu, “Standard analysis for transfer delay in CTCS–3,” Chinese Journal of Electronics, vol. 26, no. 5, pp. 1057–1063, September 2017.Google Scholar
  4. [4]
    Y. Cao, Y. Wen, X. Meng, and W. Xu, “Performance evaluation with improved receiver design for asynchronous coordinated multipoint transmissions,” Chinese Journal of Electronics. vol. 25, no. 2, pp. 372–378, March 2016.Google Scholar
  5. [5]
    A. Karoui, K. I. Taarit, and M. Ksouri, “A new convolution approach for the time–delay identification of systems with arbitrary entries,” International Journal of Control, Automation and Systems, vol. 15, no. 6, pp. 2492–2499, December 2017.Google Scholar
  6. [6]
    X. Zhang, L. Xu, F. Ding, and T. Hayat, “Combined state and parameter estimation for a bilinear state space system with moving average noise,” Journal of the Franklin Institute, vol. 355, no. 6, pp. 3079–3103, April 2018.MathSciNetzbMATHGoogle Scholar
  7. [7]
    M. H. Li and X. M. Liu, “Auxiliary model based least squares iterative algorithms for parameter estimation of bilinear systems using interval–varying measurements,” IEEE Access, vol. 6, pp. 21518–21529, 2018.Google Scholar
  8. [8]
    M. H. Li and X. M. Liu, “The least squares based iterative algorithms for parameter estimation of a bilinear system with autoregressive noise using the data filtering technique,” Signal Processing, vo. 147, pp. 23–34, June 2018.Google Scholar
  9. [9]
    G. L. Ji, Y. Z. Wang, S. Y. Zhao, Y. L. Liu, K. K. Zhang, B. Yao, and S. Zhou, “Bayesian hybrid state estimation for unequal–length batch processes with incomplete observations,” International Journal of Control, Automation and Systems, vol. 15, no. 6, pp. 2480–2491, December 2017.Google Scholar
  10. [10]
    M. Gan, H. X. Li, and H. Peng, “A variable projection approach for efficient estimation of RBF–ARX model,” IEEE Transactions on Cybernetics, vol. 45, no. 3, pp. 462–471, March 2015.Google Scholar
  11. [11]
    M. Gan, C. L. P. Chen, G. Y. Chen, and L. Chen, “On some separated algorithms for separable nonlinear squares problems,” IEEE Transactions on Cybernetics, vol. 48, no. 10, pp. 2866–2874, October 2018.Google Scholar
  12. [12]
    Z. P. Zhou and X. F. Liu, “State and fault estimation of sandwich systems with hysteresis,” International Journal of Robust and Nonlinear Control, vol. 28, no. 13, pp. 3974–3986, September 2018.MathSciNetzbMATHGoogle Scholar
  13. [13]
    X. T. Wu, Y. Tang, and W. B. Zhang, “Input–to–state stability of impulsive stochastic delayed systems under linear assumptions,” Automatica, vol. 66, pp. 195–204, 2016.MathSciNetzbMATHGoogle Scholar
  14. [14]
    X. T. Wu, Y. Tang, and W. B. Zhang, “Stability analysis of stochastic delayed systems with an application to multiagent systems,” IEEE Transactions on Automatic Control, vol. 61, no. 12, pp. 4143–4149, December 2016.Google Scholar
  15. [15]
    J. Chen, B. Jiang, and J. Li, “Missing output identification model based recursive least squares algorithm for a distributed parameter system,” International Journal of Control, Automation and Systems, vol. 16, no. 1, pp. 150–157, February 2018.Google Scholar
  16. [16]
    L. Xu and F. Ding, “Parameter estimation for control systems based on impulse responses,” International Journal of Control Automation and Systems, vol. 15, no. 6, pp. 2471–2479, December 2017.Google Scholar
  17. [17]
    L. Xu, “The parameter estimation algorithms based on the dynamical response measurement data,” Advances in Mechanical Engineering, vol. 9, no. 11, pp. 1–12, November 2017.Google Scholar
  18. [18]
    J. L. Ding, “Recursive and iterative least squares parameter estimation algorithms for multiple–input–output–error systems with autoregressive noise,” Circuits, Systems and Signal Processing, vol. 37, no. 5, pp. 1884–1906, May 2018.MathSciNetGoogle Scholar
  19. [19]
    J. Pan, X. Jiang, X. K. Wan, and W. Ding, “A filtering based multi–innovation extended stochastic gradient algorithm for multivariable control systems,” International Journal of Control,Automation and Systems, vol. 15, no. 3, pp. 1189–1197, June 2017.Google Scholar
  20. [20]
    X. Zhang, F. Ding, A. Alsaadi, and T. Hayat, “Recursive parameter identification of the dynamical models for bilinear state space systems,” Nonlinear Dynamics, vol. 89, no. 4, pp. 2415–2429, September 2017.MathSciNetzbMATHGoogle Scholar
  21. [21]
    L. Xu and F. Ding, “Iterative parameter estimation for signal models based on measured data,” Circuits, Systems and Signal Processing, vol. 37, no. 7, pp. 3046–3069, July 2018.MathSciNetGoogle Scholar
  22. [22]
    L. Xu, W. L. Xiong, A. Alsaedi, and T. Hayat, “Hierarchical parameter estimation for the frequency response based on the dynamical window data,” International Journal of Control, Automation and Systems, vol. 16, no. 4, pp. 1756–1764, August 2018.Google Scholar
  23. [23]
    Y. J. Wang and F. Ding, “A filtering based multi–innovation gradient estimation algorithm and performance analysis for nonlinear dynamical systems,” IMA Journal of Applied Mathematics, vol. 82, no. 6, pp. 1171–1191, November 2017.MathSciNetGoogle Scholar
  24. [24]
    P. C. Young, “Refined instrumental variable estimation: maximum likelihood optimization of a unified Box–Jenkins model,” Automatica, vol. 52, pp. 35–46, February 2015.MathSciNetzbMATHGoogle Scholar
  25. [25]
    T. Söderström and U. Soverini, “Errors–in–variables identification using maximum likelihood estimation in the frequency domain,” Automatica, vol. 79, pp. 131–143, May 2017.MathSciNetzbMATHGoogle Scholar
  26. [26]
    F. Y. Chen and F. Ding, “The filtering based maximum likelihood recursive least squares estimation for multiple–input single–output systems,” Applied Mathematical Modelling, vol. 40, no. 3, pp. 2106–2118, February 2016.MathSciNetGoogle Scholar
  27. [27]
    F. Ding, H. B. Chen, L. Xu, J.Y. Dai, Q.S. Li, and T. Hayat, “A hierarchical least squares identification algorithm for Hammerstein nonlinear systems using the key term separation,” Journal of the Franklin Institute, vol. 355, no. 8, pp. 3737–3752, May 2018.MathSciNetzbMATHGoogle Scholar
  28. [28]
    X. Zhang, F. Ding, L. Xu, and E. F. Yang, “State filteringbased least squares parameter estimation for bilinear systems using the hierarchical identification principle,” IET Control Theory and Applications, vol. 12, no. 12, pp. 1704–1713, August 2018.MathSciNetGoogle Scholar
  29. [29]
    P. Ma, F. Ding, and Q. M. Zhu, “Decomposition–based recursive least squares identification methods for multivariate pseudolinear systems using the multi–innovation,” International Journal of Systems Science, vol. 49, no. 5, pp. 920–928, April 2018.MathSciNetGoogle Scholar
  30. [30]
    H. F. Xia, Y. Ji, L. Xu, and T. Hayat, “Maximum likelihood–based recursive least–squares algorithm for multivariable systems with colored noises using the decomposition technique,” Circuits, Systems and Signal Processing, vol. 38, 2019.Google Scholar
  31. [31]
    F. Ding, “Decomposition based fast least squares algorithm for output error systems,” Signal Processing, vol. 93, no. 5, pp. 1235–1242, May 2013.Google Scholar
  32. [32]
    F. Ding, “Two–stage least squares based iterative estimation algorithm for CARARMA system modeling,” Applied Mathematical Modelling, vol. 37. no. 7, 4798.4808, April 2013.Google Scholar
  33. [33]
    F. Ding, Y. J. Liu, and B. Bao, “Gradient based and least squares based iterative estimation algorithms for multiinput multi–output systems,” Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, vol. 226, no. 1, pp. 43–55, February 2012.Google Scholar
  34. [34]
    F. Ding, X. P. Liu, and G. Liu, “Gradient based and leastsquares based iterative identification methods for OE and OEMA systems,” Digital Signal Processing, vol. 20, no. 3, pp. 664–677, May 2010.Google Scholar
  35. [35]
    F. Ding and H. M. Zhang, “Gradient–based iterative algorithm for a class of the coupled matrix equations related to control systems,” IET Control Theory and Applications, vol. 8, no. 15, pp. 1588–1595, October 2014.MathSciNetGoogle Scholar
  36. [36]
    F. Liu, “A note on Marcinkiewicz integrals associated to surfaces of revolution,” Journal of the Australian Mathematical Society, vol. 104, no. 3, pp. 380–402, June 2018.MathSciNetzbMATHGoogle Scholar
  37. [37]
    F. Liu and H. X. Wu, “Singular integrals related to homogeneous mappings in triebel–lizorkin spaces,” Journal of Mathematical Inequalities, vol. 11, no. 4, pp. 1075–1097, December 2017.MathSciNetzbMATHGoogle Scholar
  38. [38]
    J. Pan, W. Li, and H. P. Zhang, “Control algorithms of magnetic suspension systems based on the improved double exponential reaching law of sliding mode control,” International Journal of Control Automation and Systems, vol. 16, no. 6, pp. 2878–2887, December 2018.Google Scholar
  39. [39]
    F. Liu and H. X. Wu, “A note on the endpoint regularity of the discrete maximal operator,” Proceedings of the American Mathematical Society, vol. 147, no. 2, pp. 583–596, February 2019.MathSciNetzbMATHGoogle Scholar
  40. [40]
    X. Li and D. Q. Zhu, “An adaptive SOM neural network method for distributed formation control of a group of AUVs,” IEEE Transactions on Industrial Electronics, vol. 65, no. 10, pp. 8260–8270, October 2018.Google Scholar
  41. [41]
    F. Z. Geng and S. P. Qian, “An optimal reproducing kernel method for linear nonlocal boundary value problems,” Applied Mathematics Letters, vol. 77, pp. 49–56, March 2018.MathSciNetzbMATHGoogle Scholar
  42. [42]
    X. Y. Li and B. Y. Wu, “A new reproducing kernel collocation method for nonlocal fractional boundary value problems with non–smooth solutions,” Applied Mathematics Letters, vol. 86, pp. 194–199, December 2018.MathSciNetGoogle Scholar
  43. [43]
    P. C. Gong, W. Q. Wang, F. C. Li, and H. Cheung, “Sparsity–aware transmit beamspace design for FDAMIMO radar,” Signal Processing, vol. 144, pp. 99–103, March 2018.Google Scholar
  44. [44]
    Z. H. Rao, C. Y. Zeng, M. H. Wu, Z. F. Wang, N. Zhan, M. Liu, and X. K. Wan, “Research on a handwritten character recognition algorithm based on an extended nonlinear kernel residual network,” KSII Transactions on Internet and Information Systems, vol. 12, no. 1, pp. 413–435, January 2018.Google Scholar
  45. [45]
    N. Zhao, R. Liu, Y. Chen, M. Wu, Y. Jiang, W. Xiong, and C. Liu, “Contract design for relay incentive mechanism under dual asymmetric information in cooperative networks,” Wireless Networks, vol. 24, no. 8, pp. 3029–3044, November 2018.Google Scholar
  46. [46]
    M. H. Li, X. M. Liu, and F. Ding, “Filtering–based maximum likelihood gradient iterative estimation algorithm for bilinear systems with autoregressive moving average noise,” Circuits, Systems, and Signal Processing, vol. 37, no. 11, pp. 5023–5048, November 2018.MathSciNetGoogle Scholar
  47. [47]
    Y. Ji and F. Ding, “Multiperiodicity and exponential attractivity of neural networks with mixed delays,” Circuits, Systems, and Signal Processing, vol. 36, no. 6, pp. 2558–2573, June, 2017.MathSciNetzbMATHGoogle Scholar

Copyright information

© Institute of Control, Robotics and Systems and The Korean Institute of Electrical Engineers and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Internet of Things EngineeringJiangnan UniversityWuxiP. R. China
  2. 2.College of Automation and Electronic EngineeringQingdao University of Science and TechnologyQingdaoP. R. China
  3. 3.Taizhou Electric Power Conversion and Control Engineering Technology Research CenterTaizhou UniversityTaizhouP. R. China
  4. 4.Department of MathematicsKing Abdulaziz UniversityJeddahSaudi Arabia

Personalised recommendations