Skip to main content
SpringerLink
Log in

Time delay system identification using controlled recurrent neural network and discrete bayesian optimization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Deep learning methods have been widely studied in system modeling due to their strong abilities in feature representation and function fitting. However, most deep learning models are end-to-end black box models, and some key information of the system (such as time delay) cannot be obtained. This paper proposes a grey box model that combines discrete bayesian optimization (DBO) and controlled recurrent neural network (CRNN), namely CRNN-DBO model, aiming at modeling and time delay identification for time delay systems. CRNN is introduced and learns to map the relationship between input data and output data by backpropagation algorithm, while the unknown time delays are modeled as hyperparameters of the mask layer in the CRNN model which are identified by DBO method. The backpropagation algorithm and the DBO method are combined to find the minimal loss value of the model, and the correct time delays as well. To ensure the convergence of the DBO method, l2 regularization term of the mask layer is added in the loss function. The effectiveness and robustness of the model are verified through simulation in the situations of short time delay, long time delay, and nonlinear time delay system. The results indicate that time delays are accurately identified, and the prediction error is smaller than other models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Normey-Rico JE, Camacho EF (2007) Control of dead-time processes. Springer, London

  2. Wu H, Tsakalis KS, Heydt GT (2004) Evaluation of time delay effects to wide-area power system stabilizer design. IEEE Trans Power Syst 19(4):1935–1941

    Article  Google Scholar 

  3. Richard JP (2003) Time-delay systems: an overview of some recent advances and open problems. Automatica 39(10):1667–1694

    Article  MathSciNet  MATH  Google Scholar 

  4. Barreto GA, Souza LgM (2016) Novel approaches for parameter estimation of local linear models for dynamical system identification. Appl Intell 44(1):149–165

    Article  Google Scholar 

  5. Yang Y, Yang B, Niu M (2017) Adaptive infinite impulse response system identification using opposition based hybrid coral reefs optimization algorithm. Appl Intell 48:1689–1706

    Article  Google Scholar 

  6. Jiménez EC, Díaz P, Ávalos O, Zaldivar D, Cisneros M (2017) Nonlinear system identification based on anfis-hammerstein model using gravitational search algorithm. Appl Intell 48:182–203

    Google Scholar 

  7. Singh S, Ashok A, Kumar M, Rawat T (2018) Adaptive infinite impulse response system identification using teacher learner based optimization algorithm. Appl Intell 49:1785–1802

    Article  Google Scholar 

  8. Li J, Zheng Y, Lin Z (2014) Recursive identification of time-varying systems: Self-tuning and matrix rls algorithms. Syst Control Lett 66:104–110

    Article  MathSciNet  MATH  Google Scholar 

  9. Qiang LIU, Jia-chen MA (2016) Subspace-based identification of discrete time-delay system. Front Inf Technol Electron Eng 17(6):566–575

    Article  Google Scholar 

  10. Liu Y, Ding R (2013) Consistency of the extended gradient identification algorithm for multi-input multi-output systems with moving average noises. Int J Comput Math 90(9):1840–1852

    Article  MathSciNet  MATH  Google Scholar 

  11. Sharma S, Padhy PK (2020) A novel iterative system identification and modeling scheme with simultaneous time-delay and rational parameter estimation. IEEE Access PP(99):1–12

    Google Scholar 

  12. Bedoui S, Ltaief M, Abderrahim K (2013) A new generalized vector observation for discrete-time delay systems identification, pp 1922–1927

  13. Bedoui S, Ltaief M, Abderrahim K (2013) Hierarchical gradient based identification of discrete-time delay systems, pp 4565–4570

  14. Chen F, Liu T (2017) Iterative identification of discrete-time output-error model with time delay. J Central South Univ 24(3):647–654

    Article  Google Scholar 

  15. Chen J, Ma J, Liu Y, Ding F (2017) Identification methods for time-delay systems based on the redundant rules. Signal Process 137:192–198

    Article  Google Scholar 

  16. Higo T, Kawaguchi T, Itamoto M, Adachi S (2018) Recursive identification of fractional time-delay systems using discrete-time model. IFAC-PapersOnLine 51(15):939–944

    Article  Google Scholar 

  17. Ghanes M, Deleon J, Barbot JP (2016) Simultaneous observation and identification for nonlinear systems under unknown time-varying delays. J Frankl Inst-Eng Appl Math 353(10):2305–2318

    Article  MathSciNet  MATH  Google Scholar 

  18. Bayrak A, Tatlicioglu E (2013) Online time delay identification and control for general classes of nonlinear systems. Trans Inst Measur Control 35(6):808–823

    Article  Google Scholar 

  19. Li J, Zong T, Lu G (2021) Parameter identification of hammerstein–wiener nonlinear systems with unknown time delay based on the linear variable weight particle swarm optimization. ISA transactions

  20. Tlili AS (2021) H\(\infty \) optimization-based stabilization for nonlinear disturbed time delay systems. J Control Autom Electr Syst 32(1):96–108

    Article  Google Scholar 

  21. Wang D, Wang Z, Chen M, Wang W (2018) Distributed optimization for multi-agent systems with constraints set and communication time-delay over a directed graph. Inf Sci 438:1–14

    Article  MathSciNet  MATH  Google Scholar 

  22. Li C, Yan H (2017) Nonlinear time-delay system identification based on multi-dimensional taylor network and ipso. In: 2017 International Conference on Grey Systems and Intelligent Services (GSIS)

  23. Li C, Yan H (2018) Identification of nonlinear time-delay system using multi-dimensional taylor network model. In: 2018 IEEE International Conference on Manipulation, Manufacturing and Measurement on the Nanoscale (3M-NANO)

  24. Ren XM, Rad AB (2007) Identification of nonlinear systems with unknown time delay based on time-delay neural networks. IEEE Trans Neural Netw 18(5):1536–1541

    Article  Google Scholar 

  25. Wang YH, Zhao YD, Hao XJ, Cheng PX (2008) Time-delay identification for linear multi-input multi-output system. In: World Congress on Intelligent Control & Automation

  26. Shahriari B, Swersky K, Wang Z, Adams RP, De Freitas N (2016) Taking the human out of the loop: A review of bayesian optimization. Proc IEEE 104(1):148–175

    Article  Google Scholar 

  27. Luong P, Gupta S, Nguyen D, Rana S, Venkatesh S (2019) Bayesian optimization with discrete variables. Springer International Publishing, Cham, pp 473–484

    Google Scholar 

  28. Rasmussen CE (2003) Gaussian processes in machine learning. In: Summer School on Machine Learning. Springer, pp 63–71

  29. Srinivas N, Krause A, Kakade SM, Seeger M (2012) Information-theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Trans Inf Theory 58(5):3250–3265

    Article  MathSciNet  MATH  Google Scholar 

  30. Liu Y, Tao T (2015) A cs recovery algorithm for model and time delay identification of miso-fir systems. Algorithms 8(3):743–753

    Article  MathSciNet  MATH  Google Scholar 

  31. Yang S, Gao T, Wang J, Deng B, Lansdell B, Linares-Barranco B (2021) Efficient spike-driven learning with dendritic event-based processing. Front Neurosci 15:97

    Google Scholar 

  32. Lobov SA, Mikhaylov AN, Shamshin M, Makarov VA, Kazantsev VB (2020) Spatial properties of stdp in a self-learning spiking neural network enable controlling a mobile robot. Front Neurosci 14:88

    Article  Google Scholar 

  33. Yang S, Wang J, Zhang N, Deng B, Pang Y, Azghadi MR (2021) Cerebellumorphic: large-scale neuromorphic model and architecture for supervised motor learning. IEEE Transactions on Neural Networks and Learning Systems

  34. Yang S, Wang J, Hao X, Li H, Wei X, Deng B, Loparo KA (2021) Bicoss: toward large-scale cognition brain with multigranular neuromorphic architecture. IEEE Transactions on Neural Networks and Learning Systems

  35. Yang S, Deng B, Wang J, Li H, Lu M, Che Y, Wei X, Loparo KA (2019) Scalable digital neuromorphic architecture for large-scale biophysically meaningful neural network with multi-compartment neurons. IEEE Trans Neural Netw Learn Syst 31(1):148–162

    Article  Google Scholar 

  36. Yang S, Wang J, Deng B, Liu C, Li H, Fietkiewicz C, Loparo KA (2018) Real-time neuromorphic system for large-scale conductance-based spiking neural networks. IEEE Trans Cybern 49(7):2490–2503

    Article  Google Scholar 

  37. Gan W, Lin J, Zhang J, Fournier-Viger P, Chao H, Yu PS (2021) Fast utility mining on sequence data. IEEE Trans Cybern 51:487–500

    Article  Google Scholar 

  38. Fournier-Viger P, Cheng C, Cheng Z, Lin JC-W, Selmaoui-Folcher N (2019) Mining significant trend sequences in dynamic attributed graphs. Knowl-Based Syst 182:104797

    Article  Google Scholar 

  39. Chen S-M, Zou X, Gunawan GC (2019) Fuzzy time series forecasting based on proportions of intervals and particle swarm optimization techniques. Inf Sci 500:127–139

    Article  MathSciNet  Google Scholar 

  40. Cheng S-H, Chen S-M, Jian W-S (2016) Fuzzy time series forecasting based on fuzzy logical relationships and similarity measures. Inf Sci 327:272–287

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grants Nos. 11972115, 11572084).

Author information

Authors and Affiliations

  1. College of Information Science and Technology, Donghua University, Shanghai, 201620, China

    Shenyi Ding, Zhijie Wang, Jue Zhang, Fang Han & Xiaochun Gu

Authors
  1. Shenyi Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhijie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fang Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaochun Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhijie Wang or Jue Zhang.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, S., Wang, Z., Zhang, J. et al. Time delay system identification using controlled recurrent neural network and discrete bayesian optimization. Appl Intell 52, 8351–8371 (2022). https://doi.org/10.1007/s10489-021-02823-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02823-3

Keywords

Search

Navigation