Abstract
One of the main purposes of nonlinear system modeling is to design model-based controllers such as model predictive control (MPC). A group of deep belief networks (DBNs) are used to approximate the function type coefficients of a state dependent autoregressive model with exogenous variables (SD-ARX), which can represent nonlinear dynamics, and thus a DBN-based state-dependent ARX (DBN-ARX) model is obtained in this paper. The DBN-ARX model has the function approximation ability of single DBN model and the nonlinear description advantage of SD-ARX model. All parameters of the DBN-ARX model are estimated by the pre-training and fine-tuning strategies and the stability condition of the model are also discussed. The proposed DBN-ARX model is a pseudo-linear ARX model identified offline, and its function type coefficients are composed of the operating-point dependent DBNs. The usefulness of the DBN-ARX model is illustrated by modeling a continuously stirred tank reactor (CSTR) time series, Box and Jenkins data, a nonlinear process and a water tank system. The four experimental results show that the one-step-ahead and multi-step-ahead prediction accuracy of the proposed DBN-ARX model is improved comparing with the modeling results of several existing models.
Similar content being viewed by others
References
Gu B, Gupta YP (2008) Control of nonlinear processes by using linear model predictive control algorithms. ISA Transactions 47(2):211–216
Huusom JK, Poulsen NK, Rgensen SB, Rgensen JB (2012) Tuning siso offset-free model predictive control based on ARX models. Journal of Process Control 22(10):1997–2007
Novák, J., Chalupa, P., & Bobál, V. (2011). MIMO model predictive control with local linear models. Wseas International Conference on Automatic Control
Hosen MA, Hussain MA, Mjalli FS (2011) Control of polystyrene batch reactors using neural network based model predictive control (NNMPC): an experimental investigation. Control Engineering Practice 19(5):454–467
Ekman M (2008) Bilinear black-box identification and MPC of the activated sludge process. Journal of Process Control 18(7–8):643–653
Dongli W, Yan Z, Xiaoyang H (2007) Radial basis function neural network-based model predictive control for freeway traffic systems. International Journal of Intelligent Systems Technologies and Applications 2(4):370
Aggelogiannaki E, Sarimveis H (2008) Nonlinear model predictive control for distributed parameter systems using data driven artificial neural network models. Computers and Chemical Engineering 32(6):1225–1237
Han HG, Zhang L, Hou Y, Qiao JF (2015) Nonlinear model predictive control based on a self-organizing recurrent neural network. IEEE Transactions on Neural Networks and Learning Systems 27(2):402
Rezaee B, Zarandi MHF (2010) Data-driven fuzzy modeling for takagi–sugeno–kang fuzzy system. Information Sciences 180(2):241–255
Box GEP, Jenkins GM (1971) Time series analysis, forecasting and control. Journal of the American Statistical Association 134(3)
Gaweda AE, Zurada JM (2003) Data-driven linguistic modeling using relational fuzzy rules. Fuzzy Systems IEEE Transactions on 11(1):121–134
Kim E, Park M, Ji S, Park M (1997) A new approach to fuzzy modeling. IEEE Trans Fuzzy Syst 5(3):328–337
Kim E, Park M, Kim S, Park M (1998) A transformed input-domain approach to fuzzy modeling. Fuzzy Systems IEEE Transactions on 6(4):596–604
Sugeno M, Tanaka K (1991) Successive identification of a fuzzy model and its applications to prediction of a complex system. Fuzzy Sets & Systems 42(3):315–334
Wang H, Kwong S, Jin Y, Wei W, Man KF (2005) Agent-based evolutionary approach for interpretable rule-based knowledge extraction. IEEE Transactions on Systems Man & Cybernetics Part C 35(2):143–155
Zhao J, Mo H, Deng Y (2020) An efficient network method for time series forecasting based on the DC algorithm and visibility relation. IEEE Access 8(1). https://doi.org/10.1109/ACCESS.2020.2964067
Liu F, Deng Y (2019) A fast algorithm for network forecasting time series. IEEE Access 7(1):102554–102560
Xu P, Zhang R, Deng Y (2018) A novel visibility graph transformation of time series into weighted networks. Chaos, Solitons & Fractals 117:201–208
Maciel L, Ballini R, Gomide F (2016) Evolving granular analytics for interval time series forecasting. Granular Computing 1(4):213–224
Zhang J, Chin KS, Lawryńczuk M (2018) Nonlinear model predictive control based on piecewise linear hammerstein models. Nonlinear Dynamics 92:1001–1021
Korda M, Mezić I (2016) Linear predictors for nonlinear dynamical systems: koopman operator meets model predictive control. Automatica 93
Ania LC, Agamennoni OE, Figueroa JL (2003) A nonlinear model predictive control system based on Wiener piecewise linear models. Journal of Process Control 13(7):655–666
Zhao M, Li N, Li SY (2009) Min-Max model predictive control for constrained nonlinear systems via multiple LPV embeddings. Science in China Series F (Information Science) 52(7):1129–1135
Ding B, Tao Z (2014) A synthesis approach for output feedback robust model predictive control based-on input–output model. Journal of Process Control 24(3):60–72
Xu W, Peng H, Zeng X, Zhou F, Tian X, Peng X (2019) Deep belief network-based AR model for nonlinear time series forecasting. Applied Soft Computing 77:605–621
Xu W, Peng H, Zeng X, Zhou F, Tian X, Peng X (2019) A hybrid modelling method for time series forecasting based on a linear regression model and deep learning. Applied Intelligence 49(8):3002–3015
Li W, Szidarovszky F (1999) An elementary result in the stability theory of time-invariant nonlinear discrete dynamical systems. Applied Mathematics and Computation 102:35–49
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science. 313(5786):504–507
Cervantes AL, Agamennoni OE, Figueroa JL (2003) A nonlinear model predictive control system based on Wiener piecewise linear models. Journal of Process Control 13:655–666
Man, H. (2014). Research on model identification and nonlinear predictive control methods of CSTR process, Ph.D. dissertation, Univ. Dalian University of Technology, Dalian, China
Gan M, Li HX, Peng H (2017) A variable projection approach for efficient estimation of RBF-ARX model. IEEE Transactions on Cybernetics 45(3):462–471
Acknowledgments
The authors would like to thank the editors and referees for their valuable comments and suggestions, which substantially improved the original manuscript. The work presented in this paper was supported by the National Natural Science Foundation of China (Grant No. 61773402, Grant No. 51575167 and Grant No. 61540037) and the Anhui Provincial Natural Science Foundation(2008085MF197).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Fine tuning process of the proposed DBN-ARX model
Fine tuning process of the proposed DBN-ARX model
Eq. (13) is used to calculate the objective function in the fine-tuning stage, and the objective function is defined as follows
where y(t) and \( \hat{y}(t) \) are actual and predicted values, respectively.
The traditional gradient decent method is used to fine tune the parameters of the DBN-ARX model. For the neuron in the \( {N}_r^{(j)} \)-th layer, Eq. (14) is used to compute the gradient for parameter updating.
where
and φ′(u) is the derivative of φ(u) with respect to u. Let
According to the chain rule of calculus, the local gradient of the j ‐ th DBN module in the last layer can be redefined by
Similarly, we have
With regard to neuron \( {n}_{N_r^{(j)}-1}^{(j)}\in \left\{1,2,\cdots, {Q}_{N_r^{(j)}-1}^{(j)}\right\} \) in the (\( {N}_r^{(j)}-1 \))-th hidden layer, Eq. (27) can be used to compute the gradient for parameter updating.
Let
then Eq. (27) becomes
and
With regard to neuron \( {n}_{N_r^{(j)}-2}^{(j)}\in \left\{1,2,\cdots, {Q}_{N_r^{(j)}-2}^{(j)}\right\} \) in the (\( {N}_r^{(j)}-2 \))-th hidden layer, Eq. (31) can be used to compute the gradient for parameter updating. Eq. (31) is given as follows.
Let
then formula (31) becomes
And similarly, we have
Therefore, as for the j ‐ th DBN module, according to the derivation process above, Eq. (35) is used to calculate the local gradient of each neuron of layer ℓj.
The updated gradients to the weight and corresponding bias can be calculated by Eq. (36) and Eq. (37).
The correction \( \varDelta {w}_{n_L^{(j)},{n}_{L-1}^{(j)},j}^{(L)} \) applied to \( {w}_{n_L^{(j)},{n}_{L-1}^{(j)},j}^{(L)} \) is defined by the delta rule, Accordingly, the use of Eqs. (24–26), Eqs. (28–30) and Eqs. (32–37) yields
where \( L\in \left\{1,2,\cdots, {N}_r^{(j)}-1,{N}_r^{(j)}\right\} \) and η > 0 is the pre-determined learning rate, and the parameters of weight and bias are calculated according to the following rules
where the initial value of weight \( {w}_{n_L^{(j)},{n}_{L-1}^{(j)},j}^{(L)} \) and bias \( {b}_{n_L^{(j)},j}^{(L)} \) are obtained in the pre-training stage. In addition, to slow down the parameters convergence speed and oscillation, the momentum constant is added in Eq. (40), and finally we use the following parameter updating strategy
where k is the number of updating iterations, α ∈ [0, 1)is a pre-determined momentum constant.
Rights and permissions
About this article
Cite this article
Xu, W., Peng, H., Tian, X. et al. DBN based SD-ARX model for nonlinear time series prediction and analysis. Appl Intell 50, 4586–4601 (2020). https://doi.org/10.1007/s10489-020-01804-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-01804-2