Echo state network (ESN), a novel recurrent neural network, has a randomly and sparsely connected reservoir. Since the reservoir size is very large, the collinearity problem may exist in the ESN. To address this problem and get a sparse architecture, an adaptive lasso echo state network (ALESN) is proposed, in which the adaptive lasso algorithm is used to calculate the output weights. The ALESN combines the advantages of quadratic regularization and adaptively weighted lasso shrinkage; furthermore, it has the oracle properties and can deal with the collinearity problem. Meanwhile, to obtain the optimal model, the selection of tuning regularization parameter based on modified Bayesian information criterion is proposed. Simulation results show that the proposed ALESN has better performance and relatively uniform output weights than some other existing methods.
Echo state network Collinearity problem Adaptive lasso algorithm Modified Bayesian information criterion Nonlinear system modeling
This is a preview of subscription content, log in to check access.
This work was supported in part by the National Natural Science Foundation of China under Grant 61533002 and Grant 61603012, in part by the Beijing Municipal Education Commission Foundation under Grant KM201710005025, in part by the Beijing Postdoctoral Research Foundation of China under Grant 2017ZZ-028 and in part by the China Postdoctoral Science Foundation-funded project.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Han HG, Wu XL, Qiao JF (2013) Real-time model predictive control using a self-organizing neural network. IEEE Trans Neural Netw Learn Syst 24(9):1425–1436CrossRefGoogle Scholar
Zhang YW, Chai TY, Li ZM, Yang CY (2012) Modeling and monitoring of dynamic processes. IEEE Trans Neural Netw Learn Syst 23(2):277–284CrossRefGoogle Scholar
Qiao JF, Han HG (2012) Identification and modeling of nonlinear dynamical systems using a novel self-organizing RBF-based approach. Automatica 48(8):1729–1734MathSciNetCrossRefzbMATHGoogle Scholar
Chang WD (2014) Recurrent neural network modeling combined with bilinear model structure. Neural Comput Appl 24(3–4):765–773CrossRefGoogle Scholar
Han HG, Wu XL, Qiao JF (2014) Nonlinear system modeling based on self-organizing fuzzy-neural-network with adaptive computation algorithm. IEEE Trans Cybern 44(4):554–564CrossRefGoogle Scholar
Li FJ, Qiao JF, Han HG, Yang CL (2016) A self-organizing cascade neural network with random weights for nonlinear system modeling. Appl Soft Comput 42:184–193CrossRefGoogle Scholar
Han HG, Zhang S, Qiao JF (2017) An adaptive growing and pruning algorithm for designing recurrent neural network. Neurocomputing 242:51–62CrossRefGoogle Scholar
Hwang CL, Jan C (2016) Recurrent neural network based multivariable adaptive control for a class of nonlinear dynamic systems with time-varying delay. IEEE Trans Neural Netw Learn Syst 27(2):388–401MathSciNetCrossRefGoogle Scholar
Geng K, Marmarelis VZ (2017) Methodology of recurrent Laguerre-Volterra network for modeling nonlinear dynamic systems. IEEE Trans Neural Netw Learn Syst 28(9):2196–2208MathSciNetGoogle Scholar
Wang L, Zhang L, Yi Z (2017) Trajectory predictor by using recurrent neural networks in visual tracking. IEEE Trans Cybern 47(10):3172–3183CrossRefGoogle Scholar
Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667):78–80CrossRefGoogle Scholar
Bacciu D, Barsocchi P, Chessa S et al (2014) An experimental characterization of reservoir computing in ambient assisted living applications. Neural Comput Appl 24(6):1451–1464CrossRefGoogle Scholar
Qiao JF, Li FJ, Han HG, Li WJ (2017) Growing echo state network with multiple subreservoirs. IEEE Trans Neural Netw Learn Syst 28(2):391–404CrossRefGoogle Scholar
Peng Y, Lei M, Li JB et al (2014) A novel hybridization of echo state networks and multiplicative seasonal ARIMA model for mobile communication traffic series forecasting. Neural Comput Appl 24(3–4):883–890CrossRefGoogle Scholar
Oubbati M, Palm G (2010) A neural framework for adaptive robot control. Neural Comput Appl 19(1):103–114CrossRefGoogle Scholar
Skowronski MD, Harris JG (2007) Automatic speech recognition using a predictive echo state network classifier. Neural Netw 20(3):414–423CrossRefzbMATHGoogle Scholar
Xia Y, Jelfs B, Van Hulle MM (2011) An augmented echo state network for nonlinear adaptive filtering of complex noncircular signals. IEEE Trans Neural Netw 22(1):74–83CrossRefGoogle Scholar
Xu M, Han M (2016) Adaptive elastic echo state network for multivariate time series prediction. IEEE Trans Cybern 46(10):2173–2183CrossRefGoogle Scholar
Dutoit X, Schrauwen B, Van Campenhout J, Stroobandt D, Van Brussel H, Nuttin M (2009) Pruning and regularization in reservoir computing. Neurocomputing 72(7–9):1534–1546CrossRefGoogle Scholar
Reinhart RF, Steil JJ (2012) Regularization and stability in reservoir networks with output feedback. Neurocomputing 90:96–105CrossRefGoogle Scholar
Rodan A, Tiňo P (2011) Minimum complexity echo state network. IEEE Trans Neural Netw 22(1):131–144CrossRefGoogle Scholar
Han HG, Zhang L, Hou Y, Qiao JF (2016) Nonlinear model predictive control based on a self-organizing recurrent neural network. IEEE Trans Neural Netw Learn Syst 27(2):402–415MathSciNetCrossRefGoogle Scholar