Abstract
A novel multi-reservoir echo state network incorporating the scheme of extracting features from multiple-size input time slices is proposed in this paper. The proposed model, Multi-size Input Time Slices Echo State Network (MITSESN), uses multiple reservoirs, each of which extracts features from each of the multiple input time slices of different sizes. We compare the prediction performances of MITSESN with those of the standard echo state network and the grouped echo state network on three benchmark nonlinear time-series datasets to show the effectiveness of our proposed model. Moreover, we analyze the richness of reservoir dynamics of all the tested models and find that our proposed model can generate temporal features with less linear redundancies under the same parameter settings, which provides an explanation about why our proposed model can outperform the other models to be compared on the nonlinear time-series prediction tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Gallicchio, C., Micheli, A.: A Markovian characterization of redundancy in echo state networks by PCA. In: ESANN. Citeseer (2010)
Gallicchio, C., Micheli, A.: Richness of deep echo state network dynamics. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2019. LNCS, vol. 11506, pp. 480–491. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20521-8_40
Gallicchio, C., Micheli, A., Pedrelli, L.: Deep reservoir computing: a critical experimental analysis. Neurocomputing 268, 87–99 (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. Ger. Natl. Res. Cent. Inf. Technol. GMD Tech. Rep. 148(34), 13 (2001)
Li, Z., Tanaka, G.: Deep echo state networks with multi-span features for nonlinear time series prediction. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–9. IEEE (2020)
Li, Z., Tanaka, G.: HP-ESN: echo state networks combined with hodrick-prescott filter for nonlinear time-series prediction. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–9. IEEE (2020)
Liu, Y., Gong, C., Yang, L., Chen, Y.: DSTP-RNN: a dual-stage two-phase attentionbased recurrent neural network for long-term and multivariate time series prediction. Expert Syst. Appl. 143, 113082 (2020)
Lorenz, E.N.: Deterministic nonperiodic flow. J. Atmos. Sci. 20(2), 130–141 (1963)
Mackey, M.C., Glass, L.: Oscillation and chaos in physiological control systems. Science 197(4300), 287–289 (1977)
Medsker, L., Jain, L.C.: Recurrent Neural Networks: Design and Applications. CRC Press, Boca Raton (1999)
Menezes, J.M., Barreto, G.A.: A new look at nonlinear time series prediction with narx recurrent neural network. In: 2006 Ninth Brazilian Symposium on Neural Networks (SBRN 2006), pp. 160–165. IEEE (2006)
Shen, L., Chen, J., Zeng, Z., Yang, J., Jin, J.: A novel echo state network for multivariate and nonlinear time series prediction. Appl. Soft Comput. 62, 524–535 (2018)
Song, Z., Wu, K., Shao, J.: Destination prediction using deep echo state network. Neurocomputing 406, 343–353 (2020)
Suykens, J.A., Vandewalle, J.: The KU leuven time series prediction competition. In: Suykens J.A.K., Vandewalle J. (eds) Nonlinear Modeling, pp. 241–253. Springer, Boston (1998). https://doi.org/10.1007/978-1-4615-5703-6_9
Tanaka, G., et al.: Recent advances in physical reservoir computing: a review. Neural Netw. 115, 100–123 (2019)
Tikhonov, A.N., Goncharsky, A., Stepanov, V., Yagola, A.G.: Numerical Methods for the Solution of III-Posed Problems, vol. 328. Springer Science & Business Media, Berlin (2013)
Weigend, A.S.: Time Series Prediction: Forecasting The Future And Understanding The Past Routledge, Abingdon-on-Thames (2018)
Yu, Z., Liu, G.: Sliced recurrent neural networks. arXiv preprint arXiv:1807.02291 (2018)
Acknowledgements
This work was partly supported by JSPS KAKENHI Grant Number 20K11882 and JST-Mirai Program Grant Number JPMJMI19B1, Japan (GT), and partly based on results obtained from Project No. JPNP16007, commissioned by the New Energy and Industrial Technology Development Organization (NEDO).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Z., Tanaka, G. (2021). A Multi-Reservoir Echo State Network with Multiple-Size Input Time Slices for Nonlinear Time-Series Prediction. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13109. Springer, Cham. https://doi.org/10.1007/978-3-030-92270-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-92270-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92269-6
Online ISBN: 978-3-030-92270-2
eBook Packages: Computer ScienceComputer Science (R0)