Abstract
Chaotic dynamics are the paradigm of complex and unpredictable evolution due to their built-in feature of amplifying arbitrarily small perturbations. The forecasting of these dynamics has attracted the attention of many scientists since the discovery of chaos by Lorenz in the 1960s. In the last decades, machine learning techniques have shown a greater predictive accuracy than traditional tools from nonlinear time-series analysis. In particular, artificial neural networks have become the state of the art in chaotic time series forecasting. However, how to select their structure and the training algorithm is still an open issue in the scientific community, especially when considering a multi-step forecasting horizon. We implement feed-forward and recurrent architectures, considering different training methods and forecasting strategies. The predictors are evaluated on a wide range of problems, from low-dimensional deterministic cases to real-world time series.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdulkadir, S. J., Alhussian, H., & Alzahrani, A. I. (2018). Analysis of recurrent neural networks for henon simulated time-series forecasting. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 10.1-8, 155–159.
Akritas, P., Antoniou, I., & Ivanov, V. V. (2000). Identification and prediction of discrete chaotic maps applying a Chebyshev neural network. Chaos, Solitons and Fractals, 11.1-3, 337–344.
Antonik, P., et al. (2018). Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronization and cryptography. Physical Review E, 98.1, 012215.
Atsalakis, G., Skiadas, C., & Nezis, D. (2008). Forecasting Chaotic time series by a Neural Network. In Proceedings of the 8th International Conference on Applied Stochastic Models and Data Analysis, Vilnius, Lithuania. (Vol. 30, p. 7782).
Atsalakis, G., & Tsakalaki, K. (2012). Simulating annealing and neural networks for chaotic time series forecasting. Chaotic Model. Simul., 1, 81–90.
Bakker, R., et al. (2000). Learning chaotic attractors by neural networks. Neural Computation, 12.10, 2355–2383.
Bollt, E. M. (2000). Model selection, confidence and scaling in predicting chaotic time-series. International Journal of Bifurcation and Chaos, 10.06, 1407–1422.
Bompas, S., Georgeot, B., & Guéry-Odelin, D. (2020). Accuracy of neural networks for the simulation of chaotic dynamics: Precision of training data versus precision of the algorithm. arXiv:2008.04222.
Bonnet, D., Labouisse, V., & Grumbach, A. (1997). \(\delta \)-NARMA neural networks: A new approach to signal prediction. IEEE Transactions on Signal Processing, 45.11, 2799–2810.
Borra, F., Vulpiani, A., & Cencini, M. (2020). Effective models and predictability of chaotic multiscale systems via machine learning. Physical Review E, 102.5, 052203.
Bradley, E., & Kantz, H. (2015). Nonlinear time-series analysis revisited. Chaos: An Interdisciplinary Journal of Nonlinear Science, 25.9, 097610.
Brajard, J. et al. (2020). Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the Lorenz 96 model. Journal of Computational Science, 44, 101171.
Butcher, J. B., et al. (2013). Reservoir computing and extreme learning machines for non-linear time-series data analysis. Neural Networks, 38, 76–89.
Canaday, D., Griffith, A., & Gauthier, D. J. (2018). Rapid time series prediction with a hardware-based reservoir computer. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28.12, 123119.
Cannas, B., & Cincotti, S. (2002). Neural reconstruction of Lorenz attractors by an observable. Chaos, Solitons and Fractals, 14.1, 81–86.
Cannas, B., et al. (2001). Learning of Chua’s circuit attractors by locally recurrent neural networks. Chaos, Solitons and Fractals, 12.11, 2109–2115.
Cao, L., et al. (1995). Predicting chaotic time series with wavelet networks. Physica D: Nonlinear Phenomena, 85.1-2, 225–238.
Casdagli, M. (1989). Nonlinear prediction of chaotic time series. Physica D: Nonlinear Phenomena, 35.3, 335–356.
Cechin, A. L., Pechmann, D. R., & de Oliveira, L. P. (2008). Optimizing Markovian modeling of chaotic systems with recurrent neural networks. Chaos, Solitons and Fractals, 37.5, pp. 1317–1327.
Chandra, R., & Zhang, M. (2012). Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction. Neurocomputing, 86, 116–123.
Chen, P., et al. (2020). Autoreservoir computing for multistep ahead prediction based on the spatiotemporal information transformation. Nature Communications, 11.1, 1–15.
Chen, Z. (2010). A chaotic time series prediction method based on fuzzy neural network and its application. In International Workshop on Chaos-Fractal Theories and Applications (pp. 355–359). IEEE.
Cheng, W., et al. (2021). High-efficiency chaotic time series prediction based on time convolution neural network. Chaos, Solitons and Fractals, 152, 111304.
Covas, E., & Benetos, E. (2019). Optimal neural network feature selection for spatial-temporal forecasting. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29.6, 063111.
Dercole, F., Sangiorgio, M., & Schmirander, Y. (2020). An empirical assessment of the universality of ANNs to predict oscillatory time series. IFAC-PapersOnLine, 53.2, 1255–1260.
Ding, H.-L., et al. (2009). Prediction of chaotic time series using L-GEM based RBFNN. In 2009 International Conference on Machine Learning and Cybernetics (Vol. 2, pp. 1172–1177). IEEE.
Doan, N. A. K., Polifke, W., & Magri, L. (2019). Physics-informed echo state networks for chaotic systems forecasting. In International Conference on Computational Science (pp. 192–198). Springer.
Dudul, S. V. (2005). Prediction of a Lorenz chaotic attractor using two-layer perceptron neural network. Applied Soft Computing, 5.4, pp. 333–355.
Fan, H., et al. (2020). Long-term prediction of chaotic systems with machine learning. Physical Review Research, 2.1, 012080.
Faqih, A., Kamanditya, B., & Kusumoputro, B. (2018). Multi-step ahead prediction of Lorenz’s Chaotic system using SOM ELM- RBFNN. In 2018 International Conference on Computer, Information and Telecommunication Systems (CITS) (pp. 1–5). IEEE.
Farmer, J. D., & Sidorowich, J. J. (1987). Predicting chaotic time series. Physical Review Letters, 59.8, 845.
M Galván, I., & Isasi, P. (2001). Multi-step learning rule for recurrent neural models: An application to time series forecasting. Neural Processing Letters, 13.2, 115–133.
Gao, Y., & Joo Er, M. (2005). NARMAX time series model prediction: Feedforward and recurrent fuzzy neural network approaches. Fuzzy Sets and Systems, 150.2, 331–350.
Gers, F. A., Eck, D., & Schmidhuber, J. (2002). Applying LSTM to time series predictable through time-window approaches. In Neural Nets WIRN Vietri-01 (pp. 193–200). Springer.
Glorot, X., Bordes, A., & Bengio, Y. (2011). Domain adaptation for large-scale sentiment classification: A deep learning approach. Proceedings of the 28 th International Conference on Machine Learning, Bellevue, WA, USA.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
Guariso, G., Nunnari, G., & Sangiorgio, M. (2020). Multi-step solar irradiance forecasting and domain adaptation of deep neural networks. Energies, 13.15,3987.
Guerra, F. A., & dos Coelho, L. S. (2008). Multi-step ahead nonlinear identification of Lorenz’s chaotic system using radial basis neural network with learning by clustering and particle swarm optimization. Chaos, Solitons and Fractals, 35.5, 967–979.
Guo, Y., et al. (2020). Transfer learning of chaotic systems. arXiv:2011.09970.
Haluszczynski, A., & Räth, C. (2019). Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29.10, 103143.
Han, L., Ding, L., & Qi, L. (2005). Chaotic Time series nonlinear prediction based on support vector machines. Systems Engineering - Theory and Practice, 9.
Han, M., & Wang, Y. (2009). Analysis and modeling of multivariate chaotic time series based on neural network. Expert Systems with Applications, 36.2, 1280–1290.
Han, M. et al. (2004). Prediction of chaotic time series based on the recurrent predictor neural network. IEEE Transactions on Signal Processing, 52.12, 3409–3416.
Hassanzadeh, P., et al. (2019). Data-driven prediction of a multi-scale Lorenz 96 chaotic system using a hierarchy of deep learning methods: Reservoir computing, ANN, and RNN-LSTM. In Bulletin of the American Physical Society, C17-009.
He, T., et al. (2019). Quantifying exposure bias for neural language generation. arXiv:1905.10617
Hussein, S., Chandra, R., & Sharma, A. (2016). Multi-step- ahead chaotic time series prediction using coevolutionary recurrent neural networks. In IEEE Congress on Evolutionary Computation (CEC) (pp. 3084–3091). IEEE.
Inoue, H., Fukunaga, Y., & Narihisa, H. (2001). Efficient hybrid neural network for chaotic time series prediction. In International Conference on Artificial Neural Networks (pp. 712–718). Springer.
Inubushi, M., & Goto, S. (2020). Transfer learning for nonlinear dynamics and its application to fluid turbulence. Physical Review E, 102.4, 043301.
Jiang, J., & Lai, Y.-C. (2019). Model-free prediction of spatiotemporal dynamical systems with recurrent neural networks: Role of network spectral radius. Physical Review Research, 1.3, 033056.
Jones, R. D., et al. (1990). Function approximation and time series prediction with neural networks. In 1990 IJCNN International Joint Conference on Neural Networks (pp. 649–665). IEEE.
Jüngling, T. (2019). Reconstruction of complex dynamical systems from time series using reservoir computing. In IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 1–5). IEEE.
Karunasinghe, D. S. K., & Liong, S.-Y. (2006). Chaotic time series prediction with a global model: Artificial neural network. Journal of Hydrology, 323.1-4, 92–105.
Kuremoto, T., et al. (2003). Predicting chaotic time series by reinforcement learning. In Proceedings of the 2nd International Conferences on Computational Intelligence, Robotics and Autonomous Systems (CIRAS 2003).
Kuremoto, T. (2014). Forecast chaotic time series data by DBNs. In 7th International Congress on Image and Signal Processing (pp. 1130–1135). IEEE.
Lei, Y., Hu, J., & Ding, J. (2020). A hybrid model based on deep LSTM for predicting high-dimensional chaotic systems. arXiv:2002.00799.
Lellep, M., et al. (2020). Using machine learning to predict extreme events in the HĂ©non map. In Chaos: An Interdisciplinary Journal of Nonlinear Science, 30.1, 013113.
Leung, H., Lo, T., & Wang, S. (2001). Prediction of noisy chaotic time series using an optimal radial basis function neural network. IEEE Transactions on Neural Networks, 12.5, 1163–1172.
Levine, M. E., & Stuart, A. M. (2021). A framework for machine learning of model error in dynamical systems. arXiv:2107.06658.
Li, Q., & Lin, R.-C. (2016). A new approach for chaotic time series prediction using recurrent neural network. Mathematical Problems in Engineering, 3542898.
Lim, T. P., & Puthusserypady, S. (2006). Error criteria for cross validation in the context of chaotic time series prediction. Chaos: An Interdisciplinary Journal of Nonlinear Science, 16.1, 013106.
Lin, T.-N., et al. (1997). A delay damage model selection algorithm for NARX neural networks. IEEE Transactions on Signal Processing, 45.11, 2719–2730.
LĂłpez-Caraballo, C. H., et al. (2016). Mackey-Glass noisy chaotic time series prediction by a swarm-optimized neural network. Journal of Physics: Conference Series, 720, 1. IOP Publishing.
Lorenz, E. N. (1963). Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20.2, 130–141.
Lu, Z., Hunt, B. R., & Ott, E. (2018). Attractor reconstruction by machine learning. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28.6, 061104.
Lu, Z., et al. (2017). Reservoir observers: Model-free inference of unmeasured variables in chaotic systems. Chaos: An Interdisciplinary Journal of Nonlinear Science, 27.4, 041102.
Ma, Q.-L. (2007). Chaotic time series prediction based on evolving recurrent neural networks. In International Conference on Machine Learning and Cybernetics (Vol. 6, pp. 3496–3500). IEEE.
Maathuis, H. et al. (2017). Predicting chaotic time series using machine learning techniques. In Preproceedings of the 29th Benelux Conference on Artificial Intelligence (BNAIC 2017) (pp. 326–340).
Madondo, M., & Gibbons, T. (2018). Learning and modeling chaos using lstm recurrent neural networks. MICS 2018 Proceedings Paper 26.
Maguire, L. P., et al. (1998). Predicting a chaotic time series using a fuzzy neural network. Information Sciences, 112.1-4, 125–136.
Mariet, Z., & Kuznetsov, V. (2019). Foundations of sequence-to-sequence modeling for time series. In The 22nd International Conference on Artificial Intelligence and Statistics, 408–417.
Masnadi-Shirazi, M., & Subramaniam, S. (2020). Attractor Ranked Radial Basis function network: A nonparametric forecasting Approach for chaotic Dynamic Systems. Scientific Reports, 10.1, 1–10.
Xin-Ying, W. Min, H. (2012). Multivariate chaotic time series prediction based on extreme learning machine. Acta Physica Sinica, 8.
Mukherjee, S., Osuna, E., & Girosi, F. (1997). Nonlinear prediction of chaotic time series using support vector machines. In Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop (pp. 511–520). IEEE.
Nakai, K., & Saiki, Y. (2019). Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable. arXiv:1903.05770.
Navone, H. D., & Ceccatto, H. A. (1995). Learning chaotic dynamics by neural networks. Chaos, Solitons and Fractals, 6, 383–387.
Okuno, S., Aihara, K., & Hirata, Y. (2019). Combining multiple forecasts for multivariate time series via state-dependent weighting. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29.3, 033128.
Patel, D., et al. (2021). Using machine learning to predict statistical properties of non-stationary dynamical processes: System climate, regime transitions, and the effect of stochasticity. Chaos: An Interdisciplinary Journal of Nonlinear Science, 31.3, 033149.
Pathak, J., et al. (2018). Hybrid forecasting of chaotic processes: Using machine learning in conjunction with a knowledge-based model. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28.4, 041101.
Pathak, J., et al. (2018). Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Physical Review Letters, 120.2, 024102.
Pathak, J., et al. (2017). Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data. Chaos: An Interdisciplinary Journal of Nonlinear Science, 27.12, 121102.
Penkovsky, B., et al. (2019). Coupled nonlinear delay systems as deep convolutional neural networks. Physical Review Letters, 123.5, 054101.
Principe, J. C., & Kuo, J.-M. (1995). Dynamic modelling of chaotic time series with neural networks. Proceedings of the 7th International Conference on Neural Information Processing Systems, 311–318.
Principe, J. C., Rathie, A., Kuo, J.-M. (1992). Prediction of chaotic time series with neural networks and the issue of dynamic modeling. International Journal of Bifurcation and Chaos, 2.04, 989–996.
Principe, J. C., Wang, L., & Kuo, J.-M. (1998). Non-linear dynamic modelling with neural networks. In Signal Analysis and Prediction (pp. 275–290). Springer.
Ranzato, M., et al. (2015). Sequence level training with recurrent neural networks. arXiv:1511.06732.
Sangiorgio, M. (2021). Deep learning in multi-step forecasting of chaotic dynamics. Ph.D. thesis. Department of Electronics, Information and Bioengineering, Politecnico di Milano.
Sangiorgio, M., & Dercole, F. (2020). Robustness of LSTM neural networks for multi-step forecasting of chaotic time series. Chaos, Solitons and Fractals, 139, 110045.
Sangiorgio, M., Dercole, F., & Guariso, G. (2021). Forecasting of noisy chaotic systems with deep neural networks. Chaos, Solitons & Fractals, 153, 111570.
Shi, X., et al. (2017). Chaos time-series prediction based on an improved recursive Levenberg-Marquardt algorithm. Chaos, Solitons and Fractals, 100, 57–61.
Shi, Z., & Han, M. (2007). Support vector echo-state machine for chaotic time-series prediction. IEEE Transactions on Neural Networks, 18.2, 359–372.
Shukla, J. (1998). Predictability in the midst of chaos: A scientific basis for climate forecasting. Science, 282.5389, 728–731.
Su, L., & Li, C. (2015). Local prediction of chaotic time series based on polynomial coefficient autoregressive model. Mathematical Problems in Engineering, 901807.
Su, L.-Y. (2010). Prediction of multivariate chaotic time series with local polynomial fitting. Computers and Mathematics with Applications, 59.2, 737–744.
Teng, Q., & Zhang, L. (2019). Data driven nonlinear dynamical systems identification using multi-step CLDNN. AIP Advances, 9.8, p. 085311.
Todorov, Y., Koprinkova-Hristova, P., & Terziyska, M. (2017). Intuitionistic fuzzy radial basis functions network for modeling of nonlinear dynamics. In 2017 21st International Conference on Process Control (PC) (pp. 410–415). IEEE.
Van Truc, N., & Anh, D. T. (2018). Chaotic time series prediction using radial basis function networks. 2018 4th International Conference on Green Technology and Sustainable Development (GTSD) (pp. 753–758). IEEE.
Verdes, P. F., et al. (1998). Forecasting chaotic time series: Global versus local methods. Novel Intelligent Automation and Control Systems, 1, 129–145.
Vlachas, P. R., et al. (2018). Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks. In Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences (Vol. 474.2213, p. 20170844).
Vlachas, P. R. et al. (2020). Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics. Neural Networks, 126, 191–217.
Wan, Z. Y., et al. (2018). Data-assisted reduced-order modeling of extreme events in complex dynamical systems. PLoS One, 13.5, e0197704.
Wang, R., Kalnay, E., & Balachandran, B. (2019). Neural machine-based forecasting of chaotic dynamics. Nonlinear Dynamics, 98.4, 2903–2917.
Weng, T., et al. (2019). Synchronization of chaotic systems and their machine-learning models. Physical Review E, 99.4, 042203.
Woolley, J. W., Agarwal, P. K., & Baker, J. (2010). Modeling and prediction of chaotic systems with artificial neural networks. International Journal for Numerical Methods in Fuids, 63.8, 989–1004.
Wu, K. J., & Wang, T. J. (2013). Prediction of chaotic time series based on RBF neural network optimization. Computer Engineering, 39.10, 208–216.
Wu, X., et al. (2014). Multi-step prediction of time series with random missing data. Applied Mathematical Modelling, 38.14, 3512–3522.
Xin, B., & Peng, W. (2020). Prediction for chaotic time series-based AE-CNN and transfer learning. Complexity, 2680480.
Yanan, G., Xiaoqun, C., & Kecheng, P. (2020). Chaotic system prediction using data assimilation and machine learning. In E3S Web of Conferences (Vol. 185, p. 02025).
Yang, H. Y. et al. (2006). Fuzzy neural very-short-term load forecasting based on chaotic dynamics reconstruction. Chaos, Solitons and Fractals, 29.2, 462–469.
Yang, F.-P., & Lee, S.-J. (2008). Applying soft computing for forecasting chaotic time series. In 2008 IEEE International Conference on Granular Computing (pp. 718–723), IEEE.
Yeh, J.-P. (2007). Identifying chaotic systems using a fuzzy model coupled with a linear plant. Chaos, Solitons and Fractals, 32.3, 1178–1187.
Yeo, K. (2019). Data-driven reconstruction of nonlinear dynamics from sparse observation. Journal of Computational Physics, 395, 671–689.
Yosinski, J. et al. (2014). How transferable are features in deep neural networks? In Proceedings of the 28th Conference on Neural Information Processing Systems, 27, 3320–3328.
Yu, R., Zheng, S., & Liu, Y. (2017). Learning chaotic dynamics using tensor recurrent neural networks. Proceedings of the ICML. In ICML 17 Workshop on Deep Structured Prediction.
Yuxia, H., & Hongtao, Z. (2012). Chaos optimization method of SVM parameters selection for chaotic time series forecasting. Physics Procedia, 25, 588–594.
Zhang, C., et al. (2020). Predicting phase and sensing phase coherence in chaotic systems with machine learning. Chaos: An Interdisciplinary Journal of Nonlinear Science, 30.8, 083114.
Zhang, J.-S., & Xiao, X.-C. (2000). Predicting chaotic time series using recurrent neural network. Chinese Physics Letters, 17.2, 88.
Zhang, J., Shu-Hung Chung, H., & Lo, W.-L. (2008). Chaotic time series prediction using a neuro-fuzzy system with time-delay coordinates. IEEE Transactions on Knowledge and Data Engineering, 20.7, 956–964.
Zhu, Q., Ma, H., & Lin, W. (2019). Detecting unstable periodic orbits based only on time series: When adaptive delayed feedback control meets reservoir computing. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29.9, 093125.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Sangiorgio, M., Dercole, F., Guariso, G. (2021). Introduction to Chaotic Dynamics’ Forecasting. In: Deep Learning in Multi-step Prediction of Chaotic Dynamics. SpringerBriefs in Applied Sciences and Technology(). Springer, Cham. https://doi.org/10.1007/978-3-030-94482-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-94482-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-94481-0
Online ISBN: 978-3-030-94482-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)