Abstract
When working with time series, it is often beneficial to have an idea as to how complex the signal is. Periodic, chaotic and random signals (from least to most complex) may each be approached in different ways, and knowing when a signal can be identified as belonging to one of these categories can reveal a lot about the underlying system. In the field of time series analysis, permutation entropy has emerged as one of the premier measures of time series complexity due to its ability to be calculated from data alone. We propose an alternative method for calculating complexity based on the machine learning paradigm of reservoir computing, and how the outputs of these neural networks capture similar information regarding signal complexity. We observe similar behaviour in our proposed measure to both the Lyapunov exponent and permutation entropy for well known dynamical systems. Additionally, we assess the dependence of our measure on key hyperparameters of the model, drawing conclusions about the invariance of the measure and possible implications on informing network structure.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Appeltant, L., et al.: Information processing using a single dynamical node as complex system. Nature Comm. 2, 468 (2011). https://doi.org/10.1038/ncomms1476
Bandt, C., Pompe, B.: Permutation entropy: a natural complexity measure for time series. Phys. Rev. Lett. 88, 174102 (2002). https://doi.org/10.1103/PhysRevLett.88.174102
Bianchi, F.M., Scardapane, S., Lokse, S., Jenssen, R.: Reservoir computing approaches for representation and classification of multivariate time series. IEEE Trans. Neural Netw. Learn. Syst. 32(5), 2169–2179 (2021). https://doi.org/10.1109/TNNLS.2020.3001377
Cao, Y., Tung, W.w., Gao, J.B., Protopopescu, V.A., Hively, L.M.: Detecting dynamical changes in time series using the permutation entropy. Phys. Rev. E. 70, 046217 (2004). https://doi.org/10.1103/PhysRevE.70.046217
Carroll, T.L.: Using reservoir computers to distinguish chaotic signals. Phys. Rev. E. 98, 052209 (2018). https://doi.org/10.1103/PhysRevE.98.052209
Cellucci, C.J., Albano, A.M., Rapp, P.E.: Comparative study of embedding methods. Phys. Rev. E 67, 066210 (2003). https://doi.org/10.1103/PhysRevE.67.066210
Costa, F.G.d., Duarte, F.S., Vallim, R.M., Mello, R.F.d.: Multidimensional surrogate stability to detect data stream concept drift. Expert Syst. Appl. 87(C), 15–29 (2017). https://doi.org/10.1016/j.eswa.2017.06.005
Eckmann, J.P., Ruelle, D.: Ergodic theory of chaos and strange attractors. Rev. Mod. Phys. 57, 617–656 (1985). https://doi.org/10.1103/RevModPhys.57.617
Hirata, Y.: Recurrence plots for characterizing random dynamical systems. Commun. Nonlinear Sci. Numer. Simul. 94, 105552 (2021). https://doi.org/10.1016/j.cnsns.2020.105552
Jaeger, H.: The echo state approach to analysing and training recurrent neural networks-with an erratum note. Ger. Natl. Res. Center for Inf. Technol. GMD Tech. Rep. 148 (2001)
Larger, L., Baylón-Fuentes, A., Martinenghi, R., Udaltsov, V.S., Chembo, Y.K., Jacquot, M.: High-speed photonic reservoir computing using a time-delay-based architecture: Million words per second classification. Phys. Rev. X 7, 011015 (2017). https://doi.org/10.1103/PhysRevX.7.011015
Lee, G.C., Loo, C.K., Liew, W.S., Wermter, S.: Self-organizing kernel-based convolutional echo state network for human actions recognition. In: Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN’20), pp. 591–596 (2020). https://www2.informatik.uni-hamburg.de/wtm/publications/2020/LLLW20/ES2020-99-7.pdf
Lu, Z., Hunt, B.R., Ott, E.: Attractor reconstruction by machine learning. Chaos 28(6), 061104 (2018). https://doi.org/10.1063/1.5039508
Lu, Z., Pathak, J., Hunt, B., Girvan, M., Brockett, R., Ott, E.: Reservoir observers: model-free inference of unmeasured variables in chaotic systems. Chaos 27(4), 041102 (2017). https://doi.org/10.1063/1.4979665
Lymburn, T., Khor, A., Stemler, T., Corréa, D.C., Small, M., Jüngling, T.: Consistency in echo-state networks. Chaos 29(2), 023118 (2019). https://doi.org/10.1063/1.5079686
Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002). https://doi.org/10.1162/089976602760407955
McCullough, M., Small, M., Iu, H.H.C., Stemler, T.: Multiscale ordinal network analysis of human cardiac dynamics. Philos. Trans. Royal Soc. A: Math. Phys. Eng. Sci. 375(2096), 20160292 (2017). https://doi.org/10.1098/rsta.2016.0292
Yamane, T., et al.: Dimensionality reduction by reservoir computing and its application to IoT edge computing. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11301, pp. 635–643. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04167-0_58
Pathak, J., Lu, Z., Hunt, B., Girvan, M., Ott, E.: Using machine learning to replicate chaotic attractors and calculate lyapunov exponents from data. Chaos 27(12), 121102 (2017)
Pathak, J., Hunt, B., Girvan, M., Lu, Z., Ott, E.: Model-free prediction of large spatiotemporally chaotic systems from data: a reservoir computing approach. Phys. Rev. Lett. 120, 024102 (2018). https://doi.org/10.1103/PhysRevLett.120.024102
Pesin, Y.B.: Dimension theory in dynamical systems contemporary views and applications. Chicago lectures in mathematics series, University of Chicago Press, Chicago (1997)
Qing-Fang, M., Yu-Hua, P., Pei-Jun, X.: A new method of determining the optimal embedding dimension based on nonlinear prediction. Chin. Phys. 16(5), 1252–1257 (2007). https://doi.org/10.1088/1009-1963/16/5/014
Shahriari, Z., Small, M.: Permutation entropy of state transition networks to detect synchronization. Int. J. Bifurcat. Chaos 30(10), 2050154 (2020). https://doi.org/10.1142/S0218127420501540
Tanaka, G., et al.: Recent advances in physical reservoir computing: a review. Neural Netw. 115, 100–123 (2019). https://doi.org/10.1016/j.neunet.2019.03.005
Thorne, B., Jüngling, T., Small, M., Corrêa, D., Zaitouny, A.: Reservoir time series analysis: using the response of complex dynamical systems as a universal indicator of change. Chaos: Interdisc. J. Nonlinear Sci. 32(3), 033109 (2022). https://doi.org/10.1063/5.0082122
Vertechi, P., Brendel, W., Machens, C.K.: Unsupervised learning of an efficient short-term memory network. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems. vol. 27. Curran Associates, Inc. (2014). https://proceedings.neurips.cc/paper/2014/file/333222170ab9edca4785c39f55221fe7-Paper.pdf
Verzelli, P., Alippi, C., Livi, L.: Learn to synchronize, synchronize to learn. Chaos: Interdisc. J. Nonlinear Sci. 31(8), 083119 (2021). https://doi.org/10.1063/5.0056425
Verzelli, P., Alippi, C., Livi, L., Tiňo, P.: Input-to-state representation in linear reservoirs dynamics. IEEE Trans. Neural Netw. Learn. Syst. PP1–12 (2021). https://doi.org/10.1109/TNNLS.2021.3059389
Xu, X., Zhang, J., Small, M.: Superfamily phenomena and motifs of networks induced from time series. Proc. Natl. Acad. Sci. 105(50), 19601–19605 (2008). https://doi.org/10.1073/pnas.0806082105
Acknowledgements
BT is partially supported by the Australian Government Research Training Program and a University Postgraduate Award at The University of Western Australia. BT, MS, and DC are partially supported by the Australian Research Council through the Centre for Transforming Maintenance Through Data Science (grant number IC180100030) funded by the Australian Government.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Thorne, B., Jüngling, T., Small, M., Corrêa, D., Zaitouny, A. (2022). A Novel Approach to Time Series Complexity via Reservoir Computing. In: Aziz, H., Corrêa, D., French, T. (eds) AI 2022: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13728. Springer, Cham. https://doi.org/10.1007/978-3-031-22695-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-22695-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22694-6
Online ISBN: 978-3-031-22695-3
eBook Packages: Computer ScienceComputer Science (R0)