Abstract
High-dimensional signals generated by dynamical systems arise in many fields of science. For example, many biomedical signals can be modeled by a few latent physiologically related variables measured indirectly through a large set of noisy sensors. In such applications, a notable challenge in analyzing and processing the observed data is that the generating system is typically unknown. In this chapter, this problem is addressed through geometric analysis by applying manifold learning. Specifically, we show how using manifold learning in a purely data-driven manner, with minimal prior knowledge or model assumptions, we can both discover the hidden state, dynamics, and observation function, and also attain a compact linear description of the full system. The main assumption is that the accessible high- dimensional data (the observations of the system) lie on an underlying nonlinear manifold of lower dimensions. Furthermore, when applying manifold learning to time series, critical information is typically overlooked. Time series are processed as data sets of samples, ignoring their embodied dynamics and temporal order. We address the challenge of incorporating the time dependencies into manifold learning and present a purely data-driven scheme. First, an intrinsic representation is derived without prior knowledge of the system by applying diffusion maps. Second, we show that even for highly nonlinear systems, the dynamics of the constructed representation is approximately linear, and present accordingly two filtering frameworks, one based on a linear observer and another based on the Kalman filter. These filtering methods enable us to directly incorporate the inherent dynamics and time dependencies between consecutive system observations into the diffusion maps coordinates. We show that this approach is analogous to Koopman spectral analysis, since applying diffusion maps to the given measurements generates a parametrization of the state space in a known analytic form of the dynamics with a linear drift. We demonstrate the benefits of the approach on simulated data and on real recordings from various applications.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Berry, T., Giannakis, D., Harlim, J.: Nonparametric forecasting of low-dimensional dynamical systems. Phys. Rev. E 91(3), 032,915 (2015)
Berry, T., Harlim, J.: Nonparametric uncertainty quantification for stochastic gradient flows. SIAM/ASA J. Uncertain. Quantif. 3(1), 484–508 (2015)
Berry, T., Sauer, T.: Adaptive ensemble Kalman filtering of non-linear systems. Tellus A: Dyn. Meteorol. Oceanogr. 65(1), 20,331 (2013)
Budišić, M., Mohr, R., Mezić, I.: Applied Koopmanism a. Chaos: Interdiscip. J. Nonlinear Sci. 22(4), 047,510 (2012)
Coifman, R., Kevrekidis, I., Lafon, S., Maggioni, M., Nadler, B.: Diffusion maps, reduction coordinates, and low dimensional representation of stochastic systems. Multiscale Model. Simul. 7(2), 842–864 (2008)
Coifman, R., Lafon, S.: Diffusion maps. Appl. Comput. Harmon. Anal. 21, 5–30 (2006)
Črnjarić-Žic, N., Maćešić, S., Mezić, I.: Koopman operator spectrum for random dynamical system (2017). arXiv:1711.03146
Deisenroth, M.P., Turner, R.D., Huber, M.F., Hanebeck, U.D., Rasmussen, C.E.: Robust filtering and smoothing with gaussian processes. IEEE Trans. Autom. Control 57(7), 1865–1871 (2012)
Giannakis, D.: Data-driven spectral decomposition and forecasting of ergodic dynamical systems. Appl. Comput. Harmonic Anal. (2017)
Kalman, R.: A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82, 34–45 (1960)
Lohmiller, W., Slotine, J.: On contraction analysis for non-linear systems. Automatica 34(6), 683–696 (1998)
Mezić, I.: Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dyn. 41(1), 309–325 (2005)
Nadler, B., Lafon, S., Coifman, R., Kevrekidis, I.G.: Diffusion maps, spectral clustering and eigenfunctions of Fokker-Planck operators. In: Neural Information Processing Systems (NIPS), vol. 18 (2005)
Nadler, B., Lafon, S., Coifman, R., Kevrekidis, I.G.: Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Appl. Comput. Harmon. Anal. 21, 113–127 (2006)
Shnitzer, T., Talmon, R., Slotine, J.J.: Diffusion maps Kalman filter (2017). arXiv:1711.09598
Shnitzer, T., Talmon, R., Slotine, J.J.E.: Manifold learning with contracting observers for data-driven time-series analysis. IEEE Trans. Signal Process. 65(4), 904–918 (2017)
Singer, A., Coifman, R.: Non-linear independent component analysis with diffusion maps. Appl. Comput. Harmon. Anal. 25, 226–239 (2008)
Sinha, S., Bowen, H., Vaidya, U.: On robust computation of Koopman operator and prediction in random dynamical systems (2018). arXiv:1803.08562
Surana, A., Banaszuk, A.: Linear observer synthesis for nonlinear systems using Koopman operator framework. IFAC-PapersOnLine 49(18), 716–723 (2016)
Takeishi, N., Kawahara, Y., Yairi, T.: Learning Koopman invariant subspaces for dynamic mode decomposition. In: Advances in Neural Information Processing Systems, pp. 1130–1140 (2017)
Takeishi, N., Kawahara, Y., Yairi, T.: Subspace dynamic mode decomposition for stochastic Koopman analysis. Phys. Rev. E 96(3), 033,310 (2017)
Takens, F.: Dynamical systems and turbulence (detecting strange attractors in fluid turbulence) (1981)
Wang, Y., Brubaker, M.A., Chaib-draa, B., Urtasun, R.: Bayesian filtering with online Gaussian process latent variable models. UAI pp. 849–857 (2014)
Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Shnitzer, T., Talmon, R., Slotine, JJ. (2020). Manifold Learning for Data-Driven Dynamical System Analysis. In: Mauroy, A., Mezić, I., Susuki, Y. (eds) The Koopman Operator in Systems and Control. Lecture Notes in Control and Information Sciences, vol 484. Springer, Cham. https://doi.org/10.1007/978-3-030-35713-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-35713-9_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35712-2
Online ISBN: 978-3-030-35713-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)