Skip to main content

Data-Driven Model Reduction and Transfer Operator Approximation

Journal of Nonlinear Science volume 28, pages 985–1010 (2018)Cite this article

Abstract

In this review paper, we will present different data-driven dimension reduction techniques for dynamical systems that are based on transfer operator theory as well as methods to approximate transfer operators and their eigenvalues, eigenfunctions, and eigenmodes. The goal is to point out similarities and differences between methods developed independently by the dynamical systems, fluid dynamics, and molecular dynamics communities such as time-lagged independent component analysis, dynamic mode decomposition, and their respective generalizations. As a result, extensions and best practices developed for one particular method can be carried over to other related methods.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. A general time-homogeneous Itô stochastic differential equation is given by \(\mathrm {d}\mathbf {X}_{t}=-\alpha (\mathbf {X}_{t})\,\mathbf {X}_{t}\,\mathrm {d}t+\sigma (\mathbf {X}_{t})\,\mathrm {d}\mathbf {W}_{t}\), where \(\alpha :\mathbb {R}^{d}\rightarrow \mathbb {R}^{d}\) and \(\sigma :\mathbb {R}^{d}\rightarrow \mathbb {R}^{d\times d}\) are coefficient functions, and \(\{\mathbf {W}_{t}\}_{t\ge 0}\) is a d-dimensional standard Wiener process.

  2. We call a stochastic process \(\{\mathbf {X}_{t}\}_{t\ge 0}\) time-homogeneous, or autonomous, if it holds for every \(t\ge s\ge 0\) that the distribution of \(\mathbf {{X}}_{t}\) conditional to \(\mathbf {X}_{s}=x\) only depends on x and \((t-s)\). It is the stochastic analogue of the flow of an autonomous (time-independent) ordinary differential equation.

  3. For a measure-theoretic discussion of this construction, please refer to Klus et al. (2016). For our purposes, it is sufficient to equip \(\mathbb {X}\) with the standard Lebesgue measure. In particular, if not stated otherwise, measurability of a set \(\mathbb {A\subset X}\) is meant with respect to the Borel \(\sigma \)-algebra.

  4. These conditions are called interchangeably absolute continuity,\(\mu \)-compatibility, or null preservingness.

  5. Algorithm for Multiple Unknown Signals Extraction.

  6. The easiest way to accomplish this is by adding the observables \(x_{i}\), \(i=1,\dots ,d\), to the set of basis functions.

  7. A process \(\{\mathbf {X}_{t}\}_{t\ge 0}\) is called Feller-continuous if the mapping \(x\mapsto \mathbb {E}[g(\mathbf {X}_{t})\vert \mathbf {X}_{0}=x]\) is continuous for any fixed continuous function g. This implies, that the Koopman operator of a Feller-continuous process has a well-defined restriction from \(L^{\infty }(\mathbb {X})\) to the set of continuous functions. Any stochastic process generated by an Itô stochastic differential equation with Lipschitz-continuous coefficients is Feller-continuous (Øksendal 2003, Lemma 8.1.4).

References

  • Bandle, C.: Isoperimetric inequalities and applications. Pitman, Monographs and studies in mathematics (1980)

  • Baxter, J.R., Rosenthal, J.S.: Rates of convergence for everywhere-positive Markov chains. Stat. Probab. Lett. 22(4), 333–338 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  • Brunton, S.L., Proctor, J.L., Tu, J.H., Kutz, J.N.: Compressed sensing and dynamic mode decomposition. J. Comput. Dyn. 2(2), 165–191 (2015)

    MathSciNet  MATH  Google Scholar 

  • Brunton, B.W., Johnson, L.A., Ojemann, J.G., Kutz, J.N.: Extracting spatial-temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition. J. Neurosci. Methods 258, 1–15 (2016)

    Article  Google Scholar 

  • Budišić, M., Mohr, R., Mezić, I.: Applied koopmanism. Appl. Chaos An Interdiscip. J. Nonlinear Sci. 22(4), 047510 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Chen, K.K., Tu, J.H., Rowley, C.W.: Variants of dynamic mode decomposition: boundary condition, Koopman, and Fourier analyses. J. Nonlinear Sci. 22(6), 887–915 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Coifman, R.R., Kevrekidis, I.G., Lafon, S., Maggioni, M., Nadler, B.: Diffusion maps, reduction coordinates, and low dimensional representation of stochastic systems. Multiscale Model. Simul. 7(2), 842–864 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Dellnitz, M., Junge, O.: On the approximation of complicated dynamical behavior. SIAM J. Numer. Anal. 36(2), 491–515 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Deuflhard, P., Weber, M.: Robust Perron cluster analysis in conformation dynamics. Linear Algebra Appl. 398, 161–184 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Djurdjevac, N., Sarich, M., Schütte, C.: Estimating the eigenvalue error of Markov state models. Multiscale Model. Simul. 10(1), 61–81 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Ferguson, A.L., Panagiotopoulos, A.Z., Kevrekidis, I.G., Debenedetti, P.G.: Nonlinear dimensionality reduction in molecular simulation: the diffusion map approach. Chem. Phys. Lett. 509(1), 1–11 (2011)

    Article  Google Scholar 

  • Froyland, G., Padberg, K.: Almost-invariant sets and invariant manifolds-connecting probabilistic and geometric descriptions of coherent structures in flows. Physica D 238, 1507–1523 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Froyland, G., Padberg-Gehle, K.: Almost-invariant and finite-time coherent sets: directionality, duration, and diffusion. In: Bahsoun, W., Bose, C.H., Froyland, G. (eds.) Ergodic Theory. Open Dynamics, and Coherent Structures, pp. 171–216. Springer, New York (2014)

    Chapter  Google Scholar 

  • Froyland, G., Junge, O., Koltai, P.: Estimating long term behavior of flows without trajectory integration: the infinitesimal generator approach. SIAM J. Numer. Anal. 51(1), 223–247 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Froyland, G., Gottwald, G., Hammerlindl, A.: A computational method to extract macroscopic variables and their dynamics in multiscale systems. SIAM J. Appl. Dyn. Syst. 13(4), 1816–1846 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Giannakis, D.: Data-driven spectral decomposition and forecasting of ergodic dynamical systems. rXiv e-prints (2015)

  • Hopf, E.: The general temporally discrete Markoff process. J. Ration. Mech. Anal. 3(1), 13–45 (1954)

    MathSciNet  MATH  Google Scholar 

  • Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley, Hoboken (2001)

    Book  Google Scholar 

  • Jovanović, M.R., Schmid, P.J., Nichols, J.W.: Sparsity-promoting dynamic mode decomposition. Phys. Fluids 26(2), 024103 (2014)

    Article  Google Scholar 

  • Junge, O., Koltai, P.: Discretization of the Frobenius–Perron operator using a sparse Haar tensor basis: the Sparse Ulam method. SIAM J. Numer. Anal. 47, 3464–3485 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Klus, S., Gelß, P., Peitz, S. and Schütte, Ch.: Tensor-based dynamic mode decomposition. ArXiv e-prints (2016)

  • Klus, S. and Schütte, CH.: Towards tensor-based methods for the numerical approximation of the Perron–Frobenius and Koopman operator. J. Comput. Dyn. 3(2) (2016)

  • Klus, S., Koltai, P., Schütte, Ch.: On the numerical approximation of the Perron–Frobenius and Koopman operator. J. Comput. Dyn. 3(1), 51–79 (2016)

    MathSciNet  MATH  Google Scholar 

  • Koopman, B.: Hamiltonian systems and transformation in Hilbert space. Proc. Ntl. Acad. Sci. USA 17(5), 315 (1931)

    Article  MATH  Google Scholar 

  • Korda, M. and Mezić, I.: On convergence of Extended Dynamic Mode Decomposition to the Koopman operator. arXiv preprint rXiv:1703.04680 (2017)

  • Krengel, U.: Ergodic Theorems, Volume 6 of de Gruyter Studies in Mathematics, vol. 6. Walter de Gruyter & Co., Berlin (1985)

    Google Scholar 

  • Kutz, J.N., Brunton, S.L., Brunton, B.W., Proctor, J.L.: Dynamic mode decomposition: data-driven modeling of complex systems. SIAM (2016)

  • Lasota, A.: Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics volume 97 of Applied Mathematical Sciences, 2nd edn. Springer, Berlin (1994)

    Book  Google Scholar 

  • Leimkuhler, B., Chipot, Ch., Elber, R., Laaksonen, A., Mark, A., Schlick, T., Schütte, Ch., Skeel, R.: New Algorithms for Macromolecular Simulation (Lecture Notes in Computational Science and Engineering). Springer, New York (2006)

    Book  MATH  Google Scholar 

  • Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, pp. 689–696. ACM (2009)

  • McGibbon, R.T., Pande, V.S.: Variational cross-validation of slow dynamical modes in molecular kinetics. J. Chem. Phys. 142(12), 03B621 (2015)

    Article  Google Scholar 

  • Mezić, I.: Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dyn. 41(1), 309–325 (2005)

    MathSciNet  MATH  Google Scholar 

  • Mezić, I.: Analysis of fluid flows via spectral properties of the Koopman operator. Ann. Rev. Fluid Mech. 45(1), 357–378 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Molgedey, L., Schuster, H.G.: Separation of a mixture of independent signals using time delayed correlations. Phys. Rev. Lett. 72, 3634–3637 (1994)

    Article  Google Scholar 

  • Nadler, B., Lafon, S., Coifman, R.R., Kevrekidis, I.G.: Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Appl. Comput. Harmonic Anal. 21(1), 113–127 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Noé, F., Clementi, C.: Kinetic distance and kinetic maps from molecular dynamics simulation. J. Chem. Theory Comput. 11(10), 5002–5011 (2015)

    Article  Google Scholar 

  • Noé, F., Nüske, F.: A variational approach to modeling slow processes in stochastic dynamical systems. Multiscale Model. Simul. 11(2), 635–655 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Noé, F., Wu, H., Prinz, J.-H., Plattner, N.: Projected and hidden Markov models for calculating kinetics and metastable states of complex molecules. J. Chem. Phys. 139, 184114 (2013)

    Article  Google Scholar 

  • Nüske, F., Keller, B.G., Pérez-Hernández, G., Mey, A.S.J.S., Noé, F.: Variational approach to molecular kinetics. J. Chem. Theory Comput. 10(4), 1739–1752 (2014)

    Article  Google Scholar 

  • Nüske, F., Keller, B.G., Pérez-Hernández, G., Mey, A.S.J.S., Noé, F.: Variational approach to molecular kinetics. J. Chem. Theory Comput. 10, 1739–1752 (2014)

    Article  Google Scholar 

  • Nüske, F., Schneider, R., Vitalini, F., Noé, F.: Variational tensor approach for approximating the rare-event kinetics of macromolecular systems. J. Chem. Phys. 144(5), 054105 (2016)

    Article  Google Scholar 

  • Øksendal, B.: Stochastic Differential Equations, 6th edn. Springer, Berlin (2003)

    Book  MATH  Google Scholar 

  • Pavliotis, G.A.: Stochastic Processes and Applications: Diffusion Processes, the Fokker–Planck and Langevin Equations volume 60 of Texts in Applied Mathematics. Springer, Berlin (2014)

    Book  Google Scholar 

  • Pazy, A.: Semigroups of Linear Operators and Applications to Partial Differential Equations. Springer, Berlin (1983)

    Book  MATH  Google Scholar 

  • Pérez-Hernández, G., Paul, F., Giorgino, T., De Fabritiis, G., Noé, F.: Identification of slow molecular order parameters for Markov model construction. J. Chem. Phys. 139(1), 07B604 (2013)

    Article  Google Scholar 

  • Prinz, J.-H., Wu, H., Sarich, M., Keller, B., Senne, M., Held, M., Chodera, J.D., Schütte, C., Noé, F.: Markov models of molecular kinetics: generation and validation. J. Chem. Phys. 134, 174105 (2011)

    Article  Google Scholar 

  • Röblitz, S., Weber, M.: Fuzzy spectral clustering by PCCA+: application to Markov state models and data classification. Adv. Data Anal. Classif. 7(2), 147–179 (2013)

    Article  MATH  Google Scholar 

  • Rowley, C.W., Mezić, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of nonlinear flows. J. Fluid Mech. 641, 115–127 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Sarich, M., Noé, F., Schütte, C.: On the approximation quality of Markov state models. Multiscale Model. Simul. 8, 1154–1177 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech. 656, 5–28 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Schütte, Ch. and Sarich, M.: Metastability and Markov State Models in Molecular Dynamics: Modeling, Analysis, Algorithmic Approaches. Number 24 in Courant Lecture Notes. American Mathematical Society, (2013)

  • Schütte, Ch., Fischer, A., Huisinga, W., Deuflhard, P.: A direct approach to conformational dynamics based on hybrid Monte Carlo. J. Comput. Phys. 151(1), 146–168 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Schwantes, C.R., Pande, V.S.: Improvements in Markov state model construction reveal many non-native interactions in the folding of NTL9. J. Chem. Theory. Comput. 9, 2000–2009 (2013)

    Article  Google Scholar 

  • Shadden, S.C., Lekien, F., Marsden, J.E.: Definition and properties of Lagrangian coherent structures from finite-time Lyapunov exponents in two-dimensional aperiodic flows. Phys. D Nonlinear Phenom. 212(3), 271–304 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Tong, L., Soon, V.C., Huang, Y.F., Liu, R.: AMUSE: a new blind identification algorithm. In: IEEE International Symposium on Circuits and Systems, pp. 1784–1787 (1990)

  • Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L. ,Kutz, J.N.: On dynamic mode decomposition: theory and applications. J. Comput. Dyn. 1(2), (2014)

  • Ulam, S.M.: A Collection of Mathematical Problems. Interscience Publisher, Hoboken (1960)

    MATH  Google Scholar 

  • Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Williams, M.O., Rowley, C.W., Kevrekidis, I.G.: A kernel-based method for data-driven Koopman spectral analysis. J. Comput. Dyn. 2(2), 247–265 (2015)

    MathSciNet  MATH  Google Scholar 

  • Wu, H., Nüske, F., Paul, F., Klus, S., Koltai, P., Noé, F.: Variational Koopman models: slow collective variables and molecular kinetics from short off-equilibrium simulations. J. Chem. Phys. 146(15), 154104 (2017)

    Article  Google Scholar 

  • Ziehe, A. and Müller, K.-R.: TDSEP—an efficient algorithm for blind separation using time structure. In: CANN 98, pp. 675–680. Springer Science and Business Media (1998)

Download references

Acknowledgements

This research has been partially funded by Deutsche Forschungsgemeinschaft (DFG) through grant CRC 1114 “Scaling Cascades in Complex Systems,” Project A04 “Efficient calculation of slow and stationary scales in molecular dynamics” and Project B03 “Multilevel coarse graining of multi-scale problems”, and by the Einstein Foundation Berlin (Einstein Center ECMath). Furthermore, we would like to thank the reviewers for their helpful comments and suggestions.

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, Freie Universität Berlin, Berlin, Germany

    Stefan Klus, Feliks Nüske, Péter Koltai, Hao Wu, Christof Schütte & Frank Noé

  2. Department of Chemical and Biological Engineering and Program in Applied and Computational Mathematics, Princeton University, Princeton, NJ, USA

    Ioannis Kevrekidis

  3. Zuse Institute Berlin, Berlin, Germany

    Ioannis Kevrekidis & Christof Schütte

Authors
  1. Stefan Klus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feliks Nüske
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Péter Koltai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ioannis Kevrekidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Christof Schütte
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Frank Noé
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefan Klus.

Additional information

Communicated by Clarence W. Rowley.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Klus, S., Nüske, F., Koltai, P. et al. Data-Driven Model Reduction and Transfer Operator Approximation. J Nonlinear Sci 28, 985–1010 (2018). https://doi.org/10.1007/s00332-017-9437-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00332-017-9437-7

Keywords

  • Koopman operator
  • Perron-Frobenius operator
  • Model reduction
  • Data-driven methods

Mathematics Subject Classification

  • 37M10
  • 37M25
  • 37L65
  • 34L16