Advertisement

Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces

  • Stefan KlusEmail author
  • Ingmar Schuster
  • Krikamol Muandet
Article
  • 42 Downloads

Abstract

Transfer operators such as the Perron–Frobenius or Koopman operator play an important role in the global analysis of complex dynamical systems. The eigenfunctions of these operators can be used to detect metastable sets, to project the dynamics onto the dominant slow processes, or to separate superimposed signals. We propose kernel transfer operators, which extend transfer operator theory to reproducing kernel Hilbert spaces and show that these operators are related to Hilbert space representations of conditional distributions, known as conditional mean embeddings. The proposed numerical methods to compute empirical estimates of these kernel transfer operators subsume existing data-driven methods for the approximation of transfer operators such as extended dynamic mode decomposition and its variants. One main benefit of the presented kernel-based approaches is that they can be applied to any domain where a similarity measure given by a kernel is available. Furthermore, we provide elementary results on eigendecompositions of finite-rank RKHS operators. We illustrate the results with the aid of guiding examples and highlight potential applications in molecular dynamics as well as video and text data analysis.

Keywords

Reproducing kernel Hilbert spaces Koopman operator Perron–Frobenius operator Eigendecompositions Kernel mean embeddings 

Mathematics Subject Classification

37M10 46E22 34L16 37L65 

Notes

Acknowledgements

This research has been partially funded by Deutsche Forschungsgemeinschaft (DFG) through grant CRC 1114 “Scaling Cascades in Complex Systems”. Krikamol Muandet acknowledges fundings from the Faculty of Science, Mahidol University and the Thailand Research Fund (TRF). We would like to thank the reviewers for their helpful comments.

References

  1. Bach, F.R., Jordan, M.I.: Kernel independent component analysis. J. Mach. Learn. Res. 3, 1–48 (2003)MathSciNetzbMATHGoogle Scholar
  2. Baker, C.: Mutual information for Gaussian processes. SIAM J. Appl. Math. 19(2), 451–458 (1970)MathSciNetzbMATHGoogle Scholar
  3. Baker, C.: Joint measures and cross-covariance operators. Trans. Am. Math. Soc. 186, 273–289 (1973)MathSciNetzbMATHGoogle Scholar
  4. Baxter, J.R., Rosenthal, J.S.: Rates of convergence for everywhere-positive Markov chains. Stat. Probab. Lett. 22(4), 333–338 (1995)MathSciNetzbMATHGoogle Scholar
  5. Berlinet, A., Thomas-Agnan, C.: Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer Academic Publishers, Berlin (2004)zbMATHGoogle Scholar
  6. Berry, T., Giannakis, D., Harlim, J.: Nonparametric forecasting of low-dimensional dynamical systems. Phys. Rev. E 91, 032915 (2015).  https://doi.org/10.1103/PhysRevE.91.032915 Google Scholar
  7. Bittracher, A., Koltai, P., Klus, S., Banisch, R., Dellnitz, M., Schütte, C.: Transition manifolds of complex metastable systems: theory and data-driven computation of effective dynamics. Accepted for publication in JNLS, (2017)Google Scholar
  8. Bo, L., Ren, X., Fox, D.: Kernel descriptors for visual recognition. In J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 244–252. Curran Associates, Inc., (2010)Google Scholar
  9. Brunton, S.L., Brunton, B.W., Proctor, J.L., Kutz, J.N.: Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control. PLoS ONE 11(2), 1–19 (2016) Google Scholar
  10. Case, D.A., Berryman, J.T., Betz, R.M., Cerutti, D.S., Cheatham, T.E., Darden, T.A., Duke, R.E., Giese, T.J., Gohlke, H., Goetz, A.W., Homeyer, N., Izadi, S., Janowski, P., Kaus, J., Kovalenko, A., Lee, T.S., LeGrand, S., Li, P., Luchko, T., Luo, R., Madej, B., Merz, K.M., Monard, G., Needham, P., Nguyen, H., Nguyen, H.T., Omelyan, I., Onufriev, A., Roe, D.R., Roitberg, A., Salomon-Ferrer, R., Simmerling, C.L., Smith, W., Swails, J., Walker, R.C., Wang, J., Wolf, R.M., Wu, X., York, D.M., Kollman, P.A.: AMBER 2015. University of California, San Francisco (2015)Google Scholar
  11. Diestel, J., Uhl, J.J.: Vector Measures. American Mathematical Society, Providence (1977)zbMATHGoogle Scholar
  12. Dinculeanu, N.: Vector Integration and Stochastic Integration in Banach Spaces. Wiley, Hoboken (2000)zbMATHGoogle Scholar
  13. Fukumizu, K., Bach, F.R., Jordan, M.I.: Dimensionality reduction for supervised learning with Reproducing Kernel Hilbert Spaces. J. Mach. Learn. Res. 5, 73–99 (2004)MathSciNetzbMATHGoogle Scholar
  14. Fukumizu, K., Bach, F., Gretton, A.: Statistical consistency of kernel canonical correlation analysis. J. Mach. Learn. Res. 8, 361–383 (2007a)MathSciNetzbMATHGoogle Scholar
  15. Fukumizu, K., Song, L., Gretton, A.: Kernel Bayes’ rule: Bayesian inference with positive definite kernels. J. Mach. Learn. Res. 14, 3753–3783 (2013)MathSciNetzbMATHGoogle Scholar
  16. Giannakis, D., Das, S., Slawinska, J.: Reproducing kernel Hilbert space compactification of unitary evolution groups. arXiv e-prints, (2018)Google Scholar
  17. Grünewälder, S., Lever, G., Gretton, A., Baldassarre, L., Patterson, S., Pontil, M.: Conditional mean embeddings as regressors. In: Proceedings of the 29th International Conference on Machine Learning (ICML), (2012)Google Scholar
  18. Hofmann, T., Schölkopf, B., Smola, A.: Kernel methods in machine learning. Ann. Stat. 36(3), 1171–1220 (2008)MathSciNetzbMATHGoogle Scholar
  19. Huisinga, W.: Metastability of Markovian systems: a transfer operator based approach in application to molecular dynamics. PhD thesis, Freie Universität Berlin (2001)Google Scholar
  20. Kato, T.: Perturbation Theory for Linear Operators. Springer, Berlin (1980)zbMATHGoogle Scholar
  21. Kawahara, Y.: Dynamic mode decomposition with reproducing kernels for Koopman spectral analysis. In: Advances in Neural Information Processing Systems 29, pp. 911–919. Curran Associates, Inc., (2016)Google Scholar
  22. Kloeden, P.E., Platen, E.: Numerical Solution of Stochastic Differential Equations. Stochastic Modelling and Applied Probability. Springer, Berlin (2011)Google Scholar
  23. Klus, S., Koltai, P., Schütte, C.: On the numerical approximation of the Perron–Frobenius and Koopman operator. J. Comput. Dyn. 3(1), 51–79 (2016)MathSciNetzbMATHGoogle Scholar
  24. Klus, S., Nüske, F., Koltai, P., Wu, H., Kevrekidis, I., Schütte, C., Noé, F.: Data-driven model reduction and transfer operator approximation. J. Nonlinear Sci. 28, 985–1010 (2018a).  https://doi.org/10.1007/s00332-017-9437-7 MathSciNetzbMATHGoogle Scholar
  25. Klus, S., Bittracher, A., Schuster, I., Schütte, C.: A kernel-based approach to molecular conformation analysis. J. Chem. Phys. 149, 244109 (2018b)Google Scholar
  26. Klus, S., Peitz, S., Schuster, I.: Analyzing high-dimensional time-series data using kernel transfer operator eigenfunctions. ArXiv e-prints, (2018c)Google Scholar
  27. Korda, M., Mezić, I.: On convergence of extended dynamic mode decomposition to the Koopman operator. J. Nonlinear Sci. 28(2), 687–710 (2018)MathSciNetzbMATHGoogle Scholar
  28. Lasota, A., Mackey, M. C.: Chaos, fractals, and noise: Stochastic aspects of dynamics, volume 97 of Applied Mathematical Sciences. Springer, 2nd edition, (1994)Google Scholar
  29. Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. J. Mach. Learn. Res. 2, 419–444 (2002)zbMATHGoogle Scholar
  30. McGibbon, R.T., Pande, V.S.: Variational cross-validation of slow dynamical modes in molecular kinetics. J. Chem. Phy. 142(12), 124105 (2015)Google Scholar
  31. Molgedey, L., Schuster, H.G.: Separation of a mixture of independent signals using time delayed correlations. Phys. Rev. Lett. 72, 3634–3637 (1994)Google Scholar
  32. Muandet, K., Fukumizu, K., Sriperumbudur, B., Schölkopf, B.: Kernel mean embedding of distributions: a review and beyond. Found. Trends Mach. Learn. 10(1–2), 1–141 (2017)zbMATHGoogle Scholar
  33. Noé, F., Nüske, F.: A variational approach to modeling slow processes in stochastic dynamical systems. Multiscale Model. Simul. 11(2), 635–655 (2013)MathSciNetzbMATHGoogle Scholar
  34. Nüske, F., Keller, B.G., Pérez-Hernández, G., Mey, A.S.J.S., Noé, F.: Variational approach to molecular kinetics. J. Chem. Theory Comput. 10(4), 1739–1752 (2014)Google Scholar
  35. Pérez-Hernández, G., Paul, F., Giorgino, T., De Fabritiis, G., Noé, F.: Identification of slow molecular order parameters for Markov model construction. J. Chem. Phys. 139(1), 015102 (2013)Google Scholar
  36. Rowley, C.W., Mezić, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of nonlinear flows. J. Fluid Mech. 641, 115–127 (2009)MathSciNetzbMATHGoogle Scholar
  37. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech. 656, 5–28 (2010)MathSciNetzbMATHGoogle Scholar
  38. Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, Cambridge (2001)Google Scholar
  39. Schölkopf, B., Smola, A., Müller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)Google Scholar
  40. Schuster, I., Mollenhauer, M., Klus, S., Muandet, K.: Kernel conditional density operators. arXiv e-prints, (2019)Google Scholar
  41. Schütte, C.: Conformational dynamics: modelling, theory, algorithm, and application to biomolecules. Habilitation Thesis (1999)Google Scholar
  42. Schütte, C., Sarich, M.: Metastability and Markov State Models in Molecular Dynamics: Modeling, Analysis, Algorithmic Approaches. Number 24 in Courant Lecture Notes. American Mathematical Society (2013)Google Scholar
  43. Schwantes, C.R., Pande, V.S.: Modeling molecular kinetics with tICA and the kernel trick. J. Chem. Theory Comput. 11(2), 600–608 (2015)Google Scholar
  44. Sejdinovic, D., Strathmann, H., Garcia, M.L., Andrieu, C., Gretton, A.: Kernel adaptive Metropolis–Hastings. In: International conference on machine learning, pp. 1665–1673 (2014)Google Scholar
  45. Smola, A., Gretton, A., Song, L., Schölkopf, B.: A Hilbert space embedding for distributions. In: Proceedings of the 18th International Conference on Algorithmic Learning Theory, pp. 13–31. Springer, Berlin (2007)Google Scholar
  46. Song, L., Huang, J., Smola, A., Fukumizu, K.: Hilbert space embeddings of conditional distributions with applications to dynamical systems. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 961–968 (2009)Google Scholar
  47. Song, L., Fukumizu, K., Gretton, A.: Kernel embeddings of conditional distributions: a unified kernel framework for nonparametric inference in graphical models. IEEE Signal Process. Mag. 30(4), 98–111 (2013)Google Scholar
  48. Sriperumbudur, B., Gretton, A., Fukumizu, K., Lanckriet, G., Schölkopf, B.: Injective Hilbert space embeddings of probability measures. In: The 21st Annual Conference on Learning Theory, pp. 111–122. Omnipress (2008)Google Scholar
  49. Steinwart, I., Christmann, A.: Support Vector Machines, 1st edn. Springer Publishing Company, Incorporated, Berlin (2008)zbMATHGoogle Scholar
  50. Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L., Kutz, J.N.: On dynamic mode decomposition: theory and applications. J. Comput. Dyn. 1(2), 391 (2014)MathSciNetzbMATHGoogle Scholar
  51. Ulam, S .M.: A Collection of Mathematical Problems. Interscience Publisher, New York (1960)zbMATHGoogle Scholar
  52. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015a)MathSciNetzbMATHGoogle Scholar
  53. Williams, M.O., Rowley, C.W., Kevrekidis, I.G.: A kernel-based method for data-driven Koopman spectral analysis. J. Comput. Dyn. 2(2), 247–265 (2015b)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Stefan Klus
    • 1
    Email author
  • Ingmar Schuster
    • 2
  • Krikamol Muandet
    • 3
  1. 1.Department of Mathematics and Computer ScienceFreie Universität BerlinBerlinGermany
  2. 2.Zalando Research, Zalando SEBerlinGermany
  3. 3.Max Planck Institute for Intelligent SystemsTübingenGermany

Personalised recommendations