Abstract
Experimentation as a setting for learning demands adaptability on the part of the decision-making system. It is typically infeasible for agents to have complete a priori knowledge of an environment, their own dynamics, or the behavior of other agents. In order to achieve autonomy in robotic applications, learning must occur incrementally, and ideally as a function of decision-making by exploiting the underlying control system. Most artificial intelligence techniques are ill suited for experimental settings because they either lack the ability to learn incrementally or do not have information measures with which to guide their learning. This chapter examines the Koopman operator, its application in active learning, and its relationship to alternative learning techniques, such as Gaussian processes and kernel ridge regression. Additionally, examples are provided from a variety of experimental applications of the Koopman operator in active learning settings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abraham, I., Murphey, T.D.: Active learning of dynamics for data-driven control using koopman operators. IEEE Trans. Robot. 35(5), 1071–1083 (2019)
Abraham, I., Torre, G., Murphey, T.: Model-based control using Koopman operators. In: Robotics: Science and Systems (2017)
Ansari, A., Murphey, T.D.: Sequential action control: Closed-form optimal control for nonlinear and nonsmooth systems. IEEE Trans. Robot. 32 (2017)
Berrueta, T.A., Pervan, A., Fitzsimons, K., Murphey, T.D.: Dynamical system segmentation for information measures in motion. IEEE Robot. Autom. Lett. 4(1), 169–176 (2019)
Bishop, C.: Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, Berlin (2006)
Brunton, S., Brunton, B., Proctor, J., Kutz, J.: Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control. PLOS ONE 11(2), 1–19 (2016)
Budišić, M., Mohr, R., Mezić, I.: Applied Koopmanism. Chaos: Interdiscip. J. Nonlinear Sci. 22(4), 047, 510 (2012)
Campello, R., Moulavi, D., Sander, J.: Density-based clustering based on hierarchical density estimates. Advances in Knowledge Discovery and Data Mining, pp. 160–172. Springer, Berlin (2013)
Chen, B.W., Abdullah, N.N.B., Park, S., Gu, Y.: Efficient multiple incremental computation for kernel ridge regression with Bayesian uncertainty modeling. Futur. Gener. Comput. Syst. 82, 679–688 (2018)
Chirikjan, G.S.: Stochastic Models, Information Theory, and Lie Groups, vol. 2. Springer, Berlin (2012)
Dias, J., Leitao, J.: A method for computing the information matrix of stationary Gaussian processes. In: 1996 8th European Signal Processing Conference (EUSIPCO 1996), pp. 1–4 (1996)
Engel, Y., Mannor, S., Meir, R.: The kernel recursive least squares algorithm. IEEE Trans. Signal Process. 52(8), 2275–2285 (2004)
Fan, T., Murphey, T.: Online feedback control for input-saturated robotic systems on Lie groups. In: Robotics: Science and Systems (2016)
Gallager, R.: Stochastic Processes: Theory for Applications. Cambridge University Press, Cambridge (2013)
Govindarajan, N., Arbabi, H., van Blargian, L., Matchen, T., Tegling, E., Mezić, I.: An operator-theoretic viewpoint to non-smooth dynamical systems: Koopman analysis of a hybrid pendulum. In: 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 6477–6484 (2016)
Hamilton, J.: Time Series Analysis. Princeton University Press, Princeton (1994)
Heersink, B., Warren, M., Hoffmann, H.: Dynamic mode decomposition for interconnected control systems (2017). arxiv:1709.02883
Hemati, M.S., Williams, M.O., Rowley, C.W.: Dynamic mode decomposition for large and streaming datasets. Phys. Fluids 26(11), 111, 701 (2014)
Henzinger, T.A.: The theory of hybrid automata. In: Proceedings of the 11th Annual IEEE Symposium on Logic in Computer Science, pp. 278–283 (1996)
Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for control (2017). arxiv:1707.01146
Kalinowska, A., Fitzsimons, K., Dewald, J.P., Murphey, T.D.: Online user assessment for minimal intervention during task-based robotic assistance. In: Robotics: Science and Systems (2018)
Koopman, B.: Hamiltonian systems and transformation in Hilbert space. Proc. Natl. Acad. Sci. 17(5), 315–318 (1931)
Korda, M., Mezić, I.: Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control. Automatica 93, 149–160 (2018)
Lee, J., Bahri, Y., Novak, R., Schoenholz, S.S., Pennington, J., Sohl-Dickstein, J.: Deep neural networks as Gaussian processes (2018). arxiv:1711.00165
Li, Q., Dietrich, F., Bollt, E.M., Kevrekidis, I.G.: Extended dynamic mode decomposition with dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator. Chaos: Interdiscip. J. Nonlinear Sci. 27(10), 103, 111 (2017)
Mezić, I.: Analysis of fluid flows via spectral properties of the Koopman operator. Annu. Rev. Fluid Mech. 45(1), 357–378 (2013). https://doi.org/10.1146/annurev-fluid-011212-140652
Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press, Cambridge (2012)
Neal, R.M.: Priors for infinite networks. Bayesian Learning for Neural Networks, pp. 29–53. Springer, Berlin (1996)
Nguyen-Tuong, D., Peters, J., Seeger, M.: Local Gaussian process regression for real time online model learning and control. In: Proceedings of the 21st International Conference on Neural Information Processing Systems, pp. 1193–1200 (2008)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Peitz, S., Klus, S.: Koopman operator-based model reduction for switched-system control of PDEs (2017). arXiv:1710.06759
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. The MIT Press, Cambridge (2005)
Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech. 656, 5–28 (2010)
Seal, H.L.: Studies in the history of probability and statistics. XV: the historical development of the Gauss linear model. Biometrika 54(1/2), 1–24 (1967)
Shahriari, B., Swersky, K., Wang, Z., Adams, R., de Freitas, N.: Taking the human out of the loop: a review of Bayesian optimization. Proc. IEEE 104(1), 148–175 (2016)
Srivastava, J., Anderson, D.: A comparison of the determinant, maximum root, and trace optimality criteria. Commun. Stat. 3(10), 933–940 (1974)
Takeishi, N., Kawahara, Y., Tabei, Y., Yairi, T.: Bayesian dynamic mode decomposition. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pp. 2814–2821 (2017)
Tu, J., Rowley, C., Luchtenburg, D., Brunton, S., Kutz, J.: On dynamic mode decomposition: theory and applications. J. Comput. Dyn. 1, 391 (2014). https://doi.org/10.3934/jcd.2014.1.391
Umlauft, J., Beckers, T., Kimmel, M., Hirche, S.: Feedback linearization using Gaussian processes. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 5249–5255 (2017)
Van Vaerenbergh, S., Fernandez-Bes, J., Elvira, V.: On the relationship between online Gaussian process regression and kernel least mean squares algorithms. In: 2016 IEEE International Workshop on Machine Learning for Signal Processing (2016)
Williams, M., Kevrekidis, I., Rowley, C.: A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25, 1307–1346 (2015)
Williams, M.O., Rowley, C.W., Kevrekidis, I.G.: A kernel-based method for data-driven Koopman spectral analysis. J. Comput. Dyn. 2, 247–265 (2015)
Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network representations for Koopman operators of nonlinear dynamical systems (2017). arxiv:1708.06850
Acknowledgements
This work was supported by the National Science Foundation under grant CBET-1637764. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Berrueta, T.A., Abraham, I., Murphey, T. (2020). Experimental Applications of the Koopman Operator in Active Learning for Control. In: Mauroy, A., Mezić, I., Susuki, Y. (eds) The Koopman Operator in Systems and Control. Lecture Notes in Control and Information Sciences, vol 484. Springer, Cham. https://doi.org/10.1007/978-3-030-35713-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-35713-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35712-2
Online ISBN: 978-3-030-35713-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)