Abstract
This paper presents a dependent multi-output Gaussian process (GP) for modeling complex dynamical systems. The outputs are dependent in this model, which is largely different from previous GP dynamical systems. We adopt convolved multi-output GPs to model the outputs, which are provided with a flexible multi-output covariance function. We adapt the variational inference method with inducing points for approximate posterior inference of latent variables. Conjugate gradient based optimization is used to solve parameters involved. Besides the temporal dependency, the proposed model also captures the dependency among outputs in complex dynamical systems. We evaluate the model on both synthetic and real-world data, and encouraging results are observed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Álvarez, M.A., Luengo, D., Lawrence, N.D.: Linear latent force models using Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 2693–2705 (2013)
Álvarez, M.A., Lawrence, N.D.: Computationally efficient convolved multiple output Gaussian processes. Journal of Machine Learning Research 12, 1459–1500 (2011)
Álvarez, M.A., Luengo, D., Lawrence, N.D.: Latent force models. In: Proceedings of the 12th International Conference on Articicial Intelligence and Statistics, pp. 9–16 (2009)
Bonilla, E.V., Chai, K.M., Williams, C.K.I.: Multi-task Gaussian process prediction. In: Advances in Neural Information Processing Systems, vol. 18, pp. 153–160 (2008)
Damianou, A.C., Ek, C.H., Titsias, M.K., Lawrence, N.D.: Manifold relevance determination. In: Proceedings of the 29th International Conference on Machine Learning, pp. 145–152 (2012)
Damianou, A.C., Titsias, M.K., Lawrence, N.D.: Variational Gaussian process dynamical systems. In: Advances in Neural Information Processing Systems, vol. 24, pp. 2510–2518 (2011)
Deisenroth, M.P., Mohamed, S.: Expectation propagation in Gaussian process dynamical systems. In: Advances in Neural Information Processing Systems, vol. 25, pp. 2618–2626 (2012)
Hartikainen, J., Särkkä, S.: Sequential inference for latent force models (2012), http://arxiv.org/abs/1202.3730
Lawrence, N.D.: Gaussian process latent variable models for visualisation of high dimensional data. In: Advances in Neural Information Processing Systems, vol. 17, pp. 329–336 (2004)
Lawrence, N.D.: Probabilistic non-linear principal component analysis with Gaussian process latent variable models. Journal of Machine Learning Research 6, 1783–1816 (2005)
Lawrence, N.D.: Learning for larger dataset with the Gaussian process latent variable model. In: Proceedings of the 11th International Workshop on Artificial Intelligence and Statistics, pp. 243–250 (2007)
Luttinen, J., Ilin, A.: Efficient Gaussian process inference for short-scale spatio-temporal modeling. In: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics, pp. 741–750 (2012)
Opper, M., Archambeau, A.: The variational Gaussian approximation revisited. Neural Computation 21, 786–792 (2009)
Park, H., Yun, S., Park, S., Kim, J., Yoo, C.D.: Phoneme classification using constrained variational Gaussian process dynamical system. In: Advances in Neural Information Processing Systems, vol. 22, pp. 2015–2023 (2012)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Process for Machine Learning. MIT Press (2006)
Sun, S.: A review of deterministic approximate inference techniques for Bayesian machine learning. Neural Computing and Applications 23, 2039–2050 (2013)
Taylor, G.W., Hinton, G.E., Roweis, S.: Modeling human motion using binary latent variables. In: Advances in Neural Information Processing Systems, vol. 17, pp. 1345–1352 (2007)
Tipping, M.E., Bishop, C.M.: Probabilistic principal component analysis. Journal of the Royal Statistical Society 61, 611–622 (1999)
Titsias, M.K.: Variational learning of inducing variables in sparse Gaussian processes. In: Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, pp. 567–574 (2009)
Titsias, M.K., Lawrence, N.D.: Bayesian Gaussian process latent variable model. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, pp. 844–851 (2010)
Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models. In: Advances in Neural Information Processing Systems, vol. 19, pp. 1441–1448 (2006)
Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 283–398 (2008)
Wilson, A.G., Knowles, D.A., Ghahramani, Z.: Gaussian process regression networks. In: Proceedings of the 29th International Conference on Machine Learning, pp. 599–606 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Zhao, J., Sun, S. (2014). Variational Dependent Multi-output Gaussian Process Dynamical Systems. In: Džeroski, S., Panov, P., Kocev, D., Todorovski, L. (eds) Discovery Science. DS 2014. Lecture Notes in Computer Science(), vol 8777. Springer, Cham. https://doi.org/10.1007/978-3-319-11812-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-11812-3_30
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11811-6
Online ISBN: 978-3-319-11812-3
eBook Packages: Computer ScienceComputer Science (R0)