Advertisement

Variational Dependent Multi-output Gaussian Process Dynamical Systems

  • Jing Zhao
  • Shiliang Sun
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8777)

Abstract

This paper presents a dependent multi-output Gaussian process (GP) for modeling complex dynamical systems. The outputs are dependent in this model, which is largely different from previous GP dynamical systems. We adopt convolved multi-output GPs to model the outputs, which are provided with a flexible multi-output covariance function. We adapt the variational inference method with inducing points for approximate posterior inference of latent variables. Conjugate gradient based optimization is used to solve parameters involved. Besides the temporal dependency, the proposed model also captures the dependency among outputs in complex dynamical systems. We evaluate the model on both synthetic and real-world data, and encouraging results are observed.

Keywords

Gaussian process variational inference dynamical system multi-output modeling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Álvarez, M.A., Luengo, D., Lawrence, N.D.: Linear latent force models using Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 2693–2705 (2013)CrossRefGoogle Scholar
  2. 2.
    Álvarez, M.A., Lawrence, N.D.: Computationally efficient convolved multiple output Gaussian processes. Journal of Machine Learning Research 12, 1459–1500 (2011)zbMATHGoogle Scholar
  3. 3.
    Álvarez, M.A., Luengo, D., Lawrence, N.D.: Latent force models. In: Proceedings of the 12th International Conference on Articicial Intelligence and Statistics, pp. 9–16 (2009)Google Scholar
  4. 4.
    Bonilla, E.V., Chai, K.M., Williams, C.K.I.: Multi-task Gaussian process prediction. In: Advances in Neural Information Processing Systems, vol. 18, pp. 153–160 (2008)Google Scholar
  5. 5.
    Damianou, A.C., Ek, C.H., Titsias, M.K., Lawrence, N.D.: Manifold relevance determination. In: Proceedings of the 29th International Conference on Machine Learning, pp. 145–152 (2012)Google Scholar
  6. 6.
    Damianou, A.C., Titsias, M.K., Lawrence, N.D.: Variational Gaussian process dynamical systems. In: Advances in Neural Information Processing Systems, vol. 24, pp. 2510–2518 (2011)Google Scholar
  7. 7.
    Deisenroth, M.P., Mohamed, S.: Expectation propagation in Gaussian process dynamical systems. In: Advances in Neural Information Processing Systems, vol. 25, pp. 2618–2626 (2012)Google Scholar
  8. 8.
    Hartikainen, J., Särkkä, S.: Sequential inference for latent force models (2012), http://arxiv.org/abs/1202.3730
  9. 9.
    Lawrence, N.D.: Gaussian process latent variable models for visualisation of high dimensional data. In: Advances in Neural Information Processing Systems, vol. 17, pp. 329–336 (2004)Google Scholar
  10. 10.
    Lawrence, N.D.: Probabilistic non-linear principal component analysis with Gaussian process latent variable models. Journal of Machine Learning Research 6, 1783–1816 (2005)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Lawrence, N.D.: Learning for larger dataset with the Gaussian process latent variable model. In: Proceedings of the 11th International Workshop on Artificial Intelligence and Statistics, pp. 243–250 (2007)Google Scholar
  12. 12.
    Luttinen, J., Ilin, A.: Efficient Gaussian process inference for short-scale spatio-temporal modeling. In: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics, pp. 741–750 (2012)Google Scholar
  13. 13.
    Opper, M., Archambeau, A.: The variational Gaussian approximation revisited. Neural Computation 21, 786–792 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Park, H., Yun, S., Park, S., Kim, J., Yoo, C.D.: Phoneme classification using constrained variational Gaussian process dynamical system. In: Advances in Neural Information Processing Systems, vol. 22, pp. 2015–2023 (2012)Google Scholar
  15. 15.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Process for Machine Learning. MIT Press (2006)Google Scholar
  16. 16.
    Sun, S.: A review of deterministic approximate inference techniques for Bayesian machine learning. Neural Computing and Applications 23, 2039–2050 (2013)CrossRefGoogle Scholar
  17. 17.
    Taylor, G.W., Hinton, G.E., Roweis, S.: Modeling human motion using binary latent variables. In: Advances in Neural Information Processing Systems, vol. 17, pp. 1345–1352 (2007)Google Scholar
  18. 18.
    Tipping, M.E., Bishop, C.M.: Probabilistic principal component analysis. Journal of the Royal Statistical Society 61, 611–622 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Titsias, M.K.: Variational learning of inducing variables in sparse Gaussian processes. In: Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, pp. 567–574 (2009)Google Scholar
  20. 20.
    Titsias, M.K., Lawrence, N.D.: Bayesian Gaussian process latent variable model. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, pp. 844–851 (2010)Google Scholar
  21. 21.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models. In: Advances in Neural Information Processing Systems, vol. 19, pp. 1441–1448 (2006)Google Scholar
  22. 22.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 283–398 (2008)CrossRefGoogle Scholar
  23. 23.
    Wilson, A.G., Knowles, D.A., Ghahramani, Z.: Gaussian process regression networks. In: Proceedings of the 29th International Conference on Machine Learning, pp. 599–606 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Jing Zhao
    • 1
  • Shiliang Sun
    • 1
  1. 1.Department of Computer Science and TechnologyEast China Normal UniversityShanghaiP.R. China

Personalised recommendations