Learning Non-stationary System Dynamics Online Using Gaussian Processes

  • Axel Rottmann
  • Wolfram Burgard
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6376)


Gaussian processes are a powerful non-parametric framework for solving various regression problems. In this paper, we address the task of learning a Gaussian process model of non-stationary system dynamics in an online fashion. We propose an extension to previous models that can appropriately handle outdated training samples by decreasing their influence onto the predictive distribution. The resulting model estimates for each sample of the training set an individual noise level and thereby produces a mean shift towards more reliable observations. As a result, our model improves the prediction accuracy in the context of non-stationary function approximation and can furthermore detect outliers based on the resulting noise level. Our approach is easy to implement and is based upon standard Gaussian process techniques. In a real-world application where the task is to learn the system dynamics of a miniature blimp, we demonstrate that our algorithm benefits from individual noise levels and outperforms standard methods.


Prediction Accuracy Training Sample Gaussian Process Predictive Distribution Observation Noise 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rasmussen, C.E., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  2. 2.
    Goldberg, P., Williams, C., Bishop, C.: Regression with input-dependent noise: A Gaussian process treatment. In: Proc. of the Advances in Neural Information Processing Systems, vol. 10, pp. 493–499. MIT Press, Cambridge (1998)Google Scholar
  3. 3.
    Kersting, K., Plagemann, C., Pfaff, P., Burgard, W.: Most likely heteroscedastic Gaussian process regression. In: Proc. of the Int. Conf. on Machine Learning, pp. 393–400 (2007)Google Scholar
  4. 4.
    Schölkopf, B., Smola, A., Williamson, R., Bartlett, P.: New support vector algorithms. Neural Computation 12, 1207–1245 (2000)CrossRefGoogle Scholar
  5. 5.
    Bishop, C., Quazaz, C.: Regression with input-dependent noise: A bayesian treatment. In: Proc. of the Advances in Neural Information Processing Systems, vol. 9, pp. 347–353. MIT Press, Cambridge (1997)Google Scholar
  6. 6.
    Ko, J., Klein, D., Fox, D., Hähnel, D.: Gaussian processes and reinforcement learning for identification and control of an autonomous blimp. In: Proc. of the IEEE Int. Conf. on Robotics & Automation, pp. 742–747 (2007)Google Scholar
  7. 7.
    Rottmann, A., Plagemann, C., Hilgers, P., Burgard, W.: Autonomous blimp control using model-free reinforcement learning in a continuous state and action space. In: Proc. of the IEEE Int. Conf. on Intelligent Robots and Systems, pp. 1895–1900 (2007)Google Scholar
  8. 8.
    Deisenroth, M., Rasmussen, C., Peters, J.: Gaussian process dynamic programming. Neurocomputing 72(7-9), 1508–1524 (2009)CrossRefGoogle Scholar
  9. 9.
    D’Souza, A., Vijayakumar, S., Schaal, S.: Learning inverse kinematics. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 298–303 (2001)Google Scholar
  10. 10.
    Sundararajan, S., Keerthi, S.: Predictive approaches for choosing hyperparameters in Gaussian processes. Neural Computation 13(5), 1103–1118 (2001)zbMATHCrossRefGoogle Scholar
  11. 11.
    Sugiyama, M., Krauledat, M., Müller, K.R.: Covariate shift adaptation by importance weighted cross validation. J. Mach. Learn. Res. 8, 985–1005 (2007)Google Scholar
  12. 12.
    Rottmann, A., Sippel, M., Zitterell, T., Burgard, W., Reindl, L., Scholl, C.: Towards an experimental autonomous blimp platform. In: Proc. of the 3rd Europ. Conf. on Mobile Robots, pp. 19–24 (2007)Google Scholar
  13. 13.
    Zufferey, J., Guanella, A., Beyeler, A., Floreano, D.: Flying over the reality gap: From simulated to real indoor airships. Autonomous Robots 21(3), 243–254 (2006)CrossRefGoogle Scholar
  14. 14.
    Yuan, M., Wahba, G.: Doubly penalized likelihood estimator in heteroscedastic regression. Statistics and Probability Letter 69(1), 11–20 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Silverman, B.: Some aspects of the spline smoothing approach to non-parametric regression curve fitting. Journal of the Royal Statistical Society 47(1), 1–52 (1985)zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Axel Rottmann
    • 1
  • Wolfram Burgard
    • 1
  1. 1.Department of Computer ScienceUniversity of FreiburgGermany

Personalised recommendations