Closed-Loop Control with Evolving Gaussian Process Models

  • Juš Kocijan
  • Dejan Petelin
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 55)


This contribution presents a new development in the design of control system based on evolving Gaussian process (GP) models . GP models provide a probabilistic, nonparametric modelling approach for black-box identification of nonlinear dynamic systems. They can highlight areas of the input space where prediction quality is poor, due to the lack of data or its complexity, by indicating the higher variance around the predicted mean. GP models contain noticeably less coefficients to be optimised than commonly used parametric models. While GP models are Bayesian models, their output is normal distribution, expressed in terms of mean and variance. Latter can be interpreted as a confidence in prediction and used in many fields, especially in control system. Evolving GP model is the concept approach within which various ways of model adaptations can be used. Successful control system needs as much as possible data about process to be controlled . If the prior knowledge about the system to be controlled is scarce or the system varies with time or operating region, this control problem can be solved with an iterative method which adapts model with information obtained with streaming data and concurrently optimises hyperparameter values. This contribution provides: a survey of adaptive control algorithms for dynamic systems described in publications where GP models have been used for control design, a novel and improved closed-loop controller with evolving GP models and an example for the illustration of proposed control algorithm.


Dynamic systems modelling Gaussian process regression Evolving Gaussian process model Adaptive control 



This work has been supported by the Slovenian Research Agency, grant No. P2-0001


  1. 1.
    Ažman, K., Kocijan, J.: Application of Gaussian processes for black-box modelling of biosystems. ISA Trans. 46, 443–457 (2007)CrossRefGoogle Scholar
  2. 2.
    Ažman, K., Kocijan, J.: Fixed-structure Gaussian process model. Int. J. Syst. Sci. 40(12), 1253–1262 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Cornford, D., Csato, L., Opper, M.: Sequential, sparse learning in Gaussian processes. In: Proceedings of the 7th International Conference on GeoComputation, vol. 44. Southampton, UK (2003)Google Scholar
  4. 4.
    Csató, L., Opper, M.: Sparse online Gaussian processes. Neural Comput. 14(3), 641–668 (2002)CrossRefzbMATHGoogle Scholar
  5. 5.
    Deisenroth, M.P.: Efficient Reinforcement Learning using Gaussian Processes. Ph.D. thesis, Karlsruhe Institute of Technology, Karlsruhe (2010)Google Scholar
  6. 6.
    Deisenroth, M.P., Rasmussen, C.E.: PILCO: a model-based and data-efficient approach to policy search. In: Proceedings of the 28th International Conference on Machine Learning (ICML 2011). Bellevue, WA (2011)Google Scholar
  7. 7.
    Deisenroth, M.P., Rasmussen, C.E., Fox, D.: Learning to control a low-cost manipulator using data-efficient reinforcement learning. In: Proceedings of the International Conference on Robotics: Science & Systems (R:SS 2011). Los Angeles, CA (2011)Google Scholar
  8. 8.
    Deisenroth, M.P., Rasmussen, C.E., Peters, J.: Gaussian process dynamic programming. Neurocomputing 72(7–9), 1508–1524 (2009)CrossRefGoogle Scholar
  9. 9.
    Deisenroth, M., Peters, J., Rasmussen, C.: Approximate dynamic programming with Gaussian processes. In: Proceedings of American Control Conference (ACC), pp. 4480–4485. Seattle, WA (2008)Google Scholar
  10. 10.
    Deisenroth, M., Rasmussen, C.: Bayesian inference for efficient learning in control. In: Proceedings of Multidisciplinary Symposium on Reinforcement Learning (MSRL). Montreal, Canada (2009)Google Scholar
  11. 11.
    Filatov, N., Unbehauen, H.: Survey of adaptive dual control methods. IEE Proc.— Control Theory Appl. 147(1), 119–128 (2000)Google Scholar
  12. 12.
    Isermann, R., Lachman, K.H., Matko, D.: Adaptive Control Systems. Systems and Control Engineering. Prentice Hall International, New York (1992)Google Scholar
  13. 13.
    Kocijan, J.: Control algorithms based on Gaussian process models: a state-of-the-art survey. In: Kolemisevska-Gugulovska, T.D., Stankovski, M.J. (eds.) Special International Conference on Complex systems: Synergy of Control, Communications and Computing—Proceedings of COSY 2011 Papers, September 16–20, 2011, Ohrid, Macedonia. The Society for Electronics, Telecommunications, Automation, and Informatics of Macedonia, pp. 69–80. Skopje, Macedonia, Sept 2011Google Scholar
  14. 14.
    Kocijan, J.: Dynamic GP models: an overview and recent developments. In: Recent Researches in Applied Mathematics and Economics: proceedings of the 6th International Conference on Applied Mathematics. Simulation, Modelling, (ASM’12), pp. 38–43. Vougliameni, Greece (2012)Google Scholar
  15. 15.
    Kocijan, J.: Modelling and Control of Dynamic Systems Using Gaussian Process Models. Springer International Publishing, Cham (2016)Google Scholar
  16. 16.
    Kocijan, J., Girard, A., Banko, B., Murray-Smith, R.: Dynamic systems identification with Gaussian processes. Math. Comput. Model. Dyn. Syst. 11(4), 411–424 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Kocijan, J., Likar, B.: Gas-liquid separator modelling and simulation with Gaussian-process models. Simul. Model. Pract. Theory 16(8), 910–922 (2008)CrossRefGoogle Scholar
  18. 18.
    Lázaro-Gredilla, M., Quiñonero Candela, J., Rasmussen, C.E., Figueiras-Vidal, A.R.: Sparse spectrum Gaussian process regression. J. Mach. Learn. Res. 11, 1865–1881 (2010)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Murray-Smith, R., Sbarbaro, D., Rasmussen, C., Girard, A.: Adaptive, cautious, predictive control with Gaussian process priors. In: Proceedings of 13th IFAC Symposium on System Identification. Rotterdam, Netherlands (2003)Google Scholar
  20. 20.
    Murray-Smith, R., Sbarbaro, D.: Nonlinear adaptive control using nonparametric Gaussian process prior models. In: Proceedings of IFAC 15th World Congress. Barcelona (2002)Google Scholar
  21. 21.
    Narendra, K., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Networks 1(1), 4–27 (1990)CrossRefGoogle Scholar
  22. 22.
    Nguyen-Tuong, D., Peters, J.: Learning robot dynamics for computed torque control using local Gaussian processes regression. In: Symposium on Learning and Adaptive Behaviors for Robotic Systems, pp. 59–64 (2008)Google Scholar
  23. 23.
    Nguyen-Tuong, D., Seeger, M., Peters, J.: Real-time local GP model learning, chap. From Motor Learning to Interaction Learning in Robots, vol. 264, pp. 193–207. Springer (2010)Google Scholar
  24. 24.
    Petelin, D., Grancharova, A., Kocijan, J.: Evolving Gaussian process models for prediction of ozone concentration in the air. Simul. Model. Pract. Theory 33, 68–80 (2013)CrossRefGoogle Scholar
  25. 25.
    Petelin, D., Kocijan, J.: Control system with evolving Gaussian process model. In: Proceedings of IEEE Symposium Series on Computational Intelligence, SSCI 2011. IEEE, Paris (2011)Google Scholar
  26. 26.
    Quinonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6, 1939–1959 (2005)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Rasmussen, C.E., Deisenroth, M.P.: Probabilistic inference for fast learning in control. In: Recent Advances in Reinforcement Learning, Lecture Notes on Computer Science, vol. 5323, pp. 229–242. Springer (2008)Google Scholar
  28. 28.
    Rasmussen, C.E., Kuss, M.: Gaussian processes in reinforcement learning. In: Thurn, S., Saul, L., Schoelkopf, B. (eds.) Advances in Neural Information Processing Systems conference. vol. 16, pp. 751–759. MIT Press (2004)Google Scholar
  29. 29.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  30. 30.
    Sbarbaro, D., Murray-smith, R.: An adaptive nonparametric controller for a class of nonminimum phase non-linear system. In: Proceedings of IFAC 16th World Congress. Prague, Czech Republic (2005)Google Scholar
  31. 31.
    Sbarbaro, D., Murray-Smith, R., Valdes, A.: Multivariable generalized minimum variance control based on artificial neural networks and Gaussian process models. In: International Symposium on Neural Networks. Springer (2004)Google Scholar
  32. 32.
    Sbarbaro, D., Murray-Smith, R.: Self-tuning control of nonlinear systems using Gaussian process prior models. In: Murray-Smith, R., Shorten, R. (eds.) Switching and Learning in Feedback Systems. Lecture Notes in Computer Science, vol. 3355, pp. 140–157. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  33. 33.
    Seeger, M.: Low Rank Updates for the Cholesky Decomposition. University of California at Berkeley, Technical report (2008)Google Scholar
  34. 34.
    Seeger, M., Williams, C.K.I., Lawrence, N.D.: Fast forward selection to speed up sparse gaussian process regression. In: Ninth International Workshop on Artificial Intelligence and Statistics. Society for Artificial Intelligence and Statistics (2003)Google Scholar
  35. 35.
    Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: Neural Information Processing Systems (2005)Google Scholar
  36. 36.
    Wittenmark, B.: Adaptive dual control. In: Control Systems, Robotics and Automation, Encyclopedia of Life Support Systems (EOLSS), Developed under the auspices of the UNESCO. Eolss Publishers, Oxford, UK, Jan 2002Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Jožef Stefan InstituteLjubljanaSlovenia
  2. 2.University of Nova GoricaNova GoricaSlovenia

Personalised recommendations