Skip to main content
SpringerLink
Log in

Closed-Loop Control with Evolving Gaussian Process Models

  • Chapter
  • First Online:
Complex Systems

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 55))

Abstract

This contribution presents a new development in the design of control system based on evolving Gaussian process (GP) models . GP models provide a probabilistic, nonparametric modelling approach for black-box identification of nonlinear dynamic systems. They can highlight areas of the input space where prediction quality is poor, due to the lack of data or its complexity, by indicating the higher variance around the predicted mean. GP models contain noticeably less coefficients to be optimised than commonly used parametric models. While GP models are Bayesian models, their output is normal distribution, expressed in terms of mean and variance. Latter can be interpreted as a confidence in prediction and used in many fields, especially in control system. Evolving GP model is the concept approach within which various ways of model adaptations can be used. Successful control system needs as much as possible data about process to be controlled . If the prior knowledge about the system to be controlled is scarce or the system varies with time or operating region, this control problem can be solved with an iterative method which adapts model with information obtained with streaming data and concurrently optimises hyperparameter values. This contribution provides: a survey of adaptive control algorithms for dynamic systems described in publications where GP models have been used for control design, a novel and improved closed-loop controller with evolving GP models and an example for the illustration of proposed control algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    It should be noted that for calculation of the negative log marginal likelihood the inverse of the covariance matrix is needed. In our case the inverse of the covariance matrix should be calculated for every subset. But, this computational demanding task can be speeded up by calculating the Cholesky decomposition of the exceeded active set of length n only once and then downdates it for every data in it by using low-rank updates [33].

References

  1. Ažman, K., Kocijan, J.: Application of Gaussian processes for black-box modelling of biosystems. ISA Trans. 46, 443–457 (2007)

    Article  Google Scholar 

  2. Ažman, K., Kocijan, J.: Fixed-structure Gaussian process model. Int. J. Syst. Sci. 40(12), 1253–1262 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Cornford, D., Csato, L., Opper, M.: Sequential, sparse learning in Gaussian processes. In: Proceedings of the 7th International Conference on GeoComputation, vol. 44. Southampton, UK (2003)

    Google Scholar 

  4. Csató, L., Opper, M.: Sparse online Gaussian processes. Neural Comput. 14(3), 641–668 (2002)

    Article  MATH  Google Scholar 

  5. Deisenroth, M.P.: Efficient Reinforcement Learning using Gaussian Processes. Ph.D. thesis, Karlsruhe Institute of Technology, Karlsruhe (2010)

    Google Scholar 

  6. Deisenroth, M.P., Rasmussen, C.E.: PILCO: a model-based and data-efficient approach to policy search. In: Proceedings of the 28th International Conference on Machine Learning (ICML 2011). Bellevue, WA (2011)

    Google Scholar 

  7. Deisenroth, M.P., Rasmussen, C.E., Fox, D.: Learning to control a low-cost manipulator using data-efficient reinforcement learning. In: Proceedings of the International Conference on Robotics: Science & Systems (R:SS 2011). Los Angeles, CA (2011)

    Google Scholar 

  8. Deisenroth, M.P., Rasmussen, C.E., Peters, J.: Gaussian process dynamic programming. Neurocomputing 72(7–9), 1508–1524 (2009)

    Article  Google Scholar 

  9. Deisenroth, M., Peters, J., Rasmussen, C.: Approximate dynamic programming with Gaussian processes. In: Proceedings of American Control Conference (ACC), pp. 4480–4485. Seattle, WA (2008)

    Google Scholar 

  10. Deisenroth, M., Rasmussen, C.: Bayesian inference for efficient learning in control. In: Proceedings of Multidisciplinary Symposium on Reinforcement Learning (MSRL). Montreal, Canada (2009)

    Google Scholar 

  11. Filatov, N., Unbehauen, H.: Survey of adaptive dual control methods. IEE Proc.— Control Theory Appl. 147(1), 119–128 (2000)

    Google Scholar 

  12. Isermann, R., Lachman, K.H., Matko, D.: Adaptive Control Systems. Systems and Control Engineering. Prentice Hall International, New York (1992)

    Google Scholar 

  13. Kocijan, J.: Control algorithms based on Gaussian process models: a state-of-the-art survey. In: Kolemisevska-Gugulovska, T.D., Stankovski, M.J. (eds.) Special International Conference on Complex systems: Synergy of Control, Communications and Computing—Proceedings of COSY 2011 Papers, September 16–20, 2011, Ohrid, Macedonia. The Society for Electronics, Telecommunications, Automation, and Informatics of Macedonia, pp. 69–80. Skopje, Macedonia, Sept 2011

    Google Scholar 

  14. Kocijan, J.: Dynamic GP models: an overview and recent developments. In: Recent Researches in Applied Mathematics and Economics: proceedings of the 6th International Conference on Applied Mathematics. Simulation, Modelling, (ASM’12), pp. 38–43. Vougliameni, Greece (2012)

    Google Scholar 

  15. Kocijan, J.: Modelling and Control of Dynamic Systems Using Gaussian Process Models. Springer International Publishing, Cham (2016)

    Google Scholar 

  16. Kocijan, J., Girard, A., Banko, B., Murray-Smith, R.: Dynamic systems identification with Gaussian processes. Math. Comput. Model. Dyn. Syst. 11(4), 411–424 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  17. Kocijan, J., Likar, B.: Gas-liquid separator modelling and simulation with Gaussian-process models. Simul. Model. Pract. Theory 16(8), 910–922 (2008)

    Article  Google Scholar 

  18. Lázaro-Gredilla, M., Quiñonero Candela, J., Rasmussen, C.E., Figueiras-Vidal, A.R.: Sparse spectrum Gaussian process regression. J. Mach. Learn. Res. 11, 1865–1881 (2010)

    MathSciNet  MATH  Google Scholar 

  19. Murray-Smith, R., Sbarbaro, D., Rasmussen, C., Girard, A.: Adaptive, cautious, predictive control with Gaussian process priors. In: Proceedings of 13th IFAC Symposium on System Identification. Rotterdam, Netherlands (2003)

    Google Scholar 

  20. Murray-Smith, R., Sbarbaro, D.: Nonlinear adaptive control using nonparametric Gaussian process prior models. In: Proceedings of IFAC 15th World Congress. Barcelona (2002)

    Google Scholar 

  21. Narendra, K., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Networks 1(1), 4–27 (1990)

    Article  Google Scholar 

  22. Nguyen-Tuong, D., Peters, J.: Learning robot dynamics for computed torque control using local Gaussian processes regression. In: Symposium on Learning and Adaptive Behaviors for Robotic Systems, pp. 59–64 (2008)

    Google Scholar 

  23. Nguyen-Tuong, D., Seeger, M., Peters, J.: Real-time local GP model learning, chap. From Motor Learning to Interaction Learning in Robots, vol. 264, pp. 193–207. Springer (2010)

    Google Scholar 

  24. Petelin, D., Grancharova, A., Kocijan, J.: Evolving Gaussian process models for prediction of ozone concentration in the air. Simul. Model. Pract. Theory 33, 68–80 (2013)

    Article  Google Scholar 

  25. Petelin, D., Kocijan, J.: Control system with evolving Gaussian process model. In: Proceedings of IEEE Symposium Series on Computational Intelligence, SSCI 2011. IEEE, Paris (2011)

    Google Scholar 

  26. Quinonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6, 1939–1959 (2005)

    MathSciNet  MATH  Google Scholar 

  27. Rasmussen, C.E., Deisenroth, M.P.: Probabilistic inference for fast learning in control. In: Recent Advances in Reinforcement Learning, Lecture Notes on Computer Science, vol. 5323, pp. 229–242. Springer (2008)

    Google Scholar 

  28. Rasmussen, C.E., Kuss, M.: Gaussian processes in reinforcement learning. In: Thurn, S., Saul, L., Schoelkopf, B. (eds.) Advances in Neural Information Processing Systems conference. vol. 16, pp. 751–759. MIT Press (2004)

    Google Scholar 

  29. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  30. Sbarbaro, D., Murray-smith, R.: An adaptive nonparametric controller for a class of nonminimum phase non-linear system. In: Proceedings of IFAC 16th World Congress. Prague, Czech Republic (2005)

    Google Scholar 

  31. Sbarbaro, D., Murray-Smith, R., Valdes, A.: Multivariable generalized minimum variance control based on artificial neural networks and Gaussian process models. In: International Symposium on Neural Networks. Springer (2004)

    Google Scholar 

  32. Sbarbaro, D., Murray-Smith, R.: Self-tuning control of nonlinear systems using Gaussian process prior models. In: Murray-Smith, R., Shorten, R. (eds.) Switching and Learning in Feedback Systems. Lecture Notes in Computer Science, vol. 3355, pp. 140–157. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  33. Seeger, M.: Low Rank Updates for the Cholesky Decomposition. University of California at Berkeley, Technical report (2008)

    Google Scholar 

  34. Seeger, M., Williams, C.K.I., Lawrence, N.D.: Fast forward selection to speed up sparse gaussian process regression. In: Ninth International Workshop on Artificial Intelligence and Statistics. Society for Artificial Intelligence and Statistics (2003)

    Google Scholar 

  35. Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: Neural Information Processing Systems (2005)

    Google Scholar 

  36. Wittenmark, B.: Adaptive dual control. In: Control Systems, Robotics and Automation, Encyclopedia of Life Support Systems (EOLSS), Developed under the auspices of the UNESCO. Eolss Publishers, Oxford, UK, Jan 2002

    Google Scholar 

Download references

Acknowledgments

This work has been supported by the Slovenian Research Agency, grant No. P2-0001

Author information

Authors and Affiliations

  1. Jožef Stefan Institute, Jamova 39, 1000, Ljubljana, Slovenia

    Juš Kocijan & Dejan Petelin

  2. University of Nova Gorica, Vipavska 13, 5000, Nova Gorica, Slovenia

    Juš Kocijan

Authors
  1. Juš Kocijan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dejan Petelin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juš Kocijan .

Editor information

Editors and Affiliations

  1. School of Electrical Engineering an, St. Cyril and St. Methodius Univers, Skopje, Macedonia

    Georgi M. Dimirovski

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kocijan, J., Petelin, D. (2016). Closed-Loop Control with Evolving Gaussian Process Models. In: Dimirovski, G. (eds) Complex Systems. Studies in Systems, Decision and Control, vol 55. Springer, Cham. https://doi.org/10.1007/978-3-319-28860-4_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28860-4_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28858-1

  • Online ISBN: 978-3-319-28860-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation