Skip to main content
SpringerLink
Log in

Lazy Fully Probabilistic Design of Decision Strategies

  • Conference paper
  • First Online:
Advances in Neural Networks – ISNN 2014 (ISNN 2014)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8866))

Included in the following conference series:

Abstract

Fully probabilistic design of decision strategies (FPD) extends Bayesian dynamic decision making. The FPD specifies the decision aim via so-called ideal - a probability density, which assigns high probability values to the desirable behaviours and low values to undesirable ones. The optimal decision strategy minimises the Kullback-Leibler divergence of the probability density describing the closed-loop behaviour to this ideal. In spite of the availability of explicit minimisers in the corresponding dynamic programming, it suffers from the curse of dimensionality connected with complexity of the value function. Recently proposed a lazy FPD tailors lazy learning, which builds a local model around the current behaviour, to estimation of the closed-loop model with the optimal strategy. This paper adds a theoretical support to the lazy FPD and outlines its further improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bellman, R.: Adaptive Control Processes. Princeton U. Press, NJ (1961)

    MATH  Google Scholar 

  2. Berec, L., Kárný, M.: Identification of reality in Bayesian context. In: Warwick, K., Kárný, M. (eds.) Computer-Intensive Methods in Control and Signal Processing, pp. 181–193. Birkhäuser (1997)

    Google Scholar 

  3. Berger, J.: Statistical Decision Theory and Bayesian Analysis. Springer, New York (1985)

    Book  MATH  Google Scholar 

  4. Bertsekas, D.: Dynamic Programming and Optimal Control. Athena Scientific, US (2001)

    MATH  Google Scholar 

  5. Bontempi, G., Birattari, M., Bersini, H.: Lazy learning for local modelling & control design. Int. J. of Control 72(7–8), 643–658 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  6. Cappe, O., Godsill, S., Moulines, E.: An overview of existing methods and recent advances in sequential monte carlo. Proc. of the IEEE 95(5), 899–924 (2007)

    Article  Google Scholar 

  7. Daum, F.: Nonlinear filters: beyond the kalman filter. IEEE Aerospace and Electronic Systems Magazine 20(8), 57–69 (2005)

    Article  Google Scholar 

  8. Doucet, A., Johansen, A.: A tutorial on particle filtering and smoothing: Fifteen years later. In: Handbook of Nonlinear Filtering. Oxford University Press, Oxford (2011)

    Google Scholar 

  9. Feldbaum, A.: Theory of dual control. Autom. Remote Control 21(9) (1960)

    Google Scholar 

  10. Gilboa, I., Schmeidler, D.: Case-based decsion theory. The Quaterly Journal of Economics 110, 605–639 (1995)

    Article  MATH  Google Scholar 

  11. Guan, P., Raginsky, M., Willett, R.: Online Markov decision processes with Kull-back Leibler control cost. IEEE Trans. on Automatic, Control (2014)

    Google Scholar 

  12. Kárný, M.: Towards fully probabilistic control design. Automatica 32(12), 1719–1722 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  13. Kárný, M.: Adaptive systems: Local approximators? In: Workshop n Adaptive Systems in Control and Signal Processing, pp. 129–134. IFAC, Glasgow (1998)

    Google Scholar 

  14. Kárný, M.: On approximate fully probabilistic design of decision making strategies. In: Guy, T., Kárný, M. (eds.) Proceedings of the 3rd International Workshop on Scalable Decision Making, ECML/PKDD 2013. UTIA AV ČR, Prague (2013) iSBN 978-80-903834-8-7

    Google Scholar 

  15. Kárný, M.: Approximate bayesian recursive estimation. Information Sciences (2014), doi: 10.1016/j.ins.2014.01.048

    Google Scholar 

  16. Kárný, M., Guy, T.V.: Fully probabilistic control design. Systems & Control Letters 55(4), 259–265 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  17. Kárný, M., Kroupa, T.: Axiomatisation of fully probabilistic design. Information Sciences 186(1), 105–113 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  18. Kulhavý, R., Zarrop, M.B.: On a general concept of forgetting. Int. J. of Control 58(4), 905–924 (1993)

    Article  MATH  Google Scholar 

  19. Kullback, S., Leibler, R.: On information and sufficiency. Annals of Mathematical Statistics 22, 79–87 (1951)

    Article  MATH  MathSciNet  Google Scholar 

  20. Li, J., Dong, G., Ramamohanarao, K., Wong, L.: Deeps: a new instance-based lazy discovery and classification system. Machine Learning 54(2), 99–124 (2004)

    Article  MATH  Google Scholar 

  21. Loeve, M.: Probability Theory. van Nostrand, Princeton, New Jersey (1962) (Russian translation, Moscow 1962)

    Google Scholar 

  22. Macek, K., Guy, T., Kárný, M.: A lazy-learning concept of fully probabilistic decision making (2014) (unpublished manuscript)

    Google Scholar 

  23. Martín-Sánchez, J., Lemos, J., Rodellar, J.: Survey of industrial optimized adaptive control. Int. J. of Adaptive Control and Signal Processing 26(10), 881–918 (2013).

    Google Scholar 

  24. Peterka, V.: Bayesian system identification. In: Eykhoff, P. (ed.) Trends and Progress in System Identification, pp. 239–304. Pergamon Press, Oxford (1981)

    Chapter  Google Scholar 

  25. Qin, S., Badgwell, T.: A survey of industrial model predictive control technology. Control Engineering Practice 11(7), 733–764 (2003)

    Article  Google Scholar 

  26. Rao, M.: Measure Theory and Integration. John Wiley, NY (1987)

    MATH  Google Scholar 

  27. Roll, J., Nazin, A., Ljung, L.: Nonlinear system identification via direct weight optimization. Automatica 41(3), 475–490 (2004)

    Article  MathSciNet  Google Scholar 

  28. Sanov, I.: On probability of large deviations of random variables. Matematičeskij Sbornik 42, 11–44 (in russian), also in selected translations mathematical statistics and probability. I 1961, 213–244 (1957)

    Google Scholar 

  29. Savage, L.: Foundations of Statistics. Wiley, NY (1954)

    MATH  Google Scholar 

  30. Schon, T., Gustafsson, F., Nordlund, P.: Marginalized particle filters for mixed linear/nonlinear state-space models. IEEE Tran. on Signal Processing 53(7), 2279–2289 (2005)

    Article  MathSciNet  Google Scholar 

  31. Si, J., Barto, A., Powell, W., Wunsch, D. (eds.): Handbook of Learning and Approximate Dynamic Programming. Wiley-IEEE Press, Danvers (2004)

    Google Scholar 

  32. Tishby, N., Polani, D.: Information theory of decisions and actions. In: Cutsuridis, V., Hussain, A., Taylor, J. (eds.) Perception-Action Cycle. Springer Series in Cognitive and Neural Systems, pp. 601–636. Springer, New York (2011)

    Chapter  Google Scholar 

  33. Todorov, E.: Linearly-solvable Markov decision problems. In: Schölkopf, B., et al. (eds.) Advances in Neural Inf. Processing, pp. 1369–1376. MIT Press, NY (2006)

    Google Scholar 

  34. Zhu, C., Zhu, W.: Feedback control of nonlinear stochastic systems for targeting a specified stationary probability density. Automatica 47(3), 539–544 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Information Theory and Automation, Academy of Sciences of the Czech Republic, P.O.Box 18, 182 08, Prague 8, Czech Republic

    Miroslav Kárný, Karel Macek & Tatiana V. Guy

Authors
  1. Miroslav Kárný
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karel Macek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tatiana V. Guy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miroslav Kárný .

Editor information

Editors and Affiliations

  1. Wuhan, China

    Zhigang Zeng

  2. University of Macau, Macau, Macao

    Yangmin Li

  3. The Chinese University of Hong Kong, Hong Kong, Hong Kong, Hong Kong SAR

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Kárný, M., Macek, K., Guy, T.V. (2014). Lazy Fully Probabilistic Design of Decision Strategies. In: Zeng, Z., Li, Y., King, I. (eds) Advances in Neural Networks – ISNN 2014. ISNN 2014. Lecture Notes in Computer Science(), vol 8866. Springer, Cham. https://doi.org/10.1007/978-3-319-12436-0_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12436-0_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-12435-3

  • Online ISBN: 978-3-319-12436-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation