Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments

  • Yi Sun
  • Faustino Gomez
  • Jürgen Schmidhuber
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6830)


To maximize its success, an AGI typically needs to explore its initially unknown world. Is there an optimal way of doing so? Here we derive an affirmative answer for a broad class of environments.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chaloner, K., Verdinelli, I.: Bayesian experimental design: A review. Statistical Science 10, 273–304 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Fedorov, V.V.: Theory of optimal experiments. Academic Press, London (1972)Google Scholar
  3. 3.
    Itti, L., Baldi, P.F.: Bayesian surprise attracts human attention. In: NIPS 2005, pp. 547–554 (2006)Google Scholar
  4. 4.
    Lindley, D.V.: On a measure of the information provided by an experiment. Annals of Mathematical Statistics 27(4), 986–1005 (1956)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Penny, W.: Kullback-liebler divergences of normal, gamma, dirichlet and wishart densities. Tech. rep., Wellcome Department of Cognitive Neurology, University College London (2001)Google Scholar
  6. 6.
    Schmidhuber, J.: Curious model-building control systems. In: IJCNN 1991, vol. 2, pp. 1458–1463 (1991)Google Scholar
  7. 7.
    Schmidhuber, J.: Formal theory of creativity, fun, and intrinsic motivation (1990-2010). Autonomous Mental Development, IEEE Trans. on Autonomous Mental Development 2(3), 230–247 (2010)CrossRefGoogle Scholar
  8. 8.
    Settles, B.: Active learning literature survey. Tech. rep., University of Wisconsin Madison (2010)Google Scholar
  9. 9.
    Singh, S., Barto, A., Chentanez, N.: Intrinsically motivated reinforcement learning. In: NIPS 2004 (2004)Google Scholar
  10. 10.
    Storck, J., Hochreiter, S., Schmidhuber, J.: Reinforcement driven information acquisition in non-deterministic environments. In: ICANN 1995 (1995)Google Scholar
  11. 11.
    Sun, Y., Gomez, F.J., Schmidhuber, J.: Planning to be surprised: Optimal bayesian exploration in dynamic environments (2011),

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Yi Sun
    • 1
  • Faustino Gomez
    • 1
  • Jürgen Schmidhuber
    • 1
  1. 1.IDSIAMannoSwitzerland

Personalised recommendations