International Conference on Algorithmic Learning Theory

Algorithmic Learning Theory pp 364-378 | Cite as

On the Computability of Solomonoff Induction and Knowledge-Seeking

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9355)

Abstract

Solomonoff induction is held as a gold standard for learning, but it is known to be incomputable. We quantify its incomputability by placing various flavors of Solomonoff’s prior M in the arithmetical hierarchy. We also derive computability bounds for knowledge-seeking agents, and give a limit-computable weakly asymptotically optimal reinforcement learning agent.

Keywords

Solomonoff induction Exploration Knowledge-seeking agents General reinforcement learning Asymptotic optimality  Computability Complexity Arithmetical hierarchy Universal turing machine AIXI BayesExp 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Blackwell, D., Dubins, L.: Merging of opinions with increasing information. The Annals of Mathematical Statistics, 882–886 (1962)Google Scholar
  2. 2.
    Gács, P.: On the relation between descriptional complexity and algorithmic probability. Theoretical Computer Science 22(1–2), 71–93 (1983)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Hutter, M.: A theory of universal artificial intelligence based on algorithmic complexity. Technical Report cs.AI/0004001 (2000). http://arxiv.org/abs/cs.AI/0004001
  4. 4.
    Hutter, M.: New error bounds for Solomonoff prediction. Journal of Computer and System Sciences 62(4), 653–667 (2001)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Hutter, M.: Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer (2005)Google Scholar
  6. 6.
    Lattimore, T.: Theory of General Reinforcement Learning. PhD thesis, Australian National University (2013)Google Scholar
  7. 7.
    Lattimore, T., Hutter, M.: Asymptotically optimal agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 368–382. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  8. 8.
    Lattimore, T., Hutter, M., Gavane, V.: Universal prediction of selected bits. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 262–276. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  9. 9.
    Leike, J., Hutter, M.: Bad universal priors and notions of optimality. In: Conference on Learning Theory (2015)Google Scholar
  10. 10.
    Leike, J., Hutter, M.: On the computability of AIXI. In: Uncertainty in Artificial Intelligence (2015)Google Scholar
  11. 11.
    Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications. Texts in Computer Science, 3rd edn. Springer (2008)Google Scholar
  12. 12.
    Nies, A.: Computability and Randomness. Oxford University Press (2009)Google Scholar
  13. 13.
    Orseau, L.: Optimality issues of universal greedy agents with static priors. In: Hutter, M., Stephan, F., Vovk, V., Zeugmann, T. (eds.) Algorithmic Learning Theory. LNCS, vol. 6331, pp. 345–359. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  14. 14.
    Orseau, L.: Universal knowledge-seeking agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 353–367. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  15. 15.
    Orseau, L.: Asymptotic non-learnability of universal agents with computable horizon functions. Theoretical Computer Science 473, 149–156 (2013)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Orseau, L.: Universal knowledge-seeking agents. Theoretical Computer Science 519, 127–139 (2014)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Orseau, L., Lattimore, T., Hutter, M.: Universal knowledge-seeking agents for stochastic environments. In: Jain, S., Munos, R., Stephan, F., Zeugmann, T. (eds.) ALT 2013. LNCS, vol. 8139, pp. 158–172. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  18. 18.
    Rathmanner, S., Hutter, M.: A philosophical treatise of universal induction. Entropy 13(6), 1076–1136 (2011)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Solomonoff, R.: A formal theory of inductive inference. Parts 1 and 2. Information and Control 7(1), 1–22 and 224–254 (1964)Google Scholar
  20. 20.
    Solomonoff, R.: Complexity-based induction systems: Comparisons and convergence theorems. IEEE Transactions on Information Theory 24(4), 422–432 (1978)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Wood, I., Sunehag, P., Hutter, M.: (Non-)equivalence of universal priors. In: Dowe, D.L. (ed.) Solomonoff Festschrift. LNCS, vol. 7070, pp. 417–425. Springer, Heidelberg (2013) Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Australian National UniversityCanberraAustralia

Personalised recommendations