On the Computability of Solomonoff Induction and Knowledge-Seeking
Solomonoff induction is held as a gold standard for learning, but it is known to be incomputable. We quantify its incomputability by placing various flavors of Solomonoff’s prior M in the arithmetical hierarchy. We also derive computability bounds for knowledge-seeking agents, and give a limit-computable weakly asymptotically optimal reinforcement learning agent.
KeywordsSolomonoff induction Exploration Knowledge-seeking agents General reinforcement learning Asymptotic optimality Computability Complexity Arithmetical hierarchy Universal turing machine AIXI BayesExp
Unable to display preview. Download preview PDF.
- 1.Blackwell, D., Dubins, L.: Merging of opinions with increasing information. The Annals of Mathematical Statistics, 882–886 (1962)Google Scholar
- 3.Hutter, M.: A theory of universal artificial intelligence based on algorithmic complexity. Technical Report cs.AI/0004001 (2000). http://arxiv.org/abs/cs.AI/0004001
- 5.Hutter, M.: Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer (2005)Google Scholar
- 6.Lattimore, T.: Theory of General Reinforcement Learning. PhD thesis, Australian National University (2013)Google Scholar
- 9.Leike, J., Hutter, M.: Bad universal priors and notions of optimality. In: Conference on Learning Theory (2015)Google Scholar
- 10.Leike, J., Hutter, M.: On the computability of AIXI. In: Uncertainty in Artificial Intelligence (2015)Google Scholar
- 11.Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications. Texts in Computer Science, 3rd edn. Springer (2008)Google Scholar
- 12.Nies, A.: Computability and Randomness. Oxford University Press (2009)Google Scholar
- 19.Solomonoff, R.: A formal theory of inductive inference. Parts 1 and 2. Information and Control 7(1), 1–22 and 224–254 (1964)Google Scholar
- 21.Wood, I., Sunehag, P., Hutter, M.: (Non-)equivalence of universal priors. In: Dowe, D.L. (ed.) Solomonoff Festschrift. LNCS, vol. 7070, pp. 417–425. Springer, Heidelberg (2013) Google Scholar