Abstract
Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. Solomonoff’s theory of universal induction formally solves the problem of sequence prediction for unknown distributions. We unify both theories and give strong arguments that the resulting universal AIξ model behaves optimally in any computable environment. The major drawback of the AIξ model is that it is uncomputable. To overcome this problem, we construct a modified algorithm AIξ, which is still superior to any other time t and length l bounded agent. The computation time of AIξtl is of the order t·2 l.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
R. Bellman. Dynamic Programming. Princeton University Press, New Jersey, 1957.
D. P. Bertsekas. Dynamic Programming and Optimal Control, Vol. (II). Athena Scientific, Belmont, Massachusetts, 1995.
D. P. Bertsekas and J. N. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientific, Belmont, MA, 1996.
G. J. Chaitin. A theory of program size formally identical to information theory. Journal of the ACM, 22(3):329–340, 1975.
M. Feder, N. Merhav, and M. Gutman. Universal prediction of individual sequences. IEEE Transactions on Information Theory, 38:1258–1270, 1992.
P. Gács. On the symmetry of algorithmic information. Russian Academy of Sciences Doklady. Mathematics (formerly Soviet Mathematics—Doklady), 15:1477–1480, 1974.
M. Hutter. New error bounds for Solomonoff prediction. Journal of Computer and System Science, in press, 1999. ftp://ftp.idsia.ch/pub/techrep/IDSIA-11-00.ps.gz.
M. Hutter. A theory of universal artificial intelligence based on algorithmic complexity. Technical report, 62 pages, 2000. http://arxiv.org/abs/cs.AI/0004001.
M. Hutter. General loss bounds for universal sequence prediction. Technical Report, Manno(Lugano), CH, 2001. http://www.ftp.idsia.ch/pub/techrep/IDSIA-03-01.ps.gz.
L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: a survey. Journal of AI research, 4:237–285, 1996.
A. N. Kolmogorov. Three approaches to the quantitative definition of information. Problems of Information and Transmission, 1(1):1–7, 1965.
L. A. Levin. Universal sequential search problems. Problems of Information Transmission, 9: 265–266, 1973.
L. A. Levin. Laws of information conservation (non-growth) and aspects of the foundation of probability theory. Problems of Information Transmission, 10:206–210, 1974.
M. Li and P. M. B. Vitányi. An introduction to Kolmogorov complexity and its applications. Springer, 2nd edition, 1997.
J. J. Rissanen. Stochastic Complexity in Statistical Inquiry. World Scientific Publ. Co., 1989.
S. J. Russell and P. Norvig. Artificial Intelligence. A Modern Approach. Prentice-Hall, Englewood Cliffs, 1995.
R. Sutton and A. Barto. Reinforcement learning: An introduction. Cambridge, MA, MIT Press, 1998.
J. Schmidhuber. Discovering neural nets with low Kolmogorov complexity and high generalization capability. Neural Networks, 10(5):857–873, 1997.
R. J. Solomonoff. A formal theory of inductive inference: Part 1 and 2. Inform. Control, 7:1–22, 224–254, 1964.
R. J. Solomonoff. Complexity-based induction systems: comparisons and convergence theorems. IEEE Trans. Inform. Theory, IT-24:422–432, 1978.
J. Schmidhuber, J. Zhao, and M. Wiering. Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement. Machine Learning, 28:105–130, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hutter, M. (2001). Towards a Universal Theory of Artificial Intelligence Based on Algorithmic Probability and Sequential Decisions. In: De Raedt, L., Flach, P. (eds) Machine Learning: ECML 2001. ECML 2001. Lecture Notes in Computer Science(), vol 2167. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44795-4_20
Download citation
DOI: https://doi.org/10.1007/3-540-44795-4_20
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42536-6
Online ISBN: 978-3-540-44795-5
eBook Packages: Springer Book Archive