ALT 2011: Algorithmic Learning Theory pp 353-367 | Cite as
Universal Knowledge-Seeking Agents
Abstract
From a point of view of Artificial General Intelligence, RL learners like Hutter’s universal, Pareto optimal, incomputable AIXI heavily rely on the definition of the rewards, which are necessarily given by some “teacher” to define the tasks to solve. AIXI, as is, cannot therefore be said to be a fully autonomous agent.
Furthermore, it has recently been shown that AIXI can converge to a suboptimal behavior in certain situations, hence showing the intrinsic difficulty of RL, with its non-obvious pitfalls.
We propose a new model of intelligence, the Knowledge-Seeking Agent (KSA), halfway between Solomonoff Induction and AIXI, that defines a completely autonomous agent that does not require a teacher. The goal of this agent is not to maximize arbitrary rewards, but “simply” to entirely explore its world in an optimal way. A proof of strong asymptotic optimality for a class of horizon functions shows that this agent, unlike AIXI in its domain, behaves according to expectation. Some implications of such an unusual agent are proposed.
Keywords
Universal Artificial Intelligence AIXI Solomonoff Induction Artificial General IntelligencePreview
Unable to display preview. Download preview PDF.
References
- 1.Hutter, M.: A theory of universal artificial intelligence based on algorithmic complexity. Arxiv (April 2000), http://arxiv.org/abs/cs/0004001
- 2.Hutter, M.: Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probability. Springer, Heidelberg (2005)CrossRefMATHGoogle Scholar
- 3.Hutter, M.: Universal algorithmic intelligence: A mathematical top-down approach. In: Artificial General Intelligence, pp. 227–290. Springer, Heidelberg (2007)CrossRefGoogle Scholar
- 4.Jaynes, E.T., Bretthorst, G.L.: Probability theory: the logic of science. Cambridge University Press, Cambridge (2003)CrossRefGoogle Scholar
- 5.Lattimore, T., Hutter, M.: Asymptotically optimal agents. In: Proc. 22nd International Conf. on Algorithmic Learning Theory (ALT 2011), Espoo, Finland. LNCS (LNAI), vol. 6925, pp. 369–383. Springer, Berlin (2011)Google Scholar
- 6.Li, M., Vitanyi, P.: An Introduction to Kolmogorov Complexity and Its Applications. Springer, New York (2008)CrossRefMATHGoogle Scholar
- 7.Orseau, L., Ring, M.: Self-modification and mortality in artificial agents. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS, vol. 6830, pp. 1–10. Springer, Heidelberg (2011)CrossRefGoogle Scholar
- 8.Orseau, L.: Optimality issues of universal greedy agents with static priors. In: Algorithmic Learning Theory, vol. 6331, pp. 345–359. Springer, Heidelberg (2010)CrossRefGoogle Scholar
- 9.Ring, M., Orseau, L.: Delusion, survival, and intelligent agents. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS, vol. 6830, pp. 11–20. Springer, Heidelberg (2011)CrossRefGoogle Scholar
- 10.Schmidhuber, J.: Driven by compression progress: A simple principle explains essential aspects of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. In: Pezzulo, G., Butz, M.V., Sigaud, O., Baldassarre, G. (eds.) Anticipatory Behavior in Adaptive Learning Systems. LNCS, vol. 5499, pp. 48–76. Springer, Heidelberg (2009)CrossRefGoogle Scholar
- 11.Schmidhuber, J.: Artificial scientists a artists based on the formal theory of creativity. In: Proceedings of the 3d Conference on Artificial General Intelligence (AGI 2010), Lugano, Switzerland, pp. 145–150 (2010)Google Scholar
- 12.Shannon, C.E.: A mathematical theory of communication (parts I and II). Bell System Technical Journal 27, 379–423, 623–656 (1948)Google Scholar
- 13.Solomonoff, R.: Complexity-based induction systems: comparisons and convergence theorems. IEEE transactions on Information Theory 24(4), 422–432 (1978)MathSciNetCrossRefMATHGoogle Scholar
- 14.Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998) (a Bradford Book)Google Scholar
- 15.Veness, J., Ng, K.S., Hutter, M., Silver, D.: A monte carlo AIXI approximation. Arxiv (September 2009), http://arxiv.org/abs/0909.0801