Advertisement

On the Existence and Convergence of Computable Universal Priors

  • Marcus Hutter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2842)

Abstract

Solomonoff unified Occam’s razor and Epicurus’ principle of multiple explanations to one elegant, formal, universal theory of inductive inference, which initiated the field of algorithmic information theory. His central result is that the posterior of his universal semimeasure M converges rapidly to the true sequence generating posterior μ, if the latter is computable. Hence, M is eligible as a universal predictor in case of unknown μ. We investigate the existence and convergence of computable universal (semi)measures for a hierarchy of computability classes: finitely computable, estimable, enumerable, and approximable. For instance, M is known to be enumerable, but not finitely computable, and to dominate all enumerable semimeasures. We define seven classes of (semi)measures based on these four computability concepts. Each class may or may not contain a (semi)measure which dominates all elements of another class. The analysis of these 49 cases can be reduced to four basic cases, two of them being new. We also investigate more closely the types of convergence, possibly implied by universality: in difference and in ratio, with probability 1, in mean sum, and for Martin-Löf random sequences. We introduce a generalized concept of randomness for individual sequences and use it to exhibit difficulties regarding these issues.

Keywords

Random Sequence Turing Machine Computable Function Kolmogorov Complexity Approximable Measure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [Doo53]
    Doob, J.L.: Stochastic Processes. John Wiley & Sons, New York (1953)zbMATHGoogle Scholar
  2. [Hut01]
    Hutter, M.: Convergence and error bounds of universal prediction for general alphabet. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 239–250. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  3. [Hut03]
    Hutter, M.: Sequence prediction based on monotone complexity. In: Proceedings of the 16th Conference on Computational Learning Theory, COLT 2003 (2003)Google Scholar
  4. [Lam87]
    van Lambalgen, M.: Random Sequences. PhD thesis, Univ. Amsterdam (1987)Google Scholar
  5. [Lev73]
    Levin, L.A.: On the notion of a random sequence. Soviet Math. Dokl. 14(5), 1413–1416 (1973)zbMATHGoogle Scholar
  6. [LV97]
    Li, M., Vitányi, P.M.B.: An introduction to Kolmogorov complexity and its applications, 2nd edn. Springer, Heidelberg (1997)zbMATHGoogle Scholar
  7. [Sch71]
    Schnorr, C.P.: Zufälligkeit und Wahrscheinlichkeit. Springer, Berlin (1971)zbMATHGoogle Scholar
  8. [Sch02]
    Schmidhuber, J.: Hierarchies of generalized Kolmogorov complexities and nonenumerable universal measures computable in the limit. Journal of Foundations of Computer Science 13(4), 587–612 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  9. [Sol64]
    Solomonoff, R.J.: A formal theory of inductive inference: Part 1 and 2. Inform. Control 7, 1-22, 224–254 (1964)zbMATHCrossRefMathSciNetGoogle Scholar
  10. [Sol78]
    Solomonoff, R.J.: Complexity-based induction systems: comparisons and convergence theorems. IEEE Trans. Inform. Theory IT-24, 422–432 (1978)CrossRefMathSciNetGoogle Scholar
  11. [VL00]
    Vitányi, P.M.B., Li, M.: Minimum description length induction, Bayesianism, and Kolmogorov complexity. IEEE Transactions on Information Theory 46(2), 446–464 (2000)zbMATHCrossRefGoogle Scholar
  12. [Vov87]
    Vovk, V.G.: On a randomness criterion. Soviet Mathematics Doklady 35(3), 656–660 (1987)zbMATHMathSciNetGoogle Scholar
  13. [Wan96]
    Wang, Y.: Randomness and Complexity. PhD thesis, Univ. Heidelberg (1996)Google Scholar
  14. [ZL70]
    Zvonkin, A.K., Levin, L.A.: The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms. Russian Mathematical Surveys 25(6), 83–124 (1970)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Marcus Hutter
    • 1
  1. 1.IDSIAManno-LuganoSwitzerland

Personalised recommendations