Universal Convergence of Semimeasures on Individual Random Sequences

  • Marcus Hutter
  • Andrej Muchnik
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3244)


Solomonoff’s central result on induction is that the posterior of a universal semimeasure M converges rapidly and with probability 1 to the true sequence generating posterior μ, if the latter is computable. Hence, M is eligible as a universal sequence predictor in case of unknown μ. Despite some nearby results and proofs in the literature, the stronger result of convergence for all (Martin-Löf) random sequences remained open. Such a convergence result would be particularly interesting and natural, since randomness can be defined in terms of M itself. We show that there are universal semimeasures M which do not converge for all random sequences, i.e. we give a partial negative answer to the open problem. We also provide a positive answer for some non-universal semimeasures. We define the incomputable measure D as a mixture over all computable measures and the enumerable semimeasure W as a mixture over all enumerable nearly-measures. We show that W converges to D and D to μ on all random sequences. The Hellinger distance measuring closeness of two distributions plays a central role.


Random Sequence Computable Function Iterate Logarithm Kolmogorov Complexity Hellinger Distance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Hut03a]
    Hutter, M.: Convergence and loss bounds for Bayesian sequence prediction. IEEE Transactions on Information Theory 49(8), 2061–2067 (2003)CrossRefMathSciNetGoogle Scholar
  2. [Hut03b]
    Hutter, M.: On the existence and convergence of computable universal priors. In: Gavaldá, R., Jantke, K.P., Takimoto, E. (eds.) ALT 2003. LNCS (LNAI), vol. 2842, pp. 298–312. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  3. [Hut03c]
    Hutter, M.: An open problem regarding the convergence of universal a priori probability. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT/Kernel 2003. LNCS (LNAI), vol. 2777, pp. 738–740. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  4. [Hut03d]
    Hutter, M.: Sequence prediction based on monotone complexity. In: Proc. 16th Annual Conf. on Learning Theory (COLT 2003). LNCS (LNAI), pp. 506–521. Springer, Heidelberg (2003)Google Scholar
  5. [Lev73]
    Levin, L.A.: On the notion of a random sequence. Soviet Mathematics Doklady 14(5), 1413–1416 (1973)zbMATHGoogle Scholar
  6. [LV97]
    Li, M., Vitányi, P.M.B.: An introduction to Kolmogorov complexity and its applications, 2nd edn. Springer, Heidelberg (1997)zbMATHGoogle Scholar
  7. [ML66]
    Martin-Löf, P.: The definition of random sequences. Information and Control 9(6), 602–619 (1966)CrossRefMathSciNetGoogle Scholar
  8. [MP02]
    Muchnik, A.A., Positselsky, S.Y.: Kolmogorov entropy in the context of computability theory. Theoretical Computer Science 271(1-2), 15–35 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  9. [Sch02]
    Schmidhuber, J.: Hierarchies of generalized Kolmogorov complexities and nonenumerable universal measures computable in the limit. International Journal of Foundations of Computer Science 13(4), 587–612 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  10. [Sol64]
    Solomonoff, R.J.: A formal theory of inductive inference: Part 1 and 2. Information and Control 7(1-22), 224–254 (1964)zbMATHCrossRefMathSciNetGoogle Scholar
  11. [Sol78]
    Solomonoff, R.J.: Complexity-based induction systems: comparisons and convergence theorems. IEEE Transaction on Information Theory, IT 24, 422–432 (1978)zbMATHCrossRefMathSciNetGoogle Scholar
  12. [VL00]
    Vitányi, P.M.B., Li, M.: Minimum description length induction, Bayesianism, and Kolmogorov complexity. IEEE Transactions on Information Theory 46(2), 446–464 (2000)zbMATHCrossRefGoogle Scholar
  13. [Vov87]
    Vovk, V.G.: On a randomness criterion. Soviet Mathematics Doklady 35(3), 656–660 (1987)zbMATHMathSciNetGoogle Scholar
  14. [ZL70]
    Zvonkin, A.K., Levin, L.A.: The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms. Russian Mathematical Surveys 25(6), 83–124 (1970)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Marcus Hutter
    • 1
  • Andrej Muchnik
    • 2
  1. 1.IDSIAManno-LuganoSwitzerland
  2. 2.Institute of New TechnologiesMoscowRussia

Personalised recommendations