Algorithmic Statistics: Normal Objects and Universal Models

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9691)


In algorithmic statistics quality of a statistical hypothesis (a model) P for a data x is measured by two parameters: Kolmogorov complexity of the hypothesis and the probability P(x). A class of models \(S_{ij}\) that are the best at this point of view, were discovered. However these models are too abstract.

To restrict the class of hypotheses for a data, Vereshchaginintroduced a notion of a strong model for it. An object is called normal if it can be explained by using strong models not worse than without this restriction. In this paper we show that there are “many types” of normal strings. Our second result states that there is a normal object x such that all models \(S_{ij}\) are not strong for x. Our last result states that every best fit strong model for a normal object is again a normal object.


Algorithmic statistics Minimum description length Stochastic strings Total conditional complexity Sufficient statistic Denoising 



The author is grateful to professor N. K. Vereshchagin for statements of questions, remarks and useful discussions.

This work is supported by RFBR grant 16-01-00362 and partially supported by RaCAF ANR-15-CE40-0016-01 grant. The study has been funded by the Russian Academic Excellence Project ‘5-100’.


  1. 1.
    Gács, P., Tromp, J., Vitányi, P.M.B.: Algorithmic statistics. IEEE Trans. Inform. Theory 47(6), 2443–2463 (2001)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Kolmogorov, A.N.: Three approaches to the quantitative definition of information. Probl. Inf. Trans. 1(1), 1–7 (1965)MathSciNetMATHGoogle Scholar
  3. 3.
    Kolmogorov, A.N.: The complexity of algorithms, the objective definition of randomness. Usp. Matematicheskich Nauk 29(4(178)), 155 (1974). Summary of the talk presented April 16, at Moscow Mathematical SocietyMathSciNetGoogle Scholar
  4. 4.
    Li, M., Vitányi, P.: An Introduction to Kolmogorov complexity and its applications, 3rd edn., xxiii+790 p. Springer, New York (2008). (1st edn. 1993; 2nd edn. 1997), ISBN 978-0-387-49820-1Google Scholar
  5. 5.
    Shen, A.: Game arguments in computability theory and algorithmic information theory. In: Cooper, S.B., Dawar, A., Löwe, B. (eds.) CiE 2012. LNCS, vol. 7318, pp. 655–666. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    Shen, A.: Around kolmogorov complexity: basic notions and results. In: Vovk, V., Papadoupoulos, H., Gammerman, A. (eds.) Measures of Complexity: Festschrift for Alexey Chervonenkis. Springer, Heidelberg (2015)Google Scholar
  7. 7.
    Shen, A.: The concept of \((\alpha, \beta )\)-stochasticity in the Kolmogorov sense, and its properties. Sov. Math. Dokl. 271(1), 295–299 (1983)MATHGoogle Scholar
  8. 8.
    Shen, A., Uspensky, V., Vereshchagin, N.: Kolmogorov complexity and algorithmic randomness. MCCME (2013) (Russian). English translation:
  9. 9.
    Vereshchagin, N.: Algorithmic minimal sufficient statistics: A new approach. Theory Comput. Syst. 56(2), 291–436 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Vereshchagin, N., Vitányi, P.: Kolmogorov’s structure functions with an application to the foundations of model selection. IEEE Trans. Inf. Theory 50(12), 3265–3290 (2004). Preliminary version: Proceedings of the 47th IEEE Symposium on Foundations of Computer Science, pp. 751–760 (2002)CrossRefMATHGoogle Scholar
  11. 11.
    Vitányi, P., Vereshchagin, N.: On algorithmic rate-distortion function. In: Proceedings of 2006 IEEE International Symposium on Information Theory, Seattle, Washington, 9–14 July 2006Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Moscow State UniversityMoscowRussian Federation
  2. 2.National Research University Higher School of EconomicsMoscowRussian Federation

Personalised recommendations