A Divergence Formula for Randomness and Dimension

  • Jack H. Lutz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5635)


If S is an infinite sequence over a finite alphabet Σ and β is a probability measure on Σ, then the dimension of S with respect to β, written \(\dim^\beta(S)\), is a constructive version of Billingsley dimension that coincides with the (constructive Hausdorff) dimension dim(S) when β is the uniform probability measure. This paper shows that dim β (S) and its dual Dim β (S), the strong dimension of S with respect to β, can be used in conjunction with randomness to measure the similarity of two probability measures α and β on Σ. Specifically, we prove that the divergence formula

$$ {\mathrm {dim}}^\beta(R) = {\mathrm{Dim}}^\beta(R) =\frac{{\mathcal{H}}(\alpha)}{{\mathcal{H}}(\alpha) + {\mathcal{D}}(\alpha || \beta)} $$

holds whenever α and β are computable, positive probability measures on Σ and R ∈ Σ ∞  is random with respect to α. In this formula, \({\mathcal{H}}(\alpha)\) is the Shannon entropy of α, and \({\mathcal{D}}(\alpha||\beta)\) is the Kullback-Leibler divergence between α and β.


Probability Measure Shannon Entropy SIAM Journal Divergence Formula Kolmogorov Complexity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Athreya, K.B., Hitchcock, J.M., Lutz, J.H., Mayordomo, E.: Effective strong dimension, algorithmic information, and computational complexity. SIAM Journal on Computing 37, 671–705 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Billingsley, P.: Hausdorff dimension in probability theory. Illinois Journal of Mathematics 4, 187–209 (1960)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Borel, E.: Sur les probabilités dénombrables et leurs applications arithmétiques. Rend. Circ. Mat. Palermo 27, 247–271 (1909)CrossRefzbMATHGoogle Scholar
  4. 4.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. John Wiley & Sons, Inc., Chichester (2006)zbMATHGoogle Scholar
  5. 5.
    Dai, J.J., Lathrop, J.I., Lutz, J.H., Mayordomo, E.: Finite-state dimension. Theoretical Computer Science 310, 1–33 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Eggleston, H.: The fractional dimension of a set defined by decimal properties. Quarterly Journal of Mathematics 20, 31–36 (1949)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Hausdorff, F.: Dimension und äusseres Mass. Mathematische Annalen 79, 157–179 (1919) (English translation)CrossRefzbMATHGoogle Scholar
  8. 8.
    Hitchcock, J.M.: Effective Fractal Dimension Bibliography (October 2008),
  9. 9.
    Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and its Applications, 2nd edn. Springer, Berlin (1997)CrossRefzbMATHGoogle Scholar
  10. 10.
    Lutz, J.H.: Dimension in complexity classes. SIAM Journal on Computing 32, 1236–1259 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Lutz, J.H.: The dimensions of individual strings and sequences. Information and Computation 187, 49–79 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Lutz, J.H.: A divergence formula for randomness and dimension. Technical Report cs.CC/0811.1825, Computing Research Repository (2008)Google Scholar
  13. 13.
    Lutz, J.H., Mayordomo, E.: Dimensions of points in self-similar fractals. SIAM Journal on Computing 38, 1080–1112 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Martin-Löf, P.: The definition of random sequences. Information and Control 9, 602–619 (1966)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Mayordomo, E.: A Kolmogorov complexity characterization of constructive Hausdorff dimension. Information Processing Letters 84(1), 1–3 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Schnorr, C.P.: A unified approach to the definition of random sequences. Mathematical Systems Theory 5, 246–258 (1971)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Schnorr, C.P.: A survey of the theory of random sequences. In: Butts, R.E., Hintikka, J. (eds.) Basic Problems in Methodology and Linguistics, pp. 193–210. D. Reidel, Dordrecht (1977)CrossRefGoogle Scholar
  18. 18.
    Sullivan, D.: Entropy, Hausdorff measures old and new, and limit sets of geometrically finite Kleinian groups. Acta Mathematica 153, 259–277 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Tricot, C.: Two definitions of fractional dimension. In: Mathematical Proceedings of the Cambridge Philosophical Society, vol. 91, pp. 57–74 (1982)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jack H. Lutz
    • 1
  1. 1.Department of Computer ScienceIowa State UniversityAmesUSA

Personalised recommendations