A Divergence Formula for Randomness and Dimension

  • Jack H. Lutz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5635)


If S is an infinite sequence over a finite alphabet Σ and β is a probability measure on Σ, then the dimension of S with respect to β, written \(\dim^\beta(S)\), is a constructive version of Billingsley dimension that coincides with the (constructive Hausdorff) dimension dim(S) when β is the uniform probability measure. This paper shows that dimβ(S) and its dual Dimβ(S), the strong dimension of S with respect to β, can be used in conjunction with randomness to measure the similarity of two probability measures α and β on Σ. Specifically, we prove that the divergence formula

$$ {\mathrm {dim}}^\beta(R) = {\mathrm{Dim}}^\beta(R) =\frac{{\mathcal{H}}(\alpha)}{{\mathcal{H}}(\alpha) + {\mathcal{D}}(\alpha || \beta)} $$

holds whenever α and β are computable, positive probability measures on Σ and R ∈ Σ ∞  is random with respect to α. In this formula, \({\mathcal{H}}(\alpha)\) is the Shannon entropy of α, and \({\mathcal{D}}(\alpha||\beta)\) is the Kullback-Leibler divergence between α and β.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Athreya, K.B., Hitchcock, J.M., Lutz, J.H., Mayordomo, E.: Effective strong dimension, algorithmic information, and computational complexity. SIAM Journal on Computing 37, 671–705 (2007)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Billingsley, P.: Hausdorff dimension in probability theory. Illinois Journal of Mathematics 4, 187–209 (1960)MathSciNetMATHGoogle Scholar
  3. 3.
    Borel, E.: Sur les probabilités dénombrables et leurs applications arithmétiques. Rend. Circ. Mat. Palermo 27, 247–271 (1909)CrossRefMATHGoogle Scholar
  4. 4.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. John Wiley & Sons, Inc., Chichester (2006)MATHGoogle Scholar
  5. 5.
    Dai, J.J., Lathrop, J.I., Lutz, J.H., Mayordomo, E.: Finite-state dimension. Theoretical Computer Science 310, 1–33 (2004)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Eggleston, H.: The fractional dimension of a set defined by decimal properties. Quarterly Journal of Mathematics 20, 31–36 (1949)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Hausdorff, F.: Dimension und äusseres Mass. Mathematische Annalen 79, 157–179 (1919) (English translation)CrossRefMATHGoogle Scholar
  8. 8.
    Hitchcock, J.M.: Effective Fractal Dimension Bibliography (October 2008), http://www.cs.uwyo.edu/~jhitchco/bib/dim.shtml
  9. 9.
    Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and its Applications, 2nd edn. Springer, Berlin (1997)CrossRefMATHGoogle Scholar
  10. 10.
    Lutz, J.H.: Dimension in complexity classes. SIAM Journal on Computing 32, 1236–1259 (2003)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Lutz, J.H.: The dimensions of individual strings and sequences. Information and Computation 187, 49–79 (2003)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Lutz, J.H.: A divergence formula for randomness and dimension. Technical Report cs.CC/0811.1825, Computing Research Repository (2008)Google Scholar
  13. 13.
    Lutz, J.H., Mayordomo, E.: Dimensions of points in self-similar fractals. SIAM Journal on Computing 38, 1080–1112 (2008)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Martin-Löf, P.: The definition of random sequences. Information and Control 9, 602–619 (1966)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Mayordomo, E.: A Kolmogorov complexity characterization of constructive Hausdorff dimension. Information Processing Letters 84(1), 1–3 (2002)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Schnorr, C.P.: A unified approach to the definition of random sequences. Mathematical Systems Theory 5, 246–258 (1971)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Schnorr, C.P.: A survey of the theory of random sequences. In: Butts, R.E., Hintikka, J. (eds.) Basic Problems in Methodology and Linguistics, pp. 193–210. D. Reidel, Dordrecht (1977)CrossRefGoogle Scholar
  18. 18.
    Sullivan, D.: Entropy, Hausdorff measures old and new, and limit sets of geometrically finite Kleinian groups. Acta Mathematica 153, 259–277 (1984)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Tricot, C.: Two definitions of fractional dimension. In: Mathematical Proceedings of the Cambridge Philosophical Society, vol. 91, pp. 57–74 (1982)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jack H. Lutz
    • 1
  1. 1.Department of Computer ScienceIowa State UniversityAmesUSA

Personalised recommendations