Theory of Computing Systems

, Volume 43, Issue 3–4, pp 425–463 | Cite as

Dimension Extractors and Optimal Decompression



A dimension extractor is an algorithm designed to increase the effective dimension—i.e., the amount of computational randomness—of an infinite binary sequence, in order to turn a “partially random” sequence into a “more random” sequence. Extractors are exhibited for various effective dimensions, including constructive, computable, space-bounded, time-bounded, and finite-state dimension. Using similar techniques, the Kučera-Gács theorem is examined from the perspective of decompression, by showing that every infinite sequence S is Turing reducible to a Martin-Löf random sequence R such that the asymptotic number of bits of R needed to compute n bits of S, divided by n, is precisely the constructive dimension of S, which is shown to be the optimal ratio of query bits to computed bits achievable with Turing reductions. The extractors and decompressors that are developed lead directly to new characterizations of some effective dimensions in terms of optimal decompression by Turing reductions.


Dimension Kolmogorov complexity Compression Martingale Algorithmic randomness Extractor 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ambos-Spies, K.: Randomness, relativizations, and polynomial reducibilities. In: Proceedings of the First Structure in Complexity Theory Conference, pp. 23–34 (1986) Google Scholar
  2. 2.
    Athreya, K.B., Hitchcock, J.M., Lutz, J.H., Mayordomo, E.: Effective strong dimension, algorithmic information, and computational complexity. SIAM J. Comput. (to appear). Preliminary version appeared in Diekert, V., Habib, M. (eds.) Proceedings of the 21st International Symposium on Theoretical Aspects of Computer Science, Montpellier, France, March 25–27, 2004. Lecture Notes in Computer Science, pp. 632–643 Google Scholar
  3. 3.
    Barzdin’, Y.M.: Complexity of programs to determine whether natural numbers not greater than n belong to a recursively enumerable set. Sov. Math. Dokl. 9, 1251–1254 (1968) Google Scholar
  4. 4.
    Bennett, C.H.: Logical depth and physical complexity. In: Herken, R. (ed.) The Universal Turing Machine: A Half-Century Survey, pp. 227–257. Oxford University Press, London (1988) Google Scholar
  5. 5.
    Bennett, C.H., Gill, J.: Relative to a random oracle A, \(\mathsf{P}^{A}\neq\mathsf{NP}^{A}\neq\mathsf{co}\mbox{-}\mathsf{NP}^{A}\) with probability 1. SIAM J. Comput. 10, 96–113 (1981) MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Bienvenu, L.: Personal communication (2006) Google Scholar
  7. 7.
    Bienvenu, L., Doty, D., Stephan, F.: Constructive dimension and weak truth-table degrees. In: Cooper, S.B., Löwe, B., Sorbi, A. (eds.) Computation and Logic in the Real World—Third Conference of Computability in Europe. Lecture Notes in Computer Science, vol. 4497. Springer (2007) Google Scholar
  8. 8.
    Book, R.V., Lutz, J.H., Wagner, K.W.: An observation on probability versus randomness with applications to complexity classes. Math. Syst. Theory 27, 201–209 (1994) MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Bourke, C., Hitchcock, J.M., Vinodchandran, N.V.: Entropy rates and finite-state dimension. Theor. Comput. Sci. 349, 392–406 (2005) MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Buhrman, H., Fortnow, L., Newman, I., Vereshchagin, N.: Increasing Kolmogorov complexity. In: Proceedings of the 22nd Symposium on Theoretical Aspects of Computer Science. Lecture Notes in Computer Science, vol. 3404, pp. 412–421. Springer, Berlin (2005) Google Scholar
  11. 11.
    Chaitin, G.J.: A theory of program size formally identical to information theory. J. Assoc. Comput. Mach. 22, 329–340 (1975) MATHMathSciNetGoogle Scholar
  12. 12.
    Dai, J.J., Lathrop, J.I., Lutz, J.H., Mayordomo, E.: Finite-state dimension. Theor. Comput. Sci. 310, 1–33 (2004) MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Doty, D.: Every sequence is decompressible from a random one. In: Beckmann, A., Berger, U., Löwe, B., Tucker, J.V. (eds.) Logical Approaches to Computational Barriers, Second Conference on Computability in Europe, CiE 2006, Swansea, UK, June 30–July 5, 2006, Proceedings. Lecture Notes in Computer Science, vol. 3988, pp. 153–162. Springer, New York (2006) Google Scholar
  14. 14.
    Doty, D., Moser, P.: Finite-state dimension and lossy decompressors. Technical Report cs.CC/0609096, Computing Research Repository (2006) Google Scholar
  15. 15.
    Edgar, G.A.: Classics on Fractals. Westview Press, Oxford (2004) MATHGoogle Scholar
  16. 16.
    Fenner, S.A.: Gales and supergales are equivalent for defining constructive Hausdorff dimension. Technical Report cs.CC/0208044, Computing Research Repository (2002) Google Scholar
  17. 17.
    Fortnow, L., Hitchcock, J., Pavan, A., Vinodchandran, N.V., Wang, F.: Extracting Kolmogorov complexity with applications to dimension zero-one laws. In: Proceedings of the 33rd International Colloquium on Automata, Languages and Programming. Springer, New York (2006) Google Scholar
  18. 18.
    Gács, P.: Every sequence is reducible to a random one. Inf. Control 70, 186–192 (1986) MATHCrossRefGoogle Scholar
  19. 19.
    Gill, J.: Computational complexity of probabilistic Turing machines. SIAM J. Comput. 6, 675–695 (1977) MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Gu, X., Lutz, J.H.: Dimension characterizations of complexity classes. In: Proceedings of the Thirtieth International Symposium on Mathematical Foundations of Computer Science, pp. 471–479. Springer, New York (2006) Google Scholar
  21. 21.
    Hausdorff, F.: Dimension und äußeres Maß. Math. Ann. 79, 157–179 (1919). English version appears in [15], pp. 75–99 CrossRefMathSciNetGoogle Scholar
  22. 22.
    Hitchcock, J.M.: Effective fractal dimension: foundations and applications. Ph.D. Thesis, Iowa State University (2003) Google Scholar
  23. 23.
    Hitchcock, J.M.: Fractal dimension and logarithmic loss unpredictability. Theor. Comput. Sci. 304(1–3), 431–441 (2003) MATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Hitchcock, J.M.: Gales suffice for constructive dimension. Inf. Process. Lett. 86(1), 9–12 (2003) CrossRefMathSciNetGoogle Scholar
  25. 25.
    Hitchcock, J.M.: Personal communication (2006) Google Scholar
  26. 26.
    Huffman, D.A.: Canonical forms for information-lossless finite-state logical machines. In: IRE Trans. Circuit Theory CT-6 (Special Supplement), pp. 41–59 (1959). Also available in Moore, E.F. (ed.) Sequential Machine: Selected Papers, pp. 866–871. Addison-Wesley (1964) Google Scholar
  27. 27.
    Kohavi, Z.: Switching and Finite Automata Theory, 2nd edn. McGraw-Hill, New York (1978) MATHGoogle Scholar
  28. 28.
    Kučera, A.: Measure, Π10-classes and complete extensions of PA. In: Recursion Theory Week. Lecture Notes in Mathematics, vol. 1141, pp. 245–259 (1985) Google Scholar
  29. 29.
    Kučera, A.: On the use of diagonally nonrecursive functions. In: Studies in Logic and the Foundations of Mathematics, vol. 129, pp. 219–239. North-Holland (1989) Google Scholar
  30. 30.
    Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and its Applications, 2nd edn. Springer, Berlin (1997) MATHGoogle Scholar
  31. 31.
    Lutz, J.H.: Almost everywhere high nonuniform complexity. J. Comput. Syst. Sci. 44(2), 220–258 (1992) MATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    Lutz, J.H.: A pseudorandom oracle characterization of BPP. SIAM J. Comput. 22(5), 1075–1086 (1993) MATHCrossRefMathSciNetGoogle Scholar
  33. 33.
    Lutz, J.H.: Dimension in complexity classes. SIAM J. Comput. 32, 1236–1259 (2003) MATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Lutz, J.H.: The dimensions of individual strings and sequences. Inf. Comput. 187, 49–79 (2003) MATHCrossRefMathSciNetGoogle Scholar
  35. 35.
    Martin-Löf, P.: The definition of random sequences. Inf. Control 9, 602–619 (1966) CrossRefGoogle Scholar
  36. 36.
    Mayordomo, E.: A Kolmogorov complexity characterization of constructive Hausdorff dimension. Inf. Process. Lett. 84(1), 1–3 (2002) MATHCrossRefMathSciNetGoogle Scholar
  37. 37.
    Merkle, W., Mihailović, N.: On the construction of effective random sets. J. Symb. Log. 862–878 (2004) Google Scholar
  38. 38.
    Miller, J.S., Nies, A.: Randomness and computability: open questions. Technical Report, University of Auckland (2005) Google Scholar
  39. 39.
    Nies, A., Reimann, J.: A lower cone in the wtt degrees of non-integral effective dimension. In: Proceedings of IMS Workshop on Computational Prospects of Infinity (2006, to appear) Google Scholar
  40. 40.
    Papadimitriou, C.H.: Computational Complexity. Addison-Wesley, Reading (1994) MATHGoogle Scholar
  41. 41.
    Reimann, J.: Computability and fractal dimension. Ph.D. Thesis, Universität Heidelberg (2004) Google Scholar
  42. 42.
    Ryabko, B.Y.: Coding of combinatorial sources and Hausdorff dimension. Sov. Math. Dokl. 30, 219–222 (1984) MATHGoogle Scholar
  43. 43.
    Ryabko, B.Y.: Noiseless coding of combinatorial sources. Prob. Inf. Transm. 22, 170–179 (1986) MATHMathSciNetGoogle Scholar
  44. 44.
    Schnorr, C.P.: A unified approach to the definition of random sequences. Math. Syst. Theory 5, 246–258 (1971) MATHCrossRefMathSciNetGoogle Scholar
  45. 45.
    Schnorr, C.P.: Process complexity and effective random tests. J. Comput. Syst. Sci. 7, 376–388 (1973) MATHCrossRefMathSciNetGoogle Scholar
  46. 46.
    Shaltiel, R.: Recent developments in explicit constructions of extractors. Bull. EATCS 77, 67–95 (2002) MATHMathSciNetGoogle Scholar
  47. 47.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948) Google Scholar
  48. 48.
    Soare, R.I.: Recursively Enumerable Sets and Degrees. Springer, Berlin (1987) Google Scholar
  49. 49.
    Sullivan, D.: Entropy, Hausdorff measures old and new, and limit sets of geometrically finite Kleinian groups. Acta Math. 153, 259–277 (1984) MATHCrossRefMathSciNetGoogle Scholar
  50. 50.
    Tricot, C.: Two definitions of fractional dimension. Math. Proc. Camb. Philos. Soc. 91, 57–74 (1982) MATHCrossRefMathSciNetGoogle Scholar
  51. 51.
    Valdés, M.L., Mayordomo, E.: Dimension is compression. In: Jedrzejowicz, J., Szepietowski, A. (eds.) MFCS. Lecture Notes in Computer Science, vol. 3618, pp. 676–685. Springer, New York (2005) Google Scholar
  52. 52.
    Vereshchagin, N.K., Vyugin, M.V.: Independent minimum length programs to translate between given strings. Theor. Comput. Sci. 271(1–2), 131–143 (2002) MATHCrossRefMathSciNetGoogle Scholar
  53. 53.
    von Neumann, J.: Various techniques for use in connection with random digits. In: von Neumann’s Collected Works, vol. 5, pp. 768–770. Pergamon, Oxford (1963) Google Scholar
  54. 54.
    Ziv, J., Lempel, A.: Compression of individual sequences via variable-rate coding. IEEE Trans. Inf. Theory 24, 530–536 (1978) MATHCrossRefMathSciNetGoogle Scholar
  55. 55.
    Zvonkin, A.K., Levin, L.A.: The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms. Russ. Math. Surv. 25, 83–124 (1970) MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Computer ScienceIowa State UniversityAmesUSA

Personalised recommendations