The Incompressibility Method

  • Ming Li
  • Paul Vitányi
Part of the Graduate Texts in Computer Science book series (TCS)


The incompressibility of random objects yields a simple but powerful proof technique. The incompressibility method is a general-purpose tool and should be compared with the pigeon-hole principle or the probabilistic method. Whereas the older methods generally show the existence of an object with the required properties, the incompressibility argument shows that almost all objects have the required property. This follows immediately from the fact that the argument is typically used on a Kolmogorov random object. Since such objects are effectively indistinguishable, the proof holds for all such objects. Each class of objects has an abundance of objects that are Kolmogorov random relative to the class.


Random Graph Turing Machine Label Graph Kolmogorov Complexity Longe Common Subsequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

History and References

  1. Apparently, W.J. Paul [Proc. Int. Conf. Fund. Comput. Theory, L. Budach (Ed.), 1979, pp. 325-334]Google Scholar
  2. R.V. Freivalds [‘On the running time of deterministic and nondeterministic Turing machines,’ Latv. Mat. Ezhegodnik, 23 (1979) 158–165 (in Russian)]MathSciNetzbMATHGoogle Scholar
  3. W.J. Paul, J. Seiferas, and J. Simon, [J. Comput. System Sci., 23:2(1981), 108–126].MathSciNetzbMATHCrossRefGoogle Scholar
  4. W.J. Paul, which contains Section 6.1.1, page 380, was not widely circulated.Google Scholar
  5. S.O. Aanderaa [pp.75-96 in: Complexity of Computation, R. Karp (Ed.), Amer. Math. Soc, 1974].Google Scholar
  6. M.O. Rabin [Israel J. Math., 1(1963), 203–211]MathSciNetzbMATHCrossRefGoogle Scholar
  7. W.J. Paul [Inform. Contr., 53(1982), 1–8]zbMATHCrossRefGoogle Scholar
  8. S. Reisch and G. Schnitger [Proc. 23rd IEEE Found. Comput. Sci., 1982, pp. 45-52]Google Scholar
  9. F.C. Hennie using a counting argument in [Inform. Contr., 8:6(1965), 553–578].MathSciNetCrossRefGoogle Scholar
  10. W.J. Paul. Section 6.1.2 is based on [R. Beigel, W. Gasarch, M. Li, and L. Zhang,’ Addition in log2 n + O(1) steps on average: a simple analysis,’ Manuscript, 1996].Google Scholar
  11. [A.W. Burks, H.H. Goldstine, and J. von Neumann, ‘Preliminary discussion of the logical design of an electronic computing instrument. Institute for Advanced Studies,’ Report (1946). Reprinted in John von Neumann Collected Works, vol 5 (1961)].Google Scholar
  12. [B.E. Briley, IEEE Trans. Computers, C-22:5(1973)]Google Scholar
  13. [G. Schay, Amer. Math. Monthly, 102:8(1995), 725–730].MathSciNetzbMATHCrossRefGoogle Scholar
  14. [K. Hwang, Computer arithmetic: principles, architecture, and design, Wiley, New York, 1979].Google Scholar
  15. [Personal communication, 1986].Google Scholar
  16. R. Paturi, J. Simon, R. Newman-Wolfe, and J. Seiferas, [Inform. Comput., 88(1990), 88–104].MathSciNetzbMATHCrossRefGoogle Scholar
  17. [H. Buhrman, M. Li, and P.M.B. Vitányi, ‘Kolmogorov random graphs,’ Manuscript, CWI, Amsterdam, November 1995].Google Scholar
  18. [M. van Lambalgen, Random Sequences, PhD Thesis, Universiteit van Amsterdam, Amsterdam, 1987]Google Scholar
  19. [V.V. V’yugin, ‘Ergodic theorem for individual random sequences,’ Manuscript, Inst. Inform. Transm. Probl., Russ. Acad. Sci., Moscow, Russia, 1996].Google Scholar
  20. [M. Li and P.M.B. Vitányi, J. Comb. Theory, Ser. A, 66:2(1994), 226–236].zbMATHCrossRefGoogle Scholar
  21. [P. Erdős and J. Spencer, Probabilistic methods in combinatorics, Academic Press, 1974].Google Scholar
  22. [N. Alon, J. Spencer, and P. Erdős, The probabilistic method, Wiley, 1992].Google Scholar
  23. [P. Erdős, Bull. Amer. Math. Soc, 53(1947), 292–294].MathSciNetCrossRefGoogle Scholar
  24. [P. Erdős and J. Spencer, Ibid.].Google Scholar
  25. P. Erdős and A. Rényi [Publ. Hungar. Acad. Sci., 8(1963), 241–254].Google Scholar
  26. L. Moser [Combinatorial structures and their applications, Gordon and Breach, 1970, pp. 283-384].Google Scholar
  27. N. Pippenger [J. Combinat. Theory, Ser. A, 23(1977), 105–115].MathSciNetCrossRefGoogle Scholar
  28. [J. Kahn and J. Kim, Proc. 24th ACM Symp. Theory Comput., 1992, pp. 178-187].Google Scholar
  29. [I. Newman, P. Ragde, and A. Wigderson, 5th IEEE Conf. Structure in Complexity Theory, 1990, pp. 78-87].Google Scholar
  30. [R. Boppana, Proc. 21st ACM Symp. Theory Comput., 1989, pp. 320-326].Google Scholar
  31. [H. Buhrman, M. Li, P. Vitányi, ‘Kolmogorov random graphs,’ Manuscript, CWI, Amsterdam, November 1995].Google Scholar
  32. [B. Bollobas, Random graphs, Academic Press, 1985].Google Scholar
  33. [N. Alon, J. Spencer, and P. Erdős, The probabilistic method, Wiley, 1992] pp. 125-140.Google Scholar
  34. [H. Buhrman, J.H. Hoepman, and P.M.B. Vitányi, Proc. 15th ACM Symp. Principles Distribute Cornput., 1996, pp. 134-142].Google Scholar
  35. Related research (see exercises) appears in [E. Kranakis and D. Krizanc, ‘Boolean Routing on Cayley Networks,’ Proc. Snd Int’l Colloq. Structure Inform. Communication Complexity, Siena, Italy, 1996, to appear; E. Kranakis and D. Krizanc, Proc. 13th Symp. Theoret. Aspects Comput. Sci., 1996, pp. 529-540.Google Scholar
  36. E. Kranakis, D. Krizanc, and F. Luccio, Proc. 13th Symp. Math. Found. Comput. Sci., 1995, pp. 392-401].Google Scholar
  37. J.W.J. Williams [Comm. ACM, 7(1964), 347–348]. R.W. Floyd [Comm. ACM, 7(1964), 701]Google Scholar
  38. R. Schaffer and R. Sedgewick [J. Algorithms, 15(1993), 76–100].MathSciNetzbMATHCrossRefGoogle Scholar
  39. [T. Jiang and M. Li, SIAM J. Comput., 24:5(1995), 1122–1139].MathSciNetzbMATHCrossRefGoogle Scholar
  40. [M. Li and P.M.B. Vitányi, SIAM J. Comput., 24:2(1995), 398–410].MathSciNetzbMATHCrossRefGoogle Scholar
  41. [M.A. Harrison, Introduction to formal language theory, Addison-Wesley, 1978; J.E. Hopcroft and J.D. Ullman, Introduction to automata theory, languages, and computation, Addison-Wesley, 1979].Google Scholar
  42. [Inform. Contr., 69(1986), 255-260].Google Scholar
  43. H. Gallaire [Inform. Contr., 15(1969), 288–295].MathSciNetzbMATHCrossRefGoogle Scholar
  44. T. Kasami [Inform. Contr., 10(1967), 209–214]MathSciNetzbMATHCrossRefGoogle Scholar
  45. [J. Hartmanis and R. Stearns, Trans. Amer. Math. Soc., 117(1969), 285–306]MathSciNetCrossRefGoogle Scholar
  46. [F.C. Hennie and R. Stearns, J. ACM, 4(1966), 533–546].MathSciNetCrossRefGoogle Scholar
  47. P. Duris, Z. Galil, W.J. Paul, and R. Reischuk [Inform. Contr., 60(1984), 1–11]MathSciNetzbMATHCrossRefGoogle Scholar
  48. W.J. Paul [Inform. Contr., 53(1982), 1–8]zbMATHCrossRefGoogle Scholar
  49. [P.M.B. Vitányi, Inform. Process. Lett., 21(1985), 87–91 and 147-152]MathSciNetzbMATHCrossRefGoogle Scholar
  50. Z. Galil, R. Kannan, and E. Szemerédi [Proc. 18th ACM Symp. Theory Comput., 1986, pp. 39-49]Google Scholar
  51. [W. Maass, Trans. Amer. Math. Soc., 292(1985), 675–693MathSciNetzbMATHCrossRefGoogle Scholar
  52. M. Li and P.M.B. Vitányi, Inform. Comput., 78(1988), 56–85].zbMATHCrossRefGoogle Scholar
  53. P. Beame [Inform. Comput., 76(1988), 13–28]MathSciNetzbMATHCrossRefGoogle Scholar
  54. M. Li and Y. Yesha [J. ACM, 36:3(1989), 671–680]MathSciNetzbMATHCrossRefGoogle Scholar
  55. F. Meyer auf der Heide and A. Wigderson [SIAM J. Comput., 16(1987), 100–107]MathSciNetzbMATHCrossRefGoogle Scholar
  56. A. Israeli and S. Moran [private communication, 1985] and by I. Parberry [Ph.D. Thesis, 1984, Warwick University].Google Scholar
  57. L. Fortnow and S. Laplante [Inform. Comput, 123(1995), 121–126]MathSciNetzbMATHCrossRefGoogle Scholar
  58. A. Razborov [pp. 344-386 in Feasible Mathematics II, P. Clote, J. Remmel (Eds), 1995].Google Scholar
  59. J. Hástad [pp. 143-170 in Randomness and Computation, S. Micali (Ed), JAI Press, 1989].Google Scholar
  60. [A. Yao, Proc. 26th IEEE Symp. Found. Comput. Sci., 1985, pp. 1-10].Google Scholar
  61. A.M. Ben-Amram and Z. Galil [J. ACM, 39:3(1992)].Google Scholar
  62. [J. Shallit and Y. Breitbart, Proc. 11th Symp. Theoret. Aspects Comput. Sci., 1994, pp. 619-630.J. Comput. System Sci.].Google Scholar
  63. U. Vazirani and V. Vazirani [Theoret. Comput. Sci., 24(1983), 291–300]MathSciNetzbMATHCrossRefGoogle Scholar
  64. H.G. Mairson [Proc. 24th IEEE Found. Comput. Sci., 1983, pp. 40-47].Google Scholar
  65. [K. Mehlhorn, Proc. 23rd IEEE Found. Comput. Sci., 1982, 170-175].Google Scholar
  66. D. Hammer and A.Kh. Shen’ [A strange application of Kolmogorov complexity, Math. Systems Theory, to appear]Google Scholar
  67. M. Zimand [Inform. Process. Lett, 57(1996), 59–64]MathSciNetzbMATHCrossRefGoogle Scholar
  68. J. Tyszkiewicz in [‘The Kolmogorov expression complexity of logics,’ in Proc. Comput. Sci. Logic Conf., 1995]Google Scholar
  69. [Proc. 8th Intern. Conf. Database Theory, Lecture Notes in Computer Science, Vol 893, G. Gottlob and M.Y. Vardi (Eds), Springer-Verlag, 1995, pp. 97-110].Google Scholar
  70. [R. Book, SIAM J. Comput., 23(1994), 1275–1282].MathSciNetzbMATHCrossRefGoogle Scholar
  71. K.W. Regan [Proc. 10th IEEE Conf. Structure in Complexity Theory, 1995, pp. 50-64].Google Scholar

Copyright information

© Springer Science+Business Media New York 1997

Authors and Affiliations

  • Ming Li
    • 1
  • Paul Vitányi
    • 2
  1. 1.Department of Computer ScienceUniversity of WaterlooWaterlooCanada
  2. 2.Centrum voor Wiskunde en InformaticaSJ AmsterdamThe Netherlands

Personalised recommendations